My R program produced a dataframe1 and write it into databaseA(eg.oracle database),but another databaseB(eg.MySQL) want this dataframe1,
How do I synchronization databaseA's dataframe1 to MySQL databaseB using the URL from databaseB? Do I need to make asynchronous requests using any R packages?
my R program on the offline server? Do I have to add the URL to my previous R program？
databaseB's URL:http://10.234.200.173:8088/Server/data/forecast/synchronize?kpi_no=$kpi_id, which offered by my other department colleague.
Related
I am trying to build a fully automated and sustainable reporting tool in Power BI. I have built a report in Power BI that among other things uses R scripting at one point to create a data export to my local C: drive which is the following code:
# 'dataset' holds the input data for this script
.libPaths(.libPaths()[3])
require(gdata)
write.table(trim(dataset), file="C:\\Users\\Username\\OneDrive\\Folder\\Inventory Log.csv", sep=",", row.names=FALSE, append=TRUE, col.names=FALSE)
plot(dataset);
While all my other data is connected to PBI via OneDrive or online sources, this is still connected to my local machine. I have personal gateway setup but that still requires my local machine to be physically on during the scheduled refresh on the PBI service.
I have tried used the Microsoft365R Package but my R knowledge and experience is still limited so I wasn't able to come up with a solution that would allow file="OneDrive Path" within the write.table() function to successfully execute on Power BI Desktop, let alone Power BI Service.
The goal is to fully automate and not require me to have my computer on during the weekends or a non work day.
Is it possible to write a csv to a OneDrive file? If so, what are some ways that have worked successfully?
Any ideas? Thank you for any help you can provide!
Microsoft365R author here. Disclaimer: I'm not familiar with PowerBI, but I assume you can run R scripts inside it and connect to the Internet etc.
There's a few things needed to get this to run unattended.
A function to upload a data frame as CSV to Onedrive, without requiring you to login, is as follows:
upload <- function(dataset, upload_path, drive_id, ...)
{
outfile <- tempfile()
on.exit(unlink(outfile))
write.table(dataset, outfile, ...)
library(Microsoft365R)
library(AzureGraph)
gr <- create_graph_login(
tenant="{yourtenant}",
app="{client_id}",
password="{client_secret}",
auth_type="client_credentials"
)
gr$get_drive(drive_id)$upload_file(outfile, upload_path)
}
On the AAD side, create an app registration and give it a client secret (password). You then give it the Microsoft Graph application permissions necessary to read drives--most likely "Files.Readwrite.All".
Note down the client ID and client secret for your app registration. These are the values you plug into the R function above.
You can get your drive ID with the following R code.
drv <- get_business_onedrive(tenant="yourtenant")
drv$properties$id
You'll probably need the help of your friendly local admin to get all this done, if only because most orgs lock down the ability to register apps.
I'm trying to read in SAS data into R similarly to this quick tutorial here: https://josezea.wordpress.com/2019/05/02/connect-sas-server-from-r/
The issue is that the server I'm trying to connect to uses an IOM protocol, which doesn't seem to be supported in the RCurl package. Does anyone have any suggestions to reading data from a SAS Server with these protocols in R? It can be reading from a file pathway or a library, either works for my scenario. Thanks!
Below is the code I attempted to run in R:
library(RCurl)
library(haven)
protocol <- "IOM"
server <- "server.com:5555"
userpwd <- "username:password"
sasfileRoute <- "/path_to_data/bonus_schedule.sas7bdat"
## Read Data as data frame
url <- paste0(protocol, "://", server, sasfileRoute)
binary_sasdata <- getBinaryURL(url = url, userpwd=userpwd)
df_data = read_sas(binary_sasdata)
I think you're misunderstanding what the linked page does. It shows how to use R to read in a SAS dataset - but not to connect to SAS.
SAS datasets are usually stored as .sas7bdat files. You should connect via SFTP or network share or similar to access datasets; this won't work if the datasets are stored in a LASR server or other in-memory location of course.
If you need to connect to SAS itself (to execute code or to access in-memory data), you can do so if the SAS server is a SAS Viya server. See R SWAT for more information on that; it uses SAS's APIs to do what you need.
Otherwise, you will have to run the SAS executable from inside R (if you have access to that version of SAS), or have a SAS user export your data for you from inside SAS. I am not familiar with a way to connect to SAS 9 via R directly, and the link in comments seems out of date (CRAN at least doesn't seem to have that package any more).
SASPy does allow Python to do something similar with SAS 9.4, so perhaps that's a better route if you have SAS 9.4.
IOM is a SAS protocol used by its Integration Technologies product. Instead of IOM, find the network path to the file and use it. See resources using IOM (C#, Java, PowerShell, etc.) on the SAS website. Usually, IOM is used for code submission and SAS object management.
I have developed a relatively simply R shiny app that does the following:
I input a txt file.
The txt file is processed.
A new object is created: timestamp + a number.
I save it in my sql database for instance or I save it as a .txt on my server.
I want to configure an API such as on a specific request it would pull the new object through the R shiny server. I believe I could use plumber but that means to run 2 servers and somehow coordinate the data flow.
Is there a simple solution such as I could get the data using my R shiny server using an api tool and a simple GET?
Thank you
I'm working on a process improvement that will use SQL in r to work with large datasets. Currently the source data is stored in several different MS Access databases. My initial approach was to use RODBC to read all of the source data into r, and then use sqldf() to summarize the data as needed. I'm running out of RAM before I can even begin use sqldf() though.
Is there a more efficient way for me to complete this task using r? I've been looking for a way to run a SQL query that joins the separate databases before reading them into r, but so far I haven't found any packages that support this functionality.
Should your data be in a database dplyr (a part of the tidyverse) would be the tool you are looking for.
You can use it to connect to a local / remote database, push your joins / filters / whatever there and collect() the result as a data frame. You will find the process neatly summarized on http://db.rstudio.com/dplyr/
What I am not quite certain of - but it is not a R issue but rather an MS Access issue - is the means for accessing data across multiple MS Access databases.
You may need to write custom SQL code for that & pass it to one of the databases via DBI::dbGetQuery() and have MS Access handle the database link.
The link you posted looks promising. If it doesn't yield the intended results, consider linking one Access DB to all the others. Links take almost no memory. Union the links and fetch the data from there.
# Load RODBC package
library(RODBC)
# Connect to Access db
channel <- odbcConnectAccess("C:/Documents/Name_Of_My_Access_Database")
# Get data
data <- sqlQuery(channel , paste ("select * from Name_of_table_in_my_database"))
These URLs may help as well.
https://www.r-bloggers.com/getting-access-data-into-r/
How to connect R with Access database in 64-bit Window?
I am trying to connect to a socket.io data source using R.
Specifically I am trying to connect to CoinCap https://github.com/CoinCapDev/CoinCap.io.
I started by trying the websockets package from here but I could not get a connection. Maybe it is not socket.io compliant.
The best example appears to be in this post which asks the same question.
It seems the answer was to create a socket.io server as a middleman and then connect to R.
The problem is that I am not nearly as advanced as jeromefroe and have no experience with sockets or javascript and I have do not understand how the server that he created works or how to build or start it.
jeromefroe provides his javascript server code in the post, and I don't know what to do with it.
I am trying to collect data in R and use for analysis.
Can somebody help me get the connection running and/or help me set up the sever like jeromefroe did for the connection?
If I understand your question correctly, you are trying to "collect data in R and use for analysis". The website provides the REST URLs and so it is a matter of doing a http GET to retrieve data. An example usage of the httr package as follows. The result retrieved is in json format. Hence, you need jsonlite package to convert into a R data structure.
library(httr)
library(jsonlite)
resp <- httr::GET("http://coincap.io/coins")
jsonlite::fromJSON(rawToChar(resp$content))