I have two databases in Azure and each has 5 tables. I can perform data wrangling inside Azure using Kusto but I would rather prefer using RStudio. I wish to connect R with Azure such that I can run a script in R and return results without importing the actual datasets.
Please help, where do I start? I have zero knowledge of such connections.
Thank you in advance.
Assuming you have already installed R and RStudio. Please follow below steps:
Open ODBC Data Source through Start Window and Add a User Data Source under 'User DSN' as below. Follow through the next buttons until finish and test the connections.
Go to RStudio and create new connection, you should see the Data Source added in above step. You should see the connection and table listed under Azure Sql Database that you connected with.
Run the query like below in Console:
dbGetQuery(con, "SELECT * from dbo.xxxx")
You should be able to see the result set accordingly. You can play with queries in the way you want.
Related
I'm new to R Shiny and I would like to host my application on shinyapps.io. My app.R file gets data from my local SQL database, then I use R to manipulate that data for my R Shiny app. When I upload to shinyapps.io I get error code 1 that seems rather ambiguous based on my google searches.
The current workflow for my project is:
Scrape an API and clean/manipulate the data in R.
Using R and the dbWriteTable function, write the cleaned data into my local SQL Express db. Some data is overwritten, some is appended.
Steps 1 & 2 are automated to run either hourly or daily using Windows task scheduler.
Access the data in SQL and store in R for my app using the following code for the various SQL tables. I am not using any SQL queries in R to manipulate the SQL data before it is stored in R.
con <- dbConnect(odbc(),
Driver = "SQL Server",
Server = "laptop\\SQLEXPRESS",
Database = "myDB",
Trusted_Connection = "True")
tradingLog <- as.data.frame(tbl(con,"tradingLog"))
The above code is being repeated within the app.R shiny script for different SQL tables to supply the data for my app, but my novice understanding is that having the local database will be a problem when I want to host the app on shiny apps online. The application is fully functional running on my computer but breaks when on shinyapps, so that has led me to believe I would need to host my SQL database.
To accomplish this I think I can use Google Cloud, which I have found some resources on how to import my SQL database. If Google Cloud is a viable option, I would prefer to use it for the project because my application is using RgoogleMaps so I already setup the Google Cloud account. From here I assume I can change the driver and server settings in the dbConnect code above and this may address the shinyapps problem?
Any advice or direction on how address the problem would be appreciated. My goal is to be able to share this project with team members via shinyapps.io while accessing the SQL database I have setup that has the various scraping tasks scheduled which feed new data into the database.
Try this Database basics - dplyr and DBI
I have started using DBI and RODBC packages as a way to talk to the ODBC interface and write a dataframe as a ODBC table to be accessible during queries.
My problem is, while I can write a table by using either dbWriteTableor sqlSave, I can't access it.
When I explore the available tables on my ODBC connection, my test table appears in my personal schema but when I try to access it via SELECT or even desc the "table or view does not exist" error appears.
The problem is only accessing the database because I can properly update or remove the table using either ODBC R package or even using SQL Developer.
PS: If I create the table using the import function in SQL Developer I can properly access the table but my goal is to properly access it after writing it using an R function
Recently tableau gave the functionality of R connection in their release 8.1. I want to know if there is any way i can call an entire table created in R to tableau. Or an .rds object which contains the dataset into Tableau?
There is a tutorial on the Tableau website for this and a blog on r-bloggers which discuss. The tutorial has a number of comments and one of them (in early Dec I think) asks how to get an rds file in. You need to start Rserve and then execute a script on it to get your data.
Sorry I can't be more help as I only looked into it briefly and put it on the back-burner but if you get stuck they seem to come back quickly if you post a comment on the page:
http://www.tableausoftware.com/about/blog/2013/10/tableau-81-and-r-25327
Just pointing out that the Tableau Data Extract API might be useful here, even if the current version of R integration doesn't yet meet your needs. (Note, that link is to the version 8.1 docs released in late 2013 - so look for the latest version to see what functionality they've added since)
If what you want to do is to manipulate data in R and then send a table of data to Tableau for visualization, you could first try the simple step of exporting the data from R as a CSV file and then visualizing that data in Tableau. I know that's not sexy, but its always good to make sure you've got a way to get the output result you need before investing time in optimizing the process.
If that gets the effect you want, but you just want to automate more of the steps, then take a look at the Tableau Data Extract API. You could use that library to generate a Tableau Data Extract instead of a CSV file. If you have something in production that needs updates, then you could presumably create a python script or JVM program to read your RDS file periodically and generate a revised extract.
Let us assume your data.frame/ tibble etc (say dataset object) is ready in R/ RStudio and you want to connect it with Tableau
1. In RStudio (or R terminal), execute the following steps:
install.packages("Rserve")
library(Rserve)
Rserve() ##This gets the R connection service up and running
2. Now go to Tableau (I am using 10.3.2):
Help > Settings and Performances > Manage External Service Connection
Enter localhost in the Server field and click on Test Connection.
You have now established a connection between R and Tableau.
3. Come back to RStudio. Now we need a .rdatafile that will consist of our R object(s). In this case, dataset. This is the R object that we want to use in Tableau. Enter this in the R console:
save(dataset, file="objectName.rdata")
4. Switch to Tableau now.
Connect To a File > Statistical File
Go to your working directory where the newly created objectName.rdata resides. From the drop down list of file type, select R files (*.rdata, *.rda) and select your object. This will open the object you created in R in Tableau. Alternatively, you can drag and drop your object directly to Tableau's workspace.
For some reason neither SQL managment studio nor import and export wizard works on my friends network due to some security policy
i asked a question about OSQL two days ago and StackOverflow guys told me it is going to be obselete and that i should use SQLCMD instead.
now i am stuck with this screen and cant do anything :(
to cut long things short my question is very precise:
using SQLCMD..
1-need to import and export command from AND to MDB/MDF
2-need attach / detach commands
mokokamello
below is how to attach new database and make sure your database is attached
you also will need to check these links
http://msdn.microsoft.com/en-us/library/ms165673.aspx
http://msdn.microsoft.com/en-us/library/ms188031.aspx
AND THE MOST IMPORTANT LINK
http://support.microsoft.com/kb/224071
if you want to dettach
sp_detach_db DatabaseName
Go
SQLCMD allows you to enter Transact-SQL statements to the chosen server.
The T-SQL command for connecting to an external data source is OPENDATASOURCE
More info here:
You should be able to construct a series of queries using this in order to pull in, and export the data you are working with.
The SProc to detatch a database is sp_detach_db, and to attach sp_attach_db
More information here, here and here
With unixODBC you can use a simple command line utility called "isql" to test your connection and permissions of some queries. Without having to write extra code or install libs or bloated programs, is there a simple way to open up X data source send some sql commands and be done with it?
Doing this on the command line would be preferable.
One way to create a quick test query in Windows via an ODBC connection is using the DQY format.
To achieve this, create a DQY file (e.g. test.dqy) containing the magic first two lines (XLODBC and 1) as below, followed by your ODBC connection string on the third line and your query on the fourth line (all on one line), e.g.:
XLODBC
1
Driver={Microsoft ODBC for Oracle};server=DB;uid=scott;pwd=tiger;
SELECT COUNT(1) n FROM emp
Then, if you open the file by double-clicking it, it will open in Excel and populate the worksheet with the results of the query.
Make a file SOMEFILENAME.udl then double click on it and set it up as an ODBC connection object, username, pwd, target server
You can use the "Test Connection" feature after creating the ODBC connection through Control Panel > Administrative Tools > Data Sources.
To test a SQL command itself you could try:
http://www.sqledit.com/odbc/runner.html
http://www.sqledit.com/sqlrun.zip
Or (perhaps easier and more useful in the long run) you can make a test ASP.NET or PHP page in a couple minutes to run SQL statement yourself through IIS.
For ad hoc queries, the ODBC Test utility is pretty handy. Its design and interface is more oriented toward testing various parts of the ODBC API. But it works quite nicely for running queries and showing the output. It is part of the Microsoft Data Access Components.
To run a query, you can click the connect button (or use ctrl-F), choose a data source, type a query, then ctrl-E to execute it and ctrl-R to display the results (e.g., if it is a SELECT or something that returns a cursor).
a simple way is:
create a fake "*.UDL" file on desktop
(UDL files are described here:
https://msdn.microsoft.com/en-us/library/e38h511e(v=vs.71).aspx.
in case you can also customized it as explained there. )
It's been a while but since I precisely have the answer to the question, I'll share it and maybe someone will benefit from it.
Jaime de Los Hoyos wrote a very nice program to precisely do that: ODBC Query Tool.
Unfortunately, Jaime's website is defunct but you can still find the program and its source code at this location:
https://sourceforge.net/projects/odbc-query-tool/files/latest_release/
The program is GUI based and consists of a single executable file, no need to install anything on the machine.
Jaime's profile : https://stackoverflow.com/users/878998/jaime-de-los-hoyos-m
Jaime's posts on a forum talking about his program : https://forum.powerbasic.com/forum/user-to-user-discussions/source-code/48266-odbc-query-tool-retrieve-information-from-any-database-easily