Using RODBC in R to connect to Azure SQL - r

I am using R 3.2.4 and El captain, I wanted to connect to Azure SQL using the RODBC package in R, I did the following to install it
brew install unixodbc
install.packages("RODBC",type="source")
And also
brew install freetds --with-unixodbc
The output of above command:
Warning: unixodbc-2.3.4 already installed
Warning: freetds-0.95.80 already installed
But whenever I try to connect using the following:
library("RODBC")
con = odbcDriverConnect(
'driver = {SQL Server};
Server = xxxxxx;
Database = xxxxx;
User Id= xxxxx;
Password= xxxxx;')
I get the following Error:
[RODBC] ERROR: state IM007, code 59478176, message [iODBC][Driver Manager]No data source or driver specified, dialog prohibited

This is my first post, so please be forgiving. I have got this set up working for a windows environment and an Azure SQL server that use Active Directory.
Plan of attack for this problem:
First try to connect to the Azure database using the SQL server management Studio.
Secondly, try to connect to the Azure database using the odbcad32. If that works you can create a User DSN, which for the example's sake I will call example.
Finally use odbcDriverConnect("DSN=example;") in R
First of all try to login to the Azure database using Sql Server Management Studio. This involves two substeps.
On the target database your user domain and name may be different from the user domain and name on the machine from which you are trying to connect to the target database. Should that be the case then create a credential using the credential manager in windows.
Creating a windows credential to match your identity with that used by the Azure database
Verify the method of authentication from the dialog you get when connecting with SQL server management studio. On my machine I thus saw that the Azure SQL server uses a thing called 'Active Directory Integration' .
Select the method of authentication that applies
Secondly instead of directly testing using RODBC calls it is more convenient to use the ODBC administrator. Run 'odbcad32'. You have to choose the driver.
If you want to use Active Directory for Authentication you need the 'ODBC Driver 13 for SQL Server'. The other drivers, such as 'SQL Server' and the 'Native Client' do not support Active Directory.
Create a user DSN and test it.
The end result should look like this.
If you do not get the desired success you may instead get the following error messages.
If this occurs simply go ahead and install the required software
Go ahead and install the Sign in assistant.
Another error that may pop up is the error saying that you have not installed the Active Directory Authentication Library for SQL Server. Surely this is not true if you have the set up working via SQL server management studio. The problem is that odbcad32 cannot find adalsql.dll. The solution is to add a few registry entries.
Which registry entries to add
Addendum: Below some people note they got it working with the 11.0 driver instead of the 13.0 one. They are perfectly right, but that is because they use user / password authentication. If your server uses active directory integration you do really need the 13.0 or higher versions of the ODBC driver.

So after a lot of trail and error, here is what I have:
1) Ditch RODBC and use RSQLServer
install.packages("RSQLServer")
Create a config file (use sublime) called 'sql.yaml', with the following contents:
SQL_PROD:
server: 11.1.111.11
type: &type sqlserver
port: &port 1433
domain: &domain companyname
user: &user winusername
password: &pass winpassword
useNTLMv2: &ntlm true
SQL_DEV:
server: 11.1.111.15
type: *type
port: *port
domain: *domain
user: *user
password: *pass
useNTLMv2: *ntlm
AW:
server: <yourservername>
type: sqlserver
user: <username>
password: <password>
port: 1433
2) Save this YAML file in the following location(run the following in R: Sys.getenv("HOME")) e.g.: if username is dave its "/Users/dave"
3)
#############
# DBI #
#############
# Note we do not attach the RSQLServer package.
library(DBI)
# Connect to AW server in ~/sql.yaml
aw <- dbConnect(RSQLServer::SQLServer(), "AW", database = 'db')
# RSQLServer only returns tables with type TABLE and VIEW.
dbListTables(aw)

I installed the "odbc" library and setup the connection as described on the RStudio pages. The 11.0 driver worked for me.
library(odbc)
con <- dbConnect(odbc(),
Driver = "SQL Server Native Client 11.0",
Server = "servername",
Database = "databasename",
UID = "username",
#PWD = rstudioapi::askForPassword("Database password"),
PWD = "pwd",
Port = 1433)

Related

pyodbc DSN-less connection with AS400 / IBM midrange

I want to connect via pyodbc with a AS400 Database.
I configured it via the DSN syntax and it is working, but I need it DSN less.
This my current code:
connection = pyodbc.connect(
driver='{iSeries Access ODBC Driver}',
system='192.***.***.***',
uid='U******',
pwd='p0*****')
But always I got the error:
('HY000', '[HY000] [IBM][System i Access ODBC Driver]Missing system name needed for connection. (30188) (SQLDriverConnect); [HY000] [IBM][System i Access ODBC Driver]Missing system name needed for connection. (30188)')
I already tried it with the older driver "IBM i Access ODBC Driver" but same error is comming up.
Did someone also had this error?
Thanks
in my case - I just need to use capital letters for the functions variables:
connection = pyodbc.connect(
DRIVER='{iSeries Access ODBC Driver}',
SYSTEM='192.***.***.***',
UID='U******',
PWD='p0*****')

Cannot Connect to Azure SQL when deploying to Shinyapps.io

I have built an R shiny app/dashboard which runs perfectly on my local Windows 10 machine. I am now ready to deploy it to the web. I have chosen to do this using Shinapps.io. The app deploys, but immediately disconnects when trying to access it. Checking the log file yields this error:
Error : nanodbc/nanodbc.cpp:1021: 00000: [FreeTDS][SQL Server]Unable
to connect: Adaptive Server is unavailable or does not exist
[FreeTDS][SQL Server]Unable to connect to data source [FreeTDS][SQL
Server]Unknown host machine name.
My current implementation for connecting to the database is as follows:
conn_args <- config::get("dataconnection")
is_local<-Sys.getenv('SHINY_PORT')==""
message(paste("is_local: ", is_local))
if(is_local){
con <- odbc::dbConnect(odbc::odbc(),
Driver = "ODBC Driver 13 for SQL Server",
Server = conn_args$server,
Database = conn_args$database,
Uid = conn_args$uid,
PWD = conn_args$pwd,
Port = conn_args$port,
TrustServerCertificate="no")
} else {
#con <- RODBC::odbcDriverConnect("Driver=FreeTDS;TDS_Version=7.2;Server=tcp:#######;Port=1433;Database=######;Uid=######;Pwd=#######;Encrypt=yes;TrustServerCertificate=no;Connection Timeout=30")
#con <- RODBC::odbcDriverConnect("Driver=FreeTDS;TDS_Version-7.2;Server=tcp:#######,1433;Database=###########;Uid=######;Pwd=######;Encrypt=yes;TrustServerCertificate=no;Connection Timeout=30;")
con <- odbc::dbConnect(
odbc::odbc(),
Driver = "FreeTDS",
Server = conn_args$server,
Database = conn_args$database,
UID = conn_args$uid,
PWD = conn_args$pwd,
Port = 1433,
TrustServerCertificate="no",
TDS_Version = 7.4
)
}
As ShinyApps.io runs a Linux instance, I'm using FreeTDS as the driver on deployment. I have adjusted the TDS_Version with 7.0, 7.1, 7.2, 7.4, 8.0, and 9.0, as well as removing that parameter completely. Initally I tried using "SQLServer" as the driver name as per the Shinyapps.io documentation suggestion but that failed, too. I even tried using the RODBC package instead of odbc, but that went nowhere either.
I have set my Azure firewall to let in all necessary IP addresses. I even briefly opened the firewall completely, but that did not yield different results.
I have checked every thread and tutorial I can find, including:
Cannot connect to Microsoft Azure from shinyapps.io
Connecting Azure SQL databse to shinyapps.io
https://groups.google.com/g/shinyapps-users/c/hs4bQHsk9JU
https://docs.rstudio.com/shinyapps.io/applications.html#accessing-databases-with-odbc
Besides the arguments passed to the driver parameter(s), my connection arguments remain the same for local and web deployment. What am I missing? Is Azure SQL simply not compatible with Shinyapps.io?
Found the answer here: https://community.rstudio.com/t/unable-to-connect-to-azure-sql-database-in-shinyapps-io/45723
In the Azure portal, the ODBC connection string formats the server name like so: Server=tcp:servername.database.windows.net
Although this works locally, shinyapps.io does not like that format. I simply had to remove "tcp:" prefix from the server name, and the connection went through: Server=servername.database.windows.net
I also changed my driver to "SQLServer", although "FreeTDS" is still a viable option. If you use "FreeTDS", ensure that you have this extra parameter: TDS_Version=7.2

Add Snowflake ODBC connection in /etc/odbc.ini

I'm attempting to set up an ODBC connection on an EC2 server on our companies network. Have been reading Snowflake documentation here.
Following the documentation, I used yum to download and install the driver by adding a file /etc/yum.repos.d/snowflake-odbc.repo and then populating it with:
[snowflake-odbc]
name=snowflake-odbc
baseurl=https://sfc-repo.snowflakecomputing.com/odbc/linux/2.22.1/
gpgkey=https://sfc-repo.snowflakecomputing.com/odbc/Snowkey-37C7086698CB005C-gpg
I then ran yum install snowflake-odbc
This appears to have added a connection to our existing /etc/odbc.ini file:
[snowflake]
Description=SnowflakeDB
Driver=SnowflakeDSIIDriver
Locale=en-US
SERVER=SF_ACCOUNT.snowflakecomputing.com
PORT=443
SSL=on
ACCOUNT=SF_ACCOUNT
Chancing my luck I tested this connection in it's current format but it doesn't work. Seems I need to locate some more info / name value pairs and I'm unsure where to get them:
Driver= wants a path to a .so file that I thought would have been installed with my steps outlined above. Not sure where to find this file or what path to use?
When I tested the connection the error I received said I was missing a UID.
The section of the documentation on dns entries has additional details of which settings can be added to odbc.ini, including an example
Driver = /usr/jsmith/snowflake_odbc/lib/libSnowflake.so
Description =
server = yz23456.us-east-1.snowflakecomputing.com
role = analyst
database = sales
warehouse = analysis
Does anyone know where can I find out which settings I need to add to get the odbc connection to work?

Missing letters of database objects being returned in DBI SQL Server ODBC connection

Unfortunately, I will not be able to create a good repro for this issue without sharing confidential creds to the database I am having issues with. Hopefully I have enough information below to flag any obvious problems that ODBC experts will understand.
Background
I am running a MacBook Pro with the following specs:
Model Name: MacBook Pro
Model Identifier: MacBookPro15,1
Processor Name: 6-Core Intel Core i7
Processor Speed: 2.6 GHz
Number of Processors: 1
Total Number of Cores: 6
L2 Cache (per Core): 256 KB
L3 Cache: 9 MB
Hyper-Threading Technology: Enabled
Memory: 32 GB
Boot ROM Version: 1037.0.78.0.0 (iBridge: 17.16.10572.0.0,0)
My ODBC connection is set using FreeTDS as specified here.
The relevant portion of freetds.conf is as follows:
# The POC SQL Server
[POC]
host = 172.22.238.154
port = 1433
tds version = 7.3
My odbcinst.ini file is as follows:
[FreeTDS]
Description=FreeTDS Driver for Linux & SQL Server
Driver=/usr/local/lib/libtdsodbc.so
Setup=/usr/local/lib/libtdsodbc.so
UsageCount=1
My odbc.ini file is specified as follows:
[POC]
Description = Connecton to Partners for our children SQL Server
Driver = FreeTDS
Servername = POC
I am trying to make a connection to a SQL Server 2012 database (via VPN) using the following connection information in R:
con <- DBI::dbConnect(odbc::odbc()
,dsn = "POC"
,uid = Sys.getenv("MSSQL_UN")
,database = "CA_ODS"
,pwd = Sys.getenv("MSSQL_PW"))
This generates the following connection object:
> con
<OdbcConnection> POC2
Database: CA_ODS
Microsoft SQL Server Version: 11.00.7001
In general, this connection works as expected. I can query the database using DBI::dbGetQuery(con, "select * from MyTable"), dplyr::tbl(con, MyTable), etc. without issue.
Problem
RStudio, however, is only displaying every other letter of the database objects, and truncating the object names after the first several letters. The following screenshot should illustrate the issue well:
The database I am trying to connect to is called CA_ODS. However, the RStudio object browser is only displaying every other letter of the database name (i.e. the DB is listed as C_D).
This does not appear to be limited to RStudio per se either. While the results of the actual database queries work fine as described above, the returned names from the INFORMATION_SCHEMA appear to match the information in the object browser. Below, when run directly from SQL Server Management Studio, the returned TABLE_CATALOG is CA_ODS, TABLE_SCHEMA is ndacan, etc. When run via the DB connection, however, I get the following.
> DBI::dbGetQuery(con, "SELECT * FROM INFORMATION_SCHEMA.TABLES WHERE
TABLE_SCHEMA='ndacan'")
TABLE_CATALOG TABLE_SCHEMA TABLE_NAME TABLE_TYPE
1 C_D naa f21v BASE TABLE
Question
Any suggestions as to how I can respecify my ODBC connection in R or in my FreeTDS configs to get the full name of database objects returned?
As noted in #r2evans comments, this appears to be an issue with odbc, running in R 3.6.0, on a Mac.
In general, it appears that this can be fixed by reinstalling odbc from source install.packages("odbc", type = 'source').
As also noted in the comments, I had recently upgraded my Mac to Catalina. Prior to installing odbc from source I needed to first reinstall XCode using xcode-select --install from my terminal.
As can be seen in the screen capture below, I am now getting the full object names displayed from the odbc connection.

Problems connecting remotely to PostgreSQL on Heroku from R using RPostgreSQL

I'm using the RPostgreSQL 0.4 library (compiled on R 2.15.3) on R 2.15.2 under Windows 7 64-bit to interface to PostgreSQL. This works fine when connecting to my PostgreSQL databases on localhost. I'm trying to get my R code to run with a remote PostgreSQL database on Heroku. I can connect to Heroku's PostgreSQL database from the psql command shell on my machine, and it connects without a problem. I get the message:
psql (9.2.3, server 9.1.9)
WARNING: psql version 9.2, server version 9.1.
Some psql features might not work.
WARNING: Console code page (437) differs from Windows code page (1252)
8-bit characters might not work correctly. See psql reference
page "Notes for Windows users" for details.
SSL connection (cipher: DHE-RSA-AES256-SHA, bits: 256)
Clearly, psql uses SSL to connect. When I try to connect using the RPostgreSQL library routine dbConnect(), however, supplying exactly the same credentials using dname=, host=, port=, user=, password=, the connection fails with the complaint:
Error in postgresqlNewConnection(drv, ...) :
RS-DBI driver: (could not connect <user>#<hostname> on dbname <dbname>)
Calls: source ... .valueClassTest -> is -> is -> postgresqlNewConnection -> .Call
Execution halted
I know that Heroku insists on an SSL connection if you want to access their database remotely, so it seems likely that the R interface routine dbConnect() isn't trying SSL. Is there something else that I can do to get a remote connection from R to PostgreSQL on Heroku to work?
To get the JDBC URL for your heroku instance:
Get your hostname, username and password using [pg:credentials].
Your jdbc URL is going to be:
jdbc:postgresql://[hostname]/[database]?user=[user]&password=[password]&ssl=true&sslfactory=org.postgresql.ssl.NonValidatingFactory
Proceed as you would normally with JDBC.
Apparently there is a way using RJDBC. See:
http://ryepup.unwashedmeme.com/blog/2010/11/17/working-with-r-postgresql-ssl-and-mssql/
Please note that in order to connect to Heroku database with JDBC externally, it is important to set the sslfactory parameter as well. Hope Heroku team goes through it and modifies their documentation.
String dbUri = "jdbc:postgresql://ec2-54-243-202-174.compute-1.amazonaws.com:5432/**xxxxxxx**";
Properties props = new Properties();
props.setProperty("user", "**xxxxx**");
props.setProperty("password", "**xxxxx**");
props.setProperty("ssl", "true");//ssl to be set true
props.setProperty("sslfactory", "org.postgresql.ssl.NonValidatingFactory");// sslfactory to be set as shown above
Connection c=DriverManager.getConnection(dbUri,props);
See answer to related Q at https://stackoverflow.com/a/38942581. The suggestion of using RPostgres (https://github.com/rstats-db/RPostgres) instead of RPostgreSQL resolved this same issue for me.

Resources