I am using the R Pool package to connect to both SQL Server and Oracle databases within the same application. If I create the Oracle pool first I am unable to connect to SQL Server and get the following error:
Error in .verify.JDBC.result(jc, "Unable to connect JDBC to ", url) :
but if I create the SQL Server pool first I have no problems and can subsequently connect to Oracle.
wd <- getwd()
driver1 = JDBC("com.microsoft.sqlserver.jdbc.SQLServerDriver", classPath = file.path(wd,"mssql-jdbc-7.2.2.jre8.jar",":",wd,"jtds-1.3.1a.jar"))
connection1 = dbPool(drv=driver1, url=**, user=**, password=**, minsize=1, maxsize=5)
driver2 = JDBC("oracle.jdbc.driver.OracleDriver", classPath = file.path(wd,"ojdbc6.jar"))
connection2 = dbPool(drv=driver2, url=**, user=**, password=**, minsize=1, maxsize=5)
Has anyone experienced something similar?
Related
I'm trying to connect my on-premise R environment to an Azure Databricks backend using sparklyr and jdbc. I need to perform operations in databricks and then collect the results locally. Some limitations:
No RStudio available, only a terminal
No databricks-connect. Only odbc or jdbc.
The configuration with odbc + dplyr is working, but it seems too complicated, so I would like to use jdbc and sparklyr. Also, if I use RJDBC it works, but it would be great to have the tidyverse available for data manipulation. For that reason I would like to use sparklyr.
I've the jar file for Databricks (DatabricksJDBC42.jar) in my current directory. I downloaded it from: https://www.databricks.com/spark/jdbc-drivers-download. This is what I got so far:
library(sparklyr)
config <- spark_config()
config$`sparklyr.shell.driver-class-path` <- "./DatabricksJDBC42.jar"
# something in the configuration should be wrong
sc <- spark_connect(master = "https://adb-xxxx.azuredatabricks.net/?o=xxxx",
method = "databricks",
config = config)
spark_read_jdbc(sc, "table",
options = list(
url = "jdbc:databricks://adb-{URL}.azuredatabricks.net:443/default;transportMode=http;ssl=1;httpPath=sql/protocolv1/o/{ORG_ID}/{CLUSTER_ID};AuthMech=3;UID=token;PWD={PERSONAL_ACCESS_TOKEN}",
dbtable = "table",
driver = "com.databricks.client.jdbc.Driver"))
This is the error:
Error: java.lang.IllegalArgumentException: invalid method toDF for object 17/org.apache.spark.sql.DataFrameReader fields 0 selected 0
My intuition is that the sc might not be not working. Maybe a problem in the master parameter?
PS: this is the solution that works via RJDBC
databricks_jdbc <- function(address, port, organization, cluster, token) {
location <- Sys.getenv("DATABRICKS_JAR")
driver <- RJDBC::JDBC(driverClass = "com.databricks.client.jdbc.Driver",
classPath = location)
con <- DBI::dbConnect(driver, sprintf("jdbc:databricks://%s:%s/default;transportMode=http;ssl=1;httpPath=sql/protocolv1/o/%s/%s;AuthMech=3;UID=token;PWD=%s", address, port, organization, cluster, token))
con
}
DATABRICKS_JAR is an environment variable with the path "./DatabricksJDBC42.jar"
Then I can use DBI::dbSendQuery(), etc.
Thanks,
I Tried multiple configurations for master. So far I know that jdbc for the string "jdbc:databricks:..." is working. The JDBC connection works as shown in the code of the PS section.
Configure R studios with azure databricks -> go to cluster -> app -> set up azure Rstudio .
For information refer this third party link it has detail information about connecting azure databricks with R
Alternative approach in python:
Code:
Server_name = "vamsisql.database.windows.net"
Database = "<database_name"
Port = "1433"
user_name = "<user_name>"
Password = "<password"
jdbc_Url = "jdbc:sqlserver://{0}:{1};database={2}".format(Server_name, Port,Database)
conProp = {
"user" : user_name,
"password" : Password,
"driver" : "com.microsoft.sqlserver.jdbc.SQLServerDriver"
}
df = spark.read.jdbc(url=jdbc_Url, table="<table_name>", properties=conProp)
display(df)
Output:
I have built an R shiny app/dashboard which runs perfectly on my local Windows 10 machine. I am now ready to deploy it to the web. I have chosen to do this using Shinapps.io. The app deploys, but immediately disconnects when trying to access it. Checking the log file yields this error:
Error : nanodbc/nanodbc.cpp:1021: 00000: [FreeTDS][SQL Server]Unable
to connect: Adaptive Server is unavailable or does not exist
[FreeTDS][SQL Server]Unable to connect to data source [FreeTDS][SQL
Server]Unknown host machine name.
My current implementation for connecting to the database is as follows:
conn_args <- config::get("dataconnection")
is_local<-Sys.getenv('SHINY_PORT')==""
message(paste("is_local: ", is_local))
if(is_local){
con <- odbc::dbConnect(odbc::odbc(),
Driver = "ODBC Driver 13 for SQL Server",
Server = conn_args$server,
Database = conn_args$database,
Uid = conn_args$uid,
PWD = conn_args$pwd,
Port = conn_args$port,
TrustServerCertificate="no")
} else {
#con <- RODBC::odbcDriverConnect("Driver=FreeTDS;TDS_Version=7.2;Server=tcp:#######;Port=1433;Database=######;Uid=######;Pwd=#######;Encrypt=yes;TrustServerCertificate=no;Connection Timeout=30")
#con <- RODBC::odbcDriverConnect("Driver=FreeTDS;TDS_Version-7.2;Server=tcp:#######,1433;Database=###########;Uid=######;Pwd=######;Encrypt=yes;TrustServerCertificate=no;Connection Timeout=30;")
con <- odbc::dbConnect(
odbc::odbc(),
Driver = "FreeTDS",
Server = conn_args$server,
Database = conn_args$database,
UID = conn_args$uid,
PWD = conn_args$pwd,
Port = 1433,
TrustServerCertificate="no",
TDS_Version = 7.4
)
}
As ShinyApps.io runs a Linux instance, I'm using FreeTDS as the driver on deployment. I have adjusted the TDS_Version with 7.0, 7.1, 7.2, 7.4, 8.0, and 9.0, as well as removing that parameter completely. Initally I tried using "SQLServer" as the driver name as per the Shinyapps.io documentation suggestion but that failed, too. I even tried using the RODBC package instead of odbc, but that went nowhere either.
I have set my Azure firewall to let in all necessary IP addresses. I even briefly opened the firewall completely, but that did not yield different results.
I have checked every thread and tutorial I can find, including:
Cannot connect to Microsoft Azure from shinyapps.io
Connecting Azure SQL databse to shinyapps.io
https://groups.google.com/g/shinyapps-users/c/hs4bQHsk9JU
https://docs.rstudio.com/shinyapps.io/applications.html#accessing-databases-with-odbc
Besides the arguments passed to the driver parameter(s), my connection arguments remain the same for local and web deployment. What am I missing? Is Azure SQL simply not compatible with Shinyapps.io?
Found the answer here: https://community.rstudio.com/t/unable-to-connect-to-azure-sql-database-in-shinyapps-io/45723
In the Azure portal, the ODBC connection string formats the server name like so: Server=tcp:servername.database.windows.net
Although this works locally, shinyapps.io does not like that format. I simply had to remove "tcp:" prefix from the server name, and the connection went through: Server=servername.database.windows.net
I also changed my driver to "SQLServer", although "FreeTDS" is still a viable option. If you use "FreeTDS", ensure that you have this extra parameter: TDS_Version=7.2
I am trying to query an SQL Server database deployed on Azure from RStudio.
I established a connection. I can see all of the tables and get a preview of the data from the RStudio GUI. However, when I try to query the data I get an error.
library(dplyr)
library(odbc)
con <- dbConnect(odbc(),
Driver = "SQL Server",
Server = "#.database.windows.net",
Database = "loremipsum",
UID = "loremipsum",
PWD = "loremipsum",
Port = 1433)
First query I tried:
users <- as.data.frame(sqlQuery(con, "SELECT * FROM AspNetUsers"))
The error I got:
Error in sqlQuery(con, "SELECT * FROM AspNetUsers") :
first argument is not an open RODBC channel
I also tried this query
result <- dbSendQuery(con, "SELECT * FROM AspNetUsers")
first_100 <- dbFetch(result, n = 100)
With the following error:
Error in result_fetch(res#ptr, n) :
nanodbc/nanodbc.cpp:2966: 07009: [Microsoft][ODBC SQL Server Driver]Invalid Descriptor Index
It strange because the connection must be valid, given that it gets access to the data through the GUI.
Any hints?
Congratulations you have solved the issue:
I eventually solved the issue by switching to another libary - RODBC. It seems that thats a bug in the way odbc handles character columns as point:R DBI ODBC error: nanodbc/nanodbc.cpp:3110: 07009: [Microsoft][ODBC Driver 13 for SQL Server]Invalid Descriptor Index
I post this as answer and this can be beneficial to other community members.
I'd like to able manage table on SQL Server via my R script and RevoScaleR library.
I have Machine Learning Services installed on a local instance and Microsoft R Client as an interpreter for R. I can get a connection to the server.
However, it seems, I can't create a table on the server.
I've tried:
> predictionSql = RxSqlServerData(table = "PredictionLinReg", connectionString = connStr)
> predict_linReg = rxPredict(LinReg_model, input_data, outData = predictionSql, writeModelVars=TRUE)
...which returns:
Error in rxCompleteClusterJob(hpcServerJob, consoleOutput,
autoCleanup) : No results available - final job state: failed
Help would be appreciated. New to R.
This is my first time connecting to Vertica. I have already connected to a MySQL database sucessfully by using RODBC library.
I have the database setup in vertica and I installed the windows 64-bit ODBC driver from https://my.vertica.com/download-community-edition/
When I tried to connect to vertica using R, I get the below error:
channel = odbcDriverConnect(connection = "Server=myserver.edu;Database=mydb;User=mydb;Password=password")
Warning messages:
1: In odbcDriverConnect(connection = "Server=myserver.edu;Database=mydb;User=mydb;Password=password") :
[RODBC] ERROR: state IM002, code 0, message [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified
2: In odbcDriverConnect(connection = "Server=myserver.edu;Database=mydb;User=mydb;Password=password") :
ODBC connection failed
Can someone tell me how to fix this? Or is there any other ways to connect to vertica using R?
It may not be the fastest, but I prefer to use the Vertica JDBC driver from R. Getting the ODBC drivers working is a little messy across different operating systems. If you already have a Java Runtime Environment (JRE) installed for other applications then this is fairly straightforward.
Download the Vertica JDBC drivers for your Vertica server version from the MyVertica portal. Place the driver (a .jar file) in a reasonable location for your operating system.
Install RJDBC into your workspace:
install.packages("RJDBC",dep=TRUE)
In your R script, load the RJDBC module and create an instance of the Vertica driver, adjusting the classPath argument to point to the location and filename of the driver you downloaded:
library(RJDBC)
vDriver <- JDBC(driverClass="com.vertica.jdbc.Driver", classPath="full\path\to\driver\vertica_jdbc_VERSION.jar")
Make a new connection using the driver object, substituting your connection details for the host, username and password:
vertica <- dbConnect(vDriver, "jdbc:vertica://host:5433/db", "username", "password")
Then run your SQL queries:
myframe = dbGetQuery(vertica, "select Address,City,State,ZipCode from MyTable")
You have to use double slash in the classPath arguement in JDBC function.
for example,
vDriver <- JDBC(driverClass="com.vertica.jdbc.Driver",
classPath="C:\\Program Files\\Vertica Systems\\JDBC\\vertica-jdk5-6.1.2-0.jar")
worked for me, while just copying and pasting the route failed.