I am trying to use pool to connect to my DuckDB/SQLite database .
con <- pool::dbPool(
drv = duckdb::duckdb(),
dbname = "data",
host = "FINAL_data.duckdb")
when I then type
dplyr::tbl(con,"data")
I can't open the database.
Error in dbplyr::as.sql(from, con) : unused argument (con)
So I try:
dplyr::tbl("data")
Error in UseMethod("tbl") :
no applicable method for 'tbl' applied to an object of class "character"
So I went back to
con <- dbConnect(duckdb::duckdb(), "FINAL_data.duckdb")
bigdf<-tbl(con,"data")
And this won't open the database now at all (which previously worked fine). The DB is still on disk and about 24G in size. Any thoughts as to what's going on are really appreciated!
Error in .local(conn, statement, ...) :
duckdb_prepare_R: Failed to prepare query SELECT *
FROM "data" AS "zzz6"
WHERE (0 = 1)
Error: Catalog Error: Table with name data does not exist!
LINE 2: FROM "data" AS "zzz6"
I experienced the same issue and it seems to be an incompatibility between the pool and dbplyr version.
For version of pool 0.1.5 and up you need to used dbplyr version 2.0.0 and up.
If you use lower version of dbplyr e.g. 1.4.4 you need to downgrade your pool to e.g. 0.1.4.3
Related
I've had a working R script that uses the dbWriteTable command to write to a SQL Server table. It has worked well without issue for a while ... until the last few days.
Now when I run the dbWriteTable command, I get the following warning:
Found more than one class "blob" in cache; using the first, from namespace 'blob'
Also defined by ‘jsonlite’
Interestingly enough the table appears to write successfully.
Here is some sample code:
library("DBI")
db_test <- dbConnect(
odbc(),
driver = "SQL Server",
server = "test_server",
port = 1234,
database = "test_db"
)
dbWriteTable(
conn = db_test,
name = SQL("dbo.swc_test_write_table"),
value = df_test,
overwrite = TRUE
)
I've tried explicitly naming the package, DBI::dbWriteTable, but it throws the same warning. For reference, I'm not using using the jsonlite package, but I have it installed.
Any thoughts on why this is happening?
This seems to be a bug, which is caused by the jsonlite package in its latest release 1.7.3.
See the bug report https://github.com/jeroen/jsonlite/issues/373
It seems to be fixed upstream and as of now there's also an updated version available, Changelog from 1.7.3 to 1.8.0.
Same problem here, seems to be interfering with encoding, possibly
EDIT:
Sorry for the "me too" post. I checked around a bit more and it's caused for me at least by the loading of the tidyverse library. Loading only DBI and odbc for me resolves the warning.
I'm having trouble connecting to a DB2 database via ODBC. I'm on a Windows system, and have configured a Data Source Name within the ODBC Administrator. When I test the connection there I get Connection tested successfully.. I can also successfully test the connection within IBM's DB2 Configuration Assistant, using both CLI and ODBC.
I'm not able to connect within R. I've tried both the RODBC & odbc packages, the result is the same. My intent is to execute a simple query to verify the connection. When I run the following R script I get an error. Here's my pseudocode.
library('RODBC')
myQuery <- 'SELECT COLUMN1, COLUMN2 FROM DATABASE.TABLE FETCH FIRST 10 ROWS ONLY;'
cnxn <- odbcConnect('myDSN')
data <- sqlQuery(channel=cnxn, query=myQuery)
odbcCloseAll()
Here's the error that I get.
Error in sqlQuery(channel = cnxn, query = myQuery) :
first argument is not an open RODBC channel
In addition: Warning messages:
1: In RODBC::odbcDriverConnect("DSN=myDSN") :
[RODBC] ERROR: state 58031, code -1031, message [IBM][CLI Driver] SQL1031N The database directory cannot be found on the indicated file system. SQLSTATE=58031
2: In RODBC::odbcDriverConnect("DSN=myDSN") : ODBC connection failed
I've learned through experimentation that my script never gets to the point of sending the query. This error is generated at the odbcConnect command.
I don't have access to the server itself, only the database. Is there anything that I can do or try to resolve this on my own, without having to go through support?
EDIT:
I've now cataloged my database, and test connection is successful in 3 places, ODBC Data Source Administrator, Db2 Command Line & Db2 Configuration Assistant. I know that there's no issue with permissions, as I can execute queries via IBM Query Management Facility. I believe this is an issue with either my driver or my system's PATH statements, but I'm not sure how to trace that down.
Taking a non-RODBC approach, the below method works for connecting R and DB2. Assuming you know all the information below, you'll need to download an IBM DB2 jar file and locate it, in this case, in a folder on my machine called "IBM".
Note: there are two types of available jar files, db2jcc.jar and db2jcc4.jar. The below example is using db2jcc.jar.
library(rJava)
library(RJDBC)
library(DBI)
#Enter the values for you database connection
dsn_driver = "com.ibm.db2.jcc.DB2Driver"
dsn_database = "" # e.g. "BLUDB"
dsn_hostname = "" # e.g.: "awh-yp-small03.services.dal.bluemix.net"
dsn_port = "" # e.g. "50000"
dsn_protocol = "TCPIP" # i.e. "TCPIP"
dsn_uid = "" # e.g. "dash104434"
dsn_pwd = "" # e.g. "7dBZ39xN6$o0JiX!m"
jcc = JDBC("com.ibm.db2.jcc.DB2Driver", "C:/Program Files/IBM/SQLLIB/java/db2jcc.jar");
jdbc_path = paste("jdbc:db2://", dsn_hostname, ":", dsn_port, "/", dsn_database, sep="");
conn = dbConnect(jcc, jdbc_path, user=dsn_uid, password=dsn_pwd)
query = "SELECT *
FROM Table
FETCH FIRST 10 ROWS ONLY";
rs = dbSendQuery(conn, query);
df = fetch(rs, -1);
df
According to the DB2 Manual here
SQL1031N The database directory cannot be found on the indicated file system.
Explanation
The system database directory or local database directory could not be found. A database has not been created or it was not cataloged correctly.
The command cannot be processed.
User response
Verify that the database is created with the correct path specification. The Catalog Database command has a path parameter which specifies the directory where the database resides.
sqlcode: -1031
sqlstate: 58031
I have to following code
drv <- RPostgreSQL::PostgreSQL()
con <- DBI::dbConnect(drv, dbname = 'dbname', user = 'user',
host = 'host.name', port = 5432, password = 'password')
When I run it on server (Ubuntu server 16.04 with latest updates) running the database I get the following error:
Error in .valueClassTest(ans, "data.frame", "dbGetQuery") :
invalid value from generic function ‘dbGetQuery’, class “NULL”, expected “data.frame”
But when I run R from commandline with sudo, it works, when I run it from different my laptop connecting to the DB on the server it also works. So it shouldn't be connection problem. I am thinking about problem with access rights to some libraries/executables/configs on the system? Any help will be appreciated.
When I run the dbConnect multiple times and it ends with the error, when I run drv_info <- RPostgreSQL::dbGetInfo(drv), I still get multiple connectionIds in the drv_info:
drv_info <- RPostgreSQL::dbGetInfo(drv)
> drv_info
$drvName
[1] "PostgreSQL"
$connectionIds
$connectionIds[[1]]
<PostgreSQLConnection>
$connectionIds[[2]]
<PostgreSQLConnection>
$fetch_default_rec
[1] 500
$managerId
<PostgreSQLDriver>
$length
[1] 16
$num_con
[1] 2
$counter
[1] 2
Found a source of confusion, but not necessarily the root problem. (I was assuming RPostgres, while you are using RPostgreSQL (github mirror).)
If you check the source, you'll find that the method is calling postgresqlNewConnection, which includes a call to dbGetQuery. The problem you're seeing is that your call to dbConnect is failing (my guess is at line 100) and returns something unexpected, but postgresqlNewConnection continues.
I see three options for you:
try calling dbConnect(..., forceISOdate=FALSE) to bypass that one call to dbGetQuery (note that this doesn't fix the connection problem, but it at least will not give you the unexpected query error on connection attempt);
raise an issue for the package maintainers; or
switch to using RPostgres, still DBI-based and actively developed (it looks like RPostgreSQL has not had significant commits in the last few years, not sure if that's a sign of good code stability or development stagnation)
One lesson you may take from this is that you should check the value returned from dbConnect; if is.null(con), something is wrong. (Again, this does not solve the connection problem.)
I'm pretty new to R and may be I'm re-asking this question again, but I'm posting this as I didn't find an appropriate answer.
I'm trying to connect to Oracle DB that sits on a remote location from R using RODBC. I went through a lot of RODBC configuration documents but nothing clearly specifies connection to remote Oracle DB and what is the root cause.
I also pondered on using ROracle as DBI but found out that RODBC fares much better for multiple reasons.
So, here's my code:
> # Load RODBC package
> library(RODBC)
> # Create a connection to the database
> dbconnection <- odbcDriverConnect("Driver={Oracle ODBC Driver}; Server=my_hostname; Database=my_db_name; Uid=my_uid; Pwd=my_pwd")
> # Check that connection is working
> odbcGetInfo(dbconnection)
> close(dbconnection)
And the TNS Config that I use [values altered for obvious reasons]:
my_db_name =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = my_hostname)(PORT = my_port))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = my_service_name)
)
)
Note: The other thing here is that I don't have a local Oracle DB setup. I use SQL Developer to connect to the remote Oracle DB with the given TNS Config.
I get the below error:
Warning messages:
1: In odbcDriverConnect("Driver={Oracle ODBC Driver}; Server=my_hostname; Database=my_db_name; Uid=my_uid; Pwd=my_pwd") :
[RODBC] ERROR: state 01000, code 0, message [unixODBC][Driver Manager]Can't open lib 'Oracle ODBC Driver' : file not found
2: In odbcDriverConnect("Driver={Oracle ODBC Driver}; Server=my_hostname; Database=my_db_name; Uid=my_uid; Pwd=my_pwd") :
ODBC connection failed
After installing R, I have only installed RODBC using the install.packages("RODBC") command. Also, I'm running this on Mac OS.
I know I'm missing something here. Specifically the Driver={Oracle ODBC Driver} might need to modified to the actual drivers name. But I don't know what it should be and googling didn't help much.
Is there anything else that I need to install or configure?
Just updated to R 3.0 and updated all the packages, including DBI. To my surprise, a script that I often use stopped working.
I am unable to connect to a MySQL database using dbConnect. The code script instantly, so only a few lines will reproduce the problem
> require("RMySQL")
> m = dbDriver("MySQL")
> dbConnect(m, user = 'user', password = 'pass', dbname = 'dbname', host = 'localhost', client.flag = CLIENT_MULTI_STATEMENTS)
Error in as.integer(from) :
cannot coerce type 'S4' to vector of type 'integer'
Calls: dbConnect ... mysqlNewConnection -> isIdCurrent -> as -> asMethod
Also tried it as:
dbConnect(MySQL(), user = 'user', password = 'pass', dbname = 'dbname', host = 'localhost', client.flag = CLIENT_MULTI_STATEMENTS)
but the same problem
Also tried removing other parameters, but the same issue from the dbDriver.
What changed in the DBI package with the latest update? How can I fix this?
I noticed that the DBI package is orphaned so don't know who to ask.
I had the same issue with R 3.0.1 on ubuntu.
Installing the latest version of the RMySQL-package resolved the problem:
> install.pacakges("RMySQL")
Make sure to restart R after the installation.
I'm still digging into the issue, but I think I've identified multiple causes of this issue. At their root, they all have to do with R expecting an S4 object but getting back an integer instead. I believe these are generally a result of the connection failing to establish.
Why is it failing? One thing I've noticed is that if you fail to close to many of your connections (~16 [see the number of maximum connections specified in the driver handle call] open) DBI won't/can't open a new connection. Make sure you are calling dbDisconnect as needed. Usually, this sort of problem results in a sensible error message, however sometimes results in the above referenced error. If possible access the DB through an abstraction layer, e.g. dplyr as some will monitor the db connections and kill them if they are inactive. Whereas, AFIK if you open a connection in a function and the function breaks, you have no way to close the open connection unless you returned the driver object from your initial call to dbConnect. In this case you have no choice but to restart your instance of R (possibly resetting your machine and clearing your workspace as well).
The other issue I recently encountered is that if RMySQL masks RPostgreSQL, then RPostgreSQL will fail. The reverse does not appear to be the case, but because others have mentioned RPostgreSQL in here as showing the same error message, it seemed worthy of note.
Update
The latest version of RMySQL (0.10.1) seems to have finished off RPostgreSQL - RPostgreSQL now fails to work regardless of load order. The same people working on RMySQL appear to be working on RPostgres (https://github.com/rstats-db/RPostgres) and this conflict appears to be a non-issue if using that package instead of RPostgreSQL. Specifically, use RPostgres::Postgres() in the place of RPostgreSQL::PostgreSQL() when specifying the driver in dbConnect. Other packages, e.g. dplyr, currently assume RPostgreSQL, so this issue can still bite (but it seems a resolution is in the works (https://github.com/rstats-db/RMySQL/issues/28).