Does anyone know how to WRITE an R Data Frame to a new Snowflake database table? I have a successful Snowflake ODBC connection created in R, and can successfully query from Snowflake. The connection command is: conn <- DBI::dbConnect(odbc::odbc(), "Snowflake").
Now, I want to WRITE a data frame created in R back to Snowflake as a table. I used the following command: dbWriteTable(conn, "database.schema.tablename", R data frame name). Using this command successfully connects with Snowflake, but I get the following error message: "Error in new_result(connection#ptr, statement) : nanodbc/nanodbc.cpp:1344: 22000: Cannot perform CREATE TABLE. This session does not have a current database. Call 'USE DATABASE', or use a qualified name."
I am using a qualified database name in my "database.schema.tablename" argument in the dbWriteTable function. I don't see how to employ "USE DATABASE" in my R function. Any ideas?? Thank you!!
The API for DBI::dbWriteTable(…) requires passing either the literal table name as a string, or as a properly quoted identifier:
dbWriteTable(conn, name, value, ...)
conn: A DBIConnection object, as returned by dbConnect().
name: A character string specifying the unquoted DBMS table name, or the result of a call to dbQuoteIdentifier().
value: a data.frame (or coercible to data.frame).
dbWriteTable(conn, "database.schema.tablename", R data frame name)
Your code above will attempt to create a table literally named "database.schema.tablename", using the database and schema context associated with the connection object.
For example, if your connection had a database DB and schema SCH set, this would have succeeded in creating a table called DB.SCH."database.schema.tablename".
To define the database, schema and table names properly, use the DBI::Id class object with the right hierarchal order:
table_id <- Id(database="database", schema="schema", table="tablename")
dbWriteTable(conn, table_id, R data frame name)
Behind the scenes, the DBI::dbWriteTable(…) function recognizes the DBI::Id class argument type for name, and converts it into a quoted identifier format via DBI::dbQuoteIdentifier(…) (as a convenience).
Related
I am creating a database for a large monitoring project. I have already set up the schema on the database and am now trying to populate the tables. I am using R's DBI:: package to transfer data from R to SQL. I have been successful in transferring all datatype except for my spatial data. For the tabular data I have been using DBI::dbWriteTable() However, from scouring other posts, it seems like loading spatial data is better done using the sf::st_write() function in the sf:: package. However, I am getting several errors:
In my local instance of SQL Server, I am getting an error that I am not providing a valid instance of datatype geometry. The reproducible example below will throw this error.
On my network instance of SQL Server, I am getting an error Invalid object name 'spatial_ref_sys' Unfortunately, I was unable to reproduce this error with example data.
N.B.: In the code below, you will need to replace the name of your local instance of sql server in the connection strings
##Loading Necessary Packages##
library(DBI)
library(sf)
library(spData)
##Getting Example Data from R's spData package##
data("us_states")
##Creating a test database in a local instance of MS SQL Server##
con<-dbConnect(odbc::odbc(), .connection_string="driver={SQL Server Native Client 11.0};
server=localsqlserver;trusted_connection=yes")
dbSendQuery(con, "CREATE DATABASE test;")
dbDisconnect(con)
##Changing the connection string to connect directly to test database##
con2<-dbConnect(odbc::odbc(), .connection_string="driver={SQL Server Native Client 11.0};
server=localsqlserver;database=test;trusted_connection=yes")
##Writing tabular data to new table in test##
DF<-us_states_df
dbWriteTable(con2, "States", DF)
##Adding a column for spatial data##
dbSendQuery(con2, "ALTER TABLE dbo.States ADD geom geometry")
##Writing spatial data to new column##
geom_tmp<-us_states$geometry
geom<-st_transform(geom_tmp, "+init=epsg:2992")
st_write(obj=geom, dsn=con2, layer = Id(schema="dbo", table="States"), driver="MSSQLSpatial", append=TRUE)
My goal at the end of the day is simply to add the spatial data in geom to the geom column in test.dbo.states I am open to other avenues that might accomplish this. Thanks in advance for any help.
Take Care,
-Sean
After much tinkering I think I found the solution. Admittedly it's a bit of a workaround, but it isn't too ugly. Instead of trying to write a single column, I wrote the entire table using sf::st_write(). Importantly, while I could not find a way to write geometries directly into SQL, I found out that I could write a Well-Known-Text to SQL. Once it was in the SQL database I used the geometery::STGeomFromText() stored procedure to convert from WKT to geometry. Below is the updated code:
N.B.: Change the server to the name of your sql server instance in the connection strings below for reproducibility
##Loading Necessary Packages##
library(DBI)
library(sf)
library(spData)
##Getting Example Data from R's spData package##
data("us_states")
##Creating a test database in a local instance of MS SQL Server##
con<-dbConnect(odbc::odbc(), .connection_string="driver={SQL Server Native Client 11.0};
server=localsqlserver;trusted_connection=yes")
dbSendQuery(con, "CREATE DATABASE test;")
dbDisconnect(con)
##Changing the connection string to connect directly to test database##
con2<-dbConnect(odbc::odbc(), .connection_string="driver={SQL Server Native Client 11.0};
server=localsqlserver;database=test;trusted_connection=yes")
##Writing tabular data to new table in test##
DF<-as.data.frame(us_states)
geom<-DF$geometry
DF[,"geom"]<-st_as_text(st_transform(geom,"+init=epsg:2992"))
##Writing table to database##
dbWriteTable(con2, Id(schema="dbo", table="States"), DF[,-7])
##Writing a SQL Statement to create new column with geometry datatype##
##Adding a column for spatial data##
dbSendQuery(con2, "ALTER TABLE dbo.States ADD geom2 geometry")
##Writing spatial data to new column##
dbSendQuery(con2, "UPDATE dbo.States
Set geom2 = geometry::STGeomFromText(geom, 2992)")
##Dropping the WKT column##
dbSendQuery(con2, "ALTER TABLE dbo.States
DROP COLUMN geom")
##View the results##
DB<-dbGetQuery(con2, "SELECT * FROM dbo.States")
DB
I scan through SQL database and load evey table by ODBC then would like to store it in a file of file name same as table name. I compose filename by paste(path,variablename,Sys.Date(),sep="_"). I also like to distinguish data in R by storing value of tables in a variable of same name as corresponding SQL table. I achieve this by loading data to a temporary variable then assigning its content to a variable which name is stored in variablename with assign(variablename,temporarytable) function.
I would like to save an R variable with save() function, but need to refer to its name stored in variablename variable. When using get(variablename) I got its content. When using save(get(variablename),file=paste(...,variablename,...)) I got an error that the object ‘get(variablename)’ cannot be found.
What is the problem with get() in save()? How can I get a variable content saved in this situation?
ps
I scan through SQL database tables with for loop. variablename variable stores SQL table name in particular iteration. assign(variablename,temporarytable) helped to load data to an object of required name.
It can be solved like this:
save(list = variablename, file = paste(...,variablename,...))
But I would still itch to know why save(get(variablename), ...) does not work.
Maybe, you can save the data to an object.
temp <- get(variablename)
save(temp,file=file.path(...,variablename,...))
I can not append data frame into the an existing table, because the table in access has white space.
e.g.
Access table variable name: Dlr Name
data frame variable name: DlrName
I try to replicate access table variable name to my df using varTypes=varTypes, but when I run the following code
sqlSave(access, Contracts_CurrMth, "T_Contracts", append=TRUE, rownames=FALSE, fast=FALSE, safer=TRUE, verbose=TRUE)
Looks like R automatically delete the space in my df variable name and The error msg is
HYS22 -1507 [Microsoft][ODBC Microsoft Access Driver] The INSERT INTO statement contains the following unknown field name: DlrName
Anyone can help me on this please?
Some more background info:
R 32bit, access 2010, 32bit, both data table are same format
my data frame table is like this data frame
my access table is like this access table
I need to write data to a postgres DB table using R. If a data exist for an ID in the table, data should be updated otherwise new data should append to the table.
I tried this using 'RPostgreSQL' Package I got this error message
dbWriteTable(con, 'credit', credit,overwrite=TRUE,row.names=FALSE,append=TRUE)<br>
Error in postgresqlWriteTable(conn, name, value, ...) :overwrite and append cannot both be TRUE
You cannot use overwrite and append at once. If use overwrite command as follows, it will truncate the table and rewrite the data.
dbWriteTable(con, 'credit', credit,overwrite=TRUE,row.names=FALSE)
If use append it will Add the rows. But it won't update your results.
dbWriteTable(con, 'credit', credit,row.names=FALSE,append=TRUE)
I have started to use these packages
require(RPostgreSQL)
require(sqldf)
require(caroline)
in order to process data between PostgreSQL an R.
I created a schema public in my database with some tables (one is called "data") and attributes. I imported the table "data" back into R as the data frame "test".
After manipulating it a bit I tried to upload it back to PostgreSQL DB into the (empty) table "data". However, I always get a error which I cant find a solution for:
dbWriteTable2(con, "data", test, fill.null = T, add.id=T,
row.names=FALSE, pg.update.seq=FALSE)
[1] "loading draft table to database"
[1] FALSE
Warning message:
In postgresqlWriteTable(conn, name, value, ...) :
table data exists in database: aborting assignTable
I know that the table "data" exists. Therefore I use dbWriteTable2 function, which seems to be able to insert the data frame into the corresponding attributes within the table "data".
When droping table data from PostgreSQL DB and rerun I get:
dbWriteTable2(con, "data", test, fill.null = T, add.id=T,
row.names=FALSE, pg.update.seq=FALSE)
Error in `[.data.frame`(dbGetQuery(conn, paste("select a.attname from pg_attribute a, pg_class c, pg_tables t, pg_namespace nsp", :
undefined columns selected
Just to sum up, how can I upload a data frame into an (empty) existing table within a PostgreSQL DB.