I'm using RPostgreSQL to read and write data. Reading from any schema works perfectly, but I'm not able to write to non-public schemas. For example, the following code places a table in the public schema, with the name myschema.tablex
# write dataframe to postgres
drv <- dbDriver("PostgreSQL")
con <- dbConnect(drv, host="localhost", user="postgres", password="zzzz", dbname="mydatabase", port="5436")
if(dbExistsTable(con,"myschema.tablex")) {
dbRemoveTable(con,"myschema.vkt_tablex")}
dbWriteTable(con,"myschema.tablex", dataframe, row.names=F)
What I want to do, is to place the table tablex in the schema myschema. I've also tried to name the schema in the connection: dbname="mydatabase.myschema" and trying the argument schemaname which I saw referred to in an earlier bug.
None of these approaches work, so I'm wondering if there is another method that I can use.
Use this:
library(RPostgreSQL)
drv <- dbDriver("PostgreSQL")
con <- dbConnect(drv, dbname = "db", host = "host", port = 5432,
user = "user", password = "pwd")
dbWriteTable(con, c("yourschema", "yourtable"), value = yourRdataframe)
dbDisconnect(con)
More details: https://stat.ethz.ch/pipermail/r-sig-db/2011q1/001043.html
The default schema where objects are created is defined by the search_path. One way would be to set it accordingly. For instance:
SET search_path = myschema, public;
I quote the manual:
When objects are created without specifying a particular target
schema, they will be placed in the first schema listed in the search
path. An error is reported if the search path is empty.
You can also make this the default for a role, so it is set automatically for every connection made by this role. More:
How does the search_path influence identifier resolution and the "current schema"
In case a reader is using the newer package RPostgres to do this, the code to specify schemas is:
dbCreateTable(conn = con, name = Id(schema = "yourschema", table = "yourtable"), fields = yourRdataframe)
Related
I am using pool to handle connections to my Snowflake warehouse. I have created a connection to my database and can read data in a pre-existing table with no issues e.g:
my_pool <- dbPool(odbc::odbc(),
Driver = "Snowflake",
Server = Sys.getenv('WH_URL'),
UID = Sys.getenv('WH_USER'),
PWD = Sys.getenv('WH_PW'),
Warehouse = Sys.getenv('WH_WH'),
Database = "MY_DB")
my_data<-tbl(my_pool, in_schema(sql("schema_name"), sql("table_name"))) %>%
collect()
I would like to save back to a table (table_name) and I believe the best way to do this is with pool::dbWriteTable:
# Create some data to save to db
data<-data.frame("user_email" = "tim#apple.com",
"query_run" = "arrivals_departures",
"data_downloaded" = FALSE,
"created_at" = as.character(Sys.time()))
# Define where to save the data
table_id <- Id(database="MY_DB", schema="MY_SCHEMA", table="TABLE_NAME")
# Write to database
pool::dbWriteTable(my_pool, table_id, data, append=TRUE)
However this returns the error:
Error in new_result(connection#ptr, statement, immediate) :
nanodbc/nanodbc.cpp:1594: 00000: SQL compilation error:
Object 'MY_DB.MY_SCHEMA.TABLE_NAME' already exists.
I have read/write/update permissions for this database for the user specified in my_pool.
I have explored the accepted answers here and here to create the above attempt and can't figure out what I'm doing wrong. It's probably something simple that I've forgotten to do - any thoughts?
EDIT: Wondering if my issue is anything to do with: https://github.com/r-dbi/odbc/issues/480
I am sure this question is very basic, but this is the first time I am using R connected to a server, so a few things still confuse me.
I used ODBC Data Sources on Windows to create a DNS, and used
con <- dbConnect(odbc::odbc(), "TEST_SERVER")
this worked, and now under the connection tab I can see the server, and if I double click I can see the databases and tables that exist in the server. How would I go about reading something inside one of those databases?
For Example, if the database name is db1, and the table name is t1, what is the code needed to read that table into local memory? I would prefer using dbplyr as I am familiar with the syntax. I am just unsure how to refer to a particular database and table after making the connection to the server.
I haven't used dbplyr before, but you can query the database using dbGetQuery.
test <- dbGetQuery(
con,
"SELECT *
FROM db1.t1
"
)
You can also pass the database into the connection string.
con <- dbConnect(
drv = odbc(),
dsn = "TEST_SERVER",
database = "db1"
)
And then your query would just be "SELECT * FROM t1".
EDIT: To query the table using dbplyr:
tbl1 <- tbl(con, "t1")
qry <- tbl1 %>% head() %>% collect()
I like to use RODBC-
con <- RODBC::odbcConnect(dsn = 'your_dsn',
uid = 'userid',
pwd = 'password')
table_output <- RODBC::sqlQuery(con, 'SELECT * FROM Table')
I have a connection to our database:
con <- dbConnect(odbc::odbc(), "myHive")
I know this is successful because when I run it, in the top right of RStudio I can see all of our databases and tables.
My question is, how can I select a specific database table combination? The documentation shows a user sleecting a single table, "flights" but I need to do the equivilent of somedatabase.sometable.
Tried:
mytable <- tbl(con, "somedb.sometable")
Error in new_result(connection#ptr, statement) :
nanodbc/nanodbc.cpp:1344: 42S02: [Hortonworks][SQLEngine] (31740) Table or view not found: HIVE..dp_enterprise.uds_order
Then tried:
mytable <- tbl(con, "somedb::sometable")
Error in new_result(connection#ptr, statement) :
nanodbc/nanodbc.cpp:1344: 42S02: [Hortonworks][SQLEngine] (31740) Table or view not found: HIVE..somedb::sometable
I tried removing the quotes "" too.
Within the connections pane of RStudio I can see somedb.sometable. It's there! How can I save it to variable mytable?
You select the database when creating the connection and the table when creating the tbl (with the from argument).
There is no standard interface to dbConnect, so the exact way to pass the database name depends on the DBDriver you use. Indeed DBI::dbConnect is simply a generic dispatching to the driver-specific dbConnect.
In your case, the driver is odbc so you can check out the documentation for odbc::dbConnect and you'll see the relevant argument is database.
This will work:
con <- dbConnect(odbc::odbc(), "myHive", database = "somedb")
df <- tbl(con, from = "sometable")
With most other drivers (e.g. RMariaDB, RMySQL, RPostgres, RSQLite), the argument is called dbname, so you'd do this:
con <- dbConnect(RMariaDB::MariaDB(), dbname = "somedb")
df <- tbl(con, from = "sometable")
I think I found it, use in_schema
mytable <- tbl(con, in_schema("somedb", "sometable"))
This returns a list not a tbl though so I'm not sure.
I am querying data from a SQL Server database to R Studio. Some columns contain cyrillic letters that should be used in further analysis. However they are encoded in wrong way, so I can not use them. Due to work privacy I am gonna create reproducible example that shows the problem.
library(odbc)
library(pool)
library(DBI)
poolX <- dbPool(drv = odbc::odbc(),
Driver = "ODBC Driver 17 for SQL Server",
Database = "database",
Server = "server",
UID = "user",
PWD = "123456")
Connection works well and let R Studio query data from needed database. Database contains table with characters.
Column City contains city names written on Russian.
It's shown in SQL Server as:
City = Алматы, Астана
However when I query this column to RStudio cell it's written in this form:
City = <c0><eb><ec><e0><f2><fb>,<c0><f1><f2><e0><ed><e0>
Also R shows it in different form
unique(City)
#[1] "\xc0\xeb\xec\xe0\xf2\xfb"
#[2] "\xc0\xf1\xf2\xe0\xed\xe0"
Interesting point is, if I import data from the SQL Server database to Excel and upload to R Studio, it works well
I need direct connection from database to RStudio, so I have to fix this issue.
Any help is welcome. What is the problem?
You can set your locale to Russian before you import from MSSQL with
Sys.setlocale(locale = "Russian")
If you do not want set everything to Russian, you can just set the language with
Sys.setlocale(category = "LC_CTYPE", locale = "Russian")
Example:
> City = "Алматы, Астана"
> data.frame(City)
City
1 <U+0410><U+043B><U+043C><U+0430><U+0442><U+044B>, <U+0410><U+0441><U+0442><U+0430><U+043D><U+0430>
> Sys.setlocale(category = "LC_CTYPE", locale = "Russian")
[1] "Russian_Russia.1251"
>
> data.frame(City)
City
1 Алматы, Астана
I'm working to streamline some database connections. Using the odbc package, I have successfully established a connection with one of my databases like so:
library(odbc)
con <- dbConnect(odbc::odbc(), "db_name",
UID = "username",
PWD = "password")
This works, and the database schema displays in the Connection Pane as anticipated (using RStudio Server 1.1.383)
However, I need to call this connection within a user-defined function that decrypts our users credentials. A minimal example:
db_Connect_mod <- function(userid,
password,
...){
# Needed Processes, but ommitted for simplicity of this question
# ...
con <- dbConnect(odbc::odbc(), "db_name",
UID = userid,
PWD = password)
return(con)
}
So then I run:
con <- db_Connect_mod(userid, password, ...)
The actual database connection con is successful, but it no longer appears in the RStudio Connection Pane.
I know that odbc uses a Connections Contract, but it doesn't seem that it carries over to my new function. Is there a way to force the Connections Contract to carry over to the top-level function?
I have looked at using odbc:::on_connection_opened(con, code = "..."), which seems to work, but is not as functional as inheriting the Connections Contract from odbc within my new function and would rather not be reliant on a non-exported function.
I believe this behavior is due to changes from this odbc github issue
Didn't seem to be much interest, but posting the work-around I've been using:
I am using match.call() to gather the arguments, and then parsing that into the odbc:::on_connection_opened() function as previously discussed. Probably not best practices, but hey it works.
I added the logical argument connection_pane to easily turn off or on this feature:
internal_package::db_Connect_mod <- function(userid,
password,
connection_pane,
...){
# Needed Processes, but ommitted for simplicity of this question
# ...
con <- dbConnect(odbc::odbc(), "db_name",
UID = userid,
PWD = password)
if(connection_pane){
code <- c(match.call()) # This saves what was typed into R
odbc:::on_connection_opened(
con,
paste(c("library(internal_package)",
paste("con <-", gsub(", ", ",\n\t", code))),
collapse = "\n"))
}
return(con)
}
Imagine this could look a lot prettier with glue or stringr for further improvements