RPostgreSQL and DBI: "operator does not exist: uuid = text" - r

When using dbReadTable to read in database tables that uses UUID as the primary key, I get the following warning message.
1: In postgresqlExecStatement(conn, statement, ...) :
RS-DBI driver warning: (unrecognized PostgreSQL field type uuid (id:2950) in column 0)
When I modify the table I loaded and try to update the database using, I get the following error message:
Error in postgresqlExecStatement(conn, statement, ...) :
RS-DBI driver: (could not Retrieve the result : ERROR: operator does not exist: uuid = text
I get that the UUID type is not available in R, but is there a way that we can make the database believe the character vector "unique_id" is UUID instead of text?
Code:
library(RPostgreSQL)
library(postGIStools)
pgdrv <- dbDriver(drvName = "PostgreSQL")
# === open connection
db <- DBI::dbConnect(pgdrv,
dbname="database",
host="localhost", port=5432,
user = 'postgres')
# === get tables
users <- dbReadTable(db, "app_users")
# === interaction with tables
users$employee_has_quit[1:5] <- TRUE
# === update tables
postgis_update(conn = db,
df = users,
tbl = "app_users",
id_cols = "unique_id",
update_cols = "employee_has_quit")
# === close conncetion
DBI::dbDisconnect(db)

The problem is a bug in postGIStools. You can see the code they're using to generate this error here
query_text <- paste(query_text, ") AS", tbl_tmp, "(",
paste(quote_id(colnames(df)), collapse = ", "), ")",
"WHERE", paste(paste0(tbl_q, ".", id_q), "=",
paste0(tbl_tmp, ".", id_q),
collapse = " AND "))
Simply put, that won't work. They should be suing placeholders. It assumes that the input type can be the result of make_str_quote (by proxy of df_q and quote_str). That's a faulty assumption as seen here,
CREATE TABLE foo ( a uuid );
INSERT INTO foo VALUES ( quote_literal(gen_random_uuid()) ) ;
ERROR: column "a" is of type uuid but expression is of type text
LINE 1: INSERT INTO foo VALUES ( quote_literal(gen_random_uuid()) ) ...
^
HINT: You will need to rewrite or cast the expression.
My suggestion is you follow the docs,
Note: This package is deprecated. For new projects, we recommend using the sf package to interface with geodatabases.
You may be able to work around this by doing this
CREATE CAST (varchar AS uuid)
WITH INOUT
AS IMPLICIT;

Related

Insert R dataframe into SQL (RODBC) - error table not found

I would like to drop my whole dataframe from R preferably using RODBC with sqlSave statement (not sqlQuery). Here is my sample code.
library(RODBC)
myconn <- odbcDriverConnect("some connection string")
mydf <- data.frame(col_1 = c(1,2,3), col_2 = c(2,3,4))
sqlSave(myconn, mydf, tablename = '[some_db].[some_schema].[my_table]', append = F, rownames = F, verbose=TRUE)
odbcClose(myconn)
After I execute it, I get back error message:
Error in sqlColumns(channel, tablename) :
‘my_table’: table not found on channel
When I check in SQL Server, an empty table is present.
If I run the same code again, I get error message:
Error in sqlSave(myconn, mydf, tablename = "[some_db].[some_schema].[my_table]", :
42S01 2714 [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]There is already an object named 'my_table' in the database.
[RODBC] ERROR: Could not SQLExecDirect 'CREATE TABLE [some_db].[some_schema].[my_table] ("col_1" float, "col_2" float)'
Any suggestions on how to troubleshoot?
UPDATE
In SSMS I can run the following commands successfully:
CREATE TABLE [some_db].[some_schema].[my_table] (
test int
);
drop table [some_db].[some_schema].[my_table]
Here are details of connection string:
Driver=ODBC Driver 17 for SQL Server; Server=someserveraddress; Uid=user_login; Pwd=some_password
To avoid the error, you could specify the database in the connection string:
Driver=ODBC Driver 17 for SQL Server; Server = someserveraddress; database = some_db; Uid = user_login; Pwd = some_password
and avoid using brackets:
sqlSave(myconn, mydf, tablename = 'some_schema.my_table', append = F, rownames = F, verbose=TRUE)

Write to Snowflake VARIANT column from R

I am trying to load data to snowflake using the following code, but getting an error.
con <- DBI::dbConnect(
drv = odbc::odbc(),
driver = "SnowflakeDSIIDriver",
server = "<>",
authenticator = 'externalbrowser',
warehouse = "<>",
database = "<>",
UID = "<>",
role = "<>"
)
DBI::dbAppendTable(con, name = DBI::Id(schema = "<>", table = "<>"), value = tmp[1:2,])
tmp was downloaded from Snowflake, the same table using RStudio:
```{sql connection=con, output.var = 'tmp'}
select top 10 *
FROM <>
```
The error seems to be stemming from a VARIANT column where I store a JSON string.
Error in new_result(connection#ptr, statement, immediate) :
nanodbc/nanodbc.cpp:1374: 22000: SQL compilation error:
Expression type does not match column data type, expecting VARIANT but got VARCHAR(2) for column FEATURES
I had this once and it was an invalid JSON (missing brackets somewhere). Probably this helps.

Query a Postgresql table using variable in R

I have a few tablea in my postgresql database with names similar to: "114114.KP". I would like to dynamically call the table from R using RPostgres package.
I know that the name of the table makes it difficult to process. But is there a way to do it pls?
What I did so far, with i = "114114.KP" :
if(dbExistsTable(con5, i) == TRUE){
ExistingPortfolios <- dbSendQuery(con3, paste("SELECT * from \"", i,"\""))
ExistingPortfolios <- fetch(ExistingPortfolios)
}
But this returns an error (which is normal):
> ExistingPortfolios <- dbSendQuery(con3, paste("SELECT * from \"", i,"\""))
Error in postgresqlExecStatement(conn, statement, ...) :
RS-DBI driver: (connection with pending rows, close resultSet before continuing)

Programmatically building SQL Query R/Shiny/RODBC

I'm building a SQL Query statement using inputDateRange() in R/Shiny. My issue is in handling various strings to include the dates into the WHERE condition of the SQL:
Here is my code:
t.query <- paste0("Select [sensor_name], [temperature] from [dbo].
[temperature_sensor] where network_id = '24162' and date > "
, sQuote(format(input$my.dateRange[1], format="%d-%m-%Y"))
, " and date < "
, sQuote(format(input$my.dateRange[2], format="%d-%m-%Y"))
)
Now the statement closes with a single quote and I receive the error below:
42000 102 [Microsoft][ODBC Driver 13 for SQL Server][SQL
Server]Incorrect syntax near '‘'. [RODBC] ERROR: Could not
SQLExecDirect 'Select [sensor_name], [temperature] from
[dbo].[temperature_sensor] where network_id = '24162' and date >
‘18-09-2017’ and date < ‘22-09-2017’'
I need to close the string with " as I started it in "select ...., I tried to explicitly add """ or dQuote("") to concatenate " but I'm still encountering an error.
Any advice is highly appreciated?
I'd recommend using RODBCext, which will allow you to parameterize your query as
library(RODBCext)
channel <- odbcConnect(...) # make your connection object here
Data <-
sqlExecute(channel = channel,
query = "Select [sensor_name], [temperature]
from [dbo].[temperature_sensor]
where network_id = ? and date between ? and ?",
data = list('24162',
format(input$my.dateRange[1],
format = "%Y-%m-%d"),
format(input$my.dateRange[2],
format = "%Y-%m-%d")),
fetch = TRUE,
stringsAsFactors = FALSE)
This approach has a lot of advantages, including removing the frustration of matching quotes (which you shouldn't do because of the next reason), and protecting your data against SQL injection.

Making an iterative query to an SQL database

I'm trying to get to make a query from R to a MySQL database. The query iterates over a list, and therefore changes dynamically. Each query based on each element in the list will also in general result in multiple rows being extract. The database I'm using can be downloaded from here: http://www.ghtorrent.org/msr14.html
In the end of the day all the results should end up in the same output, and look like this:
pull_req_id,user,action,created_at
12359,arthurnn,opened,1380126837
12359,rafaelfranca,discussed,1380127245
12359,arthurnn,discussed,1380127676
...
The code that I have now looks like this, but it's not working:
library(DBI)
library(RMySQL)
m <- dbDriver("MySQL");
con <- dbConnect(m, user='msr14', password='msr14', host='localhost', dbname='msr14');
all_rails_projects <- dbGetQuery(con, 'SELECT * FROM projects WHERE name = "rails";')
all_rails_prs <- dbGetQuery(con, 'SELECT id FROM pull_requests WHERE base_repo_id = 78852;')
out <- nrow(all_rails_prs)
names(out) <- as.list(all_rails_prs)
df <- c('pull_req_id', 'user', 'action', 'created_at')
out <- numeric(length(df))
names(out) <- df
for (i in nrow(all_rails_prs)) {
SQL <- paste("select user, action, created_at from
(
select prh.action as action, prh.created_at as created_at, u.login as user
from pull_request_history prh, users u
where prh.pull_request_id ='", all_rails_prs[i,], "'",
" and prh.actor_id = u.id
union
select ie.action as action, ie.created_at as created_at, u.login as user
from issues i, issue_events ie, users u
where ie.issue_id = i.id
and i.pull_request_id ='", all_rails_prs[i,], "'",
" and ie.actor_id = u.id
union
select 'discussed' as action, ic.created_at as created_at, u.login as user
from issues i, issue_comments ic, users u
where ic.issue_id = i.id
and u.id = ic.user_id
and i.pull_request_id ='", all_rails_prs[i,], "'",
"union
select 'reviewed' as action, prc.created_at as created_at, u.login as user
from pull_request_comments prc, users u
where prc.user_id = u.id
and prc.pull_request_id ='", all_rails_prs[i,], "'",
") as actions
order by created_at;", sep = "")
res <- dbGetQuery(con, SQL)
out[i] <- dbFetch(res, n = -1)
}
This generates the following error message:
Error in (function (classes, fdef, mtable) :
unable to find an inherited method for function ‘dbFetch’ for signature ‘"data.frame"’
In addition: Warning message:
In mysqlExecStatement(conn, statement, ...) :
RS-DBI driver warning: (unrecognized MySQL field type 7 in column 2 imported as character)
I've tried different variants, but they all result in some kind of error, so it seems as if I'm simply not setting up the query structure the right way. Anyone has any advice?
According to the docs, dbGetQuery calls fetch by default if the query is successful.
res is already a database and you can put it into out directly without having to fetch the records.
Also, if you want to store the results in a dataframe and not a list, you might want to try:
#get the results
res<-dbGetQuery(con, SQL)
#if it's not null, add the request id and rbind it to the out dataframe
if(!is.null(res)){
out<-rbind(out,cbind(rep(all_rails_prs[i,],nrow(res)),res))
}
There might also be an error in your for syntax, you might need for (i in 1:nrow(all_rails_prs))

Resources