I am trying to write a prepared statement with dbSendQuery. My issue is that the data frame of inputs are converted to numeric values, but two of the three inputs are dates. This results in the following error message:
Warning: Error in postgresqlExecStatement: RS-DBI driver: (could not Retrieve the result : ERROR: invalid input syntax for type timestamp: "17624"
)
My code is as follows:
query = dbSendQuery(con,"
SELECT
***AL LOT OF TABLE AND JOINS***
WHERE
users_terminals.user_user_id = $1 and
planning_stops.planned_arrival >= $2 and
planning_stops.planned_arrival <= $3"
,
data.frame(user$users_users_id,
datefrom,
dateto))
tmp = dbFetch(query)
dbClearResult(query)
The numeric value of datefrom is 17624, so this make me think that $2 is replaced by as.numeric(datefrom) when I run the command. Also, user$users_users_id is a numeric value and I do not get an error for that one. Probably the whole data frame is converted to numeric.
I have created a workaround, but it is not an optimal situation and I would like to understand what happens here. The workaround I created is:
query = dbSendQuery(con,"
SELECT
***AL LOT OF TABLE AND JOINS***
WHERE
users_terminals.user_user_id = $1 and
EXTRACT(day from planning_stops.planned_arrival - '1970-01-01') >= $2 and
EXTRACT(day from planning_stops.planned_arrival - '1970-01-01') <= $3"
,
data.frame(user$users_users_id,
datefrom,
dateto))
tmp = dbFetch(query)
dbClearResult(query)
Could anyone help me out here? The workaround works for now, but it does not seem to be optimal.
Thanks.
UPDATE
I have read something about sqlInterpolate, so I thought let's give it a try. However, I still receive an error message:
Error in postgresqlExecStatement(conn, statement, ...) :
RS-DBI driver: (could not Retrieve the result : ERROR: operator does not exist: timestamp without time zone >= integer
LINE 57: ... planning_stops.planned_arrival >= 2018-04...
^
HINT: No operator matches the given name and argument type(s). You might need to add explicit type casts.
)
My code is now:
query = sqlInterpolate(con,"
SELECT
***AL LOT OF TABLE AND JOINS***
WHERE
users_terminals.user_user_id = ?id1 and
planning_stops.planned_arrival >= ?date1 and
planning_stops.planned_arrival <= ?date2"
,
id1 = user$users_users_id,
date1 = datefrom,
date2 = dateto)
tmp = dbGetQuery(con,
query)
Still not working though.. It seems sqlInterpolate converts the inputs to integer.
Just pass the dates as strings and convert them to DATE in the query:
query = sqlInterpolate(con,"SELECT ... WHERE
users_terminals.user_user_id = ?id1 AND
planning_stops.planned_arrival >= ?date1::DATE AND
planning_stops.planned_arrival <= ?date2::DATE"
,
id1 = user$users_users_id,
date1 = strftime(datefrom, "%Y-%m-%d"),
date2 = strftime(dateto, "%Y-%m-%d"))
tmp = dbGetQuery(con, query)
If you would like to pass a timestamp instead, just use the appropriate format in strftime() and convert to timestamp in the query.
Related
I am trying to query data from ClickHouse database from R with subset.
Here is the example
library(data.table)
library(RClickhouse)
library(DBI)
subset <- paste(traffic[,unique(IDs)][1:30], collapse = ',')
conClickHouse <- DBI::dbConnect('here is the connection')
DataX <- dbgetdbGetQuery(conClickHouse, paste0("select * from database
and IDs in (", subset ,") ", sep = "") )
As a result I get error:
DB::Exception: Type mismatch in IN or VALUES section. Expected: FixedString(34).
Got: UInt64: While processing (IDs IN ....
Any help is appreciated
Thanks to the comment of #DennyCrane,
"select * from database where toFixedString(IDs,34) in
(toFixedString(ID1, 34), toFixedString(ID2,34 ))"
This query subset properly
https://clickhouse.tech/docs/en/sql-reference/functions/#strong-typing
Strong Typing
In contrast to standard SQL, ClickHouse has strong typing. In other words, it doesn’t make implicit conversions between types. Each function works for a specific set of types. This means that sometimes you need to use type conversion functions.
https://clickhouse.tech/docs/en/sql-reference/functions/type-conversion-functions/#tofixedstrings-n
select * from (select 'x' B ) where B in (select toFixedString('x',1))
DB::Exception: Types of column 1 in section IN don't match: String on the left, FixedString(1) on the right.
use casting toString or toFixedString
select * from (select 'x' B ) where toFixedString(B,1) in (select toFixedString('x',1))
I have an issue trying to load bigquery data in R and using a date filter (the filter date is variable).
projectid = "projectid"
sql <- sprintf("SELECT *
from [table]
where num_start <21 and t < %s ", paste("'", Sys.Date() -12, "'", sep=""))
I have the following error:
Error: 4.34 - 4.45: Could not parse '2019-01-04' as a timestamp. Required format is YYYY-MM-DD HH:MM[:SS[.SSSSSS]] [invalidQuery]
Then I changed Sys.Date() to Sys.time() the query works but I would like to fetch the data n days prior to a the current day.
You must be using legacy-SQL. Instead, change your query to use standard-SQL and it will for fine then. For example:
WITH
input AS (
SELECT
TIMESTAMP_MILLIS(1547636619070) AS ts)
SELECT
*
FROM
input
WHERE
ts > '2019-01-04'
I want to send to oracle via ROracle query with bind parameters which inculde range of dates for a date column.
I try to run :
idsample <- 123
strdate <- "TO_DATE('01/02/2017','DD/MM/YYYY')"
enddate <- "TO_DATE('01/05/2017','DD/MM/YYYY')"
res <- dbGetQuery(myconn,"SELECT * FROM MYTABLE WHERE MYID = :1 AND MYDATE BETWEEN :2 AND :3", data=data.frame(MYID =idsample , MYDATE=c(strdate,enddate )))
but I get error :
"bind data does not match bind specification"
I could find no documentation which covers using more than one positional parameter, but if one parameter corresponds to a single column of a data frame, then by this logic three parameters should correspond to three columns:
idsample <- 123
strdate <- "TO_DATE('01/02/2017', 'DD/MM/YYYY')"
enddate <- "TO_DATE('01/05/2017', 'DD/MM/YYYY')"
res <- dbGetQuery(myconn,
paste0("SELECT * FROM MYTABLE WHERE MYID = :1 AND ",
"MYDATE BETWEEN TO_DATE(:2, 'DD/MM/YYYY') AND TO_DATE(:3, 'DD/MM/YYYY')"),
data=data.frame(idsample, strdate, enddate))
Note that there is nothing special about strdate and enddate from the point of view of the API, such that they should be passed as vector.
Edit:
The problem with making TO_DATE a parameter is that it will probably end up being escaped as a string. In other words, with my first approach you would end up with the following in your WHERE clause:
WHERE MYDATE BETWEEN
'TO_DATE('01/02/2017','DD/MM/YYYY')' AND 'TO_DATE('01/05/2017','DD/MM/YYYY')'
In other words, the TO_DATE function calls ends up being a string. Instead, bind the date strings only.
I'm building a SQL Query statement using inputDateRange() in R/Shiny. My issue is in handling various strings to include the dates into the WHERE condition of the SQL:
Here is my code:
t.query <- paste0("Select [sensor_name], [temperature] from [dbo].
[temperature_sensor] where network_id = '24162' and date > "
, sQuote(format(input$my.dateRange[1], format="%d-%m-%Y"))
, " and date < "
, sQuote(format(input$my.dateRange[2], format="%d-%m-%Y"))
)
Now the statement closes with a single quote and I receive the error below:
42000 102 [Microsoft][ODBC Driver 13 for SQL Server][SQL
Server]Incorrect syntax near '‘'. [RODBC] ERROR: Could not
SQLExecDirect 'Select [sensor_name], [temperature] from
[dbo].[temperature_sensor] where network_id = '24162' and date >
‘18-09-2017’ and date < ‘22-09-2017’'
I need to close the string with " as I started it in "select ...., I tried to explicitly add """ or dQuote("") to concatenate " but I'm still encountering an error.
Any advice is highly appreciated?
I'd recommend using RODBCext, which will allow you to parameterize your query as
library(RODBCext)
channel <- odbcConnect(...) # make your connection object here
Data <-
sqlExecute(channel = channel,
query = "Select [sensor_name], [temperature]
from [dbo].[temperature_sensor]
where network_id = ? and date between ? and ?",
data = list('24162',
format(input$my.dateRange[1],
format = "%Y-%m-%d"),
format(input$my.dateRange[2],
format = "%Y-%m-%d")),
fetch = TRUE,
stringsAsFactors = FALSE)
This approach has a lot of advantages, including removing the frustration of matching quotes (which you shouldn't do because of the next reason), and protecting your data against SQL injection.
I'm having trouble passing NULL as an INSERT parameter query using RPostgres and RPostgreSQL:
In PostgreSQL:
create table foo (ival int, tval text, bval bytea);
In R:
This works:
res <- dbSendQuery(con, "INSERT INTO foo VALUES($1, $2, $3)",
params=list(ival=1,
tval= 'not quite null',
bval=charToRaw('asdf')
)
)
But this throws an error:
res <- dbSendQuery(con, "INSERT INTO foo VALUES($1, $2, $3)",
params=list(ival=NULL,
tval= 'not quite null',
bval=charToRaw('asdf')
)
)
Using RPostgres, the error message is:
Error: expecting a string
Under RPostgreSQL, the error is:
Error in postgresqlExecStatement(conn, statement, ...) :
RS-DBI driver: (could not Retrieve the result : ERROR: invalid input
syntax for integer: "NULL"
)
Substituting NA would be fine with me, but it isn't a work-around - a literal 'NA' gets written to the database.
Using e.g. integer(0) gives the same "expecting a string" message.
You can use NULLIF directly in your insert statement:
res <- dbSendQuery(con, "INSERT INTO foo VALUES(NULLIF($1, 'NULL')::integer, $2, $3)",
params=list(ival=NULL,
tval= 'not quite null',
bval=charToRaw('asdf')
)
)
works with NA as well.
One option here to workaround the problem of not knowing how to articulate a NULL value in R which the PostgresSQL pacakge will be able to successfully translate is to simply not specify the column whose value you want to be NULL in the database.
So in your example you could use this:
res <- dbSendQuery(con, "INSERT INTO foo (col2, col3) VALUES($1, $2)",
params=list(tval = 'not quite null',
bval = charToRaw('asdf')
)
)
when you want col1 to have a NULL value. This of course assumes that col1 in your table is nullable, which may not be the case.
Thanks all for the help. Tim's answer is a good one, and I used it to catch the integer values. I went a different route for the rest of it, writing a function in PostgreSQL to handle most of this. It looks roughly like:
CREATE OR REPLACE FUNCTION add_stuff(ii integer, tt text, bb bytea)
RETURNS integer
AS
$$
DECLARE
bb_comp bytea;
rows integer;
BEGIN
bb_comp = convert_to('NA', 'UTF8'); -- my database is in UTF8.
-- front-end catches ii is NA; RPostgres blows up
-- trying to convert 'NA' to integer.
tt = nullif(tt, 'NA');
bb = nullif(bb, bb_comp);
INSERT INTO foo VALUES (ii, tt, bb);
GET DIAGNOSTICS rows = ROW_COUNT;
RETURN rows;
END;
$$
LANGUAGE plpgsql VOLATILE;
Now to have a look at the RPostgres source and see if there's an easy-enough way to make it handle NULL / NA a bit more easily. Hoping that it's missing because nobody thought of it, not because it's super-tricky. :)
This will give the "wrong" answer if someone is trying to put literally 'NA' into the database and mean something other than NULL / NA (e.g. NA = "North America"); given our use case, that seems very unlikely. We'll see in six months time.