I'm trying to find use RODBCext syntax for multiple parms in the same query with at least one of them part of an IN clause.
This is the approach I'm taking, but it doesn't like the IN part:
Error: unexpected symbol in "AND col2 "
This is how I'm specifying the parms:
prm <- data.frame( '2', c('B01', 'B12', 'B23', 'B34', 'B45'))
This is the SQL"
dataResult <- sqlExecute, connHandle, query = "
select col1, col2, col3
from tblA
where col1 = ?
and col2 IN ?
"
, prm
, fetch = TRUE)
odbcClose(connHandle)
I've looked at the CRAN pdf
and
the vignettes
I resolved this by changing the syntax to use OR rather than IN. This makes it harder to read, but the code runs successfully.
Related
I need to submit the following query through my R code:
select t."date"
from db.table t
where t."date" > '2016-01-01';
To generate this quoted string properly, I tried:
sqlquery <- dbQuoteString(ANSI()
, paste0("select t.", '"', "date", '"',"
from db.table t
where t.", '"',"date", '"'," > '2016-01-01';")
)
Which output is:
<SQL> 'select t."date"
from db.table t
where t."date" > ''2016-01-01'';'
So I can use:
data <- DBI::dbGetQuery(con, sqlquery)
However, there are double '' instead of a single one ' (around 2016-01-01).
How do I overcome that?
Several layers to this onion.
If you need to quote something, use dQuote or sQuote, they handle both the beginning and end for you. For instance,
dQuote("date")
# [1] "\"date\""
The use of dbQuoteString is quoting your whole query as if it is a string literal. Note the ' before select, indicating that it is a string literal surrounding in single quotes (a common SQL way of delineating string literals). You could also just as easily written
dbQuoteString(ANSI(), "Four score and seven years ago ... said by \"Lincoln\" in '1863'")
# <SQL> 'Four score and seven years ago ... said by "Lincoln" in ''1863'''
The reason you see '' is because it is trying to escape the single quotes that SQL uses to surround string literals. This produces a string, not something that can be executed as a query. In fact, dbQuoteString is something you should be using for your original query, instead of literal quotes and/or my dquote in point 1:
DBI::SQL(paste("select t.", DBI::dbQuoteIdentifier(DBI::ANSI(), "date"), "from db.table t where t.", DBI::dbQuoteIdentifier(DBI::ANSI(), "date"), ">", DBI::dbQuoteString(DBI::ANSI(), "2016-01-01")))
# <SQL> select t. "date" from db.table t where t. "date" > '2016-01-01'
(Admittedly, DBI::SQL is not strictly necessary here, it's really just a character string with a wrapper that makes it print more prettily on the R console.)
Consider not manually encoding data and such into your queries, minor mistakes break queries (or possibly worse, though unlikely in the general case). I strongly urge you to read through https://solutions.rstudio.com/db/best-practices/run-queries-safely/. There, they identify the use of dbQuote* functions as "Manual escaping", which is the last method recommended (least-preferred while still somewhat safe). Other options include:
Parameterized queries, using ? placeholders for data in the query.
ret <- DBI::dbGetQuery(con, 'select t."date" from db.table where "date" > ?', params = list("2016-01-01"))
Use glue::glue_sql:
mydate <- "2016-01-01"
sql <- glue::glue_sql(.con = con, 'select t."date" from db.table where "date" > {mydate}')
sql
# <SQL> select t."date" from db.table where "date" > '2016-01-01'
ret <- DBI::dbGetQuery(con, sql)
Use DBI::sqlInterpolate:
sql <- DBI::sqlInterpolate(con, 'select t."date" from db.table where "date" > ?mydate', mydate = "2016-01-01")
sql
# <SQL> select t."date" from db.table where "date" > '2016-01-01'
ret <- DBI::dbGetQuery(con, sql)
I am trying to query data from ClickHouse database from R with subset.
Here is the example
library(data.table)
library(RClickhouse)
library(DBI)
subset <- paste(traffic[,unique(IDs)][1:30], collapse = ',')
conClickHouse <- DBI::dbConnect('here is the connection')
DataX <- dbgetdbGetQuery(conClickHouse, paste0("select * from database
and IDs in (", subset ,") ", sep = "") )
As a result I get error:
DB::Exception: Type mismatch in IN or VALUES section. Expected: FixedString(34).
Got: UInt64: While processing (IDs IN ....
Any help is appreciated
Thanks to the comment of #DennyCrane,
"select * from database where toFixedString(IDs,34) in
(toFixedString(ID1, 34), toFixedString(ID2,34 ))"
This query subset properly
https://clickhouse.tech/docs/en/sql-reference/functions/#strong-typing
Strong Typing
In contrast to standard SQL, ClickHouse has strong typing. In other words, it doesn’t make implicit conversions between types. Each function works for a specific set of types. This means that sometimes you need to use type conversion functions.
https://clickhouse.tech/docs/en/sql-reference/functions/type-conversion-functions/#tofixedstrings-n
select * from (select 'x' B ) where B in (select toFixedString('x',1))
DB::Exception: Types of column 1 in section IN don't match: String on the left, FixedString(1) on the right.
use casting toString or toFixedString
select * from (select 'x' B ) where toFixedString(B,1) in (select toFixedString('x',1))
I'm building a SQL Query statement using inputDateRange() in R/Shiny. My issue is in handling various strings to include the dates into the WHERE condition of the SQL:
Here is my code:
t.query <- paste0("Select [sensor_name], [temperature] from [dbo].
[temperature_sensor] where network_id = '24162' and date > "
, sQuote(format(input$my.dateRange[1], format="%d-%m-%Y"))
, " and date < "
, sQuote(format(input$my.dateRange[2], format="%d-%m-%Y"))
)
Now the statement closes with a single quote and I receive the error below:
42000 102 [Microsoft][ODBC Driver 13 for SQL Server][SQL
Server]Incorrect syntax near '‘'. [RODBC] ERROR: Could not
SQLExecDirect 'Select [sensor_name], [temperature] from
[dbo].[temperature_sensor] where network_id = '24162' and date >
‘18-09-2017’ and date < ‘22-09-2017’'
I need to close the string with " as I started it in "select ...., I tried to explicitly add """ or dQuote("") to concatenate " but I'm still encountering an error.
Any advice is highly appreciated?
I'd recommend using RODBCext, which will allow you to parameterize your query as
library(RODBCext)
channel <- odbcConnect(...) # make your connection object here
Data <-
sqlExecute(channel = channel,
query = "Select [sensor_name], [temperature]
from [dbo].[temperature_sensor]
where network_id = ? and date between ? and ?",
data = list('24162',
format(input$my.dateRange[1],
format = "%Y-%m-%d"),
format(input$my.dateRange[2],
format = "%Y-%m-%d")),
fetch = TRUE,
stringsAsFactors = FALSE)
This approach has a lot of advantages, including removing the frustration of matching quotes (which you shouldn't do because of the next reason), and protecting your data against SQL injection.
I'm having trouble passing NULL as an INSERT parameter query using RPostgres and RPostgreSQL:
In PostgreSQL:
create table foo (ival int, tval text, bval bytea);
In R:
This works:
res <- dbSendQuery(con, "INSERT INTO foo VALUES($1, $2, $3)",
params=list(ival=1,
tval= 'not quite null',
bval=charToRaw('asdf')
)
)
But this throws an error:
res <- dbSendQuery(con, "INSERT INTO foo VALUES($1, $2, $3)",
params=list(ival=NULL,
tval= 'not quite null',
bval=charToRaw('asdf')
)
)
Using RPostgres, the error message is:
Error: expecting a string
Under RPostgreSQL, the error is:
Error in postgresqlExecStatement(conn, statement, ...) :
RS-DBI driver: (could not Retrieve the result : ERROR: invalid input
syntax for integer: "NULL"
)
Substituting NA would be fine with me, but it isn't a work-around - a literal 'NA' gets written to the database.
Using e.g. integer(0) gives the same "expecting a string" message.
You can use NULLIF directly in your insert statement:
res <- dbSendQuery(con, "INSERT INTO foo VALUES(NULLIF($1, 'NULL')::integer, $2, $3)",
params=list(ival=NULL,
tval= 'not quite null',
bval=charToRaw('asdf')
)
)
works with NA as well.
One option here to workaround the problem of not knowing how to articulate a NULL value in R which the PostgresSQL pacakge will be able to successfully translate is to simply not specify the column whose value you want to be NULL in the database.
So in your example you could use this:
res <- dbSendQuery(con, "INSERT INTO foo (col2, col3) VALUES($1, $2)",
params=list(tval = 'not quite null',
bval = charToRaw('asdf')
)
)
when you want col1 to have a NULL value. This of course assumes that col1 in your table is nullable, which may not be the case.
Thanks all for the help. Tim's answer is a good one, and I used it to catch the integer values. I went a different route for the rest of it, writing a function in PostgreSQL to handle most of this. It looks roughly like:
CREATE OR REPLACE FUNCTION add_stuff(ii integer, tt text, bb bytea)
RETURNS integer
AS
$$
DECLARE
bb_comp bytea;
rows integer;
BEGIN
bb_comp = convert_to('NA', 'UTF8'); -- my database is in UTF8.
-- front-end catches ii is NA; RPostgres blows up
-- trying to convert 'NA' to integer.
tt = nullif(tt, 'NA');
bb = nullif(bb, bb_comp);
INSERT INTO foo VALUES (ii, tt, bb);
GET DIAGNOSTICS rows = ROW_COUNT;
RETURN rows;
END;
$$
LANGUAGE plpgsql VOLATILE;
Now to have a look at the RPostgres source and see if there's an easy-enough way to make it handle NULL / NA a bit more easily. Hoping that it's missing because nobody thought of it, not because it's super-tricky. :)
This will give the "wrong" answer if someone is trying to put literally 'NA' into the database and mean something other than NULL / NA (e.g. NA = "North America"); given our use case, that seems very unlikely. We'll see in six months time.
My original question
When I execute the following query in SQLite, I get this error:
Query Error: misuse of aggregate: sum() Unable to execute statement
When I change the name of the "Loan" column to something like loan_amount the error goes away and my query works fine. Why is there a problem with "Loan"?
select
t.*
, coalesce(sum(ded0.after_tax_ded_amt), 0) as "Loan"
, coalesce(sum(ded1.after_tax_ded_amt), 0) as ee_advance_amount
from totals t
left join totals as ded0
on t.ee_ssn = ded0.ee_ssn
and t.deduction_code = "Loan"
and ded0.deduction_code = "Loan"
left join totals as ded1
on t.ee_ssn = ded1.ee_ssn
and t.deduction_code = "EE Advance"
and ded1.deduction_code = "EE Advance"
group by t.ee_ssn;
Mid-post revelation
I'm pretty sure I figured out why I get the error, is it because I am comparing to "Loan" in the on-clause of my joins?
If so, how can I still use the word "Loan" for my column name in the output of my query?
I'd guess that your real problem is quote misuse. Single quotes in SQL are for quoting string literals, double quotes are for quoting column and table names that need to be case sensitive or contain odd characters. SQLite is fairly forgiving of odd syntax so it is probably making a guess about what "Loan" means and guessing incorrectly. Try this:
select
t.*
, coalesce(sum(ded0.after_tax_ded_amt), 0) as "Loan"
, coalesce(sum(ded1.after_tax_ded_amt), 0) as ee_advance_amount
from totals t
left join totals as ded0
on t.ee_ssn = ded0.ee_ssn
and t.deduction_code = 'Loan'
and ded0.deduction_code = 'Loan'
left join totals as ded1
on t.ee_ssn = ded1.ee_ssn
and t.deduction_code = 'EE Advance'
and ded1.deduction_code = 'EE Advance'
group by t.ee_ssn;