Having issues with pasting a variable into the query string for RMariaDB. I can return a query without paste and find the proper where statement I am looking for within the dataframe I query (ex. MIN). When I try to use a variable in the query it fails. I have searched stackoverflow up and down and read the dbgetquery docs but nothing seems to be working. I am sure it is something simple, just can't seem to find it.
library(RMariaDB)
team <- "MIN"
# This returns entire database with MIN in tm column.
filename <- dbGetQuery(conn, "select * from nhl_lab_lines_today")
# These will all give me a [1054] error.
test <- paste("select * from nhl_lab_lines_today WHERE tm = ",paste(team,collapse=", "),sep ="")
test <- paste("select * from nhl_lab_lines_today WHERE tm = team")
test <- paste("select * from nhl_lab_lines_today WHERE tm =", team,sep=" ")
filename <- dbGetQuery(conn, test)
dbGetQuery(con, paste0("select * from nhl_lab_lines_today WHERE tm = '", team ,"'"))
Related
I would like to work on a large database table. The idea was to read some rows, process them, append the result to another table, and so on. In code:
stmt <- "SELECT * FROM input_table WHERE cond"
rs <- DBI::dbSendQuery(con, stmt)
while (!DBI::dbHasCompleted(rs)) {
current_set <- DBI::dbFetch(rs, 50000)
res <- process(current_set)
dbWriteTable(con, "output_table", value=res, append=TRUE)
}
DBI::dbClearResult(rs)
However, I get the message "Closing open result set, pending rows". Is there any way to save the intermediate output?
I would like to work with sqlite and later on Postgres.
Just for reference, I ended up with a solution using a LIMIT / OFFSET construct. Not sure if it efficient, but it is fast enough for my case (700k rows).
batchsize <- 50000
stmt <- "SELECT * FROM input_table WHERE cond"
lim <- paste("LIMIT", batchsize, ";")
finished <- FALSE
i <- 0
while (!finished) {
curr_stmt <- paste(stmt, lim)
current_set <- dbGetQuery(con, curr_stmt)
res <- process(current_set)
dbWriteTable(con, "output_table", value=res, append=TRUE)
finished <- nrow(current_set) < batchsize
i <- i + nrow(current_set)
lim <- paste("LIMIT", batchsize, "OFFSET", i, ";")
}
I am trying to get the row counts of all my tables with a query and I want to save the results in a dataframe. Right now, it only saves one value and I'm not sure what the issue is. Thanks for any help.
schema <- "test"
table_prefix <- "results_"
row_count <- list()
for (geo in geos){
table_name <- paste0(schema, ".", table_prefix, geo)
queries <- paste("SELECT COUNT(*) FROM", table_name)
}
for (x in queries){
row_count <- dbGetQuery(con, x)
}
I'm running the database query by using sqldf in Shiny in R. But getting error.
ui.R:
observeEvent (input$uploadForTest_1, {
inFile=input$uploadForTest_1
inFileName=input$uploadForTest_1$name
file <-"tss.txt"
tmp = paste("audio/street", inFileName, sep = "/")
res <- read.csv.sql(file,header=FALSE,sql = "select * from file where V1=tmp",sep="\t")
print(res)
})
I'm successfully running the following query:
res <- read.csv.sql(file,header=FALSE,sql = "select * from file where V1='audio/street/b098.wav'",sep="\t")
But, if I run the query that is mentioned in ui.R it is giving me error that tmp column doesn't exists:
Warning: Error in result_create: no such column: tmp 86:
I dont want to use string in my query. I want to use variable name. Because I don't want to hard code string in query. Can I use variable name in query instead of string. If yes, then how can I do this? I didn't find solution to my problem in Internet. Thanks.
Preface read.csv.sql with fn$, and use '$tmp' in the SQL statement.
fn$read.csv.sql(file, sql = "select * from file where V1 = '$tmp'",
header = FALSE, sep = "\t")
See ?fn and the gsubfn vignette for more info. Note that sqldf automatically loads the gsubfn package so it will already be available.
You could use sprintf. Another option would be to paste together a string, but I find sprintf far more elegant for this task.
> tmp <- "audio/street/somefile.txt"
> tmp <- "audio/street/somefile.txt"
> "select * from file where V1=tmp"
[1] "select * from file where V1=tmp"
> sprintf("select * from file where V1='%s'", tmp)
[1] "select * from file where V1='audio/street/somefile.txt'"
I have a query in R that extracts data from SQL Server that takes column values as command line arguments. But it does not result in any output.
library(RODBC)
argv <- commandArgs(TRUE)
dbhandle <- odbcDriverConnect('driver={SQL Server};server=<srvr_nm>; database=<db>; trusted_connection=true')
res <- sqlQuery(dbhandle, 'select * from table where col = \'argv[1]\'')
This is how I am calling it
C:\Users\uid>"C:\Program Files\R\R-3.1.0\bin\x64\Rscript.exe" --slave --vanilla "c:\R\script.R" "abc"
(even if I remove the quotation from command line argument while passing that also dies not help)
The output that I get is:
<0 rows> (or 0-length row.names)
When I saw what was getting passed had quotation mark... e.g. "abc"... The value stored in table is abc (without quotation). I tried to remove the quotation with
as.name(argv[1])
but it also did not help...
Then i inserted a value in table with quotation like "abc" (instead of abc)... but still it is not getting selected.
Can you help me in the query.
Try fn$ in the gsubfn package:
library(gsubfn)
argv <- gsub('"', '', commandArgs(TRUE)) # remove double quotes
# ...
res <- fn$sqlQuery(dbhandle, "select * from table where col = '`argv[1]`' ")
or replace last line with:
argv1 <- argv[1]
res <- fn$sqlQuery(dbhandle, "select * from table where col = '$argv1' ")
Unless sqlQuery has some special internal query parsing, the string 'argv[1]' will not evaluate to the value of argv[1] but remain unparsed.
Try any of these instead
res <- sqlQuery(dbhandle,
paste('select * from table where col = \'', argv[1], '\'', sep=""))
res <- sqlQuery(dbhandle,
sprintf('select * from table where col = \'%s\'', argv[1]))
I'm trying to upload a dataframe to a SQL Server table, I tried breaking it down to a simple SQL query string..
library(RODBC)
con <- odbcDriverConnect("driver=SQL Server; server=database")
df <- data.frame(a=1:10, b=10:1, c=11:20)
values <- paste("(",df$a,",", df$b,",",df$c,")", sep="", collapse=",")
cmd <- paste("insert into MyTable values ", values)
result <- sqlQuery(con, cmd, as.is=TRUE)
..which seems to work but does not scale very well. Is there an easier way?
[edited] Perhaps pasting the names(df) would solve the scaling problem:
values <- paste( " df[ , c(",
paste( names(df),collapse=",") ,
")] ", collapse="" )
values
#[1] " df[ , c( a,b,c )] "
You say your code is "working".. I would also have thought one would use sqlSave rather than sqlQuery if one wanted to "upload".
I would have guessed this would be more likely to do what you described:
sqlSave(con, df, tablename = "MyTable")
This worked for me and I found it to be simpler.
library(sqldf)
library(odbc)
con <- dbConnect(odbc(),
Driver = "SQL Server",
Server = "ServerName",
Database = "DBName",
UID = "UserName",
PWD = "Password")
dbWriteTable(conn = con,
name = "TableName",
value = x) ## x is any data frame
Since insert INTO is limited to 1000 rows, you can dbBulkCopy from rsqlserver package.
dbBulkCopy is a DBI extension that interfaces the Microsoft SQL Server popular command-line utility named bcp to quickly bulk copying large files into table. For example:
url = "Server=localhost;Database=TEST_RSQLSERVER;Trusted_Connection=True;"
conn <- dbConnect('SqlServer',url=url)
## I assume the table already exist
dbBulkCopy(conn,name='T_BULKCOPY',value=df,overwrite=TRUE)
dbDisconnect(conn)