I have a table in a SQLite database DB that I want to import in R. Yet one column integer of table is a SQLite integer datatype that might assume values > 2^31-1, then more than what supported by the R integer dataype. Yet RSQLite will proceed to import all the values from the column integer as integers into a R dataframe:
require(RSQLite)
con <- dbConnect("SQLite", dbname = DB)
query <- dbSendQuery(con, "SELECT * FROM table")
result <- fetch(query, n = -1, encoding="utf-8")
dbClearResult(query)
dbDisconnect(con)
return(result)
Is there anyway to specify the datatype of the column of the target dataframe in R?
Related
I use DBI package to write values to my database tables. Database is PostgreSQL.
My data looks like this. Some of my values have 0 digits, some have 1:
I get this data from reading csv using xlsx library.
I use this code to write data to my table:
DBI::dbWriteTable(conn = con,
name = Id(schema = "schema", table = 'table'),
value = df,
append=T,)
But in database I end up with this:
Column types of min_limit and max_limit in database are numeric.
I tried to use:
DBI::dbWriteTable(conn = con,
name = Id(schema = "schema", table = 'table'),
value = format(df, digits = 2),
append=T,)
But this gives me error:
> Error while preparing parameters ERROR: column "row_names" of relation "table" does not exist
What do I need to do to write rounded to 2 digits values to database table?
I want tu insert this data frame to my table datatable...
library(RODBC)
conn <- odbcConnect("CData Azure Source")
sqlTables(conn)
df <- data.frame(a=1:10, b=10:1, c=11:20)
values <- paste("(",df$a,",", df$b,",",df$c,")", sep="", collapse=",")
cmd <- paste("insert into datatable values ", values)
result <- sqlQuery(conn, cmd, as.is=FALSE)
print(result)
But I'm having this problem...
[1] "42000 -1 Malformed SQL Statement: Expected '('. Found: values\r\nStatement:insert into datatable values (1,10,11),(2,9,12),(3,8,13),(4,7,14),(5,6,15),(6,5,16),(7,4,17),(8,3,18),(9,2,19),(10,1,20)"
[2] "[RODBC] ERROR: Could not SQLExecDirect 'insert into datatable values (1,10,11),(2,9,12),(3,8,13),(4,7,14),(5,6,15),(6,5,16),(7,4,17),(8,3,18),(9,2,19),(10,1,20)'"
Can any anybody point where i'm wrong...
Regarding ROracle library in R...
Every time I create a table using dbWriteTable, when I select it inside an Oracle Client I have to put quotes around the table name like:
select * from "table_name"
That is super annoying, how to get rid of the quotes? What do I need to tweak so that this works:
select * from table_name
I tried a bunch of things but this worked:
dbWriteTable(conn, "MYTABLE", df, rownames = F, append = T, overwrite = F)
What is nice about this is that in case MYTABLE does not exist, it creates it for you. Now you can do:
select * from mytable;
//Edit:
If you create a table with this, it is important to ensure that your column names are upper case.
colnames(df) <- toupper(colnames(df)) # in R
dbWriteTable(conn, "MYTABLE", df, rownames = F, append = T, overwrite = F)
-- in SQL
select col1 from mytable; -- works
If you write the column names as lowercase, you will need to quote the columns in your select statement:
select "col1" from mytable; -- works
select col1 from mytable; -- doesn't work
I have an SQLite database connection to a database file. I want to extract some data from one of the tables, do some processing in R and then create a temporary table on the same connection from the processed data. It needs to be a temp table because users may not have write access to the database, but I want to be able to query this new data alongside the data already in the database.
so, for example:
require(sqldf)
db <- dbConnect(SQLite(), "tempdb")
dbWriteTable(db, "iris", iris)
# do some processing in R:
d <- dbGetQuery(db, "SELECT Petal_Length, Petal_Width FROM iris;")
names(d) <- c("length_2", "width_2")
d <- exp(d)
and then I want to make a temporary table in the connection db from d
I know I could do:
dbWriteTable(conn=db, name= "iris_proc", value = d)
but I need it in a temp table and there doesn't seem to be an option for this in dbWriteTable.
One workaround I thought of was to add a temp table and then add columns and update them:
dbGetQuery(db, "CREATE TEMP TABLE iris_proc AS SELECT Species FROM iris;")
dbGetQuery(db, "ALTER TABLE iris_proc ADD COLUMN length_2;")
But then I can't get the data from d into the columns:
dbGetQuery(db, paste("UPDATE iris2 SET length_2 =", paste(d$length_2, collapse = ", "), ";"))
Error in sqliteExecStatement(con, statement, bind.data) :
RS-DBI driver: (error in statement: near "4.05519996684467": syntax error)
I imagine that, even if I get this to work, it will be horribly inefficient.
I thought there might have been some way to do this with read.csv.sql but this does not seem to work with open connection objects.
Use an in-memory database for the temporary table:
library(RSQLite)
db <- dbConnect(SQLite(), "tempdb")
dbWriteTable(db, "iris", iris)
d <- dbGetQuery(db, "SELECT Petal_Length, Petal_Width FROM iris")
d <- exp(d)
dbGetQuery(db, "attach ':memory:' as mem")
dbWriteTable(db, "mem.d", d, row.names = FALSE) # d now in mem database
dbGetQuery(db, "select * from iris limit 3")
dbGetQuery(db, "select * from mem.d limit 3")
dbGetQuery(db, "select * from sqlite_master")
dbGetQuery(db, "select * from mem.sqlite_master")
How do I update data in a postgresql db through R with new data?
I've tried
dbGetQuery(con,"UPDATE table SET column1=:1,column2=:2, column3=:3
where id=:4", data=Rdata[,c("column1", "column3", "column3","id")])
I also tried with the colons replaced with $ but that didn't work either. I keep getting:
Error in postgresqlExecStatement(conn, statement, ...) :
unused argument(s)
I figured it out using:
update <- function(i) {
drv <- dbDriver("PostgreSQL")
con <- dbConnect(drv, dbname="db_name", host="localhost", port="5432", user="chris", password="password")
txt <- paste("UPDATE data SET column_one=",data$column_one[i],",column_two=",data$column_two[i]," where id=",data$id[i])
dbGetQuery(con, txt)
dbDisconnect(con)
}
registerDoMC()
foreach(i = 1:length(data$column_one), .inorder=FALSE,.packages="RPostgreSQL")%dopar%{
update(i)
}
At least the RODBC has a specific function sqlUpdate:
sqlUpdate updates the table where the rows already exist. Data frame
dat should contain columns
with names that map to (some of) the columns in the table
See http://cran.r-project.org/web/packages/RODBC/RODBC.pdf