Writing and Updating DB2 tables with r - r

I can't figure out how to update an existing DB2 database in R or update a single value in it.
I can't find much information on this topic online other than very general information, but no specific examples.
library(RJDBC)
teachersalaries=data.frame(name=c("bob"), earnings=c(100))
dbSendUpdate(conn, "UPDATE test1 salary",teachersalaries[1,2])
AND
teachersalaries=data.frame(name=c("bob",'sally'), earnings=c(100,200))
dbSendUpdate(conn, "INSERT INTO test1 salary", teachersalaries[which(teachersalaries$earnings>200,] )

Have you tried passing a regular SQL statement like you would in other languages?
dbSendUpdate(conn, "UPDATE test1 set salary=? where id=?", teachersalary, teacherid)
or
dbSendUpdate(conn,"INSERT INTO test1 VALUES (?,?)",teacherid,teachersalary)
Basically you specify the regular SQL DML statement using parameter markers (those question marks) and provide a list of values as comma-separated parameters.

Try this, it worked for me well.
dbSendUpdate(conn,"INSERT INTO test1 VALUES (?,?)",teacherid,teachersalary)
You just need to pass a regular SQL piece in the same way you do in any programing langs. Try it out.

To update multiple rows at the same time, I have built the following function.
I have tested it with batches of up to 10,000 rows and it works perfectly.
# Libraries
library(RJDBC)
library(dplyr)
# Function upload data into database
db_write_table <- function(conn,table,df){
# Format data to write
batch <- apply(df,1,FUN = function(x) paste0("'",trimws(x),"'", collapse = ",")) %>%
paste0("(",.,")",collapse = ",\n")
#Build query
query <- paste("INSERT INTO", table ,"VALUES", batch)
# Send update
dbSendUpdate(conn, query)
}
# Push data
db_write_table(conn,"schema.mytable",mydataframe)
Thanks to the other authors.

Related

Inserting R code output(dataframe )in multiple columns of sql table at once

For example I made an SQL table with column names "Names", "Class", "age" and I have a data-frame which I made using R code:
data_structure1<- as.data.frame("Name")
data_structure1
data_structure2<-as.data.frame("Class")
data_structure2
data_structure3<-as.data.frame("age")
data_structure3
final_df<- cbind(data_structure1,data_structure2,data_structure3)
final_df
#dataframe "Name" contains multiple entries as different names of students.
#dataframe "Class" contains multiple entries as classes in which student are studying like 1,2,3,4 etc.
#dataframe "age" contains multiple entries as ages of students.
I want to insert final_df in my SQL table containing column names as "Names", "Class", "age".
Below is the command which I am using for inserting only one column in one column of SQL table and this is working fine for me.
title<- sqlQuery(conn,paste0("INSERT INTO AMPs(Names) VALUES('",Names, "')"))
title
But this time I want to insert a dataframe of multiple columns in multiple columns of SQL table.
Please help me with this. Thanks in advance.
Using the DBI package and a package for your database (e.g., RPostgres) you can do something like this, assuming the table already exists:
AMPs <- data.frame(name = c("Foo", "Bar"), Class = "Student", age = c(21L, 22L))
conn <- DBI::dbConnect(...) # read doc to set parameters
## Create a parameterized INSERT statement
db_handle <- DBI::dbSendStatement(
conn,
"INSERT INTO AMPs (Name, Class, age) VALUES ($1, $2, $3)"
)
## Passing the data.frame
DBI::dbBind(db_handle, params = unname(as.list(AMPs)))
DBI::dbHasCompleted(db_handle)
## Close statement handle and connection
DBI::dbClearResult(db_handle)
DBI::dbDisconnect(conn)
You may also want to look at DBI::dbAppendTable.
Edit:
The example above works with PostgreSQL, and I see that the OP tagged the question with 'sql-server'. I don't have means for testing it with SQL Server, but the DBI interface aims at being general so I believe it should work. One important thing to note is the syntax, and, in particular, the syntax for the bind parameters. In PostgreSQL the parameters are defined as $1 for the first parameter, $2 for the second and so on. This might be different for other databases. I think SQL Server is using ?, and that that the binding is done based on position - i.e., the first ? is bind to the first supplied parameter, the second ? to the second and so on.

RSQLite dbGetQuery with input from Data Frame

I have a database called "db" with a table called "company" which has a column named "name".
I am trying to look up a company name in db using the following query:
dbGetQuery(db, 'SELECT name,registered_address FROM company WHERE LOWER(name) LIKE LOWER("%APPLE%")')
This give me the following correct result:
name
1 Apple
My problem is that I have a bunch of companies to look up and their names are in the following data frame
df <- as.data.frame(c("apple", "microsoft","facebook"))
I have tried the following method to get the company name from my df and insert it into the query:
sqlcomp <- paste0("'SELECT name, ","registered_address FROM company WHERE LOWER(name) LIKE LOWER(",'"', df[1,1],'"', ")'")
dbGetQuery(db,sqlcomp)
However this gives me the following error:
tinyformat: Too many conversion specifiers in format string
I've tried several other methods but I cannot get it to work.
Any help would be appreciated.
this code should work
df <- as.data.frame(c("apple", "microsoft","facebook"))
comparer <- paste(paste0(" LOWER(name) LIKE LOWER('%",df[,1],"%')"),collapse=" OR ")
sqlcomp <- sprintf("SELECT name, registered_address FROM company WHERE %s",comparer)
dbGetQuery(db,sqlcomp)
Hope this helps you move on.
Please vote my solution if it is helpful.
Using paste to paste in data into a query is generally a bad idea, due to SQL injection (whether truly injection or just accidental spoiling of the query). It's also better to keep the query free of "raw data" because DBMSes tend to optimize a query once and reuse that optimized query every time it sees the same query; if you encode data in it, it's a new query each time, so the optimization is defeated.
It's generally better to use parameterized queries; see https://db.rstudio.com/best-practices/run-queries-safely/#parameterized-queries.
For you, I suggest the following:
df <- data.frame(names = c("apple", "microsoft","facebook"))
qmarks <- paste(rep("?", nrow(df)), collapse = ",")
qmarks
# [1] "?,?,?"
dbGetQuery(con, sprintf("select name, registered_address from company where lower(name) in (%s)", qmarks),
params = tolower(df$names))
This takes advantage of three things:
the SQL IN operator, which takes a list (vector in R) of values and conditions on "set membership";
optimized queries; if you subsequently run this query again (with three arguments), then it will reuse the query. (Granted, if you run with other than three companies, then it will have to reoptimize, so this is limited gain);
no need to deal with quoting/escaping your data values; for instance, if it is feasible that your company names might include single or double quotes (perhaps typos on user-entry), then adding the value to the query itself is either going to cause the query to fail, or you will have to jump through some hoops to ensure that all quotes are escaped properly for the DBMS to see it as the correct strings.

How to unittest PostGIS database queries with R

I've been playing around with database queries in R that are executed on a Postgres database with the PostGIS extension. This means I use some of the PostGIS functions that do not have an R equivalent. If it wasn't for that, I could probably just execute the same function on a local test dataframe instead of a database connection, but due to the PostGIS functions, that's not possible.
Is there a simple approach to create test data in a test database and run the query on that and assess the outcome? I have a WKB column which R does not directly support, so I'm not even sure a simple copy_to could work with inserting a character vector into a geometry column, not to speak of resolving potential key constraints.
A local sqlite database does not work because it does not provide these functions.
Has someone found a viable solution to this problem?
It sounds like you can not collect tables from postgresql back into R, hence your comparison has to happen in sql.
I would do the following:
define text strings to generate sql tables
execute the strings to generate the tables
run your code
make your comparison
For doing a comparison in sql that two tables are identical I would follow the method in this question or this one.
This would look something like this:
# Define text strings
create_string = 'CREATE TABLE test1 (code VARCHAR(4), size INTEGER);'
insert_string = 'INSERT INTO test1 (code, size) VALUES ('AAA', 123);'
# Execute strings
db_con = create_connection()
dbExecute(db_con, create_string)
dbExecute(db_con, insert_string)
# optional validate new table with contents now exists in datbase
# run code
test1 = tbl(db_con, "test1")
test2 = my_function_to_test_that_does_nothing(test1)
# comparison
num_records_not_in_both = test1 %>%
full_join(test2, by = colnames(test2), suffix = c("_1","_2") %>%
filter(is.na(id_1) | is.na(id_2)) %>%
ungroup() %>%
summarise(num = n()) %>%
collect()
require(num_records_not_in_both == 0)
# optional delete test functions

insert additional WHERE condition into a .sql file using R

I have a .sql file that I use with SQL Server Management Studio. I use the same file within my R script to pull the data directly into R (as below) and this works well.
query <- paste(readLines("SQL_FILE.sql"), collapse="\n") # read sql query from hard drive
con <- odbcConnect(dsn ="DATABASE_NAME") # connect to database
dt <- sqlQuery(con, query, rows_at_time = 1, stringsAsFactors = FALSE)
What I need is to insert an additional condition of which the values are being genarated in R environment, to the beginning of the WHERE clause in the .sql file using R. I solved this with below approach:
queryStart <- "SELECT * FROM ANALYSIS_VIEW A WHERE 1=1 AND A.COLUMN_X IN("
filteringValuesForX <- c("FILTERING_VALUE_X1", "FILTERING_VALUE_X2")
queryEnd <- ") AND A.COLUMN_Y = 'FILTERING_VALUE_Y1';"
query <-
paste0(queryStart,
toString(paste("'",filteringValuesForX ,"'", sep='')),
queryEnd)
query
And the output is:
"SELECT * FROM ANALYSIS_VIEW A WHERE 1=1 AND A.COLUMN_X IN('FILTERING_VALUE_X1', 'FILTERING_VALUE_X2') AND A.COLUMN_Y = 'FILTERING_VALUE_Y1';"
However, I am looking for a better solution because of below reasons:
It is not dynamic; when I update the .sql file using SQL Server Management Studio, I need to update queryStart and queryEnd variables manually as well.
The actual SQL script is very long, and I don't want to see all the SQL code in the R script.
Note: There are other WHERE clauses in the original .sql file. However, I want this update only for one specific WHERE clause. To point out this specific one, I added the statement "1=1".
Any suggestions?
I think I found a way!
But before accepting my solution as an answer, I will wait for one more day to see if there is a better answer. So, please, feel free to suggest other approaches.
replaceWithThis <- paste(" A.COLUMN_X IN(", toString(paste("'",filteringValuesForX ,"'", sep='')), ")", " ", collapse = "\n")
query <- sub(x = query, pattern = "1=1 AND A.COLUMN_X IN('FILTERING_VALUE_X1', 'FILTERING_VALUE_X2') ", replacement = replaceWithThis)
query
For this example, to avoid the duplicete WHERE condition, I've used the pattern = "1=1 AND A.COLUMN_X IN('FILTERING_VALUE_X1', 'FILTERING_VALUE_X2') " . In original data it is enough to replace 1=1.
The output is:
"SELECT * FROM ANALYSIS_VIEW A WHERE 1=1 AND A.COLUMN_X IN('FILTERING_VALUE_X1', 'FILTERING_VALUE_X2') AND A.COLUMN_Y = 'FILTERING_VALUE_Y1';"

R/ROracle How to get dbListTables from specific SCHEMA

How to get the list of table names from database for certain scheme?
tabellen <- dbListTables(con, all=T)
gives all the tables from database, but i would like to specify the scheme. I read in the ROracle package that i can specify scheme like:
tabellen <- dbListTables(con, schema="K")
However i get an empty character...
when i use sql command:
rs <- dbSendQuery(con, "SELECT * FROM ALL_TABLES WHERE OWNER ='K'")
data <- fetch(rs)
It works but i get a table, not a list what i would prefer too.. Is there a way to get directly the list of tables? [SOLVED] - too much programming..I wrote scheme instead of schema...Thanks for pointing it out, my bad, sorry for that
And additionally how i can get the name of columns for certain table which i choosed [NOT SOLVED]
Thanks for help

Resources