In my project I have a lot of functions to work with database looking like this:
some_function <- function(smth) {
con <- dbConnect(SQLite(), db)
-- something --
dbDisconnect(con)
return(smth)
}
Is there any way to reduce the code and write something like a Python decorator with connection and disconnetion from db?
like this:
#conenct-disconnect
some_funcion <- function(smth){
-- something --
}
Or may be another way to do it?
A function operator
with_con <- function(f, db) {
function(...) {
con <- dbConnect(SQLite(), db)
f(..., `con` = con)
on.exit(dbDisconnect(con))
}
}
some_func <- function(query, con) {
#so something with x and the connection
dbGetQuery(con, query)
}
connected_func <- with_con(some_func, db)
connected_func('SELECT * FROM table')
Related
I am writing a script that has to run on its own (a cron job), and need to connect to a MySQL database and perform some operations. I am using the package RMySQL in order to do so.
Since it has to be run automatically, I'd like the script to recursively test whether the connection was successful via checking if there are actually any tables in the connection. So far, with no luck I have tried the following:
library(RMySQL)
con <- dbConnect(MySQL(), dbname="test")
FCSQL <- function(connection) {
if (length(dbListTables(connection)) == 0 ) {
con <- dbConnect(MySQL(), dbname="test")
connection <- FCSQL(connection)
FCSQL(connection)
} else {
return("ON")
}
}
FCSQL(con)
#the error for the previous example:
#Error in .local(conn, statement, ...) :
#connection with pending rows, close resultSet before continuing
A slight modification with the same error:
FCSQL <- function(connection) {
if (length(dbListTables(connection)) == 0 ) {
con <- dbConnect(MySQL(), dbname="test")
FCSQL(con)
} else {
return("ON")
}
}
FCSQL(con)
#Error in .local(conn, statement, ...) :
#connection with pending rows, close resultSet before continuing
I'm fairly new to R, so forgive me if this is a amateur question. I still don't get parts of how the R language works and I haven't used closures enough to really build intuition on how to approach this problem.
I want to wrap up opening and closing a database connection in my R project in a clean way. I have a variety of scripts set aside that all use a common DB connection configuration file (I don't put it in my repo, it's a local file only), all of which need to connect to the same MySQL database.
The end goal is to do something like :
query <- db_open()
out <- query("select * from example limit 10")
db_close()
This is what I wrote so far (all my scripts load these functions from another .R file) :
db_open <- function() {
db_close()
db_conn <<- dbConnect(MySQL(), user = db_user, password = db_pass, host = db_host)
query <- function(...) { dbGetQuery(db_conn, ...) }
return(query)
}
db_close <- function() {
result <- tryCatch({
dbDisconnect(db_conn)
}, warning = function(w) {
# ignore
}, error = function(e) {
return(FALSE)
})
return(result)
}
I'm probably thinking of this in an OOP way when I shouldn't be, but sticking db_conn in the global environment feels unnecessary or even wrong.
Is this a reasonable way to accomplish what I want? Is there a better way that I'm missing here?
Any advice is appreciated.
You basically had it, you just need to move the query function into its own function. Regarding keeping db_conn, there really is no reason not to have it in the global environment.
db_open <- function() {
db_close()
db_conn <<- dbConnect(MySQL(), user='root', password='Use14Characters!', dbname='msdb_complex', host='localhost')
}
db_close <- function() {
result <- tryCatch({
dbDisconnect(db_conn)
}, warning = function(w) {
# ignore
}, error = function(e) {
return(FALSE)
})
return(return)
}
query <- function(x,num=-1)
{
q <- dbSendQuery(db_conn, x)
s <- fetch(q, num);
}
Then you should be able to do something like:
query <- db_open()
results <- query("SELECT * FROM msenrollmentlog", 10)
db_close()
I'm using RPostgreSQL and sqldf inside my function like this:
MyFunction <- function(Connection) {
options(sqldf.RPostgreSQL.user = Connection[1],
sqldf.RPostgreSQL.password = Connection[2],
sqldf.RPostgreSQL.dbname = Connection[3],
sqldf.RPostgreSQL.host = Connection[4],
sqldf.RPostgreSQL.port = Connection[5])
# ... some sqldf() stuff
}
How do I test that connection is valid?
You can check that an existing connection is valid using isPostgresqlIdCurrent.
conn <- dbConnect("RPgSQL", your_database_details)
isPostgresqlIdCurrent(conn)
For testing new connections, I don't think that there is a way to know if a connection is valid without trying it. (How would R know that the database exists and is available until it tries to connect?)
For most analysis purposes, just stopping on an error and fixing the login details is the best approach. So just call dbConnect and don't worry about extra check functions.
If you are creating some kind of application where you need to to handle errors gracefully, a simple tryCatch wrapper should do the trick.
conn <- tryCatch(conn <- dbConnection(wherever), error = function(e) do_something)
My current design uses tryCatch:
Connection <- c('usr','secret','db','host','5432')
CheckDatabase <- function(Connection) {
require(sqldf)
require(RPostgreSQL)
options(sqldf.RPostgreSQL.user = Connection[1],
sqldf.RPostgreSQL.password = Connection[2],
sqldf.RPostgreSQL.dbname = Connection[3],
sqldf.RPostgreSQL.host = Connection[4],
sqldf.RPostgreSQL.port = Connection[5])
out <- tryCatch(
{
sqldf("select TRUE;")
},
error=function(cond) {
out <- FALSE
}
)
return(out)
}
if (!CheckDatabase(Connection)) {
stop("Not valid PostgreSQL connection.")
} else {
message("PostgreSQL connection is valid.")
}
One approach is to simply try executing the code, and catching any errors with a nice informative error message. Have a look at the documentation of tryCatch to see the details regarding how this works.
The following blog post provides an introduction to the exception-based style of programming.
I'm working through an R tutorial. I've been working on a function and one of the parts of the function is to take an argument and use it to define a directory in which to find data. It must then load that data.
As it stands the following works:
getmonitor <- function(id, directory){
csvfile <- function(id) {
if (id < 10) {
paste0(0,0,id,".csv")
} else if (id < 100) {
paste0(0,id,".csv")
} else paste0(id,".csv")
}
foo <- read.csv(csvfile(id))
}
Fine. But I now have to use the "directory" parameter to define the directory where the csv file must be read from. I've tried various things here to no avail.
Currently, the code works if the assumption is that the data are in the working directory. I need to say "go to the directory called (directory) and then read.csv.
The directory with all of the data files is called "specdata" and the parameter for directory is thus "specdata".
I tried the following:
getmonitor <- function(id, directory){
csvfile <- function(id) {
if (id < 10) {
paste0(0,0,id,".csv")
} else if (id < 100) {
paste0(0,id,".csv")
} else paste0(id,".csv")
}
filepath <- append(directory,"/",csvfile(id))
foo <- read.csv(filepath)
}
But then I received an error message "Error in !after : invalid argument type
"
I have tried a few various things and if I cut and paste all the code it would probably be more messy than help.
What would be a logical way to do this? Am I on the right track with append? What else should I sue if not? I need to take the parameter "directory" and then load data from that directory.
getmonitor <- function(id, directory=getwd(), ...){
csvfile <- sprintf("%03d.csv", id)
filepath <- file.path(directory, csvfile)
foo <- read.csv(filepath, ...)
foo
}
I'm using a proprietary library that has an "openConnection" function that I use as such:
conn <- openConnection("user", "pass")
# do some stuff with 'conn' that may return early or throw exceptions
closeConnection(conn)
What's the R idiom for making sure that the connection gets closed no matter how the current method gets exited. In C++ it would be RAII, in Java it probably would be a "finally" block. What is it in R?
Typically, just a call to on.exit is used, but you need to do it inside a function.
f <- function() {
conn <- openConnection("user", "pass")
on.exit(close(conn))
# use conn...
readLines(conn)
} # on.exit is run here...
A common case is when you get passed a connection or file name, and you should only create (and close) the connection if you're given a file name:
myRead <- function(file) {
conn <- file
if (!inherits(file, "connection")) {
conn <- file(file, "r")
on.exit(close(conn))
} # else just use the connection...
readLines(conn)
} # on.exit runs here...
# Try it out:
cat("hello\nworld\n", file="foo.txt")
myRead("foo.txt") # file
myRead(stdin()) # connection