How to append a R data frame into Redshift? - r

I'm trying to upload a data frame into SQL, but I keep receiving errors. I've tried the following funciont
DBI::dbWriteTable(rposgre_rsql, name = "my_schema.the_table", value = base, row.names = FALSE, append=TRUE)
that functions returns the error
RPosgreSQL error: could not Retrieve the result : ERROR: syntax error at or near "STDIN"
so i tried:
insert\<- "INSERT INTO my_schema.the_table VALUES base"
M2_results\<- RPostgreSQL::dbSendQuery(conn= rposgre_rsql,statement= insert)
but it returns
RPosgreSQL error: could not Retrieve the result : ERROR: syntax error at or near "base"
I'm positive the connection works, since I can either select the table or use "dbExistsTable.R", but I don't understand why it doesn't work with INSERT INTO. The connection is for a corporate enviroment, maybe it's a permission issue? Also, I don't quite understand what's "STDIN" is.

Related

R DBI issue with accessing list fields of a remote table

I am trying to get the fields of a table in an Arctic database. For that I've been successful at creating a jdbcConnection object which is of class "JDBCConnection", but once I write the following code to get the fields of the AR_LOT table (dbListFields comes from DBI package),
dbListFields(jdbcConnection, name = Id(schema = "ARCTIC", table = "AR_LOT"))
I get the following error message.
Error in dbSendQuery(conn, paste("SELECT * FROM ", dbQuoteIdentifier(conn, :
Unable to retrieve JDBC result set
JDBC ERROR: ORA-00933: SQL command not properly ended
Statement: SELECT * FROM "ARCTIC"."AR_LOT" LIMIT 0
I also tried with RJDBC's function dbGetFields, but I'm also running into an error.
Error in .jcall(md, "Ljava/sql/ResultSet;", "getColumns", .jnull("java/lang/String"), :
method getColumns with signature (Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;)Ljava/sql/ResultSet; not found
The weird thing is that dbReadTable from DBI package works just fine.
Can anyone please help me understand these error messages a little more clearly? Thanks in advance

RODBC connection issue

I am trying to use RODBC to connect to an access database. I have used the same structure several times in this project with success. However, in this instance it is now failing and I cannot figure out why. The code is not really reprex as I can't provide the DB, but...
This works for a single table:
library(magrittr);library(RODBC)
#xWalk_path is simply the path to the accdb
#xtabs generated by querying the available tables
x=1
tab=xtabs$TABLE_NAME[x]
temp<-RODBC::odbcConnectAccess2007(xWalk_path)%>%
RODBC::sqlFetch(., tab, stringsAsFactors = FALSE)
odbcCloseAll()
#that worked perfectly
However, I really want to use this in a a function so I can read several similar tables into a list. As a function it does not work:
xWalk_ls<- lapply(seq_along(xtabs$TABLE_NAME), function(x, xWalk_path=xWalk_path, tab=xtabs$TABLE_NAME[x]){
#print(tab) #debug code
temp<-RODBC::odbcConnectAccess2007(xWalk_path)%>%
RODBC::sqlFetch(., tab, stringsAsFactors = FALSE)
return(temp)
odbcCloseAll()
})
#error every time
The above code will return the error:
Warning in odbcDriverConnect(con, ...) :
[RODBC] ERROR: Could not SQLDriverConnect
Warning in odbcDriverConnect(con, ...) : ODBC connection failed
Error in RODBC::sqlFetch(., tab, stringsAsFactors = FALSE) :
first argument is not an open RODBC channel
I am baffled. I accessed the db to pull table names and generate the xtabs variable using sql Tables. Also, earlier in my code I used a similar code structure (not identical, but same core: sqlFetch to retrieve a table into a list) nd it worked without a problem. Only difference between then and now is that: Then I was opening and closing different .accdb files, but pulling the same table name from each. Now, I am opening and closing the same .accdb file but pulling different sheet names each time.
Am I somehow opening and closing this too fast and it is getting irritated with me? That seems unlikely, because if I force it to print(tab) as the first line of the function it will only print the first table name. If it was getting annoyed about the speed of opening an closing I would expect it to print 2 table names before throwing the error.
return returns its argument and exits, so the remaining code (odbcCloseAll()) won't be executed and the opened file (AccessDB) remains locked as you supposed.

Insert URL path into database using dbplyr

I'm trying to insert a url into a postgresql database using
db_insert_into(con, "url", "http://www.google.com")
Error in file(fn, open = "r") : cannot open the connection
In addition: Warning message:
In file(fn, open = "r") :
cannot open file 'http:/www.google.com': No such file or directory
How can I solve this?
You need to somehow specify both the table name and the field name. I'm going to guess that "url" is the field name, and the table name is as yet undefined here. But it doesn't matter, frankly, take the solution and adapt as needed.
The expectation of db_insert_into is that the values (third argument) is a data.frame or something that can easily be converted to such. So you can probably do:
newdata <- data.frame(url = "http://www.google.com", stringsAsFactors = FALSE)
db_insert_into(con, "tablename", newdata)
If you're lazy or playing code-golf, you might be able to do it with:
db_insert_into(con, "tablename", list(url = "http://google.com"))
since some of the underlying S3 or S4 methods around dbplyr sometimes check if (!is.data.frame(values)) values <- as.data.frame(values). (But I wouldn't necessarily rely on that, it's usually better to be explicit.)

"embedded nul in string" error with SparkR::collect

I'm pulling in data from a api and keep getting the following error.
I put together the sql query and am connecting to the instance for pulling the data. However, when I run collect, it given me an error.
soql_query = paste("SELECT Id, subject FROM Table")
myDF2 <- read.df(sqlContext, source="...", username=sf_username, password=sf_password, version=apiVersion, soql=soql_query)
temp2 <- SparkR::collect(myDF2)
Error in rawToChar(string) :
embedded nul in string: 'VOID: \xe5,nq\b\x92ƹ\xc8Y\x8b\n\nAdd a new comment by Asako:\0\xb3\xe1\xf3Ȓ\xfd\xa0\bE\xe4\t06/29 09:23'
In addition: Warning message:
closing unused connection 6 (col)
I've gone through and identified what column it is. It contains a lot of string data and sentences, so the error partially makes sense.
I was wondering if there was any way to get around this issue.

Error in fstRead R

I am using the new 'fst' package in R for a few weeks to write and read tables in the .fst format. Sometimes I cannot read a table that I've just write having the following message :
> tab=read.fst("Tables R/tab.fst",as.data.table=TRUE)
Error in fstRead(fileName, columns, from, to) :
Unknown type found in column.
Do you know why this happens ? Is there an other way to retrieve the table ?

Resources