Getting below error while using vertica copy table from local.
Please suggest
Error:Unable to retrieve JDBC result set for COPY
Monetisation_Base_table FROM LOCAL 'E://testCSV.csv' delimiter ','
([Vertica]JDBC A ResultSet was expected but not generated
from query "COPY Monetisation_Base_table FROM LOCAL 'E://testCSV.csv'
delimiter ','". Query not executed. )
Code Used:
library(RJDBC)
vDriver <- JDBC(driverClass="com.vertica.jdbc.Driver", classPath="full\path\to\driver\vertica_jdbc_VERSION.jar")
vertica <- dbConnect(vDriver, "jdbc:vertica://30.0.9.163:5433/db", "sk14930IU", "Snapdeal_40")
myframe = dbGetQuery(vertica, "COPY Monetisation_Base_table FROM LOCAL 'E://testCSV.csv' delimiter ','"")
dbSendUpdate should do the work in this case .
Related
I'm trying to save text into SQL Server using R and ODBC Driver 17 for SQL Server. I have this simple script:
test <- data.table(dbGetQuery(con, "select LOCATIONNAME from LOCATION where LOCATION = 'AT2331'"))
name <- test[, LOCATIONNAME]
sql <- paste0('INSERT INTO LOCATION_TEST (TEST) VALUES (\'', name, '\')')
dbGetQuery(con, sql)
The data in the original table is VÖSENDORF and it's shown properly in R, but the result in the database is VÖSENDORF. The column type is nvarchar and server collation is SQL_Latin1_General_CP1_CI_AS. How can I fix this?
I also tried
sql2 <- iconv(sql, from="UTF-8", to="UTF-16LE")
But that says:
embedded nul in string
Edit:
Using 'N' prefix doesn't change the result:
sql <- paste0("INSERT INTO LOCATION_TEST (TEST) VALUES (N'", name, "')")
If I use just a hard coded value, it works fine:
sql <- paste0("INSERT INTO LOCATION_TEST (TEST) VALUES (N'VÖSENDORF')")
This is RStudio in Windows. Output of Sys.getlocale():
"LC_COLLATE=English_United Kingdom.1252;LC_CTYPE=English_United Kingdom.1252;LC_MONETARY=English_United Kingdom.1252;LC_NUMERIC=C;LC_TIME=English_United Kingdom.1252"
Tested my code in Linux with following locale, and there it works fine:
"LC_CTYPE=en_US.UTF-8;LC_NUMERIC=C;LC_TIME=en_US.UTF-8;LC_COLLATE=en_US.UTF-8;LC_MONETARY=en_US.UTF-8;LC_MESSAGES=en_US.UTF-8;LC_PAPER=en_US.UTF-8;LC_NAME=C;LC_ADDRESS=C;LC_TELEPHONE=C;LC_MEASUREMENT=en_US.UTF-8;LC_IDENTIFICATION=C"
Tested with the latest R version, 4.2.0 (2022-04-22 ucrt) and the same issue still continues.
I am trying to write data into hive using R Studio, for which first I am storing data into HDFS and from there I want to insert data into Hive.
Data stored in HDFS as :
["TER0626974_achieved","TER0630327_achieved","TER0630520_achieved","TER0537124_achieved","TER0404705_achieved"]
Issue: Now the issue is reading this data from Hive.
CREATE EXTERNAL TABLE dbname.table_name (
id string
) ROW FORMAT SERDE 'org.apache.hive.hcatalog.data.JsonSerDe'
STORED AS TEXTFILE
LOCATION "/hdfs/path/to/file";
We are able to insert this result into hive. But when try to read, getting below error.
archive_data <- dbGetQuery(hivecon, "SELECT * from Table") Error in .jcall(rp, "I", "fetch", stride, block) :
org.apache.hive.service.cli.HiveSQLException: java.io.IOException:
org.apache.hadoop.hive.serde2.SerDeException: java.io.IOException:
Start token not found where expected
can this be the issue? JSON should start with { and not with array ([)?
I am able to establish a connection to a Microsoft SQL Server and am also able to read tables.
pool <- pool::dbPool(drv=odbc::odbc(),
dsn="MYDSN",
uid = "MYUID",
pwd = "XXXXX")
con <- poolCheckout(pool)
WVDListFull <- tbl(con, in_schema('Midas',"WVDListFull")) %>% head() %>% collect()
However I am unable to append new records to the table. Assuming that I have new records in a dataframe called x, I ttried the following code:
dbWriteTable(pool,'[Midas].[WVDListFull]', x, append=TRUE)
This gave me an error:
nanodbc/nanodbc.cpp:1587: 42000: [FreeTDS][SQL Server]CREATE TABLE permission denied in database 'ScorpioEDW'.
I do have read and write permissions on the said database. I also tried this:
dbWriteTable(con,DBI::SQL("Midas.WVDListFull"), x, append=TRUE)
Which resulted in another error:
Error: Can't unquote Midas.WVDListFull
Here Midas is the schema containing the table WVDListFull. Can someone tell me what's going on here?
I have a connection to our database:
con <- dbConnect(odbc::odbc(), "myHive")
I know this is successful because when I run it, in the top right of RStudio I can see all of our databases and tables.
My question is, how can I select a specific database table combination? The documentation shows a user sleecting a single table, "flights" but I need to do the equivilent of somedatabase.sometable.
Tried:
mytable <- tbl(con, "somedb.sometable")
Error in new_result(connection#ptr, statement) :
nanodbc/nanodbc.cpp:1344: 42S02: [Hortonworks][SQLEngine] (31740) Table or view not found: HIVE..dp_enterprise.uds_order
Then tried:
mytable <- tbl(con, "somedb::sometable")
Error in new_result(connection#ptr, statement) :
nanodbc/nanodbc.cpp:1344: 42S02: [Hortonworks][SQLEngine] (31740) Table or view not found: HIVE..somedb::sometable
I tried removing the quotes "" too.
Within the connections pane of RStudio I can see somedb.sometable. It's there! How can I save it to variable mytable?
You select the database when creating the connection and the table when creating the tbl (with the from argument).
There is no standard interface to dbConnect, so the exact way to pass the database name depends on the DBDriver you use. Indeed DBI::dbConnect is simply a generic dispatching to the driver-specific dbConnect.
In your case, the driver is odbc so you can check out the documentation for odbc::dbConnect and you'll see the relevant argument is database.
This will work:
con <- dbConnect(odbc::odbc(), "myHive", database = "somedb")
df <- tbl(con, from = "sometable")
With most other drivers (e.g. RMariaDB, RMySQL, RPostgres, RSQLite), the argument is called dbname, so you'd do this:
con <- dbConnect(RMariaDB::MariaDB(), dbname = "somedb")
df <- tbl(con, from = "sometable")
I think I found it, use in_schema
mytable <- tbl(con, in_schema("somedb", "sometable"))
This returns a list not a tbl though so I'm not sure.
Hi I am trying to update a postgresql table using RpostgreSQL package, the commands in R are executed successfully but the new data is not getting reflected in the database. Below are the commands i have executed in R
for(i in new_data$FIPS) {
drv <- dbDriver("PostgreSQL")
con <- dbConnect(drv, dbname="ip_platform", host="******", port="5432", user="data_loader", password="******")
txt <- paste("UPDATE adminvectors.county SET attributes= hstore('usco#TP-TotPop#2010'::TEXT,",new_data$TP.TotPop[new_data$FIPS == i],"::TEXT) where geoid ='",i,"'")
dbGetQuery(con, txt)
dbCommit(con)
dbDisconnect(con)
}
Can anyone let me know if I have done something wrong? Any help is highly appreciated
Simplify, simplify, simplify -- the RPostgreSQL has had unit tests for these types of operations since the very beginning (in 2008 no less) and this works (unless you have database setup issues).
See eg here in the GitHub repo for all the tests.
You are calling dbGetQuery instead of dbSendQuery and also disconnecting from the database in your for loop. You also are creating a new connection object for every loop iteration, which is not necessary. Try this:
drv <- dbDriver("PostgreSQL")
con <- dbConnect(drv, dbname="ip_platform", host="******", port="5432", user="data_loader", password="******")
for(i in new_data$FIPS) {
txt <- paste("UPDATE adminvectors.county SET attributes= hstore('usco#TP-TotPop#2010'::TEXT,",new_data$TP.TotPop[new_data$FIPS == i],"::TEXT) where geoid ='",i,"'")
dbSendQuery(con, txt)
}
dbDisconnect(con)
You shouldn't call dbCommit(con) explicitly. The enclosing transaction will always be commited when dbSendQuery returns, exactly as when you do an UPDATE with pure SQL. You don't call COMMIT unless you have created a new transaction with BEGIN TRANSACTION.
The warning "there is no transaction in progress" is PostgreSQL's way of telling you that you have issued a COMMIT statement without having issued a BEGIN TRANSACTION statement which is exactly what you are doing in your function.
Thanks for all your inputs.
The issue is with the paste() function which I used in the for loop. The paste() function has replaced the comma with a space in the query as a result the where condition is failing. I have added a sep="" attribute in the paste() and the query is now properly sent to the database and the rows are getting updated as expected.