Below is the code which i tried but there is an error please let me know where i am going wrong.
library("RJDBC")
.jinit()
.jaddClassPath("/home/r_work/hive_service.jar")
.jaddClassPath("/home/r_work/HiveJDBC4.jar")
.jaddClassPath("/home/r_work/libfb303-0.9.0.jar")
.jaddClassPath("/home/r_work/libthrift-0.9.0.jar")
.jaddClassPath("/home/r_work/log4j-1.2.14.jar")
.jaddClassPath("/home/r_work/hive_metastore.jar")
.jaddClassPath("/home/r_work/ql.jar")
.jaddClassPath("/home/r_work/slf4j-api-1.5.8.jar")
.jaddClassPath("/home/r_work/slf4j-log4j12-1.5.8.jar")
.jaddClassPath("/home/r_work/TCLIServiceClient.jar")
.jclassPath()
drv = JDBC("com.cloudera.hive.jdbc4.HS2Driver","~/r_work/hive_service.jar;~/r_work/HiveJDBC4.jar;~/r_work/libfb303-0.9.0.jar;~/r_work/libthrift-0.9.0.jar;~/r_work/log4j-1.2.14.jar;~/r_work/hive_metastore.jar;~/r_work/ql.jar;~/r_work/slf4j-api-1.5.8.jar;~/r_work/slf4j-log4j12-1.5.8.jar;~/r_work/TCLIServiceClient.jar","`")
#drv = JDBC("com.cloudera.hive.jdbc4.HS2Driver","~/r_work/HiveJDBC4.jar","`")
conn <- dbConnect(drv, "jdbc:hive2://grbd.com:10000/test_hive_db;AuthMech=1;KrbRealm=INTRANET.BARCAPINT.COM;KrbHostFQDN=grbd.com;KrbServicename=hive")
#conn <- dbConnect(drv, "jdbc:hive2://grbdwhd0017.intranet.barcapint.com:10000;AuthMech=1;KrbRealm=INTRANET.BARCAPINT.COM;KrbHostFQDN=grbdwhd0017.intranet.barcapint.com;KrbServicename=hive")
library(RHive)
> Sys.setenv(HIVE_Home= "ls /cloudera/parcels/lib/hive")
> Sys.setenv(HIVE_Lib= "ls /cloudera/parcels/lib/hive/lib")
> Sys.setenv(HIVE_HOME= "ls /cloudera/parcels/lib/hive")
> Sys.setenv(HIVE_LIB= "ls /cloudera/parcels/lib/hive/lib")
> Sys.setenv(HADOOP_HOME= "ls /cloudera/parcels/lib/hadoop")
> Sys.setenv(HADOOP_CONF = "ls /cloudera/parcels/lib/hadoop/etc/hadoop/hdfs-site.xml")
> Sys.setenv(HADOOP_LIB= "ls /cloudera/parcels/lib/hadoop/lib")
rhive.init()
> rhive.connect("grbd")
****Error: java.lang.ClassNotFoundException****
you can simply connect to hiveserver2 from R using RHIVE package
below are the commands that i had used.
Sys.setenv(HIVE_HOME="/usr/local/hive")
Sys.setenv(HADOOP_HOME="/usr/local/hadoop")
rhive.env(ALL=TRUE)
rhive.init()
rhive.connect("localhost")
Related
I'm trying to get data from AWS SQL Server.
This code works fine from local PC, but it didn't work from shiny-server (ubuntu).
library(dbplyr)
library(dplyr)
library(DBI)
con <- dbConnect(odbc::odbc(),
driver = "FreeTDS",
server = "aws server",
database = "",
uid = "",
pwd = "")
tbl(con, "shops")
dbGetQuery(con,"SELECT *
FROM shops")
"R version 3.4.2 (2017-09-28)"
packageVersion("dbplyr")
[1] ‘1.2.1.9000’
packageVersion("dplyr")
[1] ‘0.7.4’
packageVersion("DBI")
[1] ‘0.7.15’
I have next error:
tbl(con, "shops")
Error: <SQL> 'SELECT *
FROM "shops" AS "zzz2"
WHERE (0 = 1)'
nanodbc/nanodbc.cpp:1587: 42000: [FreeTDS][SQL Server]Incorrect syntax near 'shops'.
But dbGetQuery(con,"SELECT * FROM shops") works fine.
Can you explain what's going wrong?
This is more likely because the FreeTDS driver does not return the class that dbplyr expects to see in order to use the MS SQL translation. The workaround is to take the result of class(con) and then add the following lines right after you connect, but before calling tbl(). Replace the [you class name] with the results of the class(con) call:
sql_translate_env.[your class name] <- dbplyr:::`sql_translate_env.Microsoft SQL Server`
sql_select.[your class name]<- dbplyr:::`sql_select.Microsoft SQL Server`
I have R, R Studio and R-tools installed on Windows 7.
I am trying to: cd into a directory and execute a command, this is what I am getting from R, it returns M, what does that mean ?
> system(paste("cmd.exe", input = "cd c:/Users/X123/Desktop/FASTLOAD/", "&& fastload < load_journey.fl > output.txt", sep=" "))
M
>
what am I doing wrong ? I never see the command executed, thanks!
I changed it from system() to shell() and it works now, but get a bunch of warning messages, is that ok or should I do something for them to go away ?
> shell(paste("cd c:/Users/X123/Desktop/FASTLOAD/ ", "&& fastload < load_journey.fl > output.txt", sep=" "))
Warning messages:
1: running command 'C:\Windows\system32\cmd.exe /c cd c:/Users/X123/Desktop/FASTLOAD/ && fastload < load_journey.fl > output.txt' had status 4
2: In shell(paste("cd c:/Users/X123/Desktop/FASTLOAD/ ", "&& fastload < load_journey.fl > output.txt", :
'cd c:/Users/X123/Desktop/FASTLOAD/ && fastload < load_journey.fl > output.txt' execution failed with error code 4
>
I am trying to connect to hive using R JDBC library. My code looks like this:
library('DBI')
library('rJava')
library('RJDBC')
hadoop.class.path = list.files(path=c('/usr/hdp/hadoop/'), pattern='jar', full.names=T);
hadoop.lib.path = list.files(path=c('/usr/hdp/hadoop/lib/'), pattern='jar', full.names=T);
hive.class.path = list.files(path=c('/usr/hdp/hive/lib/'), pattern='jar', full.names=T);
mapred.class.path = list.files(path=c('/usr/hdp/hadoop-mapreduce'), pattern='jar', full.names=T);
cp = c(hadoop.class.path, hadoop.lib.path, hive.class.path, mapred.class.path, '/usr/hdp/hadoop-mapreduce/hadoop-mapreduce-client-core.jar')
.jinit(classpath=cp)
drv <- JDBC('org.apache.hive.jdbc.HiveDriver', '/usr/hdp/hive/lib/hive-jdbc.jar')
con <- dbConnect(drv, 'jdbc:hive2://my.cluster.net:10000/default;principal=hive/my.cluster.net#domain.com', 'hive', 'hive')
But when I run, I get the following error:
java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.security.SecurityUtil
However, I checked my /usr/hdp/hadoop/hadoop-commons.jar and found that the class org.apache.hadoop.security.SecurityUtil is there. So what else could be causing this error?
Hi I'm trying to use R to control unzipping a file.
I've added 7z into PATH, and did
7z e hat.2015-09-26T01-10-02.gz
and it worked.
Now in R, i'm in the same directory, and i tried
> command1 = paste0('7z e ', drop.file)
> command1
[1] "7z e hat.2015-09-26T01-10-02.gz"
> system(command1, intern=T)
Error in system(command1, intern = T) : '7z' not found
> system2(command1)
Warning message:
running command '"7z e hat.2015-09-26T01-10-02.gz"' had status 127
> shell(command1)
'7z' is not recognized as an internal or external command,
operable program or batch file.
Warning messages:
1: running command 'C:\Windows\system32\cmd.exe /c 7z e hat.2015-09-26T01-10-02.gz' had status 1
2: In shell(command1) :
'7z e hat.2015-09-26T01-10-02.gz' execution failed with error code 1
It's little bit raw, but try this out (:
Only Windows. It's uses CMD
ZiparEm7zip = function(sQualPasta)
{
sWDTava = getwd()
setwd(dirname(sQualPasta))
sQuem = gsub("[/]", "\\\\", sQualPasta)
dirname(sQuem)
NomeArquivo = paste0("eufaco7zip",".bat")
sNomePasta7zip = basename(sQuem)
sArquivoSaida = basename(sQuem)
sQualPasta7zip = sQuem
if(dir.exists("C:/Program Files/7-Zip/"))
{
sTexto = "set PATH=%PATH%;C:\\Program Files\\7-Zip\\"
}else if(dir.exists("C:/Program Files (x86)/7-Zip/"))
{
sTexto = "set PATH=%PATH%;C:\\Program Files (x86)\\7-Zip\\"
}else
{
stop("Error, o 7zip not installed?")
}
sTexto2 = paste0("\npushd ",sQualPasta7zip)
sTexto3 = paste0("\n7z a -r ../",sArquivoSaida," *")
sTexto4 = '\n( del /q /f "%~f0" >nul 2>&1 & exit /b 0 )'
cat(sTexto,file=NomeArquivo,append = TRUE)
cat(sTexto2,file=NomeArquivo,append = TRUE)
cat(sTexto3,file=NomeArquivo,append = TRUE)
cat(sTexto4,file=NomeArquivo,append = TRUE)
shell.exec(NomeArquivo)
while(file.exists(NomeArquivo))
{
Sys.sleep(10)
}
setwd(sWDTava)
return(TRUE)
}
Try like:
ZiparEm7zip("F:/MYFOLDER")
I'm getting an error when trying to connect (dbConnect()) from within R to MonetDB (using MonetDB.R). There is a (related SO question) and I also tryed the sugestions there ( (a) use MonetDB.R beta version 0.9.5 or (a) use the regular MonetDB.R but replace DBI package with an earlier version (0.2-7)) without success.
This is the code and errors:
library(MonetDB.R)
options( "monetdb.sequential" = TRUE )
setwd("C:/Users/lucas_000/Desktop/Curso R/importa_RAIS")
batfile <-
monetdb.server.setup(
database.directory = paste( getwd() , "/MonetDB", sep="") ,
monetdb.program.path =
ifelse(.Platform$OS.type == "windows" , "C:/Program Files/MonetDB/MonetDB5" , "" ) ,
dbname = "RAIS" , dbport = 50003
)
batfile <- "C:/Users/lucas_000/Desktop/Curso R/importa_RAIS/MonetDB/RAIS.bat"
dbname <- "RAIS"
dbport <- 50003
pid <- as.numeric(monetdb.server.start( batfile ))
# output indicates: MonetDB 5 server v11.17.21 "Jan2014-SP3"
db <- dbConnect( MonetDB.R() , "monetdb://localhost:5003/RAIS" , wait = TRUE )
Assertion Failed!
Program: C\:Program Files\RStudio\bin\x64\rsession.exe
File: mapi.c, Line 91
Expression: IS_INTEGER(port)
#Then RStudio displays an error: "R Session Aborted"
Sistem details:
Windows 8.0, on RStudio, Version 0.98.1062
> sessionInfo()
R version 3.1.1 (2014-07-10)
Platform: x86_64-w64-mingw32/x64 (64-bit)
other attached packages:
[1] R.utils_1.33.0 R.oo_1.18.0 R.methodsS3_1.6.1 downloader_0.3
[5] RCurl_1.95-4.3 bitops_1.0-6 sqlsurvey_0.6-11 survey_3.30-3
[9] KernSmooth_2.23-12 MonetDB.R_0.9.4 digest_0.6.4 DBI_0.3.1
loaded via a namespace (and not attached):
[1] tools_3.1.1
Attempted solutions and respective errors:
a) TRYING TO INSTALL MonetDB.R_0.9.5.zip (beta version)
library(devtools)
remove.packages("MonetDB.R")
install_url("http://homepages.cwi.nl/~hannes/R/MonetDB.R_0.9.5.zip")
* installing *binary* package 'MonetDB.R' ...
cp: unknown option -- )
Try `/usr/bin/cp --help' for more information.
Aviso: execução do comando 'cp -R . "C:/Users/lucas_000/Documents/R/win- library/3.1/MonetDB.R" || ( tar cd - .| (cd "C:/Users/lucas_000/Documents/R/win-library/3.1/MonetDB.R" && tar -xf -))' teve status 1
ERROR: installing binary package failed
* removing 'C:/Users/lucas_000/Documents/R/win-library/3.1/MonetDB.R'
Error: Command failed (1)
b) Keeping the MonetDB.R regular version (cran) but changing the DBI to an earlier version (also sugested on SO):
remove.packages("DBI")
library(devtools)
install_url("cran.r-project.org/src/contrib/Archive/DBI/DBI_0.2-7.tar.gz")
library(DBI)
db <- dbConnect( MonetDB.R() , "monetdb://localhost:5003/RAIS" , wait = TRUE )
# gives same error as above,
Assertion Failed!
Program: C:Program Files\RStudio\bin\x64\rsession.exe
File: mapi.c, Line 91
Expression: IS_INTEGER(port)
#Then RStudio displays an error: "R Session Aborted"
MonetDB.R 0.9.5 has been released to CRAN. Windows binary distributions should be available shortly.
The error may have been in the line where you create the dbConnect.
Your code (incorrect port):
db <- dbConnect( MonetDB.R() , "monetdb://localhost:5003/RAIS" , wait = TRUE )
Correct code:
db <- dbConnect( MonetDB.R() , "monetdb://localhost:50003/RAIS" , wait = TRUE )
Another issue with this code may have been choosing dbport to be be 50 003; I believe the default is 50 000.