After installing sparklyr package I followed the instruction here ( http://spark.rstudio.com/ ) to connect to spark. But faced with this error. Am I doing something wrong. Please help me.
sc = spark_connect( master = 'local' )
Error in file(con, "r") : cannot open the connection
In addition: Warning message:
In file(con, "r") :
cannot open file 'C:\Users\USER\AppData\Local\Temp\RtmpYb3dq4\fileff47b3411ae_spark.log':
Permission denied
But I am able to find the file at the stated location. And on opening, I found it to be empty.I
First of all, did you install sparklyr from github devtools::install_github("rstudio/sparklyr") or CRAN?
There were some issues some time ago with Windows installations.
The issue you have seems to be related to TEMP and TMP folder level permission on Windows or to file creation permission. Every time you start sc <- spark_connect(), it tries to create a folder and file to write the log files.
Make sure you have a write access to these locations.
I could observe the same error message with version 2.4.3 and 2.4.4
in different cases:
When trying to connect to a non "local" master, using spark_connect(master="spark://192.168.0.12:7077", ..),
if the master is not started or not responding at the specified master url.
when setting a specific incomplete configuration
in my case trying to set dynamicAllocation to true, without other required dynamicAllocation settings:
conf <- spark_config()
conf$spark.dynamicAllocation.enabled <- "true"
Related
here's my problem.
I access Google Analytics API via R with googleAnalyticsR. The whole thing happens in a Markdown script. Local everything runs without problems. But when I load the script on a Shiny Sever the GA token seems to expire after a while (a few hours).
The server log gives the following error message:
Warning in gzfile(file, mode) :
cannot open compressed file '.httr-oauth', probable reason 'Permission denied'
The script to create the token looks like this:
library(googleAuthR)
options(googleAuthR.client_id = "client-id",
googleAuthR.client_secret = "client-secret")
gar_auth()
I read that I should add
googleAuthR.httr_oauth_cache = FALSE
to the options. But when I do this and try to create a new Token I get follwing error:
Error: option('googleAuthR.httr_oauth_cache') must be set to
valid cache file location,
not TRUE or FALSE - (example: '.httr-oauth')
Anybody Ideas?
Thanks a lot
The issue is not that the token expires, the package googleAnalyticsR takes care of refreshing it if needed.
The problem is that the token was not created on the server (I assume you created locally then pushed to server), hence the error:
Warning in gzfile(file, mode) : cannot open compressed file '.httr-oauth', probable reason 'Permission denied'
The shinyuser which runs your app does not have the permission to open that file. I'm not a security expert so take this lightly. You run the following from the root of your application (on your server) there error will go away.
sudo chmod -R 777 .httr-oauth
The above will let shiny access the file.
So I have an Access database that I am trying to see on a windows server 2008 R2 (server name: WrenLab02); this server has the network discovery on, has the folder shared with that title, and is able to be pinged from the ubuntu computer; however now I want to call the folder that the files are in (which I tested on the server itself and works), but on the Ubuntu it keeps saying file directory not found.
On the Ubuntu computer in R:
FilePath <- file.path("//WrenLab02/Clinical/")
if (file.exists(FilePath)) {
setwd(FilePath)
} else {
dir.create(file.path(FilePath))
setwd(FilePath)
}
I know the path exists; I've tried it from another computer (which worked). Even I tried putting "smb://Wrenlab02/Clinical/" as the file director and still the setwd() cannot set the file path.
Maybe I just don't have the folder properly mapped to the Ubuntu, I tried to mount it and it is logged in and I can look at the files inside the folder and indeed the path says smb://WrenLab02/Clinical/. So I don't know where to go from here.
I figured it out; if anyone else is trying to pull files from a windows machine and you are working on a Lynx or Unbuntu machine this code should help you out.
setwd("/run/user/1000/gvfs/smb-share:server=wrenlab02,share=clinical/")
However now I am trying to use odbcConnect from RODBC package; this will connect me to a database from another server
WrenLIS <- '/run/user/1000/gvfs/smb-share:server=wrenlab01,share=administration/DATABASE/CLINICAL DATABASE/PatientContactDB.accdb'
cns <- odbcConnect(dsn="WrenLIS", uid="nferranti", pwd="H0ckey")
but it wont connect I keep getting an error that says:
Warning messages:
1: In RODBC::odbcDriverConnect("DSN=WrenLIS;UID=nferranti;PWD=H0ckey") : [RODBC] ERROR: state IM002, code 0, message [unixODBC][Driver Manager]Data source name not found, and no default driver specified
2: In RODBC::odbcDriverConnect("DSN=WrenLIS;UID=nferranti;PWD=H0ckey") : ODBC connection failed
I want to launch R help, I type ?dir.create. It is giving me this strange error:
Error in file(out, "wt") : cannot open the connection
In addition: Warning message:
In file(out, "wt") :
cannot open file
'C:\Users\XYZ~1\AppData\Local\Temp\Rtmp86bEoJ\Rtxt32dcef24de2': No
such file or directory
Please help me. What do I do to overcome this problem? Even googling this problem didn't give me much info.
This happens when the temporary directory used by R is deleted. This usually happens when user or the system cleans up the tmp directory while the R session is running. Typically happens if the R session is open for a long time.
Remedy is very simple. You don't actually need to restart R. Just re-create the temporary directory by:
tempdir()
# [1] "C:\Users\XYZ~1\AppData\Local\Temp\Rtmp86bEoJ\Rtxt32dcef24de2"
dir.create(tempdir())
NOTE: this happens to me more and more often in Windows 10... even within like 8 hour session only! Not sure why this happens. See this question: https://superuser.com/questions/1502272/windows-10-cleaning-up-my-temp-dir-too-often
Ok.Thank you all.I was able to resolve this by changing the directory and exiting R and re-logging into swirl.
Why is R not connecting to Hadoop ?
I am using R to connect to HDFS using 'rhdfs' package. The 'rJava' package is installed and rhdfs package is loaded.
The HADOOP_CMD environment variable is set in R using:
Sys.setenv(HADOOP_CMD='/usr/local/hadoop/bin')
But when hdfs.init() function is given, the following error message is generated:
sh: 1: /usr/local/hadoop/bin: Permission denied
Error in .jnew("org/apache/hadoop/conf/Configuration") :
java.lang.ClassNotFoundException
In addition: Warning message:
running command '/usr/local/hadoop/bin classpath' had status 126
Also, 'rmr2' library was loaded, and the following code was typed:
ints = to.dfs(1:100)
which generated the message given below:
sh: 1: /usr/local/hadoop/bin: Permission denied
The R-Hadoop packages are accessible only to the 'root' user and not 'hduser' (Hadoop user), since they were installed when R was run by the 'root' user.
Simple, only 2 reasons to get this type of problem
1) Wrong path
2) No privileges/permissions to that jar ok
not only that include other system paths. such as given below.
Sys.setenv(HADOOP_HOME="/home/hadoop/path")
Sys.setenv(HADOOP_CMD="/home/hadoop/path/bin/hadoop")
Sys.setenv(HADOOP_STREAMING="/home/hadoop/path/streaming-jar-file.jar")
Sys.setenv(JAVA_HOME="/home/hadoop/java/path")
Then include ibrary(rmr2) and library(rhdfs) paths, surely that error don't occur.
But your problem is Permission problem. So as a root grant all privileges (755) to you then run that jar file, surely that error don't display.
try like this.
Sys.setenv(HADOOP_CMD='/usr/local/hadoop/bin/hadoop')
Sys.setenv(JAVA_HOME='/usr/lib/jvm/java-6-openjdk-amd64')
library(rhdfs)
hdfs.init()
please give the correct HADOOP_CMD path extend with /bin/hadoop
For some reason I am suddenly not able to install packages in R (I have subsequently updated to the latest version of R and am running Windows 7). For example, if I type:
install.packages('beeswarm')
Installing package into ‘D:/Rlibs’ (as ‘lib’ is unspecified)
--- Please select a CRAN mirror for use in this session --- trying URL 'http://www.stats.bris.ac.uk/R/bin/windows/contrib/3.0/beeswarm_0.1.5.zip'
Content type 'text/html' length unknown opened URL downloaded 1859
bytes
Error in read.dcf(file.path(pkgname, "DESCRIPTION"), c("Package",
"Type")) : cannot open the connection In addition: Warning
messages: 1: In unzip(zipname, exdir = dest) : error 1 in extracting
from zip file 2: In read.dcf(file.path(pkgname, "DESCRIPTION"),
c("Package", "Type")) : cannot open compressed file
'beeswarm/DESCRIPTION', probable reason 'No such file or directory'
I have read that in Windows 7 there can be important restrictions on rights to writing to certain folders etc. so I've gone to some lengths to install R and library folders in non-default areas of my computer, and to allow myself rights to certain folders, but to no avail. Possibly also of importance is when I type:
.libPaths()
# [1] "D:/Rlibs"
# [2] "C:/Users/L.Halsey/Documents/R/win-library/3.0"
# [3] "C:/Users/L.Halsey/Documents/Documents/R-3.0.1/library"
I have created several folders in an attempt to create one that I could successfully install libraries into and set them up to be recognised by R using 'environment variables' from the start button. I don't know how to delete any of them though - not sure if this is relevant to my overall problem of not now being able to install/update packages for some reason.
The error being reported is inability to open a connection. In Windows that is often a firewall problem and is in the Windows R FAQ. The usual first attempt should be to run internet2.dll. From a console session you can use:
setInternet2(TRUE)
(You are correct in thinking this is not due to your library setup. The error says nothing about permissions.) I don't think just typing .libPaths should return that character vector since on my machine I would need to type .libPaths() to see something like that. If you wanted to reduce the number of places for libraries you can use the .libPaths function for setting the values. This would pick the second and third of the existing paths
.libPaths( .libPaths()[2:3] )
The inner call retrieves the path vector and the outer call sets it to a reduced vector.
Running RStudio as administrator fixed it for me!
I will probably duplicate a lot of other answers on the stackoverflow, but I got exactly the same error as OP, namely:
Warning messages: 1: In unzip(zipname, exdir = dest) : error 1 in extracting from zip file 2: In read.dcf(file.path(pkgname, "DESCRIPTION"), c("Package", "Type")) : cannot open compressed file 'zoo/DESCRIPTION', probable reason 'No such file or directory'
Turned out, while I as a user had permissions to write in a certain directory, R did not. In order to be sure you don't have something similar, do following:
get a usb drive, let's name it E
download package source as a .zip file and store it onto usb-drive in some directory, let's name it E:/source
Create directory for libraries on the usb drive, let's name it E:/libs
Install packages calling R command install.package from the R console and setting all relevant directories to point to your usb drive:
(here I use package zoo as an example)
install.packages("E:/source/zoo_1.7-12.zip",
destdir = 'E:/source', # no "/" after the path
lib = 'E:/libs',
repos = NULL)
Load the package from the directory, where you installed it:
library('zoo', lib.loc = 'E:/libs')
After you are sure, it works this way on your usb drive, you can start resolving directories permissions, and try out by changing the paths in the code above.
update:
In some windows environments even your usb-stick might be protected from read-write by the R. Make sure you check the permissions using the machine you are working from.
The following worked for me (based on the answer above)
install.packages("clustvarsel", lib = "C:/Users/dnentchev/My Programs/R-3.2.2/library")
I had the same problem. I turned the windows firewall off, and Run RStudio as administrator. so, that error fixed.