I have developed a package that I want to share with my colleagues at work.
I have a network drive in which I created the local repository structure that looks like this:
MyRepo
\__bin
\__windows
\__contrib
\__src
\__contrib
All folders are empty.
So I built my package with RStudio on Windows using the "Build/More/Build source package" menu, which created a tar.gz file.
Then I tried:
drat::insertPackage("../myPkg_0.0.0.9000.tar.gz",
repodir = "file://networkdrive/path/to/MyRepo",
action = "prune")
But this gives me an error:
Error: Directory file://networkdrive/path/to/MyRepo not found
Which is strange because file.exists(//networkdrive/path/to/MyRepo) returns true.
OK, then I tried:
drat::insertPackage("../myPkg_0.0.0.9000.tar.gz",
repodir = "//networkdrive/path/to/MyRepo",
action = "prune")
Without the file: in the repository path and I get another error:
tar (child): "//networkdrive/path/to/MyRepo/src/contrib/myPkg_0.0.0.9000.tar.gz: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
/usr/bin/tar: Child returned status 2
/usr/bin/tar: myPkg/DESCRIPTION: Not found in archive
/usr/bin/tar: Exiting with failure status due to previous errors
reading DESCRIPTION for package ‘myPkg’ failed with message:
cannot open the connection
But when I go in the "//networkdrive/path/to/MyRepo/src/contrib" folder, I can definitely see the myPkg_0.0.0.9000.tar.gz file that has been copied despite the error message.
Can anyone help?
> sessionInfo()
R version 3.3.3 (2017-03-06)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1
locale:
[1] LC_COLLATE=French_France.1252 LC_CTYPE=French_France.1252 LC_MONETARY=French_France.1252 LC_NUMERIC=C LC_TIME=French_France.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] drat_0.1.2 tools_3.3.3 git2r_0.18.0
I know this is old, but my colleague just came across the same problem and found this post. I believe the issue may be the lack of a trailing slash in your directory name. I have been able to recreate the error with a mapped network drive. I can resolve the issue by using "H:/MyRepo/" instead of "H:/MyRepo".
I haven't tried it with the "file://" format, but I wanted to include my answer in case someone else comes across this question.
Ok, so after some research, here is my conclusions.
It cannot be done
It's not Drat's fault
The reason why it does not work is that the tools::write_PACKAGES function does not work on network drives. Period.
I manually copied my package on the network drive, then ran setwd() to its location and executed write_PACKAGES(".", type="source") and I got the same error.
So to make this work, I just left my package.tar.gz file on a local drive, ran the tools::write_PACKAGES command locally and then moved the files to the network drive.
Adding the network drive to my repository list using options(repos = c(MyRepo = "file://networkdrive/path/to/MyRepo/")) works: RStudio and available.packages find my package.
It's not completely satisfactory, but I think it's the only way today.
I was having this problem as well and finally got to the bottom of it today.
For me, the problem was not isolated to just network locations but also occurred on C: drive. The root cause was the version of tar.exe being used to unpack the existing packages in the package directory. Calls to utils::untar are made in the tools::write_PACKAGES function.
The documentation for utils::untar explains that on Windows, external tar.exe is tried first. Sure enough, I had a version installed with Git which when used with default arguments fails when a colon is in the file name. I was able to force utils::untar to use to use the RBuildTools version of tar.exe instead by setting the environment variable TAR to "internal".
drat::insertPackage now works.
Related
This is the error message:
> h2o.init()
Error in dirname(path) : path too long
In addition: There were 12 warnings (use warnings() to see them)
This is one of the warning messages (the others are similar):
> warnings()
Warning messages:
1: In normalizePath(path.expand(path), winslash, mustWork) :
path[1]="\\FILE-EM1-06/USERDATA2$/john134/My Documents/./../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../..": The filename or extension is too long
Any idea how to work around this error?
Thanks
It seems that Windows path string is limited to (maybe) 256 length. Usually, setting a the path setwd(shorterExistingWorkDir) works and should address your issue.
I struggled with this issue quite a bit, including upgrading.
Most folks are assuming that you've literally just set an incredibly long path. I don't think this is the case (it wasn't for me, at least). It's that the PATH may be set on a network drive or other device where the underlying mapped paths are more complicated.
A related thread is here on the H2O forum:
Main issue is the user had a Windows drive that did not conform to the norm, i.e., "C://", etc. Instead, the user had a network drive
(DTCHYB-AZPX015/). This caused issues in the search for a config
file as there was no "root" (In this case, "root" is reaching your Win
drive). Since there was no "root", the path to search kept expanding
until it caused R to error out with the above exception.
The fix is to NOT search for a config when h2o.init() is called. Rather, only search for a config if a user asks to do so. My proposal
is to add a new field to h2o.init()called ignore_config. This
field will be set to TRUE by default.
When calling h2o.init() the R environment signal the launching of h2o application (actually a web server) in the backend which was installed when you install H2O package into R. The local runtime environment uses the full path of the location where H2O jar file is located. Because the packages is installed deep inside the nested folders in your file system it cross the valid limit of OS path 256 character length and fails to launch the backend H2O server and you see this error. In your case you are using external path so adds up more characters in the path to make the problem worse..
For example the h2o.jar is located in my OSX machine as below:
/Library/Frameworks/R.framework/Resources/library/h2o <-- H2O package Path
/Library/Frameworks/R.framework/Resources/library/h2o/java/h2o.jar <-- Jar Path
As you are using Windows, what you need is to find ways to reduce this path to OS limit and it will work.
The other solution is to run h2o.jar separately and then just use R to connect to H2O cluster. The steps are as below:
Download H2O 3.10.4.2 and unzip to a folder close to root so you do not hit 265 char limit again. Also install 3.10.4.2 R Package. (Try to keep the same version)
Run H2O > java -jar h2o.jar
From RStudio console try > h2o.init()
So if there is already H2O cluster running the h2o.init() will connect to a running H2O cluster instead to start one and you will by pass above problem.
If you hit any problem write here and we will help you.
When I run a Shiny App either an example or my own it starts a new window, freezes and then aborts. It tries to listening to a local port then starts a new session.
Details of my system:
My R version: 3.1.3 on MacOS Maverick
other attached packages:
[1] shiny_0.12.0 Rcpp_0.11.6
loaded via a namespace (and not attached):
[1] digest_0.6.8 htmltools_0.2.6 httpuv_1.3.2 mime_0.3 R6_2.0.1
[6] tools_3.1.3 xtable_1.7-4
What I tried so far:
From different internet forums I found a lot of suggestions:
I reinstalled Rstudio.
I updated all the packages
Tried this: install.packages(c("Rcpp", "httpuv", "shiny"))
Loaded Rcpp and httpuv separately
tried to run like this runApp('app1',host="127.0.0.1",port = 3894)
// tried other ports as well.
But nothing works.
(Please note that, I don't know whether it's relevant but GoogleVis Demo worked on my RStudio.)
#roboticman
My recommendation is first to test and confirm R is correctly setup on your machine. We can then move to harness a Shiny demonstration application and confirm if that works.
https://technophobe01.shinyapps.io/WorldBank/
First step:
Can you please run the following command via R, this should download and install shiny and its dependencies...
> install.packages("shiny", dependencies = TRUE)
If things work after this, great. If not, well, then let's check your R configuration:
Validate R Install
To check your configuration, my recommendation is you launch R on your machine and run the following tests and post back the output.
> R
> sessionInfo()
Sys.setenv(LC_COLLATE = "C", LC_TIME = "C", LANGUAGE = "en")
library("tools")
testInstalledBasic("both")
testInstalledPackages(scope = "base")
testInstalledPackages(scope = "recommended")
testInstalledBasic("internet")
Once you post the output of the above command set, we can move to work through the example 'World Bank', and see if we can determine the problem.
Take care
Try this:
library(shiny)
runExample("01_hello", host = "0.0.0.0", port = 9999).
runApp will also work:
shiny::runApp(host = "127.0.0.1", port = 9999)
I am trying to read an Excel file that is stored on a corporate intranet SharePoint site, and am doing so using the 'gdata' package from within R Studio run on a Linux server using Shiny Server.
I had good results reading the file from within a MS Windows environment using gdata, but cannot seem to get things to work when running the script on the Linux server. This was based on the info from:
http://r.789695.n4.nabble.com/trying-to-import-xls-or-xlsx-files-td3620580.html
I did modify the R script for the path to Perl on the Linux server (vs. the path to perl.exe on Windows), and this seems to work ok.
The file url is
"http://sharepoint2/ops/quality/metricspc/Metric OptIn List/temporary SPC Metric Opt-In List.xlsm"
Here is the R code:
# R read MS Excel xlsm file from SharePoint
# method using gdata - seems to work with SharePoint
# NOTE: requires 'perl' installed
#
# example from
# http://r.789695.n4.nabble.com/trying-to-import-xls-or-xlsx-files-td3620580.html
library(gdata)
fileurl =
# see fileurl above this code section - did this due to SO error message about not having 'sharepoint2' in the url
d.optin.init2 <- read.xls(fileurl,
sheet = "OPT-IN LIST",
perl = "/usr/bin/perl")
##### END CODE #####
The original (Windows-based) script used perl = "C:\\Perl64\\bin\\perl.exe"
Here is the error message that results (when run from Linux using R Studio on Shiny Server):
d.optin.init2 <- read.xls(fileurl,
+ sheet = "OPT-IN LIST",
+ perl = "/usr/bin/perl")
trying URL 'http://sharepoint2/ops/quality/metricspc/Metric OptIn List/temporary SPC Metric Opt-In List.xlsm'
Error in download.file(xls, tf, mode = "wb") :
cannot open URL 'http://sharepoint2/ops/quality/metricspc/Metric OptIn List/temporary SPC Metric Opt-In List.xlsm'
In addition: Warning message:
In download.file(xls, tf, mode = "wb") :
cannot open: HTTP status was '400 Bad Request'
Error in file.exists(tfn) : invalid 'file' argument
The path to the file (on SharePoint) is a URL (shown in the code), so I thought that the route between the Linux server and MS SharePoint might be straightforward. But since this does work for me with Windows but does not yet work for me with Linux, I wonder what I may have missed?
Thank you in advance for any insight that might be offered.
(I haven't attached the Excel file because I wasn't sure that it would help with this question.)
Best regards,
Cliff
Output from sessionInfo
sessionInfo()
R version 3.1.0 (2014-04-10)
Platform: x86_64-redhat-linux-gnu (64-bit)
locale:
1 C
attached base packages:
1 stats graphics grDevices utils datasets methods base
other attached packages:
1 gdata_2.13.3 qcc_2.5
loaded via a namespace (and not attached):
1 MASS_7.3-31 gtools_3.4.1 tools_3.1.0
Acting on the suggestions provided by hrbrmstr and also Greg, I tried both wget and also curl from the Linux command line.
wget results
curl results
I can work with our IT folks to resolve. If anyone can help me to refine the questions I might ask of them based on these results, I would welcome this input.
Thanks again to those who took the time to respond.
MORE FOLLOW-UP
#Gregory R. Warnes
I was able to use wget with this command line from the Linux server:
wget --http-user=myusername --http-passwd=mypassword (place fileurl here)
This seemed to access the file, bridging the divide between Linux server and Windows SharePoint.
Now to figure out how to include this AD authentication in the gdata R script.
The error message indicates that R's download.file() code was not able to access the URL.
To debug the issue, try accessing that URL from the Linux server from the shell command line, for example:
If wget is installed:
> wget http://sharepoint2/ops/quality/metricspc/Metric OptIn List/temporary SPC Metric Opt-In List.xlsm
or if curl is installed:
> curl http://sharepoint2/ops/quality/metricspc/Metric OptIn List/temporary SPC Metric Opt-In List.xlsm
If wget and/or curl generates an error, then you have a server configuration issue.
If wget and/or curl succeed then the problem may be with R's download.file().
-Greg
I already installed OpenCPU on a Ubuntu Server - Ubuntu 14.04.1 LTS (GNU/Linux 3.13.0-32-generic x86_64) - and everything worked perfectly without any problems.
Here I want to say that I really like this API and I am very thankful for all the effort from the people (I think mostly Jeroen Ooms) working on it.
Now I installed it again, but on another Server hosted at another provider. It is also a Ubuntu Server - Ubuntu 14.04.1 LTS (GNU/Linux 3.13.0-042stab093.4 x86_64) - and therefore I expected it to work as smoothly as before.
But now I have a problem. After the installation and starting the service, I wanted to check through my browser if everything is OK.
So I just opened http://xxx.xxx.xxx.xxx/ocpu like it worked on my other server. This time my browser doesn't show the OpenCPU API Explorer, but the following message:
Failed to set rlimit. ENOSYS
In call:
rlimit_wrapper("rlimit_as", hardlim, softlim, pid, verbose)
The server only has 1GB of physical memory, so I thought changing the "rlimit.as" to 1e9 instead of the standrd 2e9 would fix the problem (I also tried 750000000 and 500000000), but nothing helped (of course I restarted the service opencpu again after each change).
I also think that this is not the problem, because I guess the server would use virtual memory when an operation uses more than one GB.
I think the problem has to do with RAppArmor. So I tried to disable it and restart opencpu, but the problem didn't vanish:
$ sudo aa-disable usr.bin.r
Disabling /etc/apparmor.d/usr.bin.r.
Traceback (most recent call last):
File "/usr/sbin/aa-disable", line 30, in
tool.cmd_disable()
File "/usr/lib/python3/dist-packages/apparmor/tools.py", line 148, in cmd_disable
raise apparmor.AppArmorException(cmd_info[1])
apparmor.common.AppArmorException: 'Warning: unable to find a suitable fs in /proc/mounts, is it mounted?\nUse --subdomainfs to override.\n'
So does anyone know what the problem here could be or has any suggestions where to look for a solution (I tried to google already, but didn't find anything helpful)?
I don't think anything of the following is the cause of the problem, but since I'm not sure, I add these warnings anyways:
The only strange thing I encountered during the OpenCPU installation was this message (which appeared 4 times):
iptables v1.4.21: can't initialize iptables table `nat': Table does not exist (do you need to insmod?)
Perhaps iptables or your kernel needs to be upgraded.
But afterwards it said:
* Reloading nginx configuration nginx [ OK ]
OK
Setting up opencpu (1.4.4-trusty15) ...
Also when I tried to install RAppArmor separately, I got the following warning:
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LC_CTYPE = "UTF-8",
LANG = (unset)
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
Selecting previously unselected package r-cran-rapparmor.
And also this one:
Warning: unable to find a suitable fs in /proc/mounts, is it mounted?\nUse --subdomainfs to override.\n
Thanks in advance!
It looks like your new hosting provider uses some sort of virtualization system that has a shared kernel which limits all kind of linux functionality, including rlimit, iptables and probably apparmor. Is an actual cloud host, or something you setup yourself?
It would be helpful to debug this in R (outside of opencpu). On your server, start R in the console and type:
library(RAppArmor, lib="/usr/lib/opencpu/library")
rlimit_as(1e9)
rlimit_fsize(1e9)
rlimit_cpu(1e5)
I am downloading data from data.gov website and I get following two types of errors in the process:
fileUrl <- "http://catalog.data.gov/dataset/expenditures-on-children-by-families"
download.file(fileUrl,destfile=".data/studentdata.csv",method="curl")
Warning message:
In download.file(fileUrl, destfile = ".data/studentdata.csv", method = "curl") :
download had nonzero exit status
I tried to remove the method="curl" as suggested in other forum, but again I get this new error
download.file(fileUrl,destfile=".data/studentdata.csv")
Error in download.file(fileUrl, destfile = ".data/studentdata.csv") :
cannot open destfile '.data/studentdata.csv', reason 'No such file or directory'
I think there are two major factors why your curl doesn't work well.
First, the problem is on your URL. fileUrl <- "http://catalog.data.gov/dataset/expenditures-on-children-by-families". In your URL, it is not referred to a csv file. So, they won't work even if you set the destination into a csv file such as destfile = ".data/studentdata.csv"
I have an example of getting a csv dataset using the same code (different dataset):
DataURL<- "https://data.baltimorecity.gov/api/views/dz54-2aru/rows.csv?accessType=DOWNLOAD" (This link refers to a rows.csv file)
download.file(DataURL, destfile="./data/rows.csv", method="curl") (The method is quite same, using curl)
Second, previously I had the same problem that the curl does not work, even I used a proper URL that refers to a csv file. However, when I diagnosed a bit deeper, I found something interesting fact about why my curl method cannot work properly. It was my R session program. I used a 32-bit R, in which the error occurs. Later then, I tried to change the session into a 64-bit R. Amazingly, and the download status was running at that time. To see your R session architecture (whether you are using 32-bit or 64-bit), type in your R:
sessionInfo()
R version 3.5.3 (2019-03-11)
Platform: x86_64-w64-ming32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)
You have to switch your R, from 32-bit to 64-bit to avoid 'curl' call had nonzero exit status. You go to your R directory folder, and then you run a 64-bit R.
If you are using a Windows OS and installing the R in a default path folder, you can run this C:\Program Files\R\R-3.5.3\bin\x64\R.exe. (I used a version of 3.5.3, so it may be different with your version)
If you are using R-studio, you can switch the R session on the menubar Tools -> Global Options -> R version -> Change -> Use your machine's default version of R64 (64-bit) -> OK. Then restart your R-studio.
However, it depends on your OS architecture. If you are using a 32-bit OS, hence you have to find another way to solve this.
So looking at the code for download.file(...), if you specify method="curl" the function tries to use the curl shell command. If this command does not exist on your system, you will get the error above.
If you do not specify a method, the default is to use an internal R method to download, which evidently works on your system. In that case, the function is trying to put the file in .data/studentdata.csv but evidently there is not .data directory. Try taking out the ..
When this download works, you will get a text/html file, not a csv file. Your url points to a web page, not a download link. That page does have a download link, but unfortunately it is a pdf, not a csv.
Finally, if your goal is to have the data in R (is it?), and if the link actually produces a csv file, you could more easily use
df <- read.csv(fileUrl)
If I'm not very much mistaken you just have a simple typo here. I suspect you have a "data" directory, not a ".data" directory - in which case your only problem is that your destfile string needs to begin "./data", not ".data".
I was having the same problem.
Then I realized that I forget to create the "data" directory!
So try adding this above your fileURL line to create the directory first.
if(!file.exists("data")){
dir.create("data")
}
Also, if you are running a Mac, then you want to keep method="curl" when downloading a https file. I don't believe Windows has that problem hence the suggestions to remove it.
Try this:
file<-'http://catalog.data.gov/dataset/expenditures-on-children-by-families'
file<- read.csv(file)