Compression issue when saving anR project - r

I solved the problem by moving R installation directory out of disk C. Thanks Joris for the great suggestions! I think the R core team should also take this as a bug and do something against the protecting mechanism of windows xp.
Dear Community:
While using the BIOMOD packages in R, I always get the following problem:
Error in xzfile(file, "wb", compression = 9) : cannot open the connection
In addition: Warning message:
In xzfile(file, "wb", compression = 9) :
cannot initialize lzma encoder, error 5
It was said by the author of the package and also in the help file of "save" that the problem should be caused by lack of permission to write. However, as I am logging in as administative account and have assess to all operations, I have no idea what the problem is. Can anybody help me out? I really need to run the package now. Thanks in advance~
Sincerely,
Marco
Below is the illustration in the help file of "save":
The most common reason for failure is lack of write permission in
the current directory. For 'save.image' and for saving at the end
of a session this will shown by messages like
Error in gzfile(file, "wb") : unable to open connection
In addition: Warning message:
In gzfile(file, "wb") :
cannot open compressed file '.RDataTmp',
probable reason 'Permission denied'
The defaults were changed to use compressed saves for 'save' in
2.3.0 and for 'save.image' in 2.4.0. Any recent version of R can
read compressed save files, and a compressed file can be
uncompressed (by 'gzip -d') for use with very old versions of R.*
Sorry for the ommision of the information:
Here is the sessionInfo():
> sessionInfo()
R version 2.12.2 (2011-02-25)
Platform: i386-pc-mingw32/i386 (32-bit)
locale:
[1] LC_COLLATE=Chinese_People's Republic of China.936
[2] LC_CTYPE=Chinese_People's Republic of China.936
[3] LC_MONETARY=Chinese_People's Republic of China.936
[4] LC_NUMERIC=C
[5] LC_TIME=Chinese_People's Republic of China.936
attached base packages:
[1] splines stats graphics grDevices utils datasets methods
[8] base
other attached packages:
[1] BIOMOD_1.1-6.8 foreign_0.8-42 gam_1.04
[4] randomForest_4.6-2 mda_0.4-1 class_7.3-3
[7] gbm_1.6-3.1 lattice_0.19-17 MASS_7.3-11
[10] Design_2.3-0 Hmisc_3.8-3 survival_2.36-5
[13] rpart_3.1-48 nnet_7.3-1 ade4_1.4-16
[16] rgdal_0.6-33 dismo_0.5-19 rJava_0.9-0
[19] raster_1.7-47 sp_0.9-78
loaded via a namespace (and not attached):
[1] cluster_1.13.3 grid_2.12.2 tools_2.12.2
Now I found that the problem come form the lzma encoder in doing "save":
> x<-runif(100)
> save(x, file = "F:/test.gzip", compress='gzip')
> save(x, file = "F:/test.xz", compress='xz')
Error in xzfile(file, "wb", compression = 9) : cannot open the connection
>

I had a similar issue when trying to project to a new scenario (a tables containing columns corresponding to the predictor variables) after having run the modeling procedure using 8 models.
The first table (approx 250,000 rows) ran fine, and I was able to save the results as a .csv file. However the second one (approx 380,000 rows) resulted in the above error message, and some of the files were not written to the project folder.
I have since cut all the tables down to a maximum of 260,000 rows and I no longer recieve the error message. It was a bit of a pain doing it in multiple runs, but once I had written the script once, I just used find and replace in MS Word to change it for each run.

Related

Directory not found when using Drat on a network drive

I have developed a package that I want to share with my colleagues at work.
I have a network drive in which I created the local repository structure that looks like this:
MyRepo
\__bin
\__windows
\__contrib
\__src
\__contrib
All folders are empty.
So I built my package with RStudio on Windows using the "Build/More/Build source package" menu, which created a tar.gz file.
Then I tried:
drat::insertPackage("../myPkg_0.0.0.9000.tar.gz",
repodir = "file://networkdrive/path/to/MyRepo",
action = "prune")
But this gives me an error:
Error: Directory file://networkdrive/path/to/MyRepo not found
Which is strange because file.exists(//networkdrive/path/to/MyRepo) returns true.
OK, then I tried:
drat::insertPackage("../myPkg_0.0.0.9000.tar.gz",
repodir = "//networkdrive/path/to/MyRepo",
action = "prune")
Without the file: in the repository path and I get another error:
tar (child): "//networkdrive/path/to/MyRepo/src/contrib/myPkg_0.0.0.9000.tar.gz: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
/usr/bin/tar: Child returned status 2
/usr/bin/tar: myPkg/DESCRIPTION: Not found in archive
/usr/bin/tar: Exiting with failure status due to previous errors
reading DESCRIPTION for package ‘myPkg’ failed with message:
cannot open the connection
But when I go in the "//networkdrive/path/to/MyRepo/src/contrib" folder, I can definitely see the myPkg_0.0.0.9000.tar.gz file that has been copied despite the error message.
Can anyone help?
> sessionInfo()
R version 3.3.3 (2017-03-06)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1
locale:
[1] LC_COLLATE=French_France.1252 LC_CTYPE=French_France.1252 LC_MONETARY=French_France.1252 LC_NUMERIC=C LC_TIME=French_France.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] drat_0.1.2 tools_3.3.3 git2r_0.18.0
I know this is old, but my colleague just came across the same problem and found this post. I believe the issue may be the lack of a trailing slash in your directory name. I have been able to recreate the error with a mapped network drive. I can resolve the issue by using "H:/MyRepo/" instead of "H:/MyRepo".
I haven't tried it with the "file://" format, but I wanted to include my answer in case someone else comes across this question.
Ok, so after some research, here is my conclusions.
It cannot be done
It's not Drat's fault
The reason why it does not work is that the tools::write_PACKAGES function does not work on network drives. Period.
I manually copied my package on the network drive, then ran setwd() to its location and executed write_PACKAGES(".", type="source") and I got the same error.
So to make this work, I just left my package.tar.gz file on a local drive, ran the tools::write_PACKAGES command locally and then moved the files to the network drive.
Adding the network drive to my repository list using options(repos = c(MyRepo = "file://networkdrive/path/to/MyRepo/")) works: RStudio and available.packages find my package.
It's not completely satisfactory, but I think it's the only way today.
I was having this problem as well and finally got to the bottom of it today.
For me, the problem was not isolated to just network locations but also occurred on C: drive. The root cause was the version of tar.exe being used to unpack the existing packages in the package directory. Calls to utils::untar are made in the tools::write_PACKAGES function.
The documentation for utils::untar explains that on Windows, external tar.exe is tried first. Sure enough, I had a version installed with Git which when used with default arguments fails when a colon is in the file name. I was able to force utils::untar to use to use the RBuildTools version of tar.exe instead by setting the environment variable TAR to "internal".
drat::insertPackage now works.

Error when using readxl: 'exdir' does not exist

I am trying to use readxl::read_excel to import tabular data from an excel spreadsheet (.xlsx and .xls) into R. I have had success with this function when reading spreadsheets from Desktop, but I am having trouble reading from my Documents folder now.
I tried reading both .xlsx and .xls formats of the same file and got 2 different error messages.
# Check the working directory
[1] "\\\\isdowfsv04/D100Home/e525902"
# Read .xlsx
specs <- read_excel("Class Specs Sept 2016.xlsx")
Error: 'exdir' does not exist
# Read .xls
specs <- read_excel("Class Specs Sept 2016.xls")
Error: path[1]="\\isdowfsv04/D100Home/e525902/Class Specs Sept 2016.xls": Access is denied
The second message made me think I don't have permission to read/write data here, but I create folders/files here all the time. I have even used another R library, readr package, to import .csv files from this same directory.
# Using readr to import .csv version of the file works fine.
specs <- read_csv("\\\\isdowfsv04/D100Home/e525902/Class Specs Sept 2016.csv")
Based on other posts I've seen, it seems like a permission issue, but it's unclear to me how to fix it.
I tried using Sys.setenv() to change the temporary folders location to somewhere I know I have permission to write (i.e., Desktop) because I thought readxl used these paths when reading the data. But that didn't work because I got the exact same error messages.
Sys.getenv(c("TEMP", "TMP"))
TEMP TMP
"C:\\Users\\e525902\\AppData\\Local\\Temp" "C:\\Users\\e525902\\AppData\\Local\\Temp"
# Change temp file paths to the Desktop
Sys.setenv(TEMP = "C:\\Users\\e525902\\Desktop")
Sys.setenv(TMP = "C:\\Users\\e525902\\Desktop")
I am using a PC at work and the Documents folder is actually a network drive. I have used read_excel to import .xlsx files from other network folders, but this one is giving me trouble. Below is more information on the system I'm using.
Sys.info()
sysname release version
"Windows" "7 x64" "build 7601, Service Pack 1"
nodename machine login
"10004053WDT" "x86-64" "e525902"
user effective_user
"e525902" "e525902"
search()
[1] ".GlobalEnv" "package:readr" "package:readxl" "tools:rstudio" "package:stats"
[6] "package:graphics" "package:grDevices" "package:utils" "package:datasets" "package:methods"
[11] "Autoloads" "package:base"
# R version
version$version.string
version.string R version 3.2.5 (2016-04-14)
I ran into the same issue. It was occurring to me when I set my working directory to the base location of the excel files, and then in the read excel, using a relative path to get to the directory.
Once I rebooted R, and instead kept my working directory as the default, and then used the full file path in the read_excel function, my exdir error went away.
Let me know if this works for you. I was also on a network drive at work with a windows 7 64-bit system.
The current working directory is displayed by the RStudio IDE within the title region of the Console pane. You can also check your current working directory by running the command getwd() in the console.

Encoding lost when reading XML in R

I am retrieving online XML data using the XML R packages. My issue is that the UTF-8 encoding is lost during the call to xmlToList : for instance, 'é' are replaced by 'é'. This happens during the XML parsing.
Here is a code snippet, with an example of encoding lost and another where encoding is kept (depending of the data source) :
library(XML)
library(RCurl)
url = "http://www.bdm.insee.fr/series/sdmx/data/DEFAILLANCES-ENT-FR-ACT/M.AZ+BE.BRUT+CVS-CJO?lastNObservations=2"
res <- getURL(url)
xmlToList(res)
# encoding lost
url2 = "http://www.bdm.insee.fr/series/sdmx/conceptscheme/"
res2 <- getURL(url2)
xmlToList(res2)
# encoding kept
Why the behaviour about encoding is different ? I tried to set .encoding = "UTF-8" in getURL, and to enc2utf8(res) but that makes no change.
Any help is welcome !
Thanks,
Jérémy
R version 3.2.1 (2015-06-18)
Platform: i386-w64-mingw32/i386 (32-bit)
Running under: Windows 7 (build 7601) Service Pack 1
locale:
[1] LC_COLLATE=French_France.1252 LC_CTYPE=French_France.1252
[3] LC_MONETARY=French_France.1252 LC_NUMERIC=C
[5] LC_TIME=French_France.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] RCurl_1.95-4.7 bitops_1.0-6 XML_3.98-1.3
loaded via a namespace (and not attached):
[1] tools_3.2.1
You are trying to read SDMX documents in R. I would suggest to use the rsdmx package that makes easier the reading of SDMX documents. The package is available on CRAN, you can also access the latest version on Github.
rsdmx allows you to read SDMX documents by file or url, e.g.
require(rsdmx)
sdmx = readSDMX("http://www.bdm.insee.fr/series/sdmx/data/DEFAILLANCES-ENT-FR-ACT/M.AZ+BE.BRUT+CVS-CJO?lastNObservations=2")
as.data.frame(sdmx)
Another approach is to use the web-service interface to embedded data providers, and INSEE is one of them. Try:
sdmx <- readSDMX(providerId = "INSEE", resource = "data",
flowRef = "DEFAILLANCES-ENT-FR-ACT",
key = "M.AZ+BE.BRUT+CVS-CJO", key.mode = "SDMX",
start = 2010, end = 2015)
as.data.frame(sdmx)
AFAIK the package also contains issues to the character encoding, but i'm currently investigating a solution to make available soon in the package. Calling getURL(file, .encoding="UTF-8") properly retrieves data, but encoding is lost calling xml functions.
Note: I also see you use a parameter lastNObservations. For the moment the web-service interface does not support extra parameters, but it may be made available quite easily if you need it.

How to read chinese in rstudio on Linux

I encountered an issue when read the chinese file on Linux system by rstudio.
The error as below.
dt <- read.csv(file = "/home/..../aa-0912.csv", header = T , sep=",")
Error in make.names(col.names, unique = TRUE) :
invalid multibyte string at '<be><ba><b5><c3><c8><cb>'
This csv file is written by rstudio on Window system w/o specified encoding, as below:
write.csv(file = "/home/.../aa-0912.csv", data)
And I can read correctly on window but when I copy this file on my Linux system the read.csv
doesn't work.
The locale on Linux is :
Sys.getlocale()
[1] "LC_CTYPE=en_US.UTF-8;LC_NUMERIC=C;LC_TIME=en_US.UTF-8;LC_COLLATE=en_US.UTF-8;LC_MONETARY=en_US.UTF-8;LC_MESSAGES=en_US.UTF-8;LC_PAPER=en_US.UTF-8;LC_NAME=C;LC_ADDRESS=C;LC_TELEPHONE=C;LC_MEASUREMENT=en_US.UTF-8;LC_IDENTIFICATION=C"
The locale on Window is :
LC_COLLATE=English_United States.1252;LC_CTYPE=English_United States.1252;LC_MONETARY=English_United States.1252;LC_NUMERIC=C;LC_TIME=English_United States.1252
I am trying to read data by encoding="utf-8" but I got the similar error message.
Any help?
I'm not sure that this is the answer to your question.
I'll try to be as general as possible so that people having trouble in any language might have a solution:
First in the terminal local -a local would display all the available locales on your system.
Once you found the locale the right locale then on RStudio:
Sys.setlocale("LC_ALL","fr_FR.utf8")
Sorry I don't seem to have any Chinese locale on my system. Other people have had the same issues: here and here
have also a look at ?Sys.setlocale in R.

Can't use either View() or edit() functions, getting "Error in .External2(C_dataviewer, x, title) : invalid device" error message

I'm running R through the Terminal in a Mac OSX Snow Leopard system.
I get this error message when trying to use View():
Error in .External2(C_dataviewer, x, title) : invalid device In
addition: Warning messages: 1: In View(a) : locale not supported by
Xlib: some X ops will operate in C locale 2: In View(a) : X cannot set
locale modifiers 3: In View(a) : unable to create fontset
--fixed-medium-r----120------
And this error message when trying to use edit():
Error in .External2(C_dataentry, datalist, modes) : invalid device In
addition: Warning messages: 1: In edit.data.frame(a) : locale not
supported by Xlib: some X ops will operate in C locale 2: In
edit.data.frame(a) : X cannot set locale modifiers 3: In
edit.data.frame(a) : unable to create fontset
--fixed-medium-r----120------
I can't find information about this particular problem. What can I do to fix this?
sessionInfo() returns:
R version 3.0.1 (2013-05-16)
Platform: x86_64-apple-darwin10.8.0 (64-bit)
locale:
[1] C/UTF-8/C/C/C/C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
The problem might be your locale settings. Run command 'locale' from terminal and see if there are any lines with something else but 'C' locale. For example, in my case output is like this:
macbook:foo user$ locale
LANG=
LC_COLLATE="C"
LC_CTYPE="UTF-8"
LC_MESSAGES="C"
LC_MONETARY="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_ALL=
Easiest way to solve this is to set the locale to be C for just the R process ie. run R from terminal session with command 'LC_CTYPE=C R' instead of just 'R'. At least on OS/X 10.6.8 this gets rid of the error messages and allows R to display the edit window.
The cause for my problem: View(dataset) in the rchunk where we load up data.
I had it here because I loaded in data like data <-read_excel("name")
And had View there because I copied and pasted it from my console, but it's not needed.
Experienced this with RStudio Server. I just had to restart R (Ctrl/Cmd + Shift + F10) to get rid of the error.
It was possibly caused by updating some packages.
if you download the xquart, I think you need to restart your Mac to activate its work.
I had the same problem just a few minutes ago and I restart my Mac, now it works.
I had the same error in RStudio. I'm not certain if this would be the same for using R via the Terminal, but for RStudio users who get this message using the jagsUI package masks the View function which triggers this warning. See this question
Easiest workaround seems to be to call detach("package:jagsUI", unload = TRUE) once you have finished using JAGS.
Check potential conflicts between functions by calling conflicts(detail=TRUE) (Taken from this answer for finding which functions are masked in R).

Resources