I am trying to use readxl::read_excel to import tabular data from an excel spreadsheet (.xlsx and .xls) into R. I have had success with this function when reading spreadsheets from Desktop, but I am having trouble reading from my Documents folder now.
I tried reading both .xlsx and .xls formats of the same file and got 2 different error messages.
# Check the working directory
[1] "\\\\isdowfsv04/D100Home/e525902"
# Read .xlsx
specs <- read_excel("Class Specs Sept 2016.xlsx")
Error: 'exdir' does not exist
# Read .xls
specs <- read_excel("Class Specs Sept 2016.xls")
Error: path[1]="\\isdowfsv04/D100Home/e525902/Class Specs Sept 2016.xls": Access is denied
The second message made me think I don't have permission to read/write data here, but I create folders/files here all the time. I have even used another R library, readr package, to import .csv files from this same directory.
# Using readr to import .csv version of the file works fine.
specs <- read_csv("\\\\isdowfsv04/D100Home/e525902/Class Specs Sept 2016.csv")
Based on other posts I've seen, it seems like a permission issue, but it's unclear to me how to fix it.
I tried using Sys.setenv() to change the temporary folders location to somewhere I know I have permission to write (i.e., Desktop) because I thought readxl used these paths when reading the data. But that didn't work because I got the exact same error messages.
Sys.getenv(c("TEMP", "TMP"))
TEMP TMP
"C:\\Users\\e525902\\AppData\\Local\\Temp" "C:\\Users\\e525902\\AppData\\Local\\Temp"
# Change temp file paths to the Desktop
Sys.setenv(TEMP = "C:\\Users\\e525902\\Desktop")
Sys.setenv(TMP = "C:\\Users\\e525902\\Desktop")
I am using a PC at work and the Documents folder is actually a network drive. I have used read_excel to import .xlsx files from other network folders, but this one is giving me trouble. Below is more information on the system I'm using.
Sys.info()
sysname release version
"Windows" "7 x64" "build 7601, Service Pack 1"
nodename machine login
"10004053WDT" "x86-64" "e525902"
user effective_user
"e525902" "e525902"
search()
[1] ".GlobalEnv" "package:readr" "package:readxl" "tools:rstudio" "package:stats"
[6] "package:graphics" "package:grDevices" "package:utils" "package:datasets" "package:methods"
[11] "Autoloads" "package:base"
# R version
version$version.string
version.string R version 3.2.5 (2016-04-14)
I ran into the same issue. It was occurring to me when I set my working directory to the base location of the excel files, and then in the read excel, using a relative path to get to the directory.
Once I rebooted R, and instead kept my working directory as the default, and then used the full file path in the read_excel function, my exdir error went away.
Let me know if this works for you. I was also on a network drive at work with a windows 7 64-bit system.
The current working directory is displayed by the RStudio IDE within the title region of the Console pane. You can also check your current working directory by running the command getwd() in the console.
Related
is there a way to read an open excel file into R?
When an excel file is open in Excel, Excel puts a lock on the file, such as the reading method in R cannot access the file.
Can you circumvent this lock?
Thanks
Edit: this occurs under windows with original excel.
I too do not have problem opening xlsx files that are already open in excel, but if you do i have a workaround that might work:
path_to_xlsx <- "C:/Some/Path/to/test.xlsx"
temp <- tempdir()
file.copy(path_to_xlsx, to = paste0(temp, "/test.xlsx"))
df <- openxlsx::read.xlsx(paste0(temp, "/test.xlsx"))
This copies the file (Which should not be blocked) to a temporary directory, and then loads the file from there. Again, i'm not sure if this is needed, as i do not have the problem you have.
You could try something like this using the ps package. I've used it on Windows and Mac to read from files that I had downloaded from some web resource and opened in Excel with openxlsx2, but it should work with other packages or programs too.
# get the path to the open file via the ps package
library(ps)
p <- ps()
# get the pid for the current program, in my case Excel on Mac
ppid <- p$pid[grepl("Excel", p$name)]
# get the list of open files for that program
pfiles <- ps_open_files(ps_handle(ppid))
pfile <- pfiles[grepl(".xlsx", pfiles$path),]
# return the path to the file
sel <- grepl("^(.|[^~].*)\\.xlsx", basename(pfile$path))
path <- pfile$path[sel]
What do you mean by "the reading method in R", and by "cannot access the file" (i.e. what code are you using and what error message do you get exactly)? I'm successfully importing Excel files that are currently open, with something like:
dat <- readxl::read_excel("PATH/TO/FILE.xlsx")
If the file is being edited in Excel, R imports the last saved version.
EDIT: I've now tried it on both Linux and Windows and it still works, at least with version 1.3.1 of 'readxl'.
I am trying to get an application running on shinyapps.io.
First step is to load a rds file for dropbox access.
The the app is supposed to dowload one folder of the dropbox on the server.
I want to create a directory in the "static" directory of the app.
I tried
dir.create (static/...)
dir.create (./static/...)
both works fine locally, but it seems dir.create () does not work on the shinyapps server.
https://colomb.shinyapps.io/resultsgallery_software/
https://github.com/smartfigures-dar/SmartFig_Rbased_prototype/tree/dropbox_integration
Note that it seems to work on the 2 first instance and then no more:
local:
for (j in folders$path_display){
print(j)
foldercreated <-dir.create(paste0("./static",j))
}
[1] "/SFB-resultgallery"
[1] "/SFB-resultgallery/Figures"
[1] "/sfb-resultgallery/Figures/atg6-9_KD_in_brain"
[1] "/sfb-resultgallery/Figures/keep_it_short,_no_more_than1"
[1] "/sfb-resultgallery/Figures/2e"
[1] "/sfb-resultgallery/Figures/keep_it_short,_no_more_than"e
shinyapps.io log message:
2019-07-05T09:42:45.679221+00:00 shinyapps[1012452]: [1] "/SFB-resultgallery"
2019-07-05T09:42:45.679636+00:00 shinyapps[1012452]: [1] "/SFB-resultgallery/Figures"
2019-07-05T09:42:45.679740+00:00 shinyapps[1012452]: [1] "/sfb-resultgallery/Figures/atg6-9_KD_in_brain"
2019-07-05T09:42:45.679936+00:00 shinyapps[1012452]: Warnung in dir.create(paste0("./static", j))
2019-07-05T09:42:45.679938+00:00 shinyapps[1012452]: kann Verzeichnis './static/sfb-resultgallery/Figures/atg6-9_KD_in_brain' nicht erzeugen. Grund 'No such file or directory'
Short answer: never capitalise folder names.
It took forever to get to the problem and solve it, although the problem is tiny: shinyapps.io and windows deal differently with capital letters: things were saved in "Figures/" and the app tries to access "figures/", no to mention that dropbox messes with it too (file in Figures/ are seen in figures/...)
I wonder if this is somewhere in the doc...
(I am still struggling with the problem hours after finding it ...)
I have developed a package that I want to share with my colleagues at work.
I have a network drive in which I created the local repository structure that looks like this:
MyRepo
\__bin
\__windows
\__contrib
\__src
\__contrib
All folders are empty.
So I built my package with RStudio on Windows using the "Build/More/Build source package" menu, which created a tar.gz file.
Then I tried:
drat::insertPackage("../myPkg_0.0.0.9000.tar.gz",
repodir = "file://networkdrive/path/to/MyRepo",
action = "prune")
But this gives me an error:
Error: Directory file://networkdrive/path/to/MyRepo not found
Which is strange because file.exists(//networkdrive/path/to/MyRepo) returns true.
OK, then I tried:
drat::insertPackage("../myPkg_0.0.0.9000.tar.gz",
repodir = "//networkdrive/path/to/MyRepo",
action = "prune")
Without the file: in the repository path and I get another error:
tar (child): "//networkdrive/path/to/MyRepo/src/contrib/myPkg_0.0.0.9000.tar.gz: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
/usr/bin/tar: Child returned status 2
/usr/bin/tar: myPkg/DESCRIPTION: Not found in archive
/usr/bin/tar: Exiting with failure status due to previous errors
reading DESCRIPTION for package ‘myPkg’ failed with message:
cannot open the connection
But when I go in the "//networkdrive/path/to/MyRepo/src/contrib" folder, I can definitely see the myPkg_0.0.0.9000.tar.gz file that has been copied despite the error message.
Can anyone help?
> sessionInfo()
R version 3.3.3 (2017-03-06)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1
locale:
[1] LC_COLLATE=French_France.1252 LC_CTYPE=French_France.1252 LC_MONETARY=French_France.1252 LC_NUMERIC=C LC_TIME=French_France.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] drat_0.1.2 tools_3.3.3 git2r_0.18.0
I know this is old, but my colleague just came across the same problem and found this post. I believe the issue may be the lack of a trailing slash in your directory name. I have been able to recreate the error with a mapped network drive. I can resolve the issue by using "H:/MyRepo/" instead of "H:/MyRepo".
I haven't tried it with the "file://" format, but I wanted to include my answer in case someone else comes across this question.
Ok, so after some research, here is my conclusions.
It cannot be done
It's not Drat's fault
The reason why it does not work is that the tools::write_PACKAGES function does not work on network drives. Period.
I manually copied my package on the network drive, then ran setwd() to its location and executed write_PACKAGES(".", type="source") and I got the same error.
So to make this work, I just left my package.tar.gz file on a local drive, ran the tools::write_PACKAGES command locally and then moved the files to the network drive.
Adding the network drive to my repository list using options(repos = c(MyRepo = "file://networkdrive/path/to/MyRepo/")) works: RStudio and available.packages find my package.
It's not completely satisfactory, but I think it's the only way today.
I was having this problem as well and finally got to the bottom of it today.
For me, the problem was not isolated to just network locations but also occurred on C: drive. The root cause was the version of tar.exe being used to unpack the existing packages in the package directory. Calls to utils::untar are made in the tools::write_PACKAGES function.
The documentation for utils::untar explains that on Windows, external tar.exe is tried first. Sure enough, I had a version installed with Git which when used with default arguments fails when a colon is in the file name. I was able to force utils::untar to use to use the RBuildTools version of tar.exe instead by setting the environment variable TAR to "internal".
drat::insertPackage now works.
I am trying to use readxl::read_excel to import tabular data from an excel spreadsheet (.xlsx and .xls) into R. I have had success with this function when reading spreadsheets from Desktop, but I am having trouble reading from my Documents folder now.
I tried reading both .xlsx and .xls formats of the same file and got 2 different error messages.
# Check the working directory
[1] "\\\\isdowfsv04/D100Home/e525902"
# Read .xlsx
specs <- read_excel("Class Specs Sept 2016.xlsx")
Error: 'exdir' does not exist
# Read .xls
specs <- read_excel("Class Specs Sept 2016.xls")
Error: path[1]="\\isdowfsv04/D100Home/e525902/Class Specs Sept 2016.xls": Access is denied
The second message made me think I don't have permission to read/write data here, but I create folders/files here all the time. I have even used another R library, readr package, to import .csv files from this same directory.
# Using readr to import .csv version of the file works fine.
specs <- read_csv("\\\\isdowfsv04/D100Home/e525902/Class Specs Sept 2016.csv")
Based on other posts I've seen, it seems like a permission issue, but it's unclear to me how to fix it.
I tried using Sys.setenv() to change the temporary folders location to somewhere I know I have permission to write (i.e., Desktop) because I thought readxl used these paths when reading the data. But that didn't work because I got the exact same error messages.
Sys.getenv(c("TEMP", "TMP"))
TEMP TMP
"C:\\Users\\e525902\\AppData\\Local\\Temp" "C:\\Users\\e525902\\AppData\\Local\\Temp"
# Change temp file paths to the Desktop
Sys.setenv(TEMP = "C:\\Users\\e525902\\Desktop")
Sys.setenv(TMP = "C:\\Users\\e525902\\Desktop")
I am using a PC at work and the Documents folder is actually a network drive. I have used read_excel to import .xlsx files from other network folders, but this one is giving me trouble. Below is more information on the system I'm using.
Sys.info()
sysname release version
"Windows" "7 x64" "build 7601, Service Pack 1"
nodename machine login
"10004053WDT" "x86-64" "e525902"
user effective_user
"e525902" "e525902"
search()
[1] ".GlobalEnv" "package:readr" "package:readxl" "tools:rstudio" "package:stats"
[6] "package:graphics" "package:grDevices" "package:utils" "package:datasets" "package:methods"
[11] "Autoloads" "package:base"
# R version
version$version.string
version.string R version 3.2.5 (2016-04-14)
I ran into the same issue. It was occurring to me when I set my working directory to the base location of the excel files, and then in the read excel, using a relative path to get to the directory.
Once I rebooted R, and instead kept my working directory as the default, and then used the full file path in the read_excel function, my exdir error went away.
Let me know if this works for you. I was also on a network drive at work with a windows 7 64-bit system.
The current working directory is displayed by the RStudio IDE within the title region of the Console pane. You can also check your current working directory by running the command getwd() in the console.
For some reason I am suddenly not able to install packages in R (I have subsequently updated to the latest version of R and am running Windows 7). For example, if I type:
install.packages('beeswarm')
Installing package into ‘D:/Rlibs’ (as ‘lib’ is unspecified)
--- Please select a CRAN mirror for use in this session --- trying URL 'http://www.stats.bris.ac.uk/R/bin/windows/contrib/3.0/beeswarm_0.1.5.zip'
Content type 'text/html' length unknown opened URL downloaded 1859
bytes
Error in read.dcf(file.path(pkgname, "DESCRIPTION"), c("Package",
"Type")) : cannot open the connection In addition: Warning
messages: 1: In unzip(zipname, exdir = dest) : error 1 in extracting
from zip file 2: In read.dcf(file.path(pkgname, "DESCRIPTION"),
c("Package", "Type")) : cannot open compressed file
'beeswarm/DESCRIPTION', probable reason 'No such file or directory'
I have read that in Windows 7 there can be important restrictions on rights to writing to certain folders etc. so I've gone to some lengths to install R and library folders in non-default areas of my computer, and to allow myself rights to certain folders, but to no avail. Possibly also of importance is when I type:
.libPaths()
# [1] "D:/Rlibs"
# [2] "C:/Users/L.Halsey/Documents/R/win-library/3.0"
# [3] "C:/Users/L.Halsey/Documents/Documents/R-3.0.1/library"
I have created several folders in an attempt to create one that I could successfully install libraries into and set them up to be recognised by R using 'environment variables' from the start button. I don't know how to delete any of them though - not sure if this is relevant to my overall problem of not now being able to install/update packages for some reason.
The error being reported is inability to open a connection. In Windows that is often a firewall problem and is in the Windows R FAQ. The usual first attempt should be to run internet2.dll. From a console session you can use:
setInternet2(TRUE)
(You are correct in thinking this is not due to your library setup. The error says nothing about permissions.) I don't think just typing .libPaths should return that character vector since on my machine I would need to type .libPaths() to see something like that. If you wanted to reduce the number of places for libraries you can use the .libPaths function for setting the values. This would pick the second and third of the existing paths
.libPaths( .libPaths()[2:3] )
The inner call retrieves the path vector and the outer call sets it to a reduced vector.
Running RStudio as administrator fixed it for me!
I will probably duplicate a lot of other answers on the stackoverflow, but I got exactly the same error as OP, namely:
Warning messages: 1: In unzip(zipname, exdir = dest) : error 1 in extracting from zip file 2: In read.dcf(file.path(pkgname, "DESCRIPTION"), c("Package", "Type")) : cannot open compressed file 'zoo/DESCRIPTION', probable reason 'No such file or directory'
Turned out, while I as a user had permissions to write in a certain directory, R did not. In order to be sure you don't have something similar, do following:
get a usb drive, let's name it E
download package source as a .zip file and store it onto usb-drive in some directory, let's name it E:/source
Create directory for libraries on the usb drive, let's name it E:/libs
Install packages calling R command install.package from the R console and setting all relevant directories to point to your usb drive:
(here I use package zoo as an example)
install.packages("E:/source/zoo_1.7-12.zip",
destdir = 'E:/source', # no "/" after the path
lib = 'E:/libs',
repos = NULL)
Load the package from the directory, where you installed it:
library('zoo', lib.loc = 'E:/libs')
After you are sure, it works this way on your usb drive, you can start resolving directories permissions, and try out by changing the paths in the code above.
update:
In some windows environments even your usb-stick might be protected from read-write by the R. Make sure you check the permissions using the machine you are working from.
The following worked for me (based on the answer above)
install.packages("clustvarsel", lib = "C:/Users/dnentchev/My Programs/R-3.2.2/library")
I had the same problem. I turned the windows firewall off, and Run RStudio as administrator. so, that error fixed.