I have trouble with creating zip file from R. The same code worked perfectly at work with R version 3.4.2, 32 bit computer.
Now I am trying to run the same thing on R version 3.5.1, 64 bit computer, and the zip() command does not seem to work. What is going on?
zip(zipfile = "test.zip",files=list.files(getwd()))
#create zip from whole directory, on 1st machine it works, now nothing happens
I checked the source code for zip() and when I debug it, I found out that system2 command does nothing.
zip <- function (zipfile, files, flags = "-r9X", extras = "", zip = Sys.getenv("R_ZIPCMD",
"zip"))
{
if (missing(flags) && (!is.character(files) || !length(files)))
stop("'files' must a character vector specifying one or more filepaths")
args <- c(flags, shQuote(path.expand(zipfile)), shQuote(files),
extras)
if (.Platform$OS.type == "windows")
invisible(system2(zip, args))
else invisible(system2(zip, args))
}
# I run this manually when trying to debug, nothing happens;
system2(zip, args) ## zip is a parameter here, not a function
####
Browse[2]> zip
[1] "zip"
Browse[2]> args
[1] "-r9X" "\"bla.zip\""
[3] "\"[Content_Types].xml\"" "\"_rels\""
[5] "\"docProps\"" "\"xl\""
[7] ""
For example absurd call does not give an error.
system2("blablađ",2) ## does nothing but no error or warning either
I am stuck trying to understand how does system2() function works and what do I need to change to create a compressed folder.
Thanks
EDIT: After taking the account the help from comment, I got following error:
Browse[2]> system2(zip, args,stderr = T)
Error in system2(zip, args, stderr = T) : '"zip"' not found
SOLVED: After installing Rtools for version 3.5 it worked.
From the zip help:
zip(zipfile, files, flags = "-r9X", extras = "",
zip = Sys.getenv("R_ZIPCMD", "zip"))
zip A character string specifying the external command to be used.
As you can see, the zip function has an argument zip to specify the external command to be used. On my machine it is:
λ where zip
C:\Oracle\Ora11\BIN\zip.exe
C:\Program Files\Rtools\bin\zip.exe
The zip program is available in Rtools, but it is also available on any (Windows?) machine, usually.
To check whether zip is found by R, type:
> Sys.which("zip")
zip
"C:\\Oracle\\Ora11\\bin\\zip.exe"
If you get "", that means zip is not in the path, and if it is neither in the environment variable R_ZIPCMD, you have to specify its path in the zip argument.
Related
I have long-read RNA sequencing data from PacBio IsoSeq and trying to use IsoPops software (https://kellycochran.github.io/IsoPops/site_files/walkthrough.html) to generate graphs to interpret the data. I am new to R-studio so have limited proficiency. There is a walkthrough for the program which seems quite intuitive. I have successsfully installed the package on R Studio, and have opened the library and assigned variables to the files I have (which include a fasta file, gff file and abundance files). Once I've successfully assigned variables to the files, I'm supposed to generate a rawdb using the function compile_raw_db. However, when I do this function I get the error:
Error in system(command = paste("head -n1 ", filename, step = ""), intern = T) :
'head' not found
I've used the head -n1 function in linux command line, and can see that there definitely is a header and contents in the file. The error occurs for any file that I give to R studio, so my reasoning is that the programme is not recognizing the files.
Thought the files might have been empty, opened on Ubuntu and head -n2 function worked fine for each file, so I don't think it's an issue with the files.
My data was originally stored on an online shared network drive, moved files to local computer to eliminate any possibility for path errors. Still have the error message.
Have tried the same function with a number of different files, and I always have the same error message. So I think the issue is with how R studio is reading the files, and not the files themselves.
Would really appreciate some support with this, at the end of my PhD trying to analyse the last piece of my data and really struggling with this. Thanks in advance for any feedback!
> library(IsoPops)
Welcome to IsoPops version 0.3.1.
> transcript_AD3 <- "C:/R_Package_for_IsoSeq/IsoPops/IsoPops_Data/AD3-hq_transcripts.fasta"
> abundance_AD3 <- "C:/R_Package_for_IsoSeq/IsoPops/IsoPops_Data/AD3_Collapsed_Filtered_Isoform_Counts.abundance.txt"
> GFF_AD3 <- "C:/R_Package_for_IsoSeq/IsoPops/IsoPops_Data/AD3-collapse_isoforms.gff"
> rawDB <- compile_raw_db(transcript_AD3, abundance_AD3, GFF_AD3)
[1] "Loading sequences..."
Error in system(command = paste("head -n1 ", filename, step = ""), intern = T) :
'head' not found
I have a batch file test.bat on my windows desktop with the following content:
cd /d W:\r\dev\
"C:\Program Files\R\R-3.5.0\bin\i386\Rscript.exe" scripts\some_function.R
When executed, the referenced R script some_function.R begins to run and outputs messages to the console to let me know where it is. However, it stops after reaching a certain point in the R code then starts over and continues to do so until the session auto-terminates.
I've inspected the R code for an indication of why the batch file would continue to return to the beginning and I've found none. The R file works fine when run directly (from within R studio, for example).
I'm well aware that naming the batch file with a command that appears in the file can be problematic, but that's not what is happening here. I've tried multiple names for the batch file but that doesn't seem to solve the problem.
Below is the beginning of the code in the referenced file some_function.R that keeps getting re-executed:
library(data.table)
dir <- gsub(x= getwd(), pattern = "(.:/r/)(.*)",replacement = "\\1")
envir <- gsub(x= getwd(), pattern = "(.:/r/)(\\w+)",replacement = "\\2")
cat(paste("directory = "),dir,"\n","environment = ",envir,"\n")
##############################################################
# Load System Parameters #
##############################################################
cat("Loading system parameters ...","\n")
sys_param_file_path <- paste0(dir,"shared/files/system/sys_param.csv")
sys_params <- data.table(read.csv(sys_param_file_path, stringsAsFactors = FALSE))
sys_params <- sys_params[param_envir == envir,]
for(i in 1:nrow(sys_params)){
if(sys_params[i,param_type] == "environment path"){
sys_params[i,param_val := paste(dir,envir,param_val,sep = "")]
}
if(sys_params[i,param_type] == "root path"){
sys_params[i,param_val := paste(substr(dir,1,nchar(dir)-1),param_val,sep = "")]
}
}
cat(paste("Loaded",nrow(sys_params),"parameters","\n"))
and the console output when the batch file is called:
C:\Users\me\Desktop> cd /d W:\r\dev\
W:\r\dev>"C:Program Files\R\R-3.5.0\bin\i386\Rscript.exe" scripts\some_function.R
Warning message:
package 'data.table' was built under R version 3.5.1
directory = W:/r/
environment = dev
Loading system parameters ...
Loaded 32 parameters
directory = W:/r/
environment = dev
Loading system parameters ...
Loaded 32 parameters
directory = W:/r/
environment = dev
Loading system parameters ...
Loaded 32 parameters
What am I missing?
In Revolution R Enterprise console,
devtools::check("C:/Users/User/Documents/Revolution/mypackage")
produced
checking sizes of PDF files under 'inst/doc' ... NOTE
Unable to find GhostScript executable to run checks on size reduction
with no any other warnings/errors/notes. So, (even though AFAIK this note is not that much important for eventual check), I wanted to get rid of this warning (since I wanna put .PDF files into mypackage\inst\doc folder produced outside of R).
I have Ghostscript installed in my notebook. I got helped via:
> help("R_GSCMD")
R_GSCMD: Optional. The path to Ghostscript, used by dev2bitmap, bitmap and embedFonts.
Consulted when those functions are invoked.
Since it will be treated as if passed to system, spaces and shell metacharacters should be escaped.
> Sys.getenv("R_GSCMD")
[1] ""
What I did (and took error again) is:
> Sys.setenv("R_GSCMD") <- "C:\\Program Files (x86)\\gs\\gs9.19\\bin\\gswin32c.exe"
Error in Sys.setenv("R_GSCMD") <- "C:\\Program Files (x86)\\gs\\gs9.19\\bin\\gswin32c.exe" :
target of assignment expands to non-language object
Upon deepening, I found: ["These errors occur when one tries to assign a value to a variable that doesn't exist, or that R can't treat as a name. (A name is a variable type that holds a variable name."]
What I am basically trying to do is to set my GS executable (C:\Program Files (x86)\gs\gs9.19\bin\gswin32c.exe) to "R_GSCMD".
Any help would be greatly appreciated.
On consulting ?Sys.setenv it confirms my expectation that the call should instead be:
Sys.setenv(R_GSCMD = "C:\\Program Files (x86)\\gs\\gs9.19\\bin\\gswin32c.exe")
Because the gs versions change all the time, you may like a little R script for it!
system.partition = 'c:'
dirs = c('Program Files', 'Program Files (x86)')
for (dir in dirs) {
dir.list = list.dirs(file.path(system.partition, dir), recursive = FALSE)
GsinList = grepl(pattern = 'gs', x = dir.list)
if (sum(GsinList) > 0) {
gsDirectory = which(GsinList == TRUE)
GsExeFiles = list.files(
dir.list[gsDirectory],
recursive = TRUE,
pattern = 'gswin',
include.dirs = TRUE,
full.names = TRUE
)[1]
message('Gs found! ~> ',GsExeFiles)
Sys.setenv(R_GSCMD = GsExeFiles)
break
}
}
Gs found! ~> c:/Program Files/gs/gs9.21/bin/gswin64.exe
What are the workarounds on Windows to make it so Sys.which finds the proper executables? Two cases that are reoccuring problems:
convert.exe which is both a windows program and the ImageMagik program, but Sys.which only finds the windows one which is never wanted from R no matter how I seem to arrange things on my PATH.
tar.exe is packaged along with various things like git or mingw or whatever, and even when I have Rtools and Rbuildtools first in my path, the tar program from Rtools is never found, for example when installing a package from source.
So, I have resorted to writing a wrapper that calls 7-zip instead whenever I am on windows. This can't be the thing to do can it?
Edit
Actually just adding an environment variable to .Renviron: TAR=path/to/tar.exe works fine for the install.packages example, and I am having trouble remembering where else the tar.exe was biting me, but Josh answered the main one, convert.exe.
I asked a +/- identical question earlier this year over on R-devel. Among the replies was this one, by Henrik Bengtsson, who kindly provided the following useful function:
Sys.which2 <- function(cmd) {
stopifnot(length(cmd) == 1)
if (.Platform$OS.type == "windows") {
suppressWarnings({
pathname <- shell(sprintf("where %s 2> NUL", cmd), intern=TRUE)[1]
})
if (!is.na(pathname)) return(setNames(pathname, cmd))
}
Sys.which(cmd)
}
## Trying out Sys.which & Sys.which2 on my Windows box gives the following:
Sys.which("convert")
# convert
# "C:\\Windows\\system32\\convert.exe"
Sys.which2("convert")
# convert
# "C:\\Program Files\\ImageMagick-6.8.8-Q16\\convert.exe"
I'm really not sure why R-core don't just fix Sys.which() to make it actually portable, but they at least do document root cause of this behavior in ?system (whose functionality is afflicted by the same problem):
The search path for 'command' may be system-dependent: it will
include the R 'bin' directory, the working directory and the
Windows system directories before 'PATH'.
Because Sys.which() is vectorized–and since I think that is useful–I've modified the Henrik Bengtsson code in the following function sys_which(), which should be a more robust and more similar version of Sys.which():
## check if unix
is_unix <- function() grepl("unix", .Platform$OS.type, ignore.case = TRUE)
## robust version of Sys.which
sys_which <- function(x) {
if (is_unix()) {
return(Sys.which(x))
}
sys_win <- function(x) {
if (grepl("\\S", path <- Sys.which(x))) {
return(path)
}
path <- tryCatch(
suppressWarnings(system(sprintf("where %s", x), intern = TRUE)[1]),
warning = function(w) "",
error = function(e) "")
if (!grepl("\\S", path)) {
return(`names<-`("", x))
}
`names<-`(path, x)
}
vapply(x, sys_win, character(1))
}
This has the following advantages:
It's vectorized–sys_which() can handle a vector (one or more input values) e.g., sys_which(c("python", "python3"))
It's more error resistant–use of tryCatch(...) ensures any inputs that result in an error from the system call get passed on to the normal Sys.which() function.
Example:
> sys_which(c("python", "python2.7", "python3", "asdf"))
#> python python2.7 python3 asdf
#> "/usr/bin/python" "/usr/bin/python2.7" "/usr/local/bin/python3" ""
I'm trying to reproduce the example for tex2docx function in reports R package and getting the following error.
DOC <- system.file("extdata/doc_library/apa6.qual_tex/doc.tex",
package = "reports")
BIB <- system.file("extdata/docs/example.bib", package = "reports")
tex2docx(DOC, file.path(getwd(), "test.docx"), path = NULL, bib.loc = BIB)
Error Message
pandoc.exe: Error reading bibliography `C:/Users/Muhammad'
citeproc: the format of the bibliographic database could not be recognized
using the file extension.
docx file generated!
Warning message:
running command 'C:\Users\MUHAMM~1\AppData\Local\Pandoc\pandoc.exe -s C:/Users/Muhammad Yaseen/R/win-library/3.0/reports/extdata/doc_library/apa6.qual_tex/doc.tex -o C:/Users/Muhammad Yaseen/Documents/test.docx --bibliography=C:/Users/Muhammad Yaseen/R/win-library/3.0/reports/extdata/docs/example.bib' had status 23
I wonder how to get tex2docx function in reports R package working properly.
As described in the above comments, the error is caused by passing a filename/path including some spaces that are nor escaped, nor quoted. A workaround could be wrapping all file paths and names inside of shQuote before passing to the command line with system.
Code: https://github.com/trinker/reports/pull/31
Demo:
Loading package
library(reports)
Creating a dummy dir with a space in the name that would hold the bib file
dir.create('foo bar')
file.copy(system.file("extdata/docs/example.bib", package = "reports"), 'foo bar/example.bib')
Specifying the source and the copied bib file:
DOC <- system.file("extdata/doc_library/apa6.qual_tex/doc.tex", package = "reports")
BIB <- 'foo bar/example.bib'
Running the test:
tex2docx(DOC, file.path(getwd(), "test2.docx"), path = NULL, bib.loc = BIB)
Disclaimer: I tried to test this pull request, but I could not setup an environment with all the needed tools to run R CMD check with vignettes and everything else after all in 5 mins (sorry but being on vacation right now and just enjoying the siesta after lunch), so please consider this pull request as "untested" -- although it should work.