I have this windows-batchfile which I'm calling from R using the shell() command. This batchfile does some calculations and writes them on the disk but also on the screen. I'm interested in the disk-output, only. I cannot change the batchfile.
The batchfile might be something silly like:
#echo off
echo 1 + 2
#echo 1 + 2 > C:\TEMP\batchoutput.txt
exit
I tried
shell("batchfile.bat", invisible = TRUE)
1 + 2
shell("batchfile.bat", show.output.on.console = FALSE)
Error in system(cmd, intern = intern, wait = wait | intern, show.output.on.console = wait, :
formal argument "show.output.on.console" matched by multiple actual arguments
system("batchfile.bat", invisible = T)
1 + 2
system("batchfile.bat", show.output.on.console = F)
Warning message:
running command 'C:\TEMP\batchfile.bat' had status 1
Is there a way of supressing the console-output on R?
options(warn = -1)
shell("You command")
options(warn = 0)
Related
cl <- parallelly::makeClusterPSOCK(2, autoStop = TRUE)
Error:
Error in system(test_cmd, intern = TRUE, input = input) :
'CreateProcess' failed to run 'C:\Users\xxx~1\ONEDRI~1\DOCUME~1\R\R-40~1.3\bin\x64\Rscript.exe -e "try(suppressWarnings(cat(Sys.getpid(),file=\"C:/Users/LOCAL_~1/Temp/RtmpIP7vSI/worker.rank=1.parallelly.parent=19988.4e147c0a5082.pid\")), silent = TRUE)" -e "file.exists(\"C:/Users/LOCAL_~1/Temp/RtmpIP7vSI/worker.rank=1.parallelly.parent=19988.4e147c0a5082.pid\")"'
I was trying to create the clusters for parallel execution. But does not work and throws the above error.
I have been working on this for a while and I am still stuck.
I would like to call an Rscript using multiple arguments from what is essentially a command line (Snakemake file). The main difference between what I am asking and what I see on SO (How to pass list of arguments to method in R?, How can I pass an array as argument to an R script command line run?, Is it possible to pass an entire list as a command line argument in R) is that my arguments are a combination of strings, numbers, and a list.
Here is the set up in my rules (Snakemake file):
rule cluster_plots_DGE:
input:
script = 'src/scripts/create_images_DGE.R',
analyze_sc_object_output = sc_objects
params:
project = PROJECT,
method = METHOD,
rpath = RPATH,
storage=STORAGE,
components = COMPONENTS,
reso_file = resolution_file,
sample_files = integrated_seurat_objects
output: dge_files
log:
log_output = log_directory + PROJECT.lower() + '_DGE.log'
shell:
"Rscript {input.script} {params.project} {params.method} {params.rpath} {params.storage} {params.components} {params.reso_file} {params.sample_files} 2> {log.log_output}"
Here is what the call translates to:
Rscript src/scripts/create_images_DGE.R project_name ALL path_to_R_installed_libraries rds 50 data/endpoints/project_name/analysis/PCA_14/tables/project_nameR_resolution_list.txt data/endpoints/project_name/analysis/PCA_14/RDS/project_name_Standard_0.5.RDS data/endpoints/project_name/analysis/PCA_14/RDS/project_name_RPCA_0.5.RDS data/endpoints/project_name/analysis/PCA_14/RDS/project_name_SCT_0.5.RDS 2> logs/DGE_Markers/project_name_DGE.log
Where sample_files = integrated_seurat_objects is a list containing:
data/endpoints/project_name/analysis/PCA_14/RDS/project_name_Standard_0.5.RDS,
data/endpoints/project_name/analysis/PCA_14/RDS/project_name_RPCA_0.5.RDS,
data/endpoints/project_name/analysis/PCA_14/RDS/project_name_SCT_0.5.RDS
And here is the beginning of my R script:
args = commandArgs(trailingOnly=TRUE)
compo <- ''
project <- ''
method <- ''
lib_path <- ''
storage <- ''
res_file <- ''
integrated_object <- '' #list of objects
# test if there is at least 7 arguments: if not, return an error
if (length(args) < 7) {
stop('At least seven arguments must be supplied.', call.=FALSE)
} else if (length(args)==7) {
project = args[1]
method = args[2]
lib_path = args[3]
storage = args[4]
compo = args[5]
res_file = args[6]
integrated_object = args[7]
#integrated_object = eval(parse(text=args[7]))
}
print(compo)
print(project)
print(method)
print(lib_path)
print(storage)
print(res_file)
print(integrated_object)
If I use the entire integrated_seurat_objects list, this is what gets returned:
[1] ""
[1] ""
[1] ""
[1] ""
[1] ""
[1] ""
[1] ""
If I take the first entry from integrated_seurat_objects and pass that as an argument, I get (I have replaced the actual project name and paths is this post):
[1] "50"
[1] project_name
[1] "ALL"
[1] library_path_to_R_libraries
[1] "rds"
[1] "data/endpoints/project_name/analysis/PCA_14/tables/project_name_resolution_list.txt"
[1] "data/endpoints/project_name/analysis/PCA_14/RDS/project_name_Standard_0.5.RDS"
It seems do-able but I have not cracked it yet. How can I pass multiple arguments that include a list to an R script form the command line? Any assistance is always appreciated.
#MrFlick deserves credit for this answer. The issue was I was not accounting for a situation where the number of arguments would be greater than 7 (duh).
A quick very fix:
if (length(args) < 7)
{
stop('At least seven arguments must be supplied.', call.=FALSE)
}
if (length(args)==7)
{
project = args[1]
method = args[2]
lib_path = args[3]
storage = args[4]
compo = args[5]
res_file = args[6]
integrated_object = args[7]
}
if (length(args)>7)
{
project = args[1]
method = args[2]
lib_path = args[3]
storage = args[4]
compo = args[5]
res_file = args[6]
integrated_object = args[7:length(args)]
}
Thank you for your eyes #MrFlick
I want to use tryCatch() to check if packages are installed from within a loop, and then return next to break out and skip to the next iteration of the loop if the package failed to load or install. At the same time, I want to return a message to the console reporting this. I can do one, or the other, but I am having trouble figuring out how to do both at the same time. For example, this work:
package_list<-c("ggplot2", "grid", "plyr")
for(p in package_list){
# check if package can't be loaded
if(!require(p,character.only=TRUE,quietly=TRUE,warn.conflicts=FALSE)){
write(paste0("Attempting to install package: ",p), stderr())
# try to install & load the packages, give a message upon failure
tryCatch(install.packages(p,repos="http://cran.rstudio.com/"),
warning = function(e){write(paste0("Failed to install pacakge: ", p), stderr())},
error = function(e){write(paste0("Failed to install pacakge: ", p), stderr())})
tryCatch(library(p,character.only=TRUE,verbose=FALSE),
warning = function(e){write(paste0("Failed to install pacakge: ", p), stderr())},
error = function(e){write(paste0("Failed to install pacakge: ", p), stderr())})
# try to install & load the packages, skip to next loop iteration upon failure
tryCatch(install.packages(p,repos="http://cran.rstudio.com/"),warning = next)
tryCatch(library(p,character.only=TRUE,verbose=FALSE),warning = next)
}
}
But this requires running each command twice; once to fail and return a message about the failure, and then again to fail and skip to the next item in the loop.
Instead, I would much rather perform both actions with a single function, like this:
for(p in package_list){
if(!require(p,character.only=TRUE,quietly=TRUE,warn.conflicts=FALSE)){
tryCatch(install.packages(p,repos="http://cran.rstudio.com/"),
warning = function(e){print(paste("Install failed for package: ", p)); return(next)})
# ...
}
}
However, this fails because you cannot use next from within a function:
Error in value[[3L]](cond) : no loop for break/next, jumping to top level
Is there a way to both return the desired message, and issue the next command from within tryCatch() in order to perform this function?
Use message() rather than write(..., stderr()); it takes several arguments that do not have to be paste()ed together.
Use tryCatch() to return a status code, and act on the status code; the following
for (i in 1:10) {
status <- tryCatch({
if (i < 5) warning("i < 5")
if (i > 8) stop("i > 8")
0L
}, error=function(e) {
message(i, ": ", conditionMessage(e))
1L
}, warning=function(w) {
message(i, ": ", conditionMessage(w))
2L
})
if (status != 0L)
next
message("success")
}
prints
1: i < 5
2: i < 5
3: i < 5
4: i < 5
success
success
success
success
9: i > 8
10: i > 8
In man(1) env it say:
env [OPTION]... [-] [NAME=VALUE]... [COMMAND [ARG]...]
So consider print_A.sh:
#!/usr/bin/env A=b bash
echo A is $A
When I run it with ./print_A.sh it hangs.
Running it with strace ./print_A.sh I get the following log, repeating:
execve("/path/to/print_A.sh", ["/path/to/print_A.sh"...], [/* 114 vars */]) = 0
uname({sys="Linux", node="my-host", ...}) = 0
brk(0) = 0x504000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2a95556000
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=171528, ...}) = 0
mmap(NULL, 171528, PROT_READ, MAP_PRIVATE, 3, 0) = 0x2a95557000
close(3) = 0
open("/lib64/tls/libc.so.6", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\240\305\30100\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1641152, ...}) = 0
mmap(0x3030c00000, 2330696, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3030c00000
mprotect(0x3030d30000, 1085512, PROT_NONE) = 0
mmap(0x3030e2f000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x12f000) = 0x3030e2f000
mmap(0x3030e35000, 16456, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x3030e35000
close(3) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2a95581000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2a95582000
mprotect(0x3030e2f000, 16384, PROT_READ) = 0
mprotect(0x3030b14000, 4096, PROT_READ) = 0
arch_prctl(ARCH_SET_FS, 0x2a95581b00) = 0
munmap(0x2a95557000, 171528) = 0
brk(0) = 0x504000
brk(0x525000) = 0x525000
open("/usr/lib/locale/locale-archive", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=48529088, ...}) = 0
mmap(NULL, 48529088, PROT_READ, MAP_PRIVATE, 3, 0) = 0x2a95583000
close(3) = 0
As commented below, running a command in a hash-bang is not equivalent to running it on the command line, but still, why does it go into an infinite loop?
There two parts to this answer. One has already been given in a duplicate question. The answers there, however, explain the root cause for the problem, not what is actually going on.
Part 1 - What's causing this?
Hashbang parsing has never been really standardized. Here is a very good writeup by Sven Mascheck, which also includes a table with the behavior for different operating systems.
The table shows that e.g. Linux does all args in one, meaning that #!/usr/bin/env A=b bash executes env with 'A=b bash' as first argument.
Part 2 -- Why the endless loop?
What happens is that, env is executed, it sets the environment variable A='b bash' and then re-executes the original script. This results in the kernel re-interpreting the hashbang again and we get an endless env-exec loop.
After a little thinking, the problem becomes quite obvious:
A file test.sh with first line #!/bin/sh param executes /bin/sh as '/bin/sh' 'param' 'test.sh'. The script name is appended as a new command line parameter (i.e. to argv).
Thus in the example, env is actually executed as /usr/bin/env 'A=b bash'script_name.
env thus does what it's told, sets the variable, and executes script_name. This again starts hashbang interpretation and we got our loop.
Adding a demonstration of what is going on, to supplement the accepted answer:
We can make a little executable which just shows the arguments with which it was called, and put this in the shebang line:
$ cat showargs.c
#include <stdio.h>
int main(int argc, char *argv[]) {
int i;
for (i = 0; i < argc; i++)
printf("arg %d = '%s'\n", i, argv[i]);
return 0;
}
$ gcc -Wall -o /tmp/showargs showargs.c
$ cat testscript
#!/tmp/showargs A=b bash
(rest of script itself ignored here)
$ chmod +x testscript
$ ./testscript
arg 0 = '/tmp/showargs'
arg 1 = 'A=b bash'
arg 2 = './testscript'
Note that the A=b bash here is a single command-line argument.
This contrasts with:
$ /tmp/showargs A=b bash ./testscript
arg 0 = '/tmp/showargs'
arg 1 = 'A=b'
arg 2 = 'bash'
arg 3 = './testscript'
where the command line gets tokenised by the invoking shell.
Hence with env in place of showargs, we get the infinite loop as described in the accepted answer (sets A to b bash and re-invokes the same script again).
The scheme I am trying to evaluate is:
weka.classifiers.meta.AttributeSelectedClassifier -E "weka.attributeSelection.CfsSubsetEval " -S "weka.attributeSelection.BestFirst -D 1 -N 5" -W weka.classifiers.functions.SMOreg -- -C 1.0 -N 0 -I "weka.classifiers.functions.supportVector.RegSMOImproved -L 0.0010 -W 1 -P 1.0E-12 -T 0.0010 -V" -K "weka.classifiers.functions.supportVector.PolyKernel -C 250007 -E 1.0"
i.e. I am trying to run an AttributeSelectedClassifier with an SMOreg classifier inside. Every other parameter is the default value of the respective classifier.
So the R code is:
optns <- Weka_control(W = "weka.classifiers.functions.SMOreg")
ASC <- make_Weka_classifier("weka/classifiers/meta/AttributeSelectedClassifier")
model <- ASC(class ~ ., data = as.data.frame(dat), control = optns)
evaluation <- evaluate_Weka_classifier(model, numFolds = 10)
evaluation
When I run the above R code I get this error:
Error in .jcall(evaluation, "D", x, ...) : java.lang.NullPointerException
The above error happens in RWeka's evaluate.R where it tries to call the WEKA methods: "pctCorrect", "pctIncorrect", "pctUnclassified", "kappa", "meanAbsoluteError","rootMeanSquaredError","relativeAbsoluteError","rootRelativeSquaredError"
I also tried manually specifying the default values using the Weka_control object like so:
optns <- Weka_control(E = "weka.attributeSelection.CfsSubsetEval ",
S = list("weka.attributeSelection.BestFirst", D = 1,N = 5),
W = list("weka.classifiers.functions.SMOreg", "--",
C=1.0, N=0,
I = list("weka.classifiers.functions.supportVector.RegSMOImproved",
L = 0.0010, W=1,P=1.0E-12,T=0.0010,V=TRUE),
K = list("weka.classifiers.functions.supportVector.PolyKernel",
C=250007, E=1.0)))
ASC <- make_Weka_classifier("weka/classifiers/meta/AttributeSelectedClassifier")
model <- ASC(class ~ ., data = as.data.frame(dat), control = optns)
evaluation <- evaluate_Weka_classifier(model, numFolds = 10)
evaluation
and I get this error:
Error in .jcall(classifier, "V", "buildClassifier", instances) :
java.lang.Exception: Can't find class called: weka.classifiers.functions.SMOreg -- -C 1 -N 0 -I weka.classifiers.functions.supportVector.RegSMOImproved -L 0.001 -W 1 -P 1e-12 -T 0.001 -V -K weka.classifiers.functions.supportVector.PolyKernel -C 250007 -E 1
I tried your example but got a different error (where dat is my own data frame)
Error in model.frame.default(formula = class ~ ., data = dat) :
object is not a matrix
Your error may be not directly related to syntax of calling this Weka function but some issues with path setup.