R Script .bat file - Adding Source before executing - r

I have an 'R' file which inserts reads data from a database, does some calculations, then re-inserts data back into a table.
Before i execute the script, I run 'Source' as below..
I want to use Windows Task Scheduler to auto schedule this script to run. I am following the guide https://trinkerrstuff.wordpress.com/2015/02/11/scheduling-r-tasks-via-windows-task-scheduler/ - When creating the .BAT file it should look something like:
echo off
CMD BATCH C:\PATHNAME\RSCRIPT.R
What should i insert here to make sure it runs the 'Source' first?
In the R code i have
In the code i have:
#use a relative path to locate our common utilities file and source it
source("..//R-Utilities//Utilities.R")
# use check_install_package function from Utilities.R to install and load
packages
check_install_package("lubridate")
check_install_package("plyr")
check_install_package("dplyr")
check_install_package("dtplyr")
check_install_package("ISOweek")
check_install_package("stringi")
check_install_package("RODBC")
#give us access to the library of functions this script uses
source("CTB_functions.R")
But I need to click the source button before running my whole code, or i get an error (As below).
> #this automatically sets the working directory to be where this file is
> setwd(getSrcDirectory(function(x) {x}))
Error in setwd(getSrcDirectory(function(x) { :
cannot change working directory
>
> #use a relative path to locate our common utilities file and source it
> source("../R-Utilities/Utilities.R")
Error in file(filename, "r", encoding = encoding) :
cannot open the connection
In addition: Warning message:
In file(filename, "r", encoding = encoding) :
cannot open file '../R-Utilities/Utilities.R': No such file or directory
>
> # use check_install_package function from Utilities.R to install and load
packages
> check_install_package("lubridate")
Error: could not find function "check_install_package"
> check_install_package("plyr")
Error: could not find function "check_install_package"
> check_install_package("dplyr")
Error: could not find function "check_install_package"
> check_install_package("dtplyr")
Error: could not find function "check_install_package"
> check_install_package("ISOweek")
Error: could not find function "check_install_package"
> check_install_package("stringi")
Error: could not find function "check_install_package"
> check_install_package("RODBC")
Error: could not find function "check_install_package"
>
> #give us access to the library of functions this script uses
> source("CTB_functions.R")
Error in file(filename, "r", encoding = encoding) :
cannot open the connection
In addition: Warning message:
In file(filename, "r", encoding = encoding) :
cannot open file 'CTB_functions.R': No such file or directory

Given your script, there should be no need to click the “Source” button first: in fact, it would execute your script twice.
A few things about your scripts:
CMD BATCH C:\PATHNAME\RSCRIPT.R
This is missing R in front of CMD BATCH. However, You should probably use Rscript.exe instead of R CMD BATCH. It’s a modern replacement.
source("..//R-Utilities//Utilities.R")
There’s no need for double slashes: single slashes work.
source("../R-Utilities/Utilities.R")
More fundamentally, using source in this manner can quickly become complex and error-prone due to several shortcomings (e.g circular inclusion, relative paths etc). A better way of achieving this is via the ‘box’ package, which provides a vastly improved replacement for the source function, which seems to fit your use-case very closely.
In particular, your script will probably not work: source won’t find the files R-Utilities/Utilities.R and CTB_functions.R because it searches these relative to the current working directory, not relative to the script directory. Using ‘box’ fixes this.

Related

Finding and setting the working directory within a super computer

Obviously one can find and set working directories with getwd() and setwd(). This question is a bit more complicated. I'm running two files on a super computer (one is a regular R file which calls the other file (a .stan file). The way I submit work to the super computer is that I have to zip a folder containing the data, the .R file, and the .stan file. I upload this folder, and I pull this folder by setting it as one of the parameters in the super computer. I call the data using the standard read.csv() command and everything is hunky dory.
However, when I call the .stan file from the .R file, it can't access it because it needs to know the working directory.
This is the error I get:
> fit <- stan(file="ace_thresholds.stan", data=stanData, cores = 4)
Error in file(fname, "rt") : cannot open the connection
In addition: Warning messages:
1: In normalizePath(file) :
path[1]="ace_thresholds.stan": No such file or directory
2: In file(fname, "rt") :
cannot open file 'ace_thresholds.stan': No such file or directory
Error in get_model_strcode(file, model_code) :
cannot open model file "ace_thresholds.stan"
Calls: stan -> stan_model -> stanc -> get_model_strcode
Execution halted
When I tried setting the working directory to the unzipped NSG_stan folder (which is what I assumed the wd to be, I received this error:
fit <- stan(file="NSG_stan/ace_thresholds.stan", data=stanData, cores = 4)
Error in file(fname, "rt") : cannot open the connection
In addition: Warning messages:
1: In normalizePath(file) :
path[1]="NSG_stan/ace_thresholds.stan": No such file or directory
2: In file(fname, "rt") :
cannot open file 'NSG_stan/ace_thresholds.stan': No such file or directory
Error in get_model_strcode(file, model_code) :
cannot open model file "NSG_stan/ace_thresholds.stan"
Calls: stan -> stan_model -> stanc -> get_model_strcode
Execution halted
So I tried running print(getwd()) within the script and in the printout I see that the wd is
"/projects/ps-nsg/home/nsguser/ngbw/workspace/NGBW-JOB-RTOOL_TG-EBE9CDBF28BF42AF8CB6EC9355006B3E/NSG_stan"
which means that the working directory will shift with every job. So to accurately set the working directory, I'd need to set it to the current folder within the script. I looked for various posts on how to do this like the following
# install.packages("rstudioapi") # run this if it's your first time using it to install
library(rstudioapi) # load it
# the following line is for getting the path of your current open file
current_path <- getActiveDocumentContext()$path
# The next line set the working directory to the relevant one:
setwd(dirname(current_path ))
# you can make sure you are in the right directory
print( getwd() )
The issue with this is that it's a super computer, so it's difficult to install packages, because every time I want to install something, I have to email the folks associated with the super comp and that all takes time.
I've looked over this thread as well: R command for setting working directory to source file location in Rstudio. Appears to be a lot of dissent over what works. I tried a couple of them, and they didn't work.
setwd(getSrcDirectory()[1])
this.dir <- dirname(parent.frame(2)$ofile)
setwd(this.dir)
I've included the .R file below, in the event that it helps, but I think this is probably a pretty easy answer for someone with a decent amount of coding experience.
.R file (so it's the "ace_thresholds.stan" file that I either need to link to the current wd or to include the code that would set the wd, such that this "ace_thresholds.stan" call would work. Does that make sense?
Thanks much!
dat <- ace.threshold.t2.samp
dat <- subset(dat, !is.na(rw))
dat$condition <- factor(dat$condition)
dat$pid <- factor(dat$pid)
nTotal <- dim(dat)[1]
nCond <- length(unique(dat$condition))
nSubj <- length(unique(dat$pid))
intensity <- dat$rw
condition <- as.numeric(dat$condition)
pid <- as.numeric(dat$pid)
correct <- dat$correct_button == "correct"
chancePerformance <- 1/2
stanData <- list(nTotal=nTotal, nLevels=nCond, nSubj = nSubj, subject = pid, intensity=intensity, level=condition, correct=correct, chancePerformance=chancePerformance)
fit.rw <- stan(file="ace_thresholds.stan", data=stanData, cores = 4, control=list(max_treedepth=15, adapt_delta=0.90))

How to eliminate security concern while accessing to the file through program in R in Windows?

While accessing to CSV file from disk with the help of the R program, where a path to the CSV file is provided in the configuration file ( A path is like "testData/Amazon S3/Inventory/Accounts.csv" which is provided in the Configuration file and cfig[2]$save.location is variable who is having value of this path accessed from Configuration file). Few lines of code are below
path <- cfig[2]$save.location
test_data <- fread(path,stringsAsFactors = FALSE,drop=col_ignor,blank.lines.skip = TRUE)
but it gave the message below:
Taking input= as a system command ('testData/Amazon
S3/Inventory/Accounts.csv') and a variable has been used in the
expression passed to input=. Please use fread(cmd=...). There is a
security concern if you are creating an app and the app could have a
malicious user and the app is not running in a secure environment;
e.g. the app is running as root. Please read item 5 in the NEWS file
for v1.11.6 for more information and for the option to suppress this
message.
'testData' is not recognized as an internal or external
command, operable program or batch file. Warning messages:
1: In (if
(.Platform$OS.type == "unix") system else shell)(paste0("(", :
'(testData/Amazon S3/Inventory/Accounts.csv) >
C:\Users\sharmb5\AppData\Local\Temp\RtmpOa25kH\filea78b5351f1'
execution failed with error code 1.
2: In fread(cfig[2]$save.location,
stringsAsFactors = FALSE, drop = col_ignor, : File
'C:\Users\sharmb5\AppData\Local\Temp\RtmpOa25kH\filea78b5351f1' has
size 0. Returning a NULL data.table.
when the following line of code executes,
config[4]$save.location <- stri_replace_all(config[4]$save.location, cp_val, fixed = cp_key)
It gives an error like as, Error in [<-.data.table(*tmp*, j, value = list(TestCaseID = "C419760", : Supplied 14 columns to be assigned 15 items. Please see NEWS for v1.12.2.
The above error was a warning but after manually updating of packages. This warning turns into an error. What will be the reason behind this issue and how to solve it? Thanks for Advance!!!

Error with tex2docx function from reports R package

I'm trying to reproduce the example for tex2docx function in reports R package and getting the following error.
DOC <- system.file("extdata/doc_library/apa6.qual_tex/doc.tex",
package = "reports")
BIB <- system.file("extdata/docs/example.bib", package = "reports")
tex2docx(DOC, file.path(getwd(), "test.docx"), path = NULL, bib.loc = BIB)
Error Message
pandoc.exe: Error reading bibliography `C:/Users/Muhammad'
citeproc: the format of the bibliographic database could not be recognized
using the file extension.
docx file generated!
Warning message:
running command 'C:\Users\MUHAMM~1\AppData\Local\Pandoc\pandoc.exe -s C:/Users/Muhammad Yaseen/R/win-library/3.0/reports/extdata/doc_library/apa6.qual_tex/doc.tex -o C:/Users/Muhammad Yaseen/Documents/test.docx --bibliography=C:/Users/Muhammad Yaseen/R/win-library/3.0/reports/extdata/docs/example.bib' had status 23
I wonder how to get tex2docx function in reports R package working properly.
As described in the above comments, the error is caused by passing a filename/path including some spaces that are nor escaped, nor quoted. A workaround could be wrapping all file paths and names inside of shQuote before passing to the command line with system.
Code: https://github.com/trinker/reports/pull/31
Demo:
Loading package
library(reports)
Creating a dummy dir with a space in the name that would hold the bib file
dir.create('foo bar')
file.copy(system.file("extdata/docs/example.bib", package = "reports"), 'foo bar/example.bib')
Specifying the source and the copied bib file:
DOC <- system.file("extdata/doc_library/apa6.qual_tex/doc.tex", package = "reports")
BIB <- 'foo bar/example.bib'
Running the test:
tex2docx(DOC, file.path(getwd(), "test2.docx"), path = NULL, bib.loc = BIB)
Disclaimer: I tried to test this pull request, but I could not setup an environment with all the needed tools to run R CMD check with vignettes and everything else after all in 5 mins (sorry but being on vacation right now and just enjoying the siesta after lunch), so please consider this pull request as "untested" -- although it should work.

Using inst/extdata with vignette during package checking R 2.14.0

I have a package which contains a csv file which I put in inst/extdata per R-exts. This file is needed for the vignette. If I Sweave the vignette directly, all works well. When I run R --vanilla CMD check however, the check process can't find the file. I know it has been moved into an .Rcheck directory during checking and this is probably part of the problem. But I don't know how to set it up so both direct Sweave and vignette building/checking works.
The vignette contains a line like this:
EC1 <- dot2HPD(file = "../inst/extdata/E_coli/ecoli.dot",
node.inst = "../inst/extdata/E_coli/NodeInst.csv",
and the function dot2HPD accesses the file via:
ni <- read.csv(node.inst)
Here's the error message:
> tab <- read.csv("../inst/extdata/E_coli/NodeInst.csv")
Warning in file(file, "rt") :
cannot open file '../inst/extdata/E_coli/NodeInst.csv': No such file or directory
When sourcing ‘HiveR.R’:
Error: cannot open the connection
Execution halted
By the way, this is related to this question but that info seems outdated and doesn't quite cover this territory.
I'm on a Mac.
Have you tried using system.file instead of hardcoded relative paths?
EC1 <- dot2HPD(file = system.file("inst", "extdata", "E_coli", "ecoli.dot", package = "your_package+name"))
node.inst <- system.file("inst", "extdata", "E_coli", "NodeInst.csv", package = "your_package_name")

error in reading R file while submitting a r job to condor

I have a R Job that is submitted to the condor, The R file(one.R) which is submitted to the condor is reading another R file(two.R), when I submit the job to the condor its is failed and the reason for that is the submitted R(one.R) file is not reading the called R file(two.R)
Error in text file is :
Error in file(file, "rt") : cannot open the connection
Calls: read.table -> file
In addition: Warning message:
In file(file, "rt") :
cannot open file 'C:/Users/pcname/Desktop/test_case/two.R': Permission denied
Execution halted
and my submit file is
#test_input.condor
#
executable = C:\R\R-2.10.1\bin\Rscript.exe
arguments = one.R
universe = vanilla
getenv = true
#requirements = ARCH == "INTEL" && OPSYS == "WINNT60"
input = one.R
should_transfer_files = yes
transfer_executable = false
when_to_transfer_output = ON_EXIT
transfer_input_files = C:/Users/OmegaAdmin/Desktop/test_case/two.R
log = test_input.log
output = test_input.out
error = test_input.err
queue
Appreciate any ideas on this.
Thanks,
This is not an R-related problem, but a problem of accessibility. The error message seems rather clear to me: the server has no reading rights for that file. Make sure you share the file or folder you want to read in. I don't know what the setup is of the network and clusters wherever you're located, but you better contact the admins to ask how you get your files to the right places.
Also make sure that if you transfer files to the servers/cluster, you adapt your R script so it points to the right directories. Which is probably not your own harddrive...
When you say
transfer_input_files = C:/Users/OmegaAdmin/Desktop/test_case/two.R
That tells Condor to copy two.R into the current working directory when the job starts. The current working directory is a specially created workspace, not (usually) a home directory. Thus I would expect the full path to look something like
C:/condor/execute/dir_28412/two.R
However, R is actually looking in
C:/Users/pcname/Desktop/test_case/two.R
Why is R looking there? Does one.R potentially say "Find two.R in $HOME/Desktop/test_case"? Does it perhaps say, "Look in Desktop/testcase/two.R" and R has configuration that wants to look relative to the user's home directory?
The solution is almost certainly to modify one.R or your R configuration to look for two.R in the current working directory. If for some reason R changes its current working directory, the environment variable _CONDOR_SCRATCH_DIR should contain it.
On a related note, you said:
arguments = one.R
input = one.R
The first is an argument passed to Rscript.exe, which I'm guessing tells R to load and run a file called one.R. Except that script isn't present! If you want that to work, you'll need to add it to transfer_input_files. But it obviously appears to work; why? Because "input=one.R" means "take the contents of one.R and pipe it in as standard input to Rscript.exe; the same as if you'd typed those contents in." I'm guessing you can remove the arguments, or remove the input and add one.R to your transfer_input_files, removing the ambiguity.

Resources