When I run a Julia Script that prints "Hello World" on HTCondor, I get the following error
fatal: error thrown and no exception handler available.
Base.InitError(mod=:Pkg, error=Base.KeyError(key="HOME"))
The code runs without a problem on my local Ubuntu machine. I can run
eval julia --version
in a bash script on condor and the output is
julia version 0.5.0
This problem has been discussed in two places on github: one, two.
ENV["HOME"] is used in a single file and the common recommendation is to modify that. However, I cannot change the julia installation on condor.
Is there a way to fix this on the fly before running a script without sudo?
As #sujeet suggested, it is possible to set environmental parameters in condor. The issue is resolved by adding the following line in the condor submit script
Environment = "HOME=""/tmp"""
, which sets the home directory to the tmp. Julia code runs fine then (as long one is careful not to write to home before resetting it in the script itself).
Related
I have an R script that runs perfectly when copy pasted into an R session.
However, when I try to run the script from the command line (i.e., Rscript mycode.R) I keep getting an error attempt to apply non-function.
The error is coming from a function that uses AzureGraph and AzureAuth to fetch data from a Microsoft cloud directory. I can provide more detail about this function if needed, but my question is really more general.
What could cause this difference in behaviour between executing in an active R session vs. running the script from the command line.
What is the best strategy to debug this? Normally I would step through code in an R session to locate and fix errors, but obviously that will not work in this case.
Possibilities:
You have objects (functions, packages) in your interactive R session's environment that are not present in the session spawned by Rscript, for example because you automatically restore your R workspace at startup. Possible check: you can use sessionInfo() to see packages that are loaded and/or attached and ls() to list environment objects.
You have multiple R installations on your system, and the RScript command is somehow mapping to a different one than your interactive session, which is missing some packages or functions. Possible check: the version function.
The defaults of RScript are slightly different from those of an interactive session (for example, "save" and "restore" are typically disabled) and this is somehow affecting your script. Possible solution: try to add the --restore argument to the Rscript call.
How to debug? If you're going blind, good old bisection. Take your script, comment out the bottom half, see if it runs. Repeat iteratively (uncomment half of the commented section if it runs, comment out half of the uncommented section if not) until you find the line where the error is.
You can also run an individual line of code from Rscript using Rscript -e "some code" if you want to quickly check if a specific call is causing problems.
shell_exec("Rscript C:\R\R-3.2.2\bin\code.R ");
This is the call to script.On calling the above script, the error occurs.
I am trying to call my R script from the above path but no output is being shown. While checking the error logs of PHP, it says 'Rscript' is not recognized as an internal or external command, operable program or batch file.' The script is working fine on the Rstudio but not running on the command line.
Add the Rscript path to your environment variables in Windows:
Go to Control Panel\System and Security\System and click Advanced System Settings, then environment variables, click on path in the lower box, edit, add "C:\R\R-3.2.2\bin"
Restart everything. Should be good to go. Then you should be able to do
exec('Rscript PATH/TO/my_code.R')
instead of typing the full path to Rscript. Won't need the path to your my_code.R script if your php file is in the same directory.
You need to set the proper path where your RScript.exe program is located.
exec ("\"C:\\R\\R-3.2.2\\bin\\Rscript.exe\"
C:\\My_work\\R_scripts\\my_code.R my_args";
#my_args only needed if you script take `args`as input to run
other way is you declare header in your r script (my_code.r)
#!/usr/bin/Rscript
and call it from command line
./my_code.r
If you are running it in Git Bash terminal, you could follow a revised version of the idea suggested by #user5249203: in the first line of your file my_code.R, type the following
#!/c/R/R-3.2.2/bin/Rscript.exe
I assumed that your path to Rscript.exe is the one listed above C:\R\R-3.2.2\bin. For anyone having a different path to Rscript.exe in Windows, just modify the path-to-Rscript accordingly. After this modification of your R code, you could run it in the Git Bash terminal using path-to-the-code/mycode.R. I have tested it on my pc.
I faced the same problem while using r the first time in VS Code, just after installing the language package (CRAN).
I restart the application and everything worked perfectly. I think restarting would work for you as well.
I try to run an R script at regular intervals to update a webpage. The script runs fine when called from the terminal like this:
/usr/local/bin/Rscript /Users/me/path/myscript.R
However, if I try running it as a cron job, I get an error. I add the job to crontab like this:
46 10 * * * /usr/local/bin/Rscript '/Users/me/path/myscript.R' >> '/Users/me/path/mylog.log' 2>&1
The script does run in R, but aborts due to an error. Specifically, I fit some models using rstan, and get an initialization error. (The error only applies to some models, while others still run fine.) The initialization values are valid by definition, but do not seem to be used properly. It is like rstan is doing math differently (and wrong) when it is run through cron.
The session info from R is identical whether I run the script in the terminal or as a cron job. My question is what else might still differ depending on how the script is run. Could rstan be using a different version of C++ when run as a cron job? Are there other paths I may need to set to get this to work correctly?
Update: The script also works if I run it using R CMD BATCH in terminal, but not if I use R CMD BATCH in a cron job. Using launchd triggers the same issue. I also tried using CmdStan through cmdstanr, and the same same thing happens: Runs fine until added to a cron job.
Edit 2: The models I thought ran fine in cron, were not actually fine. The results were wrong, until I used the fix explained below.
It looks like I finally managed to fix this, and I'm posting my solution here for anyone who encounters the same problem.
I ran env in terminal to see my current user environment. I copy-pasted the full output to the top of my crontab file. (Simply adding the PATH variable was not sufficient. I suppose it was SHELL or perhaps both PATH and SHELL that did the trick, but I haven't explored this further.)
To edit my user's crontab, I ran crontab -e, then pressed i to edit the file, pasted everything from env at the top of the file, stopped editing by pressing ctrl + c, and quit by typing :wq and hitting enter.
I tried to run a Python script from R with:
system('python script.py arg1 arg2')
And got an error:
ImportError: No module named pandas
This was a bit of a surprise since the script was working from the terminal as expected. Having encountered this type of issue before (with knitr, whence the engine.path chunk option), I know to check:
Sys.which('python')
# python
# "/usr/bin/python"
And compare it to the command line:
$ which python
# /Users/michael.chirico/anaconda2/bin/python
(i.e., the error arises because I have pandas installed for the anaconda distribution, though TBH I don't know why I have a different distribution)
Hence I can fix my issue by running:
system('/Users/michael.chirico/anaconda2/bin/python script.py arg1 arg2')
My question is two-fold:
How does R's system/Sys.which find a different python than my terminal?
How can I fix this besides writing out the full binary path each time?
I read ?Sys.which for some hints, but to no avail. In particular, ?Sys.which suggests Sys.which is using which:
This is an interface to the system command which
This is clearly (?) untrue; to be sure, I checked Sys.which('which') and which which to confirm both are pointing to /usr/bin/which (goaded on by this tidbit):
On a Unix-alike the full path to which (usually /usr/bin/which) is found when R is installed.
To the latter, on a whim I tried Sys.setenv(python = '/Users/michael.chirico/anaconda2/bin/python') to no avail.
As some of the comments hint, this is a problem that arises because the PATH environment variable is different for programs launched by Finder (or the Dock) than it is in the Terminal. There are ways to set the PATH for Dock-launched applications, but they aren't pretty. Here's a place to start looking if you want to go that route:
https://apple.stackexchange.com/questions/51677/how-to-set-path-for-finder-launched-applications
The other thing you can do, which is probably more straightforward, is tell R to set the PATH variable when it starts up, using Sys.setenv to add the path to your desired Python instance. You can do that for just one project, for your whole user account, or for the whole system, by placing the command in a .Rprofile file in the corresponding location. More information on how to do this here:
https://stat.ethz.ch/R-manual/R-devel/library/base/html/Startup.html
I am troubleshooting an issue that I am running into on a new instance of RStudio on a virtual machine. If I run an R script expFit.R in RStudio, it has no problem finding the needed packages (in this case RODBC and minpack.lm) but if I try to run from the command line
c:\"Program Files"\R\R-3.4.1\bin\x64\Rscript.exe e:\expFit.R
I get the error 'minpack.lm' is not a valid installed package. If I move RODBC to be the first package to load, then it errors on that.
The reason that I am running from command line is that in the real application I am running it from a command line stored procedure in SQL. It doesn't work either way (stored procedure or straight command line). I made sure that the path to the packages is a Path variable.
This works on 2 of my other servers, the only difference being that RStudio is installed so that all users can use it on the server that is getting the error, while the other servers have it installed only in my account.
Any advice would be much appreciated!