I have a cron job that launches an Rscript to go fetch some data and stash it away in a database. Sometimes the source isn't ready so it waits. Sometimes the source skips a data point so the script ends up waiting until another instance is started from cron. Now that two instances are running they cause problems with each other. Is there something like the following pseudo code that I could put at the top of my scripts so they stop when they see another instance of themselves is already running:
stopifnot(Sys.running('nameofscript.r'))
One thing I thought to do would be for the script to make a temp file with a fixed name at the start and then delete that temp file at the end. This way the script can check for the existence of the file to know if it's already running. I also see there's something called flock which is probably a better solution than that but I'd prefer if there was a way R can do it without bash commands (I've never really learned any bash). If there isn't a pure R way to do this, can the R script call flock on itself or would it only work if the cron task calls a bash script that then calls the rscript. If it isn't already obvious, I don't really know how to use flock.
Related
So I wrote my father a neat little R script that pulls financial indicators on stocks, and outputs the info to a csv...
I would like to have it set up so that the script will run automatically once a day, skipping the weekends if possible. I looked around for awhile online and it seems as though the Mac "Automator" App is what I'm looking for.
However, after reading many guides and posts (like this one https://www.r-bloggers.com/how-to-source-an-r-script-automatically-on-a-mac-using-automator-and-ical/) I cannot get it to work...
In trying to replicate what this man did above I get the error that the first path is a directory; while the latter returns stuff like "cat: Rscript: No such file or directory"
So I was wondering if anyone could recommend either any good free software that will allow me to do what I would like, or how to run an R script from the /bin/bash shell
EDIT:
The suggested solution isn't really answering my problem. The issue is making this as easy as possibly for my dad to run, that way he doesn't have to do anything, specifically use the terminal. Ideally I could just schedule a task that repeats every morning, but the cronR package requires Daemon, and the others are just command line tools
I had a similar experience.
I created an automator calendar alarm
added a Execute AppleScript Action and used the following code:
on run {input, parameters}
try
tell application "R"
activate
with timeout of 90000 seconds
cmd "source(\"Dropbox/RScripts/CV19/liibre_coronabr.R\")"
end timeout
end tell
end try
return input
end run
When you save it, just choose the date and time for it to run and select the option to repeat everyday
That's it!
I want to run an R script (in Win 7) from SQL Server 2014 each time a new record is added (to perform some analysis on the data). I saw that this can be done with the xp_cmdshell command which is like running it manually from the command line.
My problems (and questions) are:
I've made out from various websites that probably the best option is to use Rscript. This would have to be used at the command line as:
C:\Program Files\R\R-3.2.3\bin\x64\Rscript "my_file_folder\my_file.r
Can I copy Rscript.exe to the folder where my script is, such that I can run my script independently, even if R is not installed? What other files do I need to copy together with Rscript.exe such that it would work independently?
My script loads some packages that contain functions that it uses. Is there a way to somehow include these in the script such that they don't have to be loaded every time (it takes about 5 sec so far and I need this script to be faster)? Or is there a way to only load these packages the first time that the script runs?
In case the overall approach I've described here is not the best one, I am open to doing it differently. Maybe there is a way to somehow package the R script together with all the required dependencies (libraries and other parts of the R software which the script would need to run independently).
What I ultimately need is a for the script to run silently, and reasonably fast, without any windows or anything else popping up, each time a new record is added to my database, do the analysis and exit.
Thanks in advance for any answers.
UPDATE:
I figured out an elegant solution to running the R script. I'm setting up a job in SQL Server and inside that job I'm using "xp_cmdshell" to run my script as a parameter to Rscript.exe, as detailed at point 1 above. I can start this job from any stored procedure and the beauty of it is that the stored procedure does not wait for the script to finish. It just triggers the job (that runs the script in a separate thread) and then it continues with its business.
But questions from points 1 and 2 still remain.
I'm currently running some queries to a database and getting back some big files as a result. I have encountered the common problem of Windows not freeing the memory, even though I 'rm()' everything and (edit) calling 'gc()'. One workaround i have found is using .rs.restartR() in Rstudio.
This though requires me to constantly watch my script, in order to continue it after the session restart. Is it possible to automate it? If not what other methods do people use to overcome this problem?
You could break the code into 2 files and write a batch file (.bat) that runs the first file through .rs.restartR() and then the remainder of the code in the next file.
You could also skip the .bat and just schedule both .R scripts to run in Task Scheduler.
Also, please see my comment regarding garbage collection (gc()).
I have a scheduled task in windows to run a R program ("ftp.R")
After many attempts and reading many SO literature, I found that the only way to make it work properly was writing this piece of code into a .bat file:
#ECHO OFF
RSCRIPT ftp.R
Everything goes fine except wqith it, when I am trying to use functions that I have in other R programs.
For example, in the "ftp.R" program I have this pice of code:
source("//BCN-01/Server/R/Main/Production/Check_Integrity.R")
In the "Check_Integrity.R" program I have some functions I need to use in "ftp.R".
The thing is that if I execute the .bat file manually, there is no problem, and the "ftp.R" runs perfectly. But if I run exactly the same .bat file but from the task scheduler the "ftp.R" is unable to find the external functions.
(I am running the code in a Windows Server 2012)
One big difference between running a batch manually / with the scheduler is that the scheduler starts the script with system32 folder as work directory. So it might be enough to add the following line to your batch file: CD %~dp0.
Another point is that the scheduler runs your batch as a different user. So it is possible that you (your user account) has access to //BCN-01/Server/R/Main/Production/ while the system user the scheduler is running your script with does not. You could also try to tell the scheduler to run the script with the same user which you are logged in when running it successfully by hand.
Finally I figured out what was wrong:
All the paths have to be defined as absolute paths: mapped units are not valid.
Althought I already I knew that (thanks to some posts I read here in SO), I missed one path mapped into one of the functions in 'Check_Integrity.R' .
I want to execute a R script in the background from the R console.
From the console , i usually run R script as source('~/.active-rstudio-document')
I have to wait until the script is completed to go ahead with my rest of work.
Instead of this i want R to be running in the background while i can continue with my work in the console.
Also i should be somehow notified when R completes the source command.
Is this possible in R ?
This might be quite useful as we often sees jobs taking long time.
PS - i want the source script to be running in the same memory space rather than a new one. Hence solutions like fork , system etc wont work for me. I am seeing if i can run the R script as a separate thread and not a separate process.
You can use system() and Rscript to run your script as an asynchronous background process:
system("Rscript -e 'source(\"your-script.R\")'", wait=FALSE)
At the end of your script, you may save your objects with save.image() in order to load them later, and notify of its completion with cat():
...
save.image("script-output.RData")
cat("Script completed\n\n")
Hope this helps!