I would like to set up a crontab that runs the emailSender.R script daily at 5pm Monday to Friday.
The script of emailSender.R is as follows:
library(rmarkdown)
rmarkdown::render("htmlmarkdown.Rmd")
library(gmailR)
gmailR::gmail(
to =c("recipient#email.com"),
subject = "Subject",
message = "Message",
username = "me#email.com",
password = "password",
attachment = "htmlmarkdown.html"
)
I then open up terminal to set up the crontab by first typing crontab -e.
Then a window pops up where I try to set up my cronjob using the following code.
0 17 * * * Rscript /Users/username/emailSender.R
Unfortunately, emailSender.R doesn't run as scheduled.
Would greatly appreciate any help on getting a crontab to schedule my R script
EDIT: After going back to my terminal and typing Rscript I am prompted:
-bash: Rscript: command not found
Perhaps I have to set Rscript in my PATH before cron can set-up the task. Unsure how to do that despite searching extensively.
Try installing the cronR package.
Once you do this, you should be able to navigate to tools > Addins where you can execute the package. It will bring up a scheduler that will allow you to schedule a time for your script to run.
If you have permission issues, go to System preferences > security > Privacy. Click the 'full disk access' and grant RStudio / R access. This should allow you to schedule jobs to run in the future.
You need to include the path of Rscript:
0 17 * * * /usr/local/bin/Rscript /Users/username/emailSender.R
Add this code at the beggining:
Sys.setenv(RSTUDIO_PANDOC="/usr/lib/rstudio-server/bin/pandoc")
Related
I am writing an R code on a Linux system using RStudio. At some point in the code, I need to use a system call to a command that will download a few thousand of files from the lines of a text file:
down.command <- paste0("parallel --gnu -a links.txt wget")
system(down.command)
However, this command takes a little while to run (a couple of hours), and the R prompt stays locked while the command runs. I would like to keep using R while the command runs on the background.
I tried to use nohup like this:
down.command <- paste0("nohup parallel --gnu -a links.txt wget > ~/down.log 2>&1")
system(down.command)
but the R prompt still gets "locked" waiting for the end of the command.
Is there any way to circumvent this? Is there a way to submit system commands from R and keep them running on the background?
Using ‘processx’, here’s how to create a new process that redirects both stdout and stderr to the same file:
args = c('--gnu', '-a', 'links.txt', 'wget')
p = processx::process$new('parallel', args, stdout = '~/down.log', stderr = '2>&1')
This launches the process and resumes the execution of the R script. You can then interact with the running process via the p name. Notably you can signal to it, you can query its status (e.g. is_alive()), and you can synchronously wait for its completion (optionally with a timeout after which to kill it):
p$wait()
result = p$get_exit_status()
Based on the comment by #KonradRudolph, I became aware of the processx R package that very smartly deals with system process submissions from within R.
All I had to do was:
library(processx)
down.command <- c("parallel","--gnu", "-a", "links.txt", "wget", ">", "~/down.log", "2>&1")
processx::process$new("nohup", down.comm, cleanup=FALSE)
As simple as that, and very effective.
I'm trying to run a daily taskscheduleR script that pulls data into R from an API. It works when I run it as a one time task but for some reason it won't work as a daily task. I keep getting the following error in the log file:
<HEAD><TITLE>Authorization Required</TITLE></HEAD>
<BODY BGCOLOR=white FGCOLOR=black>
<H1>Authorization Required</H1><HR>
<FONT FACE=Helvetica,Arial>
<B>Description: Authorization is required for access to this proxy</B>
</FONT>
<HR>
<!-- default Authorization Required response (401) -->
Here's the code:
library(httr)
library(jsonlite)
library(tidyverse)
library(taskscheduleR)
# Url to feed into GET function
url<-"https://urldefense.com/v3/__http://files.airnowtech.org/airnow/yesterday/daily_data_v2.dat__;!!J30X0ZrnC1oQtbA!Yh5wIss-mzbpMRXugALJoWEKLKcg1-7VmERQwcx2ESK0PZpM5NWNml5s9MVgwHr5LD1i5w$ "
# Sends request to AirNow API to get access to data
my_raw_result<-httr::GET(url)
# Retrieve contents of a request
my_content<-httr::content(my_raw_result,as="text")
# Parse content into a dataframe
my_content_from_delim <- my_content %>% textConnection %>% readLines %>% read.delim(text = ., sep = "|",header = FALSE)
head(my_content_from_delim)
I have been using the Rstudio add-in to create the task.
If you are trying to access this on a work computer, you may need to allow downloads from the url link. Open a browser, paste that url, click 'allow downloads', run the script.
I am not sure whether the solution I will offer will work for you, but it won't harm to try. If the problem related to the task scheduler, the following solution might work. However, if the problem of authorization issues, you may need to get some IT help from your workplace.
For the task scheduler issue, you can directly send your script to the windows task scheduler with a batch file and create a schedule for it.
To make it easy, you can use the following code. First, open a new folder and copy-paste your R script there. To run the following code, you should call you R script as My Script.r.
Then, in the same folder, create a batch file with the following codes. To create a batch file, you should copy the following code into a Notepad and save it as Run R Script.bat in the same folder.
cd %~dp0
"C:\PROGRA~1\R\R-40~1.0\bin\R.exe" -e "setwd(%~dp0)" CMD BATCH --vanilla --slave "%~dp0My Script.r" Log.txt
Here, cd %~dp0 will set the directory for the windows batch to the folder you run this batch. "C:\PROGRA~1\R\R-40~1.0\bin\R.exe" will specify your R.exe. You may need to change the path based on your system files.
-e "setwd(%~dp0)" will set the directory of R to the same folder in which the batch and script will be run.
"%~dp0My Script.r" Log.txt will define R script pathname and the log file for the batch.
Second, to create a daily schedule, we are going to create another batch file. To do so, copy and paste the following codes into a notepad and save as Daily Schedule.bat.
When you click the Daily Schedule.bat, it will create a daily task and run for the first time in one minute, and every day it will repeat itself at the same time when you first run this batch.
#echo off
for /F "tokens=1*" %%A in ('
powershell -NoP -C "(Get-Date).AddMinutes(1).ToString('MM/dd/yyyy HH:mm:ss')"
') do (
Set "MyDate=%%A"
set "MyTime=%%B"
)
::Execute path to bat path
cd %~dp0
::Create Task
SchTasks /Create /SC DAILY /TN "MY R TASK" /TR "%~dp0Run R Script.bat" /sd %MyDate% /st %MyTime%
This code will create a task called as "MY R TASK". To see whether it is scheduled, you can run the following codes on the windows prompt: taskschd.msc. This will open your task scheduler, and you can find your task there. If you want to modify or delete, you can use this task scheduler program; it has a nice GUI and easy to navigate.
For more details about the Task scheduler syntax, see the following link
If you have any questions, let me know.
I'm new to Ubuntu and would like to run a daily crontab. I'm running similar job under Windows Command Line but would like to switch to Ubuntu.
My R-Script has an Error-function which prints errors into a separate
folder everytime a job is done and an error has occured ():
# define error function
my.error.fun <- function() {
cat(geterrmessage(), file="/home/network/R/Test.txt", append=TRUE)
}
# output error
options("error"=my.error.fun)
Under crontab -e I have determined the following:
00 16 * * * /usr/bin/Rscript /home/R/network/Test.R 2> /home/R/RErrors/Test.txt
and the crontabhas been saved under /tmp/crontab.2t/crontab
where I append the errros inside the R-Script. In contrast to Windows Command Line that concept seems not to work in Ubuntu That seems not to work however. I'm not sure whether thats the correct syntax. There has been no file in the RErrors folder.
May I as for some suggestions?
I have used R in the past to do very basic calls to the commmand line. The example can be found here.
This time around, I am looking to mimic this code which runs successfully from the command line in Windows:
> cd C:\Documents and Settings\BTIBERT\My Documents\My Dropbox\Eclipse\Projects\R\MLB\retrosheet\rawdata
> bgame -y 2010 2010bos.eva >2010bos.txt
This is the code I am trying to run inside of R. I have already set the working directory inside of R.
dir <- paste("cd", getwd(), sep=" ")
system(dir)
system("bgame -y 2010 2010bos.eva >2010bos.txt")
I am sure this is user error, but what am I doing wrong? It appears to work initially, but returns the following error. I very well could be doing something wrong, but I believe I am using the same commands.
Expanded game descriptor, version 109(185) of 05/08/2008.
Type 'bgame -h' for help.
Copyright (c) 2001 by DiamondWare.
[Processing file 2010bos.eva.]
>2010bos.txt: can't open.
Warning message:
running command 'bgame -y 2010 2010bos.eva >2010bos.txt' had status 2
Any help you can provide will be appreciated.
You need to issue all commands in one system() call:
system(paste("cd",getwd() "&& bgame -y 2010 2010bos.eva >2010bos.txt",sep=" "))
You should already be in your working directory, so I'm not sure the cd getwd() is necessary. And you may need quotes around your path because it contains spaces. The error may be resolved by putting spaces around >.
If I were in your shoes, I would try this:
system("bgame -y 2010 2010bos.eva > 2010bos.txt")
UPDATE:
And you should probably heed this advice in the "Differences between Unix and Windows" section of ?system that says you should use shell:
• The most important difference is that on a Unix-alike
‘system’ launches a shell which then runs ‘command’. On
Windows the command is run directly - use ‘shell’ for an
interface which runs ‘command’ _via_ a shell (by default the
Windows shell ‘cmd.exe’, which has many differences from the
POSIX shell).
This means that it cannot be assumed that redirection or
piping will work in ‘system’ (redirection sometimes does, but
we have seen cases where it stopped working after a Windows
security patch), and ‘system2’ (or ‘shell’) must be used on
Windows.
Has no-one else found that system("dir", intern = T) for example doesn't work, but that you need system("cmd.exe /c dir", intern = T)? Only the latter works for me. I found this at the discussion site here (William Dunlap's post, about a third of the way down).
Also, it doesn't work with the "cd" command, but you can use the setwd() function within R and then the command will be executed within that directory.
I created the following functions for convenience, for executing programmes and running commands:
#the subject is an input file that a programme might require
execute <- function(programme, subject.spec = "", intern = FALSE, wait = FALSE){
if(!identical(subject.spec, "")){subject.spec <- paste0(" ", subject.spec)} #put space before the subject if it exists
system(paste0("cmd.exe /c ", programme, subject.spec), intern = intern, wait = wait)
}
command <- function(command, intern = TRUE, wait = FALSE){
system(paste("cmd.exe /c", command), intern = T, wait = wait)
}
Does it break your code when you get error 1 or does execution continue?
Whenever executing system commands through another language it is useful to print the system call before you call it to see exactly what is happening, pull up the shell you are intending to use and check for the same error. As the command is executed correctly this could be a hickup in bgame or R.
If you look at http://astrostatistics.psu.edu/datasets/R/html/base/html/shell.html you can see the variable flag passed to the system call."flag the switch to run a command under the shell. If the shell is bash or tcsh the default is changed to "-c"."
Also "the shell to be used can be changed by setting the configure variable R_SHELL to a suitable value (a full path to a shell, e.g. /usr/local/bin/bash)."
I have written an R script that pulls some data from a database, performs several operations on it and post the output to a new database.
I would like this script to run every day at a specific time but I can not find any way to do this effectively.
Can anyone recommend a resource I could look at to solve this issue? I am running this script on a Windows machine.
Actually under Windows you do not even have to create a batch file first to use the Scheduler.
Open the scheduler: START -> All Programs -> Accesories -> System Tools -> Scheduler
Create a new Task
under tab Action, create a new action
choose Start Program
browse to Rscript.exe which should be placed e.g. here:
"C:\Program Files\R\R-3.0.2\bin\x64\Rscript.exe"
input the name of your file in the parameters field
input the path where the script is to be found in the Start in field
go to the Triggers tab
create new trigger
choose that task should be done each day, month, ... repeated several times, or whatever you like
Supposing your R script is mytest.r, located in D:\mydocuments\, you can create a batch file including the following command:
C:\R\R-2.10.1\bin\Rcmd.exe BATCH D:\mydocuments\mytest.r
Then add it, as a new task, to windows task scheduler, setting there the triggering conditions.
You could also omit the batch file. Set C:\R\R-2.10.1\bin\Rcmd.exe in the program/script textbox in task scheduler, and give as Arguments the rest of the initial command: BATCH D:\mydocuments\mytest.r
Scheduling R Tasks via Windows Task Scheduler (Posted on February 11, 2015)
taskscheduleR: R package to schedule R scripts with the Windows task manager (Posted on March 17, 2016)
EDIT
I recently adopted the use of batch files again, because I wanted the cmd window to be minimized (I couldn't find another way).
Specifically, I fill the windows task scheduler Actions tab as follows:
Program/script:
cmd.exe
Add arguments (optional):
/c start /min D:\mydocuments\mytest.bat ^& exit
Contents of mytest.bat:
C:\R\R-3.5.2\bin\x64\Rscript.exe D:\mydocuments\mytest.r params
Now there is built in option in RStudio to do this, to run scheduler first install below packages
install.packages('data.table')
install.packages('knitr')
install.packages('miniUI')
install.packages('shiny')
install.packages("taskscheduleR", repos = "http://www.datatailor.be/rcube", type =
"source")
After installing go to
**TOOLS -> ADDINS ->BROWSE ADDINS ->taskscheduleR -> Select it and execute it.**
Setting up the task scheduler
Step 1) Open the task scheduler (Start > search Task Scheduler)
Step 2) Click "Action" > "Create Task"
Step 3) Select "Run only when the user is logged on", uncheck "Run with highest priveledges", name your task,
configure for "Windows Vista/Windows Server 2008"
Step 4) Under the "Triggers" tab, set when you would like the script to run
Step 5) Under the "Actions" tab, put the full location of the Rscript.exe file, i.e.
"C:\Program Files\R\R-3.6.2\bin\Rscript.exe" (include the quotes)
Put the name of your script with with -e and source() in arguments wrapping it like this:
-e "source('C:/location_of_my_script/test.R')"
Troubleshooting a Rscript scheduled in the Task Scheduler
When you run a script using the Task Scheduler, it is difficult to troubleshoot any issues because you don't get any error messages.
This can be resolved by using the sink() function in R which will allow you to output all error messages to a file that you specify. Here is how you can do this:
# Set up error log ------------------------------------------------------------
error_log <- file("C:/location_of_my_script/error_log.Rout", open="wt")
sink(error_log, type="message")
try({
# insert your code here
})
The other thing that you will have to change to make your Rscript work is to specify the full file path of any file paths in your script.
This will not work in task scheduler:
source("./functions/import_function.R")
You will need to specify the full file path of any scripts you are sourcing within your Rscript:
source("C:/location_of_my_script/functions/import_function.R")
Additionally, I would remove any special characters from any file paths that you are referencing in your R script. For example:
df <- fread("C:/location_of_my_data/file#2342.csv")
may not run. Instead, try:
df <- fread("C:/location_of_my_data/file_2342.csv")
Changing windows passwords
Beware: Changing windows passwords will pause your task scheduler script(s). You will need to log back into the task scheduler and enter your password to get them started again.
I set up my tasks via the SCHTASKS program. For running scripts on startup, you would write something along the lines of
SCHTASKS /Create /SC ONSTART /TN MyProgram /TR "R CMD BATCH --vanilla d:\path\to\script.R"
See this website for more details on SCHTASKS. More details at Microsoft's website.
You can use Windows Task Scheduler.
After following any combination of these steps and you receive the "Argument Batch Ignored" error after R.exe runs, try this, it worked for me.
In Windows Task Scheduler:
Replace BATCH "C:\Users\desktop\yourscript.R"in the arguments field
with
CMD BATCH --vanilla --slave "C:\Users\desktop\yourscript.R"