R code runs well on one computer but not on another (via Task Scheduler in RScript.exe) - r

My issue is: when I run the following code from one laptop in RScript.exe via Task Scheduler, I get the desired output; that is the email is sent. But when I run the same code on another machine in RScript.exe via Task Scheduler, it doesn't run. Another machine (machine 2) is able to send emails (when only the code for email is run), so I think the issue is with the following part.
results <- get_everything(query = q, page = 1, page_size = 2, language = "en", sort_by = "popularity", from = Yest, to = Today)
I am unable to find what is the issue here. Can someone please help me with this?
My code is:
library(readxl)
library(float)
library(tibble)
library(string)
library(data.table)
library(gt)
library(tidyquant)
library(condformat)
library(xtable)
library(plyr)
library(dplyr)
library(newsanchor)
library(blastula)
Today <- Sys.Date()
Yest <- Sys.Date()-1
results <- get_everything(query = "Inflation", page = 1, page_size = 2, language =
"en", sort_by = "popularity", from = Yest, to = Today, api_key =
Sys.getenv("NEWS_API_KEY"))
OP <- results$results_df
OP <- OP[-c(1, 5:9)]
colnames(OP) <- c("News Title", "Description", "URL")
W <- print(xtable(OP), type="html", print.results=FALSE, align = "l")
email1 <-
compose_email(
body = md(
c("<tr>", "<td>", "<table>", "<tr>", "<td>", "<b>", "Losers News", "</b>", W,
"</td>", "</tr>", "</table>","</td>", "<td>")
)
)
email1 %>%
smtp_send(
from = "abc#domain.com",
to = "pqr#domain.com",
subject = "Hello",
credentials = creds_key(
"XYZ"
)
)

Whenever you schedule jobs, consider using a command line shell such as PowerShell or Bash to handle the automation steps, capture, and log errors and messages. Rscript fails on the second machine for some unknown reason which you cannot determine since you do not receive any error messages from console using TaskScheduler.
Therefore, consider PowerShell to run all needed Rscript.exe calls and other commands and capture all errors to date-stamped log file. Below script redirects all console output to a .log file with messages. When Rscript command fails, the log will dump error or any console output (i.e., head, tail) below it. Regularly check logs after scheduled jobs.
PowerShell script (save as .ps1 file)
cd "C:\path\to\scripts"
& {
echo "`nAutomation Start: $(Get-Date -format 'u')"
echo "`nSTEP 1: myscript.R - $(Get-Date -format 'u')"
Rscript myscript.R
# ... ADD ANY OTHER COMMANDS ...
echo "`nCAutomation End: $(Get-Date -format 'u')"
} 3>&1 2>&1 > "C:\path\to\logs\automation_run_$(Get-Date -format 'yyyyMMdd').log"
Command Line (to be used in Task Scheduler)
Powershell.exe -executionpolicy remotesigned -File myscheduler.ps1
Note: Either change directory in TaskScheduler job settings where myscheduler.ps1 resides or run absolute path in -File argument.

Related

Run DOS program from R

So this command line runs fine in DOS, and the path to npx is in the path variable
npx #squoosh/cli --mozjpeg {quality:75} test.PNG
But I want to call it from R
This doesn't work:
system("npx #squoosh/cli --mozjpeg {quality:75} test.PNG")
It just returns "127"
Nor does this:
shell("npx #squoosh/cli --mozjpeg {quality:75} test.PNG")
Returns "'npx' is not recognized as an internal or external command,
operable program or batch file."
Try system2 instead, it's the recommended way for system calls.
txt <- "npx #squoosh/cli --mozjpeg {quality:75} test.PNG"
txt <- scan(text = txt, what = character())
cmd <- txt[1]
args <- txt[-1]
system2(cmd, args)
Or simply
system2(command = txt[1], args = txt[-1])

How can I commit changes to GitHub from within a R script?

I am trying to automate some basic git operations from within a R script. I am using Rstudio on Windows OS. This may be helpful for example if you wished to update GitHub when a script finishes performing some automated task.
I wrote some simple functions that utilize R's shell() function and the Window's & pipe operator to send a chain of commands to the OS terminal:
# Git status.
gitstatus <- function(dir = getwd()){
cmd_list <- list(
cmd1 = tolower(substr(dir,1,2)),
cmd2 = paste("cd",dir),
cmd3 = "git status"
)
cmd <- paste(unlist(cmd_list),collapse = " & ")
shell(cmd)
}
# Git add.
gitadd <- function(dir = getwd()){
cmd_list <- list(
cmd1 = tolower(substr(dir,1,2)),
cmd2 = paste("cd",dir),
cmd3 = "git add --all"
)
cmd <- paste(unlist(cmd_list),collapse = " & ")
shell(cmd)
}
# Git commit.
gitcommit <- function(msg = "commit from Rstudio", dir = getwd()){
cmd_list <- list(
cmd1 = tolower(substr(dir,1,2)),
cmd2 = paste("cd",dir),
cmd3 = paste0("git commit -am ","'",msg,"'")
)
cmd <- paste(unlist(cmd_list),collapse = " & ")
shell(cmd)
}
# Git push.
gitpush <- function(dir = getwd()){
cmd_list <- list(
cmd1 = tolower(substr(dir,1,2)),
cmd2 = paste("cd",dir),
cmd3 = "git push"
)
cmd <- paste(unlist(cmd_list),collapse = " & ")
shell(cmd)
}
My gitstatus, gitadd, and gitpush functions work. The gitcommit function does not work. It generates the following error:
fatal: Paths with -a does not make sense.
Warning message:
In shell(cmd) : 'd: & cd D:/Documents/R/my_path & git commit -am 'commit from Rstudio'' execution failed with error code 128
The gitpush function works because if you switch to the terminal or git within Rstudio, you can commit changes and then successfully call gitpush.
Any ideas on how to fix this issue?
...
Note: I have Git bash installed, and I can successfully use git from the Windows command terminal and Rstudio. I also tried an alternative strategy which was to have R write a temporary .bat file and then execute this, but this strategy also gets hung up on the commit step.
Solution
The answer lie within Dirk Eddelbuettel's drat package function addrepo. It was also necessary to use git2r's config function to insure that git recognizes R. git2r's functions probably provide a more robust solution for working with git from an R script in the future. In the meantime, here's how I fixed the problem.
Install git2r. Use git2r::config() to insure git recognizes R.
From Dirk's code I modified the gitcommit() function to utilize sprintf() and system() to execute a system command:
# Git commit.
gitcommit <- function(msg = "commit from Rstudio", dir = getwd()){
cmd = sprintf("git commit -m\"%s\"",msg)
system(cmd)
}
Sprintf's output looks like this:
[1] "git commit -m\"commit from Rstudio\""
Example
#install.packages("git2r")
library(git2r)
# Insure you have navigated to a directory with a git repo.
dir <- "mypath"
setwd(dir)
# Configure git.
git2r::config(user.name = "myusername",user.email = "myemail")
# Check git status.
gitstatus()
# Download a file.
url <- "https://i.kym-cdn.com/entries/icons/original/000/002/232/bullet_cat.jpg"
destfile <- "bullet_cat.jpg"
download.file(url,destfile)
# Add and commit changes.
gitadd()
gitcommit()
# Push changes to github.
gitpush()
Well, the pic looks wonky, but I think you get the point.
From what I have read in this Ask Ubuntu question, you should be using &&, not &, to separate multiple commands in the Git bash. Try doing that:
gitcommit <- function(msg = "commit from Rstudio", dir = getwd()) {
cmd_list <- list(
cmd1 = tolower(substr(dir, 1, 2)),
cmd2 = paste("cd", dir),
cmd3 = paste0("git commit -am ", "'", msg, "'")
)
cmd <- paste(unlist(cmd_list),collapse = " && ")
shell(cmd)
}
Note that your gitcommit function will output something like this:
/v & cd /var/www/service/usercode/255741827 & git commit -am 'first commit'"
I don't know what purpose the substr(dir, 1, 2) portion serves, but if my suggestion still doesn't work, then try removing it to leave just the cd and git commit commands.
I personally like the syntax and simplicity of the gert package from rOpenSci. Specifically, to commit a repo you would do:
git_add("test.txt")
git_commit("Adding a file", author = "jerry <jerry#gmail.com>")
Top push to remote:
git_push(remote = "origin", repo = ".")
And see all other useful functions with this simple syntax.

Differences in calling `system()` from within RStudio or via Rscript?

I am trying to run external tools from the MEME suite, one of this tool (jaspar2meme) producing a text file that is then use as an input of a second tool (fimo). Here is my code :
#!usr/bin/Rscript
com1 <- "meme/bin/jaspar2meme"
arg1 <- "-bundle jaspar_plant_2014.pfm"
message("jaspar2meme command: ", com1, arg1)
system2(command = com1, args = arg1, stdout = "motif.fimo", wait = T)
com2 <- paste0("meme/bin/fimo")
arg2 <- paste0("--text --oc . --verbosity 1 --thresh 1.0E-4 --bgfile bg.fimo motif.fimo Genes_up_h16.ame")
message("FIMO command: ", com2, arg2)
system2(command = com2, args = arg2, stdout = "fimoresult.txt", wait = T)
When I run this code from within RStudio (via source), it works perfectly: the file motif.fimo is produced by jaspar2meme and use by fimo to produce the resulting file fimoresult.txt.
When I run the same script via Rscript from the shell (bash), the motif.fimo is also produced as expected but is not found by fimoand the fimoresult.txt remains empty.
What I tried so far is to use either system() or system2(), using the wait=T option or not, specifying the full path to motif.fimo but without success.
I finally got it... The locale variables were different in RStudio and Rscript. The motif.fimo file produced by jaspar2meme looked the same in both cases but was apparently not. By changing the first call to system2() by :
system2(command = com1, args = arg1, stdout = "motif.fimo", wait = T, env = c("LC_NUMERIC=C"))
solve my problem.

setup_twitter_oauth, searchTwitter and Rscript

I run the following script using an installation of RStudio on a Linux-Server.
require(twitteR)
require(plyr)
setup_twitter_oauth(consumer_key='xxx', consumer_secret='xxx',
access_token='xxx', access_secret='xxx')
searchResults <- searchTwitter("#vds", n=15000, since = as.character(Sys.Date()-1), until = as.character(Sys.Date()))
head(searchResults)
tweetsDf = ldply(searchResults, function(t) t$toDataFrame())
write.csv(tweetsDf, file = paste("tweets_vds_", Sys.Date(), ".csv", sep = ""))
The script works fine, when I run it from the user-interface.
However, when I automatically run it via the terminal using crontab, I get the following error-message:
[1] "Using direct authentication"
Error in twInterfaceObj$getMaxResults :
could not find function "loadMethod"
Calls: searchTwitter -> doRppAPICall -> $
Execution halted
Why?

Is there a way to make R beep/play a sound at the end of a script?

When I run R scripts I go do something else on a different desktop. If I don't check frequently, I never know when something is finished. Is there a way to invoke a beep (like a system beep) or get R to play a sound or notify growl via some code at the end of my script?
I have a package (beepr) with the sole purpose of making notification sounds in R which should work cross-platform. Run the following to install beepr and make a sound:
install.packages("beepr")
library(beepr)
beep()
More info at github: https://github.com/rasmusab/beepr
alarm()
The alarm function. It works by sending \a to the console
On MacOSX you can let the computer speak:
system("say Just finished!")
and you can also change the artificial voice that will speak:
system("say -v Kathy Just finished!")
You can pick any voice that is available on your computer. On Yosemite you can see which voices are installed in System Preferences -> Dictation & Speech -> Text to Speech.
You should have it tweet when it's done: http://cran.r-project.org/web/packages/twitteR/index.html
alarm doesn't work on my Windows machine so I created a function that does actually make noise.
beep <- function(n = 3){
for(i in seq(n)){
system("rundll32 user32.dll,MessageBeep -1")
Sys.sleep(.5)
}
}
This clearly could only work on Windows but I don't guarantee it will even run on an arbitrary Windows computer. I've only tested it on my machine but I figured I'd post it in case anybody has the same problem with alarm that I do.
cat('Hello world!\a')
How about something reasonably OS independent for OSes with GUIs and web-browsers? It even works on RStudio Server!
browseURL('https://www.youtube.com/watch?v=QH2-TGUlwu4')
Not only that, you can also also put some epic music from Youtube when the program is done looping :) (For Ubuntu/Debian:)
system("xdg-open 'http://www.youtube.com/watch?v=9jK-NcRmVcw'")
UPDATE:
With macOS 10.9 (Mavericks) and later, you can post notifications using plain AppleScript:
theTitle <- "A Title"
theMsg <- "A message here"
cmd <- paste("osascript -e ", "'display notification ", '"', theMsg, '"', ' with title ', '"', theTitle, '"', "'", sep='')
system(cmd)
This removes the need to install terminal-notifier, referenced below.
--
I've got terminal-notifier installed on my Mac to get desktop notifications from the command line. You can then wrap up a call to the system() command like this (change the path, obviously):
notify <- function(msgString='Message from R', titleString='Message from R', speakIt=FALSE) {
cmd <- paste('~/terminal-notifier/terminal-notifier.app/Contents/MacOS/terminal-notifier -message ', '"', msgString, '" -title "', titleString, '"', sep='')
system(cmd)
if (speakIt) {
system(paste('say', msgString))
}
}
You can call the function like this
notify("R is done", "Message from R", speakIt=TRUE)
to get a message like this:
Update: Included #VLC's say command.
Please use shell.exec("url") to open some YouTube clip on Windows
Or if you're using GNU/Linux distro and have pcspkr module blacklisted (PC speaker was always annoying me), try combining system with some auditive/visual notification, e.g.
system("aplay -t wav /usr/share/sounds/phone.wav") # for auditive bell (an I mean it literary)
system("zenity --title=\"R script info\" --text=\"Script has finished with zero exit status\" --info") # for GTK dialog
You can check zenity manual if you prefer alert in, say, notification area... But, with system function, you can do pretty much anything: send an email, run some other script, reboot the machine, sudo rm -rf *.*, etc. anything... and I mean it.
But this stands only IF you're running GNU/Linux (or UNIX) distribution, otherwise, stick to Windows specific commands, though in that case, I can't give you much info...
Inspired by beepr, this is the function I'm currently using for these kind of problems :D
work_complete <- function() {
cat("Work complete. Press esc to sound the fanfare!!!\n")
on.exit(beepr::beep(3))
while (TRUE) {
beepr::beep(4)
Sys.sleep(1)
}
}
How about playing some music?
shell.exec("foo/Born.to.be.wild.mp3")
take a look at this package: RPushBullet
An R interface to the Pushbullet messaging service which provides fast
and efficient notifications (and file transfer) between computers,
phones and tablets
RPushbullet is completely free and multi platform. As for your question, you can use this library to send a Push to your browser, but obviously it becomes amazing when you need something than can notify you while you are away.
Moreover, the creator of the R package is the same of the well known Rcpp, Dirk Eddelbuettel. I'd say it's worth a shot!
Do this:
song <- function() {
for(i in 1:2) {
for(i in 1:4) {
for(i in 1:4) {
beep(7)
Sys.sleep(0.25)
beep()
Sys.sleep(0.22)
}
beep(2)
}
beep(11)
}
beep(4)
}
song()
You can jam out to it.
How about using Windows SpeechSynthesizer?
message <- "job done!"
system2(command = "PowerShell",
args = c("-Command",
"\"Add-Type -AssemblyName System.Speech;",
"$speak = New-Object System.Speech.Synthesis.SpeechSynthesizer;",
paste0("$speak.Speak('", message, "');\"")
))
This may by nicely used in iterating operations and read something like "First job done", "Second job done" etc.:
say_something <- function(message) {
message <- paste0("$speak.Speak('", message, "');\"")
system2(command = "PowerShell",
args = c("-Command",
"\"Add-Type -AssemblyName System.Speech;",
"$speak = New-Object System.Speech.Synthesis.SpeechSynthesizer;",
message
))
}
operations <- c("1st.", "2nd.", "3rd.")
lapply(operations, function(x) say_something(message=paste(x, "job done")))
*Note, that this uses system defaul language settings. The example is based on english lector, it can be changed using SelectVoice method. To check available lectors use:
system2(command = "PowerShell",
args = c("-Command",
"\"Add-Type -AssemblyName System.Speech;",
"$speak = New-Object System.Speech.Synthesis.SpeechSynthesizer;",
"$speak.GetInstalledVoices().VoiceInfo")
)
That gives:
Gender : Female
Age : Adult
Name : Microsoft Paulina Desktop
Culture : pl-PL
Id : TTS_MS_PL-PL_PAULINA_11.0
Description : Microsoft Paulina Desktop - Polish
SupportedAudioFormats : {}
AdditionalInfo : {[Age, Adult], [Gender, Female], [Language, 415], [Name, Microsoft Paulina Desktop]...}
Gender : Male
Age : Adult
Name : Microsoft David Desktop
Culture : en-US
Id : TTS_MS_EN-US_DAVID_11.0
Description : Microsoft David Desktop - English (United States)
SupportedAudioFormats : {}
AdditionalInfo : {[Age, Adult], [Gender, Male], [Language, 409], [Name, Microsoft David Desktop]...}
Finally a function to select the lector by his "name" like "David", "Paulina" or "Hazel":
say_something <- function(message , voice) {
voice <- paste0("\"$speak.SelectVoice('Microsoft ", voice, " Desktop');" )
message <- paste0("$speak.Speak('", message, "');\"")
system2(command = "PowerShell",
args = c("-Command",
"\"Add-Type -AssemblyName System.Speech;",
"$speak = New-Object System.Speech.Synthesis.SpeechSynthesizer;",
voice,
message
))
}
operations <- c("1st.", "2nd.", "3rd.")
lapply(operations, function(x) say_something(message=paste(x, "job done"), voice="David"))
Dason's answer is great, obviously, because it will work on basically any Windows machine without requiring anything except R itself.
But one thing I have been wondering is how to make these functions actually beep after a code is run.
If you do something like this:
beepR <- function(x, n = 3){
for(i in seq(n)){
system("rundll32 user32.dll,MessageBeep -1")
Sys.sleep(.5)
}
return(x)
}
You can just pipe anything to it and it will return whatever you piped, so you can do like. See the example below.
fa <- psych::fa(x, 1) |> beepR()
I'm using here the native pipe |> that was introduced in R 4.1, but you can use the %>% pipe from the maggitr package if you like; they work the exact same way. Anyway, that will beep as soon as the analysis ends, and the variable fa will contain the results of the factor analysis.
Of course, here I used the beep function that Dason provided you, but you could just as easily add anything where I indicated below.
beepR <- function(x){
# your code here #
return(x)
}
If you wish to add that function every time RStudio opens, refer to RStudio's Managing R with .Rprofile, .Renviron, Rprofile.site, Renviron.site, rsession.conf, and repos.conf to learn how to use a .Rprofile file.
You can use notify-send command:
system("notify-send \"R script finished running\"")
Because of these many ideas, I have created a solution without Internet access, because I work with a VPN client with Windows. So it plays a typical Windows sound, which is usually on any Windows operating system.
#Function with loop, press Esc to stopp
alarm2 <- function(){
while(TRUE){
system("cmd.exe",input="C:/Windows/WinSxS/amd64_microsoft-windows-shell-sounds_31bf3856ad364e35_10.0.17134.1_none_fc93088a1eb3fd11/tada.wav")
Sys.sleep(1)
}
}
Function without loop
alarm3 <- function(){
system("cmd.exe",input="C:/Windows/WinSxS/amd64_microsoft-windows-shell-sounds_31bf3856ad364e35_10.0.17134.1_none_fc93088a1eb3fd11/tada.wav")
Sys.sleep(1)
}
The following code produces a beep and does not depend on a .mp3 or .wav file:
switch(Sys.info()[['sysname']],
Linux = {
system('play -n synth 0.1 tri 1000.0')}
)
Combining some ideas on the thread, I implement this:
options(error = function() {
if (interactive()) {
# require("beepr"); beep(10)
system("mplayer /usr/share/sounds/freedesktop/stereo/dialog-warning.oga ")
system("notify-send -u normal 'R session' 'An error occured!'")
}
})
I regularly use stop() on interactive session like Rstudio, on scripts I am working and want to re-run to a point. This way I can switch to another desktop while waiting. I may test it to '.Rprofile'

Resources