Run R web scrape daily - r

I have a script written in R that pulls online and then exports it to a spreadsheet on my computer. I am attempting to create a batch file so that I can set up a scheduled task, but it only opens up the R file without actually running it. The batch file is as follows:
"C:\Program Files\R\R-3.0.2\bin\Rscript.exe" CMD BATCH
"C:\Users\xxx\OneDrive\xxx\Scraper.r"
I am very open to ideas other than creating a batch file, I just want this to work!

Try using this instead (note the x64 addition)
"C:\Program Files\R\R-3.0.2\bin\x64\Rscript.exe" C:\Users\xxx\OneDrive\xxx\Scraper.r
and without the CMD BATCH flags. That works for me.
I am not sure what is going on, but it may be you only have the 64 bit version installed, and you are trying to execute the 32 bit version. But I get a silent fail when I don't have specify the "x64" sub-directory.

Related

Running several R scripts at once using .bat file with CMD BATCH?

I wanted to use a Windows .bat file to run several R scripts and schedule it using windows scheduler.
However, when I test using windows scheduler, everything executes but the files that the R scripts should create are never created. Instead, when I double-click on the .bat file itself it works fine. My goal is for several scripts to just run overnight without me going in and running each one of them manually one-by-one.
If I were to add a second line to the .bat file that would make it run a second script, would this execute only after the first is complete? If not, would I be able to delay the second until the first finishes somehow?? For instance, my .bat file looks like this:
"C:\Program Files\R\R-3.4.2\bin\x64\R.exe" CMD BATCH C:\Users\gma\Desktop\R_Task\script1.R
"C:\Program Files\R\R-3.4.2\bin\x64\R.exe" CMD BATCH C:\Users\gma\Desktop\R_Task\script2.R
I used the answer provided by #Gautam (R taskscheduleR not executing script) to get this far

Need help getting bash/batch to work for R on Windows 10

I'm trying to get batch mode working for R on Windows 10. The ultimate goal is to run many iterations of some R code in batch on an external server.
I successfully installed bash (unix wrapper for windows 10?) on my cmd prompt. I am working through a tutorial on using batch. I'm not sure if I want this to run through cmd or through the r code directly? https://github.com/gastonstat/tutorial-R-noninteractive/blob/master/02-batch-mode.Rmd
Via the tutorial I am working on testing batch/bash with simple code myscript1.R. Then the code I enter into cmd promp/bash looks like:
R CMD BATCH "F:/Google Drive/Documents/batch/myscript1.R" "F:/Google Drive/Documents/batch/myscript1-output.R"
Currently the closest I get in the cmd/bash is that an output file is created in the right folder but blank and I am told \usr\lib\R\bin\BATCH: cannot create myscript1-output.R: Permission denied.
I have done everything possible to allow full permissions to all users and not sure what is going on. Can anyone who knows how to use batch mode or bash in R for windows advised me?
Thank you
Answer here thanks to Phil. I did not need to use "Bash" through Ubunto... I think.
Instead, I just needed to call the CMD BATCH through regular cmd with three parts:
the directory of my R.exe: "C:\Program Files\R\R-3.5.3\bin\x64\R.exe" (replace version)
CMD BATCH
directory of the project file/script: "F:\project_folders_batch\myscript1.R"
directory of the desired output (so it doesn't default to C/users/username). In this case, I output to the same folder as the script: "F:\project_folders_batch\myscript1-output.R"
Also, in case you are outputting plots or anything (I was), go ahead and cd (change directory) to the project folder before you do this. Final result in 2 steps:
cd /d "F:\projectfolder\batch"
"C:\Program Files\R\R-3.5.3\bin\x64\R.exe" CMD BATCH "F:\projectfolder\batch\myscript1.R" "F:\projectfolder\batch\myscript1-output.R"
Also mind your antivirus... it blocked access a few times.

R script to Task Scheduler

I have an R script that gets data from databases on another server and brings it into my database. I have it saved as "dataimport.R"
I followed a few answers from here and from other websites and created a batch file like this:
"C:\Program Files\R\R-3.4.0\bin\R.exe" CMD BATCH --vanilla --slave "C:\dataimport.R"
This is not working. The cmd window opens up but the tables are not recreated and I dont get any error. I wanted to run the Task Scheduler to automate the process. Any ideas on how to fix this?
I kept at it and interestingly the answer to this was this:
"C:\Program Files\R\R-3.4.0\bin\R.exe" "C:\dataimport.R"
I dont know the reason for this but as long as it works.
I've had loads of problems with this, but finally managed to get it to work. To be more comprehensive, here are some of the things I've tried (in case one of these work for other persons):
#echo off, R CMD BATCH C:\myfolder\script.R
R CMD BATCH C:\myfolder\script.R
Using the package taskschedulerR (somehow didn't work overnight)
Used the answer provided above ("C:\Program Files\R\R-3.4.0\bin\R.exe" "C:\dataimport.R")
and all kinds of variations and combinations off these. (can't remember them all exactly)
What finally worked was:
Make an R script and save it (C:\myfolder\Test.R for example)
Through notepad, fill in: "C:\Program Files\R\R-3.5.2\bin\x64\R.exe" CMD BATCH "C:\myfolder\Test.R" (also tried Rscript.exe, didn't work for me).
in Windows Task Scheduler (v1.0) do 'Create task'
fill in time triggers.
in Actions, make an action with Start a program and in the Program/Script line provide the location where your bat script is. C:\myfolder\Test.bat
in the Start in (optional) line: enter C:\myfolder\
Note: both your .bat file and .R script are in the "C:\myfolder" folder.

Automating R to run scripts

I'm basically looking for any way to automatically run R scripts just like it would run as if I was copy and pasting it into console. I've tried the package 'taskscheduleR' however it just seems to output to a log file in the directory which isn't as if I were to just run it inside the Rstudio application.
An example might be, say I want to get the last closing stock prices of 5 stocks each night, then the script in Rstudio and have the variables there and all of the code would be in the script file.
Any thoughts?
I would suggest the in-built Task Scheduler application if you using Windows.
Create a task that will run a batchscript file. This batchscript file has only 1 line which executes the Rscript you want. Set it to run each night (or whatever time you want).
I am not that well-versed in linux and MacOS but here's what I know:
Linux has cron. Add a job to crontab with your preferred timing and execute your script 'path/to/bin/r /path/to/script.r'
MacOS has Automator + iCal (for scheduling). It also has crontab like Linux.

Unable to run R script through .bat files in Windows Server

I'm trying to run a R script through a .bat file. When I run myself the commands line by line it works. But when I try to run the .bat file, it doesn't works.
This is the .bat file
cd "C:\Program Files\R\R-3.1.2\bin"
R CMD BATCH "C:\Users\Administrator\Downloads\testa_vps.R"
This is the R script
setwd('C:\Users\Administrator\Documents')
file.create('mycsv.csv')
I'm not an expert with Windows and generally try to stick to Unix-like systems for things like this, but I have found that using programs non-interactively (e.g. via .bat files) is usually less error-prone when you add the appropriate directories to your (user) PATH variable, rather than cding into the directory and calling the executable from within the .bat file. For example, among other things, my user PATH variable contains C:\PROGRA~1\R\R-3.0\bin\; - the directory that contains both R.exe and Rscript.exe - (where PROGRA~1 is an alias for Program Files which you can use in an unquoted file path, since there are no spaces in the name).
After you do this, you can check that your PATH modification was successful by typing Rscript in a new Command Prompt - it should print out usage information for Rscript rather than the typical xxx is not recognized as an internal or external command... error message.
In the directory C:\Users\russe_000\Desktop\Tempfiles, I created test_r_script.r, which contains
library(methods)
setwd("C:\Users\russe_000\Desktop\Tempfiles")
file.create("mycsv.csv")
and test_r.bat, which contains
Rscript --vanilla --no-save "C:\Users\russe_000\Desktop\Tempfiles\test_r_script.r"
Clicking on the Windows Batch File test_r ran the process successfully and produced mycsv.csv in the correct folder.
Before running test_r.bat:
After running test_r.bat:
I've never worked with a Windows server, but I don't see why the process would be fundamentally different than on a personal computer; you just may need your sysadmin to modify the PATH variable if you don't have sufficient privileges to alter environment variables.
As already suggested by #nrussel in the comments you should use RScript.exe for this.
Create a file launcher.bat with the following content:
cd C:\Users\Administrator\Documents
Rscript testa_vps.R
In addition, add C:\Program Files\R\R-[your R version]\bin\x64; or C:\Program Files\R\R-[your R version]\bin\i386to the System PATH variable in the Environment Variables menu depending if you run R on a 64-bit or 32-bit system.
I just tested the approach above successfully on a Windows Server 2008 64-bit system and mycsv.csv got created as expected.
EDIT
One important point I forgot to mention is the following: You need to specify the path in your R file in the setwd() call using \\ instead of \.
setwd('C:\\Users\\Administrator\\Documents')
Here is a screenshot of the successful run on the Windows 2008 server:
Note: I added cmd /k to the .bat file so that the cmd window stays open after clicking on the file.

Resources