keep Shell / cmd open when launching R script from VBA - r

I have an R script that is being run from an Excel workbook via a button that is linked to a VBA script.
The problem is every time the R script encounters an error - say it does not find one of the files it was supposed to read - the Shell / cmd window the R script is running in closes down instantly. The problem with that is you cannot see any clues as to why it failed. I then have to debug it manually by modifying the code and running in RStudio to find the errors - usually I have to do this for other people that do not know R.
The VBA code I use was copied and modified from one of the SO posts I found here. I am not very proficient in VBA and do not understand a lot of the code so I'm looking for a simple solution to this. My VBA code is this:
Option Explicit
Public Sub RunRscript()
ActiveWorkbook.Save
Dim shell As Object
Set shell = VBA.CreateObject("WScript.Shell")
Dim waitTillComplete As Boolean: waitTillComplete = True
Dim style As Integer: style = 1
Dim scriptPath As String
scriptPath = Range("F5").Value
Dim argument As String
argument = Range("F3").Value
Dim path As String
path = """C:\Program Files\R\R-3.4.2\bin\Rscript.exe"" """ & scriptPath & """ """
& argument & """"
ActiveWorkbook.Save
Dim errorcode As Integer
errorcode = shell.Run(path, style, waitTillComplete)
ActiveWorkbook.Save
End Sub
scriptPath points to the path of the R script and argument is an argument I pass to the R script.
I have tried passing the usual arguments to keep the cmd window open but I didn't manage to figure it out. This is also complicated by the fact that as mentioned I do not understand VBA syntax very well and the 'path' variable being very convoluted because of those endless double quotes.

You might use
errorcode = shell.Run("cmd /k " & path, style, waitTillComplete)
instead of
errorcode = shell.Run(path, style, waitTillComplete)
We open a new instance of the command-line tool, thus can use its parameters. The parameter /k keeps the cmd-window open (use /c to close). After the cmd-parameter we add the script to be executed.
But there is one problem: if you close the cmd-window, VBA throws an overflow error. I use a flag in my Excel sheet to switch between debug mode and normal mode:
debugging = Range("N5").Value 'True/False
If debugging Then
errorCode = shell.Run("cmd /k " & path, style, waitTillComplete)
Else
errorCode = shell.Run(path, style, waitTillComplete)
End If
If the script fails I rerun it setting debug mode to True in a cell of the Excel sheet.

Based on this comment:
#FreeMan, the output of the R script is actually a series of Excel workbooks that it opens, prints data in and then saves. it reads a list with the paths of all the files it needs to read as an input to generate the output data required to print in excel. Sometimes, one of the paths in that list is wrong, so the program stops halfway and there is no clue as to why for the user. (the reason could be something else as well). It would be good if I could pass the text that is generated in shell / cmd automatically to a notepad / textpad file at least, so the user could then open and check
It sounds like the best bet may be to have your R script validate the paths/files prior to attempting to do its processing. I'm barely familiar with R, but what I know of it there should be a library somewhere that will allow you to test for the existence of a path and/or a file on that path. If it doesn't exist, the script would simply put an error message in the file it was supposed to create/update indicating that it couldn't find the requested path/file, or it could write that message to an error log that could be opened by your VBA code at the completion of its processing.

Related

Passing arguments to R (not RScript) through command line/VBA

I'm developing an economic model in Excel and R. It is composed by two parts: an Excel/VBA shell, as a framework for the front-end user interface; and an Rscrit file used as a back end with R and performing all the calculations by means of the deSolve package.
I'm passing the necessary arguments from Excel/VBA by means of the following code I have found in http://shashiasrblog.blogspot.com/2013/10/vba-front-end-for-r.html
For the Excel/VBA:
Sub RunRscript()
'runs an external R code through Shell
'The location of the RScript is 'C:\R_code'
'The script name is 'hello.R'
Dim shell As Object
Set shell = VBA.CreateObject("WScript.Shell")
Dim waitTillComplete As Boolean: waitTillComplete = True
Dim style As Integer: style = 1
Dim errorCode As Integer
Dim var1, var2 As Double
var1 = Sheet1.Range("D5").Value
var2 = Sheet1.Range("F5").Value
Dim path As String
path = "RScript C:\R_code\hello.R " & var1 & " " & var2
errorCode = shell.Run(path, style, waitTillComplete)
End Sub
And for the R code:
# Accepts two numbers and adds them
# Re-directs the console output to a file 'hello.txt'
# The file is created in the directory 'C:\R_code'
args<-commandArgs(trailingOnly=T)
# cat(paste(args,collapse="\n"))
sink('C:/R_code/hello.txt',append=F,type="output")
cat('Hello World')
var1<-as.numeric(args[1])
var2<-as.numeric(args[2])
cat('\nThe result of adding',var1,'to',var2,'is',var1+var2)
sink(NULL)
It works fine, but the only drawback is that it takes almost an hour to do the calculations in the background and the only thing the user is able to see is the black screen from the command prompt without any other information on the steps the model is running and the progress of the model calculations.
What I'm intending to do is something similar to what this VBA code is doing:
Sub ShellAndWait(pathFile As String)
With CreateObject("WScript.Shell")
.Run pathFile, 1, True
End With
End Sub
'Example Usage:
Sub demo_Wait()
ShellAndWait ("R.exe")
Beep 'this won't run until Notepad window is closed
MsgBox "Done!"
End Sub
This is to say. Open the R terminal in the command prompt.
Now the only missing action is to run the Rcode in the R terminal opened in the command prompt.
Any ideas on how to do it, so the user would be able to see the progress during the execution of the model?
Kind regards,
Daniel

R Script execution from inside VBA using Shell not working

I have an R script that is working perfectly when I execute it from R Studio, but when I am trying to run it using a VBA code that calls Shell, the code runs, the command window shows up and closes, but it does not generate the result file. It does not throw any error. Can someone see what the problem is?
This is the address of the folder that has the Rscript.exe file: C:\Program Files\R\R-3.4.4\bin\x64\
VBA Code:
Sub RunRScript()
Dim shell As Object
Dim waitTillComplete As Boolean: waitTillComplete = True
Dim style As Integer: style = 1
Dim errorCode As Integer
Dim path As String
Set shell = VBA.CreateObject("WScript.Shell")
path = """C:\Program Files\R\R-3.4.4\bin\x64\RScript"" C:\Ibos\R\WF_Metrics\abc.R"
errorCode = shell.Run(path, style, waitTillComplete)
End Sub
R Script:
library(RODBC)
library(dplyr)
#library(data.table)
library(tidyr)
library(tictoc)
library(tidyverse)
library(lubridate)
library(xlsx)
library(sqldf)
#set working directory
setwd("C:/Ibos/R/WF_Metrics")
my_server="servername"
my_db="dbname"
my_username="username"
my_pwd="password"
db <- odbcDriverConnect(paste0("DRIVER={SQL Server};
server=",my_server,";
database=",my_db,";
uid=",my_username,";
pwd=",my_pwd))
sql="select * from dbo.metricsfinal"
df <- sqlQuery(db,sql)
myfile="results"
write.csv(df, file = paste0(myfile,"_test",".csv") ,row.names=FALSE)
Edit:
After Oliver's answer and some helpful comments by other, I found out that the problem is the xlsx package. Now I need to figure out how to resolve this issue. I prefer to use this package rather than look for other packages/options, any help is appreciated. Here is the error:
Error: package or namespace load failed for 'xlsx': .onLoad failed in
loadNamespace() for 'rJava', details: call:
dirname(this$RuntimeLib) error: a character vector argument
expected Execution halted
Running R from VBA is an annoying endeavor. If you want R code to run directly from VBA, i suggest looking into BERT, which is an open source add-in for Excel, letting you run and write your R code directly from excel, making debugging your code much much simpler.
That said if you are stuck with shell.run, there are some things you can do to locate your problem.
Manual debugging
In VBA either set a breakpoint or print your path to your console.
path = """C:\Program Files\R\R-3.4.4\bin\x64\RScript"" C:\Ibos\R\WF_Metrics\abc.R"
debug.print path
Open your command prompt (press the windows button and write cmd, press enter start, cmd, enter.)
Paste the line of code into your command prompt. Any error will be printed into the command line, and from this you can locate and correct the error in your script.
Less manual debugging
Now the manual debugging can be tedious and as i note below it is system specific. There are several options to automate the process slightly. These include:
Read the error code directly into VBA, pipe in any output to VBA.
Use an open source integration like BERT to let you write and debug your code somewhat directly in Excel.
Use System error messages to identify the error
The first option is the more complicated, but also a very versatile and nice option. I suggest the first answer here, which gives a link to a VBA module that can achieve this method. Note however it is a 32 bit module, and it will need a few ptrsafe markers, for the windows api to work on a 64 bit installation of excel. With a few changes it could even be used to read text output (data.frame etc) from R directly, with minimal interfering with Excel.
For the second option i suggest looking at the BERT webpage, which provides good guides for utilizing the implementation. This has the disadvantage that any computer will need to have installed BERT for your excel script to work, in addition to R being installed.
The third option is one inspired by Chip Pearson's site. When your script crashes it sends an error code to the command line, and this can be interpreted by the windows error code manager. This has the advantage that it is simple, and you will quickly be made aware if your script 'does not exist' or similar common mistakes that are not R specific.
Using this method one would change the R execution script to something like
Sub RunRScript()
Dim shell As Object
Dim waitTillComplete As Boolean: waitTillComplete = True
Dim style As Integer: style = 1
Dim errorCode As Integer
Dim path As String
Set shell = VBA.CreateObject("WScript.Shell")
path = """C:\Program Files\R\R-3.4.4\bin\x64\RScript"" C:\Ibos\R\WF_Metrics\abc.R"
errorCode = shell.Run(path, style, waitTillComplete)
if errorCode <> 0 then
errorString = GetSystemErrorMessageText(errorCode)
Err.Raise errorCode, "Run_R_Script", errorString
end if
End Sub
where GetSystemErrorMessageText(errorCode) is a call to the function in a seperate module below.
#If Win64 Then
Private Declare PtrSafe Function FormatMessage Lib "kernel32" Alias "FormatMessageA" ( _
ByVal dwFlags As Long, _
ByVal lpSource As Any, _
ByVal dwMessageId As Long, _
ByVal dwLanguageId As Long, _
ByVal lpBuffer As String, _
ByVal nSize As Long, _
ByRef Arguments As Long) As Long
#Else
Private Declare Function FormatMessage Lib "kernel32" Alias "FormatMessageA" ( _
ByVal dwFlags As Long, _
ByVal lpSource As Any, _
ByVal dwMessageId As Long, _
ByVal dwLanguageId As Long, _
ByVal lpBuffer As String, _
ByVal nSize As Long, _
ByRef Arguments As Long) As Long
#End If
Public Function GetSystemErrorMessageText(ErrorNumber As Long) As String
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
' GetSystemErrorMessageText
'
' This function gets the system error message text that corresponds
' to the error code parameter ErrorNumber. This value is the value returned
' by Err.LastDLLError or by GetLastError, or occasionally as the returned
' result of a Windows API function.
'
' These are NOT the error numbers returned by Err.Number (for these
' errors, use Err.Description to get the description of the error).
'
' In general, you should use Err.LastDllError rather than GetLastError
' because under some circumstances the value of GetLastError will be
' reset to 0 before the value is returned to VBA. Err.LastDllError will
' always reliably return the last error number raised in an API function.
'
' The function returns vbNullString is an error occurred or if there is
' no error text for the specified error number.
'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
Dim ErrorText As String
Dim TextLen As Long
Dim FormatMessageResult As Long
Dim LangID As Long
''''''''''''''''''''''''''''''''
' Initialize the variables
''''''''''''''''''''''''''''''''
LangID = 0& ' Default language
ErrorText = String$(FORMAT_MESSAGE_TEXT_LEN, vbNullChar)
TextLen = FORMAT_MESSAGE_TEXT_LEN
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
' Call FormatMessage to get the text of the error message text
' associated with ErrorNumber.
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
FormatMessageResult = FormatMessage( _
dwFlags:=FORMAT_MESSAGE_FROM_SYSTEM Or _
FORMAT_MESSAGE_IGNORE_INSERTS, _
lpSource:=0&, _
dwMessageId:=ErrorNumber, _
dwLanguageId:=LangID, _
lpBuffer:=ErrorText, _
nSize:=TextLen, _
Arguments:=0&)
If FormatMessageResult = 0& Then
''''''''''''''''''''''''''''''''''''''''''''''''''
' An error occured. Display the error number, but
' don't call GetSystemErrorMessageText to get the
' text, which would likely cause the error again,
' getting us into a loop.
''''''''''''''''''''''''''''''''''''''''''''''''''
MsgBox "An error occurred with the FormatMessage" & _
" API function call." & vbCrLf & _
"Error: " & CStr(Err.LastDllError) & _
" Hex(" & Hex(Err.LastDllError) & ")."
GetSystemErrorMessageText = "An internal system error occurred with the" & vbCrLf & _
"FormatMessage API function: " & CStr(Err.LastDllError) & ". No futher information" & vbCrLf & _
"is available."
Exit Function
End If
''''''''''''''''''''''''''''''''''''''''''''''''''''''
' If FormatMessageResult is not zero, it is the number
' of characters placed in the ErrorText variable.
' Take the left FormatMessageResult characters and
' return that text.
''''''''''''''''''''''''''''''''''''''''''''''''''''''
ErrorText = Left$(ErrorText, FormatMessageResult)
'''''''''''''''''''''''''''''''''''''''''''''
' Get rid of the trailing vbCrLf, if present.
'''''''''''''''''''''''''''''''''''''''''''''
If Len(ErrorText) >= 2 Then
If Right$(ErrorText, 2) = vbCrLf Then
ErrorText = Left$(ErrorText, Len(ErrorText) - 2)
End If
End If
''''''''''''''''''''''''''''''''
' Return the error text as the
' result.
''''''''''''''''''''''''''''''''
GetSystemErrorMessageText = ErrorText
End Function
Credit goes to Chip Pearson, although it was likely not intended for this use.
Executing shell.run(path) where path is a command to execute an R script, will now return an error message if it fails.
This does not remove manual debugging in its entirety, but the error message will help you identify errors outside of R, and often gives valuable error descriptions that will help you identify the error faster during manual debugging.
This method should thus be followed by manual debugging to the degree it is necessary.
Notes:
Running R scripts from your command line is system encoding specifically. Saving your R scripts using 'Save with encoding' and using 'System.locale' removed most of our problems when running from the command line. It might fail on UTF8 characters, however, which is a whole other hassle.
The third option is the simplest, and therefore also the least versatile. Error codes from the windows system only gives a glance into your problem and does not specify which line failed, only that Invalid function or similar was called. There is no guarantee that this error will be exactly the correct one. It does, however, give you most common mistakes like wrong paths, functions simply not working, variables called that do not exist etc.
I hope this will give some clarity and help you find your error. I suggest searching on google, the topic of using shell.run for various integrations is a topic that has been investigated and it if better options exist it is often recommended to be avoided (due to the limitations). It is however often a good place to start. Many sites like this one, shows how one can use the output from R in VBA, by saving your output to text files in R and reading from VBA. Better options do exist, but this is likely the simplest method.
Update by the Asker:
After a lot of investigation I realized the following points:
1- xlsx package uses rJava package and you need to install Java first
2- Java's 32-bit or 64-bit version has impacts on whether the rJava package and subsequently xlsx package can be loaded successfully or not. I recommend installing a 64-bit version of everything (Excel, R, Java) if you have a 64-bit windows OS.
3- It seems by default R is installed both 32-bit and 64-bit version so you have to specify which version you want to use. Choose the version here on RStudio:
Tools > Global Options > R Version
mine by default was 32-bit and Library(rJava) was throwing the error even on RStudio after I installed a new version of Java. I changed it to 64-bit and it worked.
You can use R scripts to do this too, but it seems if you run this code from inside RStudion it won't work since you need to manually change the R Version and close RStudion and launch it again to take effect
Sys.setenv(JAVA_HOME='C:\\Program Files\\Java\\jre7') # for 64-bit version
Sys.setenv(JAVA_HOME='C:\\Program Files (x86)\\Java\\jre7') # for 32-bit version
library(rJava)
4- I noticed that I have both 32-bit and 64-bit Java installed on my machine. Once I did JAVA_HOME to the environment variables, it caused a jvm.dll missing error so I deleted it and everything came back to working fine again
5- I saw in some of the post if you use another alternative for xlsx package you would not need to go through all of the hassle of making sure everything is working fine together.

Confused by difference between running R from batch file versus running it directly from shell

I recently came across a strange error, and while I managed to fix it, I cannot for the life of me figure out what was going wrong behind the scenes and I would like to understand what was going on. I am currently writing a program that takes Excel inputs and uses them to direct a program written in R, that ultimately spits out results to be fed back into Excel for user review.
My original solution utilized a batch file that consisted of a single line, calling:
Rscript "Rfilepath.R"
that was called from VBA using the below:
Dim wsh As Object
Set wsh = VBA.CreateObject("WScript.Shell")
Dim waitOnReturn As Boolean: waitOnReturn = True
Dim windowstyle As Integer: windowstyle = 1
wsh.Run Chr(34) & BatchFilePath & Chr(34), windowstyle, waitOnReturn
However, this proved unable to run the below R code (extremely simplified):
library(ggplot2)
library(gridExtra)
library(ggpubr)
x <- seq(1,10)
y <- seq(11,20)
z <- seq(6,15)
a <- ggplot(data.frame(cbind(x,y)),aes(x=x,y=y))+geom_point()
b <- ggplot(data.frame(cbind(y,z)),aes(x=y,y=z))+geom_point()
c <- ggplot(data.frame(cbind(x,z)),aes(x=x,y=z))+geom_point()
test <- ggarrange(ggarrange(a,b,nrow=2),c,ncol=2)
ggsave(file="filepath.png",plot=test)
The problem resulted when the ggarrange function was called. However, I was able to get this situation working by doing the following in VBA (essentially cutting out the batch file step):
Set wsh = VBA.CreateObject("WScript.Shell")
Dim waitTillComplete As Boolean: waitTillComplete = True
Dim style As Integer: style = 1
Dim errorCode As Integer
errorCode = wsh.Run("Rscript " & Chr(34) & RPath & Chr(34), style, waitTillComplete)
What is the difference between these two approaches and why did one work while the other one did not? It was difficult to debug due to the batch terminal immediately closing on error (probably due to running it from VBA). Any tips or recommendations for debugging these types of issues would also be greatly appreciated. Thanks!
I think you have a problem with PATH of the RScript in case one : it is lost in the double batch call.

How to keep a shell window open for the lifespan of a workbook

I can call a script in R from VBA by doing
Sub CallRScript()
Dim myShell As Object
Set myShell = VBA.CreateObject("WScript.Shell")
Dim waitTillComplete As Boolean, style As Integer, path As String
waitTillComplete = True
style = 1
path = "RScript C:\R_code\Scripts\ValuationFns.R"
Call myShell.Run(path, style, waitTillComplete)
End Sub
However, the R session closes as soon as the R script is finished running. I'd like for the R session to remain open until the excel workbook is closed, an be able to run subsequent commands in the command prompt. How might I accomplish this?

Wait for each R script to end before running next R script called by VBA code

I have a report with a front-end in Excel (inherited files). There are multiple buttons calling VBA code which then call either SQL queries, stored procedures or R script files (in the form of rcmd batch/shell, passing some parameters etc.)
R scripts are called from VBA one after another. All need some unspecified time to finish.
There are applications/wait-time in the VBA to 'ensure' that the R script is finished before next step is initiated.
This does not always work (depends on the data size etc.).
Is there a way for / code to be put at the end of the R script that would notify VBA it is time to go on?
Or for VBA to check if the specific R script is still running?
Based on #z feedback & Wait for Shell to finish, then format cells - synchronously execute a command thread.
You can use approach WScript.Shell object with Run method with waitOnReturn option set:
Please see the example of the code for Excel function:
Dim wsh As Object
Set wsh = VBA.CreateObject("WScript.Shell")
Dim waitOnReturn As Boolean: waitOnReturn = True
Dim windowStyle As Integer: windowStyle = 1
wsh.Run Chr(34) & "C:\Program Files\R\R-3.5.0\bin\rscript" & Chr(34) & " " & Chr(34) & "C:\Users\demo_user\test.R" & Chr(34), windowStyle, waitOnReturn
test.R contents:
rnorm(10000)

Resources