I have an Rscript I would like to run. It takes a very long time to run (took about 23 hours on a 64gb 32 node cluster). It keeps running into an error at this time. My boss told me to run the script locally on my macbook 4gb ram and use options(error=recover). This would stop the code at the error but allow me to enter R at the point of error and play around? Is that correct?
Anyways, I tried using Rscript options(error=recover) /path/to/file.R but I got can error -bash: syntax error near unexpected token "(". What is the proper syntax? I can't find it online...
Related
I am trying to run an executable called swat_edit.exe in R. It works perfectly when I run it directly in the command prompt, and also when I run it directly in the Terminal tab in R. However, when I try to write a function in R to run the executable, I get an error (I get a number of different errors...).
I have tried to use different methods of running the file:
1: I used system("swat_edit"), which returns the following error:
Unhandled Exception: System.IO.IOException: The handle is invalid.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.Console.set_CursorVisible(Boolean value)
at SWEdit.Program.Run(String[] args)
at SWEdit.Program.Main(String[] args)
[1] 17234
2: I used shell("swat_edit"), which returns the exact same error as (1).
3: I used shell.exec("swat_edit"). This works, but it opens the executable in a new window, which then runs for a few seconds and closes (as intended). I need the program to run in the R terminal window so it can run many iterations in the background without disrupting other things. This is not a viable option.
4: I tried using terminalSend(ID,"swat_edit") (from the rstudioapi package). This works in that it sends the command to the terminal window in R. When I move there and hit enter it executes perfectly, running in the terminal window like I want it to. However, I need to run many iterations so this is not viable either. I tried using KeyboardSimulator to go to the Terminal tab and hitting enter (which worked), but this also does not let me use the PC for other purposes while running my code.
5: I tried using terminalExecute("swat_edit"), which returns the following error code:
Error calling capture_console_output: 87
[Process completed]
[Exit code: -532462766]
6: I tried making a python file that runs swat_edit.exe, and then running that file in R. The python file works when I run it by itself, from the command prompt, or from the terminal in R. It does not, however, work when I try to run it in the R terminal using terminalExecute (same error as in (5)).
NOTE: I have another executable called swat.exe (entirely different program) that works with all of the above-mentioned methods.
So in summary: swat_edit.exe runs perfectly in command prompt and R terminal, but does not work when I try to run it using R code (either system(), shell(), or terminalExecute().
I can't figure out the difference between terminalExecute() and typing the string into terminal and hitting enter, but apparently there is something happening in between...
It will be tedious to reproduce this since it uses external programs, but if anyone has any idea about the error messages or how I can copy a string and run it in the terminal without any interference, that would be greatly appreciated.
EDIT: I found a method that solves my problem. I created a .bat file that runs swat_edit minimized. I was able to run this .bat file with the shell function (or any of the other commands I mentioned) in R. This doesn't answer why I was having the issues I described, and it doesn't let me run swat_edit in the R terminal, but it's good enough for me.
The .bat file was simply the following:
"START /MIN /WAIT C:\~\SWAT_Edit.exe"
I have a R script (script.R) that loads 25-30K documents in elasticsearch in each execution.
The point is that I can execute it in Rstudio properly. However, when I try to execute it from command line using Rscript I always get the same error:
Error: 400 - failed to parse
In addition: There were 50 or more warnings (use warnings() to see the first 50)
Execution halted
The strangest thing is that when this error occurs there are loaded different amount of documents in elastic (sometimes 1.5K, sometimes 3K, etc...). So it seems that it doesn't occurs always at same time.
Do you know what's happening? This is the Rscript execution:
/usr/bin/Rscript /Rdir/script.R
Thanks!
Finally I solved the issue using elastic::docs_bulk function instead of elastic::docs_create. It seems to work better with a huge amount of documents in elastic.
This is the first time I have tried to connect R and Tableau.
I have downloaded and installed Rserve successfully but every time I try to start Rserve is get this warning:
Starting Rserve...
"C:\Users\SIMON~1.HAR\DOCUME~1\R\WIN-LI~1\3.1\Rserve\libs\x64\Rserve.exe"
Warning message:
running command '"C:\Users\SIMON~1.HAR\DOCUME~1\R\WIN-LI~1\3.1\Rserve\libs\x64\Rserve.exe" ' had status 127
I have been searching for days and couldn't find any fix.
The Rserve() function is trying to start an application (Rserve.exe) and failed. There's a couple of things you can do.
Go to the "C:\Users\SIMON~1.HAR\DOCUME~1\R\WIN-LI~1\3.1\Rserve\libs\x64\" file directory and try loading the exe yourself, troubleshoot from there.
use the run.Rserve() function instead of Rserve(). This will use your current R session to start the Rserve server. This means the exe doesn't need to be run. This worked well for me because I am working in an environment where I don't enough privileges to run the exe. It does mean that your R session can't do anything else while the server is running, but you can always load up 2 sessions at the same time.
I'm really just looking to pick the community's brain for some leads in figuring out what is going on with the issue I'm having.
I'm writing a MR job with RHadoop (rmr2, v3.0.0) and things are great -- IO with HDFS, mapping, reducing. No problems. Life is great.
I'm trying to schedule the job with Apache Oozie, and am running into some issues:
Error in mr(map = map, reduce = reduce, combine = combine, vectorized.reduce, :
hadoop streaming failed with error code 1
I've read the rmr2 debugging guide, but nothing is really getting to the stderr because the job fails before anything even gets scheduled.
In my head, everything points to a difference in environments. However, Oozie is running the job as the same user that I'm able to run everything with via cli, and all of the R environment variables (fetched with Sys.getenv()) are the same, excepting there's some additional class path stuff set with Oozie.
I can post more of the OS or Hadoop versions and config details, but sleuthing some version-specific bugs seems like a bit of a red herring as everything runs fine at the command line.
Anybody have any thoughts what might be some helpful next steps in hunting this beast down?
UPDATE:
I overwrote the system function in the base package to log the user, the host name of the node, and the command being executed before the internal call to system. So before any system call is actually executed, I get something like the following in the stderr:
user#host.name
/usr/bin/hadoop jar /usr/lib/hadoop-mapreduce/hadoop-streaming-2.2.0.2.0.6.0-102.jar ...
When ran with Oozie, the command printed in the stderr fails with an exit status of 1. When I run the command on user#host.name, it runs successfully. So essentially the EXACT same command with the SAME user on the SAME node fails with Oozie, but runs successfully from cli.
I've tried to run command in R 2.15.2
rsaga.geoprocessor(lib="ta_channels", module=0, param=list(ELEVATION="DEMflt.sgrd", CHNLNTWRK=paste("channels", i, ".sgrd", sep=""), CHNLROUTE="channel_route.sgrd", SHAPES="channels.shp", INIT_GRID="DEMflt.sgrd", DIV_CELLS=3, MINLEN=40), show.output.on.console=FALSE)
and I'm constantly getting this warning:
Warning message:
running command '"C:/Users/Nenad/Documents/R/win-library/2.15/RSAGA/SAGA-GIS/saga_cmd.exe" ta_preprocessor 2 -DEM "DEM1.sgrd" -RESULT "DEMflt.sgrd" -MINSLOPE "0.05"' had status 1
I use windows 8 and also tried to ran R as admin.
Any idea what is the problem? Thanks!
Idk how actual it is, but I've been struggling with "had status 1" warning a lot. Especially it got really annoying when I tried to use seasonal package to conduct the X13-ARIMA-SEATS seasonal decomposition of time series. The seasonal::seas command just didn't work, because in the code of this command there is a stop condition when running a certain stuff with cmd.exe returns non-zero status. While, as it was mentioned before, 'status 1' doesn't prevent command execution, in case of seasonal package it does.
The problem in my case was caused by some mistake in Windows Registry (Win 10), that in turn caused warning System cannot find the path specified when launching CMD.exe or PowerShell, which caused warning inside R as well. So to fix it:
Press Win+R -> regedit
In HKEY_CURRENT_USER\SOFTWARE\Microsoft\Command Processor folder clean the value of Autorun record (it should be empty)
Do the same for Autorun in HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Command Processor
After these actions annoying warning with "has status 1" has gone and seasonal::seas started to work perfectly.
Hope it will be usefull for somebody.
Running system commands from R can be really tricky. In my experience, as long as the exit code is not 127 then the command did run, and you could use the intern=TRUE switch in the system command for a more verbose output. If you run the command again, the warning message could contain a errmsg attribute as well for some more info. hth