I am running an R script on databricks notebook for multiple datasets (around 500). I ordered datasets by file size to avoid errors and run the max amount of files within the shortest time because the script has large time complexity.
I was able to finish 400/500 datasets without any issues but the large files keep giving the error:
RserveException: eval failed
eval failed
The weird thing about the error is that sometimes when I run the notebook again it works without any issues for the same dataset. However, 99% of the time I get the same error for bigger files. There is no error code or any explanation when expanding the error code. I researched this problem and most people have this error with an error code, and as far as I understand it is something to do with R version or some of the libraries I installed (cluster scoped) but I cannot figure it out.
Any ideas?
Error: no more error handlers available (recursive errors ) invoking 'abort' restart
Error: option error has NULL value
whenever I try running any piece of code in R markdown. I didn't have any issues using R markdown last week. The only thing between then and now is that I ran a lot of data analysis on very large data frames(16M points) in another R script, which gave multiple warnings() which I ignored. Does that have anything to do with it? I quit Rstudio, tried restarting, tried clearing garbage with gc() but nothing works. Can't find much on this error on google.
I have an Rscript I would like to run. It takes a very long time to run (took about 23 hours on a 64gb 32 node cluster). It keeps running into an error at this time. My boss told me to run the script locally on my macbook 4gb ram and use options(error=recover). This would stop the code at the error but allow me to enter R at the point of error and play around? Is that correct?
Anyways, I tried using Rscript options(error=recover) /path/to/file.R but I got can error -bash: syntax error near unexpected token "(". What is the proper syntax? I can't find it online...
Our automated build process incorporates a grunt task that periodically (more times than I'd like) generates a 6 return code.
According to the grunt web page 6 is a "Warning". Well okay, a warning for what? It's breaking our build since we would only pass on a 0 exit case. I'm looking at wrapping this in a script so I can catch this warning condition and generate a success exit code, though without any idea what the Warning might be I'm hesitant. It appears to work when I get this exit condition but would like a better understanding on what it may be.
Any ideas?
So this turned out to be an issue with grunt-contrib-less. I didn't recognize the associated less compiler error in our build log:
[39mnon_object_property_loadError: Cannot read property 'rules' of undefined in ../../XXXX/styles/modules/SomeLessFile.less on line null, column 0: [31m
Once I found this line a google search quickly found this in GitHub:
And have updated the grunt-contrib-less package. So far so good.
check http://gruntjs.com/api/grunt.fail:
If --stack is specified on the command-line and an error object was
specified, a stack trace will be logged.
Or try to run "grunt --verbose --force"
I've often been frustrated by R's cryptic error messages. I'm not talking about during an interactive session, I mean when you're running a script. Error messages don't print out line numbers, and it's often hard to trace the offending line, and the reason for the error (even if you can find the location).
Most recently my R script failed with the the incredibly insightful message: "Execution halted." The way I usually trace such errors is by putting a lot of print statements throughout the script -- but this is a pain. I sometimes have to go through the script line by line in an interactive session to find the error.
Does anyone have a better solution for how to make R error output more informative?
EDIT: Many R-debugging things work for interactive sessions. I'm looking for help on command-line scripts run through Rscript. I'm not in the middle of an R session when the error happens, I'm at the bash shell. I can't run "traceback()"
Try some of the suggestions in this post:
General suggestions for debugging in R
Specifically, findLineNum() and traceback()/setBreakpoint().
#Nathan Well add this line sink(stdout(), type="message") at the beginning of the script and you should get in console message both script content and output along with error message so you can see it as in interactive mode in the console. (you can then also redirect to a log file if you prefer keeping the console "clean")
Have a look at my package tryCatchLog (https://github.com/aryoda/tryCatchLog).
While it is impossible to improve the R error messages directly you can save a lot of time by identifying the exact code line of the error and have actual variables at the moment of the error stored in a dump for "post mortem" analysis!
The main advantages of the tryCatchLog function over tryCatch are
easy logging of errors, warnings and messages into a file or console
warnings do not stop the program execution (tryCatch stops the execution if you pass a warning handler function)
identifies the source of errors and warnings by logging a stack trace with a reference to the source file name and line number (since traceback does not contain the full stack trace)
allows post-mortem analysis after errors by creating a dump file with all variables of the global environment (workspace) and each function called (via dump.frames) - very helpful for batch jobs that you cannot debug on the server directly to reproduce the error!
This will show a more detailed traceback, but not the line number:
options(error = function() {traceback(2, max.lines=100); if(!interactive()) quit(save="no", status=1, runLast=T)})
One way inside a script to get more info on where the error occurred is to redirect R message to the same stream as errors :
sink(stdout(), type="message")
This way you get both messages and errors in the same output so you see which line raised the error...