I run some calculations over several hours with R. After a while my memory is full of junk. The gc() and rm() command don't solve the problem. What I did is that I shut down my R session and opend a new one. This solves the memory problem. Now I want to automate this process. Is there a command to open a second R session or RGui form an existing session. Then I want to set the wd in this second session, run some code there and close it after some time. How can I do this? Alternatively, is there another way to get rid of the junk in my memory.
You may want to give Rscript a try, see Rscript --help from command line. Break down your big script into smaller parts and run them in succession using the same workspace with Rscript --restore --save yourscript.r. A new session of R will be opened for each script which may help you to keep memory use under control.
Related
I'm running a long R script, which takes 2 or 3 days to finish. I accidentally run another script, which, if it works as R usually does, will go in some queue and R will run it as soon as the first script is over. I need to stop that, as it would compromise the results from the first script. Is there a visible queue or any other way to stop R from running some code?
I'm working on an interactive session in R studio, on windows 10.
Thanks a lot for any help!
Assuming you're running in console (or interactive session in R studio, that's undetermined from your question) and that what you did was sourcing a script/pasting code and while it was running pasting another chunck of code:
What is ongoing is that you pushed data into R process input stream, it's a buffered input, so it will run each line once the previous line call has ended and free the process.
There's no easy way to play with an input buffer, that's R internal input/output system and mostly it's the Operating system which have those information in cache for now.
Asking R itself is not possible as it already has this buffer to read, any new command would go after.
Last chance thing: If you can spot your another chunck of code starting in your console, you can try pressing the esc key to stop the code running.
You may try messing with the process buffers with procexp but there's a fair chance to just make your R session segfault anyway.
To avoid that in the future, use scripts and run them on the command line separately with Rscript (present in R bin directory under windows too despite the link pointing to a linux manpage).
This would create one session per script and allow to kill them independently. That said if they both write to the same place (database, a file would create an error if accessed by two process) that won't prevent data corruption.
I am guessing OP has below problem:
# my big code, running for a long time
Sys.sleep(10); print("hello 1")
# other big code I dropped in console while R was still busy with above code
print("hello 2")
If this is the case, I don't think it is possible to stop the 2nd process from running.
I'm running Rstudio server and wondering if there is a way to run a command that may take a bit of time to complete and at the same time visually explore some of my environment's dataframes.
When I click on a dataframe it issues the view() command but if R is busy, it will not let me view the dataframe until the last command finishes. Is there a way to run the view command in parallel?
No.
The other thing you might be able is if you have the Pro version generate a parallel session
I'm currently running some queries to a database and getting back some big files as a result. I have encountered the common problem of Windows not freeing the memory, even though I 'rm()' everything and (edit) calling 'gc()'. One workaround i have found is using .rs.restartR() in Rstudio.
This though requires me to constantly watch my script, in order to continue it after the session restart. Is it possible to automate it? If not what other methods do people use to overcome this problem?
You could break the code into 2 files and write a batch file (.bat) that runs the first file through .rs.restartR() and then the remainder of the code in the next file.
You could also skip the .bat and just schedule both .R scripts to run in Task Scheduler.
Also, please see my comment regarding garbage collection (gc()).
I want to execute a R script in the background from the R console.
From the console , i usually run R script as source('~/.active-rstudio-document')
I have to wait until the script is completed to go ahead with my rest of work.
Instead of this i want R to be running in the background while i can continue with my work in the console.
Also i should be somehow notified when R completes the source command.
Is this possible in R ?
This might be quite useful as we often sees jobs taking long time.
PS - i want the source script to be running in the same memory space rather than a new one. Hence solutions like fork , system etc wont work for me. I am seeing if i can run the R script as a separate thread and not a separate process.
You can use system() and Rscript to run your script as an asynchronous background process:
system("Rscript -e 'source(\"your-script.R\")'", wait=FALSE)
At the end of your script, you may save your objects with save.image() in order to load them later, and notify of its completion with cat():
...
save.image("script-output.RData")
cat("Script completed\n\n")
Hope this helps!
I've found a lot of answers on how to run R from a Batch file, but nothing about running a Batch File from R. I know one way to do this is to use system, system2 or shell, but these methods wait for process in the Windows Command Prompt to finish before R accepts another input. I want to run a Batch File which calls a console application that runs indefinitely, and then allow R to do other things. Any help would be greatly appreciated.
The help page ?shell says how to do it. Just run
shell("MyScript.bat", wait=FALSE)