Problem Version 1, Can we make pr_fun process it's retun without waiting for ch_fun() to finish
ch_fun <- function() {Sys.sleep(10)}
pr_fun <- function() {ch_fun(); return("Done")}
pr_fun()
Proble Actual Version
R session 1 as svSocket Server
library(svSocket)
startSocketServer(port = 9875,local=FALSE)
R session 2 as svSocket client
con <- socketConnection(port = 9875,host="127.0.0.1")
evalServer(con,"Sys.sleep(20)")
R session 3 as svSocket client
con <- socketConnection(port = 9875,host="127.0.0.1")
evalServer(con,"a=10")
If we run the code lines for session 2 and while server is processing Sys.sleep call we quickly put the code lines for session 3 in session 3 and abort the call it still gets processed. We can check that on server side by checking if object "a" was created.
My point is we didn't have to wait for job to finish in session 3 still it was processed so somehow jobs were piled up on session side and we don't have to wait for jobs to finish just send them to server and abort the waiting process and move ahead. We can manually abort using Ctrl+C or Esc but how can I do that in a function. I want pr_fun to call ch_fun in server session and proceed to its return immediately.
Related
I sent API requests to get the address from a list of approximately 80,000 APIs.
I used the Google Maps API and the process finished, since the last request was 6 hours ago.
library(ggmap)
locations <- vector(mode = "list", length = nrow(data))
for(i in 1:nrow(data)){
gc <- c(data[i, 2], data[i, 1])
locations[i] <- revgeocode(as.numeric(gc))
}
It is still printing out the messages received from each API request and I'd like to use my R session to work on other problems. I can't terminate the session since the data only exists in my local environment and I haven't yet written it to a CSV or JSON file.
Here's what I've tried: I pressed stop a number of times.
Here's what I'd like to try: I could turn off my RStudio Server instance or I could turn off my EC2 instance that is hosting RStudio Server.
My concern is those will terminate the environment and cause me to lose the variables.
Do I have to just wait it out?
Context
In order to test the web capabilities of an R package I am writing, I'm attempting to serve a file locally use the httpuv package so that I can run tests using an offline copy of the page.
Issue
However, curl doesn't seem to want to play nice with httpuv - specifically, when trying to read the hosted file using curl (for example, with curl::curl() or curl::curl_fetch_memory()), the request hangs, and eventually times out if not manually interrupted.
Minimal example
# Serve a small page
server <- httpuv::startServer("0.0.0.0", port = 9359, app = list(
call = function(req) {
list(
status = 200L,
headers = list("Content-Type" = "text/html"),
body = "Some content..."
)
}
))
# Attempt to retrieve content (this hangs)
page <- curl::curl_fetch_memory(url = "http://127.0.0.1:9359")
httpuv::stopServer(server)
Current progress
Once the server has been started, running curl -v 127.0.0.1:9359 at the terminal returns content as expected. Additionally, if I open a new instance of RStudio and try to curl::curl_fetch_memory() in that new R session (while the old one is still open), it works perfectly.
Encouraged by that, I've been playing around with callr for a while, thinking maybe it's possible to launch the server in some background process, and then continue as usual. Unfortunately I haven't had any success so far with this approach.
Any insight or suggestions very much appreciated!
Isn't it a great feeling when you can come back and answer a question you asked!
From the httpuv::startServer() documentation:
startServer binds the specified port and listens for connections on an thread running in the background. This background thread handles the I/O, and when it receives a HTTP request, it will schedule a call to the user-defined R functions in app to handle the request. This scheduling is done with later(). When the R call stack is empty – in other words, when an interactive R session is sitting idle at the command prompt – R will automatically run the scheduled calls. However, if the call stack is not empty – if R is evaluating other R code – then the callbacks will not execute until either the call stack is empty, or the run_now() function is called. This function tells R to execute any callbacks that have been scheduled by later(). The service() function is essentially a wrapper for run_now().
In other words, if we want to respond to requests as soon as they are received, we have to explicitly do so using httpuv::service(). Something like the following does the trick!
s <- callr::r_session$new()
on.exit(s$close())
s$call(function() {
httpuv::startServer("0.0.0.0", port = 9359, app = list(
call = function(req) {
list(
status = 200L,
headers = list("Content-Type" = "text/html"),
body = "Some content...")
)
}
))
while (TRUE) httpuv::service()
})
# Give the server a chance to start
Sys.sleep(3)
page <- curl_fetch_memory(url = "http://127.0.0.1:9359")
I have this code in my script
sess = requests.Session()
a = requests.adapters.HTTPAdapter(max_retries=20)
sess.mount('https://', a)
If I don't explicitly close the session does it close automatically when my script exits.
The reason I am asking is because if this script is called several thousand times (Each time the previous run is closed/aborted before the next call) will I run into resource problem.
The Session object allows you to reuse the connection across multiple requests. If your Python script ends then the Session is lost, so the connection should be closed. If you want a new connection for each request you can configure keep-alive:
sess = requests.Session()
sess.config['keep_alive'] = False
I would like to run OpenCPU job asynchronously and collect its results from a different session. In Rserve + RSclient I can do the following:
RS.eval(connection, expression, wait = FALSE)
# do something while the job is running
and then when I'm ready to receive results call either:
RS.collect(connection)
to try to collect results and wait until they are ready if job is still running or:
RS.collect(connection, timeout = 0)
if I want to check the job state and let it run if it is still not finished.
Is it possible with OpenCPU to receive the tmp/*/... path with the result id before the job has finished?
It seems acording to this post that OpenCPU does not support asynchronous jobs. Every request between the browser and the OpenCPU server must be alive in order to execute a script or function and receive a response succesfully.
If you find any workaround I would be pleased to know it.
In my case, I need to run a long process (may takes a few hours) and I can't keep alive the client request until the process finishes.
I’m writing a webserver that sometimes has to pass data through a R script.
Unfortunately startup is slow, since i have to load some libraries which load other libraries etc.
Is there a way to either
load libraries, save the interpreter state to a file, and load that state fast when invoked next time? Or
maintain a background R process that can be sent messages (not just lowlevel data streams), which are delegated to asynchronous workers (i.e. sending a new message before the previous is parsed shouldn’t block)
R-Websockets is unfortunately synchronous.
Rserve and RSclient is an easy way to do create and use an Async server.
Open two R sessions.
in the first one type:
require(Rserve)
run.Rserve(port=6311L)
in the second one type:
require(RSclient)
rsc = RS.connect(port=6311L)
# start with a synchronous call
RS.eval(rsc, {x <<- vector(mode="integer")}, wait=TRUE)
# continue with an asynchronous call
RS.eval(rsc, {cat("begin")
for (i in 1:100000) x[[i]] <-i
cat("end")
TRUE
},
wait=FALSE)
# call collect until the result is available
RS.collect(rsc, timeout=1)