Shiny app is stalled when executing a big calculation - r

I'm new to shiny , couldn't get my answer anywhere tho.
I have a heavy code aggregating tables in R. I wanted to move it to the Web app and I chose Shiny as my original code is written in R so I thought it saves me lots of time.
When I run the code by
ObserveEvent(actionbutton$do,{mybigcalculation(input_tables)})
my code is running but at the time of executing my code, it's not possible to do other stuff like exploring tables in the other tabs.The rendering part works but he web app functionalities is completely frozen at the time the process is running.
Any help would be highly appreciated ? if this doesn't work I have to move to typical web app development by having backend ( e.g plumber ) and frontend (e.g React) in separate servers.
Thank you

After a long investigation, I think I found the answer. I'm writing it down here as you might face this issue quite often developing your web app by Shiny. Always there are some big processes you wanna do and wanna make sure your async works great.
There are libraries (future & promises packages) that Cheng explains them here. it says, they can take the calculations in the background and make them come back with the result while the shiny app is doing its normal job. It didn't work for me and I still had my web app stalled. But what I saw, they did increase mybigcalculation speed dramatically. Also it took it to the background makes it invisible in the console.
I found my answer in a package called Shiny.worker library.
Now my app is working fine while my expensive code is running in the background. I made my ideal execution(fast and async) by wrapping future package inside of a shiny worker library.
So it looks something like this:
load.lib <- c("promises","future","shiny.worker")
install.lib <- load.lib[!load.lib %in% installed.packages()]
for(lib in install.lib) install.packages(lib,dependencies=TRUE)
sapply(load.lib,library,character=TRUE)
plan(multisession)
initiate the worker by:
worker <- shiny.worker::initialize_worker()
then:
wrapper <- function(args) {
future::future(my_heavy_calculations(args$r))
}
reactive_arguments <- reactive({
input$start
list(r = rnorm(1))
})
resultPromise <- worker$run_job("job1", wrapper, args_reactive = reactive_arguments)
resultPromise()$result # contains the result of the calculations
resultPromise()$resolved # contains flag that informs whether the job has finished or not
I'm looking forward for any idea and suggestions that can makes the answer better.

Related

Improve profvis performance for profiling shiny app

I have a very large and complex R Shiny app and I am trying to find bottlenecks or parts of the code to improve via the profvis package.
The problem is that profvis itself is performing very slowly (because of the processes and functions in my shiny app) so that it's lagging and almost not possible to properly view & navigate through the profile or the flame graph.
Starting the profvis profile via R or via browser (firefox & chrome tested) doesn't really make a big difference here.
I am only testing one (the main) feature/calculation in my shiny app which is initiated by one action button, thus I can't really test less features or make the profile "shorter".
Any help or tips are appreciated. Especially ways to run profvis faster. Another option I tried was to only wrap parts of my code inside the shiny app with profvis, but I didn't find a way to get this work.
Thank you!

R/Shiny: View full stack/execution log

I'm debugging a Shiny web app, and would like to see the entire control flow/execution path over the course of rendering and updating the generated website.
Is there a way to capture/print/dump-to-file every line of code that is executed in the process of rendering/updating a Shiny app? It would also be good (maybe better?) to see every line of R code parsed by the running R interpreter instance; I'm not concerned about length of this output, and would prefer to get things as verbosely as possible.
I have looked into the stack tracing Shiny functions but these seem to be intended for error catching/handling/reporting. The app is not generating errors/warnings, just setting some variables to NULL at some point when they shouldn't be, so I'm not sure if this is the right approach. These stack tracing functions also seem to be more localized, designed to operate within a given reactive variable/function/render rather than following the control/execution flow across differing reactives/rendering functions in an app.
This app is a large, company-internal app so I cannot give a MRE/MWE.
I finally found the profvis package that does more-or-less exactly what I want by taking a snapshot of the execution stack at a fixed time interval (default 10ms). I'm still working on using this tool to debug, but I believe this will get me there and would also be useful to others debugging Shiny apps that need more than browser() and/or reactlog.
Specifically, I have been doing:
#install.packages("profvis")
library(profvis)
exec_log <- profvis(runApp("myShinyApp"))
...interact with the myShinyApp web page enough to trigger the bug, then interrupt execution...
print(exec_log)

R Shiny Server - set a loading animation on application startup

I've used shinyapps.io in the past and it provides a loading animation (spinner) while the application starts up. This is useful because I load 200MB of .RData-files into the memory (once on startup, not for every server()). This takes up to 40 seconds (in future, I will transition towards storing the data in a database, but for now this is what I got).
For other applications, I've used the docker image rocker/shiny and wanted to fully transition to a Docker-based approach and put all my shiny applications on one server and move away from shinyapps.io. However, the one issue I have with this application is that it does not display a loading animation while it starts so the user is left with a grey screen for a good 30-40 seconds while the data is loaded in the background.
As for the code, I load all data and then I source ui.R and server.R before running shiny::shinyApp(ui = ui, server = server).
Does any of you know a way to specify a loading animation on startup of the application (I haven't found anything in the server configuration itself but I could have overlooked something)? Or have you found a nice workaround to achieve the desired result?
So what I ended up doing was to follow this workaround suggested here: http://www.mazsoft.com/blog/post/2018/01/01/show-progress-bar-when-pre-loading-data-in-shiny-app
The idea is that we initialize all data variables with NULL. Then there is a readData() function outside of server and ui where we load all the data into the global variables and at the beginning of the server function block we check if one of our data variables is.null() which would lead us to call the readData() function.
It is an okay solution for my problem as it's a nice workaround, yet I wasn't able to figure out how to actually display a loading animation on startup, just while loading the data. I hope this helps people with a similar same problem.

Workflow for maintaining different versions of a Shiny app

Lately I have been making several very similar Shiny apps for different clients and hosting them on shinyapps.io.
Each app has a different title, different data, some differences in branding etc. but otherwise the code is very similar.
I'm having trouble maintaining these apps. When find and fix a bug I currently have to go through 5 different apps and make the change each time.
Does anyone have good suggestions on how to handle this? Git branches? I know the best solution would be to have one app and upload different data, but that's not possible unfortunately.
I'd like to keep using shinyapps.io, but I'm open to hosting the apps somewhere else if it makes my workflow better.
As I wrote in the comment shinyModules() will help you: https://shiny.rstudio.com/articles/modules.html
Shiny modules are to shiny functions, like ordinary functions are to repeating code.
Or to put it differently:
Repeating code --> function
repeating shiny function --> shiny module
As the documentation is a bit complicated here and there, i wrote a simplified example here:
Create a reactive function outside the shiny app.
You could store all the shiny modules in a file modules.R and add a global.R script to each of the apps that loads the modules (source("../modules.R"). Then you only have to update the functions within modules.R. That change of structure might take a while in the beginning. But, i think in the long run it pays off for more complex apps.
I ended up making a library that contained most of the code I needed for the apps, as suggested by Adam Spannbauer in the comments.
It's not perfect; I still have some duplication and I have to have the library on GitHub so that it will work with shinyapps.io. However, it's a big improvement on what I was doing previously.

R tcltk responsive gui during calculation

Hello dear stack overflow community.
i'm currently working on an R project for statistical calculations that involves a gui and also time consuming heuristics. in the gui shall be an button to start and stop the calculation and a textfield that reports the best error so far.
so i'm stuck with the question how to keep the gui responsive during the calculation.
some example code
require("tcltk")
result<-tclVar("")
start<-function(){
active<<-TRUE
tkconfigure(button,text="stop",command=stop)
dostuff()
}
stop<-function(){
active<<-FALSE
tkconfigure(button,text="start",command=start)
}
dostuff<-function(){#this would be the optimization function
while(active){
tclvalue(result)<-#do some stuff
}
}
toplevel<-tktoplevel()
button<-tkbutton(toplevel,text="start",command=start)
entry<-tkentry(toplevel,textvariable=result)
tkpack(button)
tkpack(entry)
in the do stuff function some multithreading stuff seems to be necessary. its a requirement to work on windows and linux. i'm hoping for ideas how to archive this. thanks in advance
I think the easiest way is to start an R script as background process using system or system2 with the wait=FALSE parameter so that the long running calculation is processed in the background.
If you want to update the UI to show intermediate results or progress the R script must write the current state into an "exchange" file that can be used to update the UI.
To update the UI you have to check for the new state periodically e. g. by using the after command of Tcl/Tk, see:
http://www.tcl.tk/man/tcl/TclCmd/after.htm
For an example of starting an R script as background process see here:
http://stackoverflow.com/questions/14208976/r-run-source-in-background
Note that R is single threaded and updating the Tk-UI must be done from the same thread (process) that created the UI since the main event loop is running here.
Also take care that you shouldn't allow the user to make conflicting changes via the UI while the background task is running (e. g. starting another background task - except you want to support this).
Canceling the background process via the UI ("cancel button") can be done best by using another "signal file" that is checked by the background process periodically.

Resources