I am building a shiny application which connects to 4 different SQL databases and 3 different APIs to download data. I have scheduled this operation of data collection through an R script which is executed every 4 hours. The script stores the data as an Rdata file comprising of 18 different dataframes.
The global.R file of the shiny application loads this data using load("data/data.Rdata") when the application is launched and the application runs fine.
However, in order to refresh the data, I presume reactivePoll is what I would need. But am not sure how to get it going. This is what I have tried in the global.R file which doesn't work:
reactivePoll(14400000,session = NULL,
checkFunc = function() {
if (file.exists("data/data.Rdata"))
file.info("data/data.Rdata")$mtime[1]
else
""
},
valueFunc = function() {
load("data/data.Rdata")
})
The problem lies in the fact that you are using load to get the data which puts the data into memory as side-effect. The example of reactivePoll uses a function that returns the values.
A discussion about combining load with reactive is available here -> Reading an RData file into Shiny Application
Related
I have a dashboard living in a Shiny Server pro that shows different analysis. The data is coming from a long query that takes around 20 minutes to be completed.
In my current set up, I have a button that updates the data:
queries new data
transforms the data
saves the data in a file .RData
saves the data in a global object (using data <<-)
Just in case, outside the server and ui functions I have a statement that checks if data object exists. In case that does not exists, it reads the data from the .RData file instead of doing the query again.
Now I would like to update the data each Monday at 5:00pm (I do not want to open the app and push the button each Monday). I think that the best way to do it is using a cron job using cronR. The code will be located in the app.R outside the server and ui functions. Now I have the following questions:
If I am using Shiny server pro how many times, the app, will create the cron job if it is located in the app.R outside the server and ui functions?
How can I replace the object data in the shiny app? In such a way that if a user open the app on Monday after 5:00 pm the data will be in place, without the need of reading the .RData file and of course not doing the query again.
What is the best practice?
Just create your cron process with cronR completely outside the shiny application and make sure it saves your data to the correct place.
Create the R code which gets your data:
library(...)
# ...
# x <- mydata
save(x, file = "NewData.Rda")
Create the cron job:
cmd <- cron_rscript("path/to/getdata.R")
cron_add(cmd, frequency = 'daily', id = 'job5', at = '05:00')
I cant't see your point 1. The app will not create the cron job if it is not named "global.R" or "ui.R" or "server.R", I think. Also, you don't have to put your code under the /srv/shiny-server/ directory.
For your point 2., check the reactiveFileReader function from the shiny library. This function checks a file's last modified time and the file is re-read if changed
data <- reactiveFileReader(5*60*1000, filePath="NewData.Rda", readFunc = load)
I have a flexdashboard that uses shiny. Here's a MRE repo and gist of the .Rmd. When I put the app live on shinyapps.io, I realized that one user's actions could affect other users. I understand that this is a scoping issue, but I'm confused about how scoping works in Flexdashboard.
This page explains scoping for 'regular' shiny apps:
You might want some objects to be visible across all sessions. For example, if you have large data structures, or if you have utility functions that are not reactive (ones that don’t involve the input or output objects), then you can create these objects once and share them across all user sessions (within the same R process), by placing them in app.R , but outside of the server function definition.
In Flexdashboard, there is no app.R file or server function. How does scoping work in these types of shiny apps?
I have several eventReactive() functions like this that get updated for User 2 when User 1 hits submit and triggers observeEvent(input$submit, {}) at the end of the file.
eventReactive(rv$run2, {
if (remote==1) {
master$df <- drop_read_csv("/dash/master.csv", stringsAsFactors = FALSE)
} else {
master$df <- read.csv("dash/master.csv", stringsAsFactors = FALSE)
}
}, ignoreNULL = TRUE)
I originally posted this to RStudio Community about 9 hours ago, but it did not generate many views (<20) or any discussion.
Moving rv <- reactiveValues() out of the global chunk did the trick.
My current workflow in a shiny application is to run a R script as a cron job periodically to pull various tables from multiple databases as well as download data from some APIs. These are then saved as a .Rdata file in a folder called data.
In my global.R file I load the data by using load("data/workingdata.Rdata"). This results in all the dataframes (about 30) loading into the environment. I know I can use the reactiveFileReader() function to refresh the data, but obviously it would have to be used in the server.R file because of an associated session with the function. Also, I am not sure if load is accepted as a readFunc in reactiveFileReader(). What should be the best strategy for the scenario here?
This example uses a reactiveVal object with observe and invalidateLater. The data is loaded into a new environment and assigned to the reactiveVal every 2 seconds.
library(shiny)
ui <- fluidPage(
actionButton("generate", "Click to generate an Rdata file"),
tableOutput("table")
)
server <- shinyServer(function(input, output, session) {
## Use reactiveVal with observe/invalidateLater to load Rdata
data <- reactiveVal(value = NULL)
observe({
invalidateLater(2000, session)
n <- new.env()
print("load data")
env <- load("workingdata.Rdata", envir = n)
data(n[[names(n)]])
})
## Click the button to generate a new random data frame and write to file
observeEvent(input$generate, {
sample_dataframe <- iris[sample(1:nrow(iris), 10, F),]
save(sample_dataframe, file="workingdata.Rdata")
rm(sample_dataframe)
})
## Table output
output$table <- renderTable({
req(data())
data()
})
})
shinyApp(ui = ui, server = server)
A few thoughts on your workflow:
In the end with your RData-approach you are setting up another data source in parallel to your databases / APIs.
When working with files there always is some housekeeping-overhead (e.g. is your .RData file completed when reading it?). In my eyes this (partly) is what DBMS are made for – taking care about the housekeeping. Most of them have sophisticated solutions to ensure that you get what you query very fast; so why reinvent the wheel?
Instead of continuously creating your .RData files and polling data with the reactiveFileReader() function you could directly query the DB for changes using reactivePoll (see this
for an example using sqlite). If your queries are long running (which I guess is the cause for your workflow) you can wrap them in a future and run them asynchronously (see this post
to get some inspiration).
Alternatively many DBMS provide something like materialized views to avoid long waiting times (according user privileges presumed).
Of course, all of this is based on assumptions, due to the fact, that your eco-system isn’t known to me, but in my experience reducing interfaces means reducing sources of error.
You could use load("data/workingdata.Rdata") at the top of server.R. Then, anytime anyone starts a new Shiny session, the data would be the most recent. The possible downsides are that:
there could be a hiccup if the data is being written at the same time a new Shiny session is loading data.
data will be stale if a session is open just before and then after new data is available.
I imagine the first possible problem wouldn't arise enough to be a problem. The second possible problem is more likely to occur, but unless you are in a super critical situation, I can't see it being a substantial enough problem to worry about.
Does that work for you?
Can we use same global.r file for 2 apps?
I am developing apps using shiny and i want to know if i can use a same global.r file for more than one app
In my global.r file i am taking the data from a url in json format and making a data frame out of it.
And also is it the best way for optimization or not?
Here's just an example function, that uses uncompressed serialization to load and update data:
append_rds <- function(new_rows) {
df <- readRDS("path/to/file.rds")
df <- rbind(df,new_rows)
# not compressing it makes it faster - but larger on disk
saveRDS(df, "path/to/file.rds", compress = FALSE)
}
new_rows <- pull_new_rows() # your download process here
append_rds(new_rows)
You would save this file as run_me_frequently.R. Then create a chron job or whatever system scheduler you use to run something like:
Rscipt path/to/update_data.R
Then in global.r you would use readRDS("path/to/file.rds") and this would always contain the most recent data based on your update schedule.
I am learning Shiny and need some help please.
I need to load a very large data.frame from a saved workspace (RData). Once loaded I need to perform analyses and output these to the UI.
I have placed the following code in server.R but it doesn't load the data and then throws an error:
shinyServer(function(input, output) {
load("c:/temp/ws1.RData")
output$balance_matrix <- renderTable({
Transaction_history
})
})
> Error in func() : object 'Transaction_history' not found
First, what am I doing wrong here?
Secondly, is this the best place to load the workspace?
Thirdly, can I load it outside the function at the top or will it not be available in the function then?
Thanks
Rather than saving and trying to load the workspace, I'd recommend saving the file individually like,
saveRDS(Transaction_history, "C:/temp/ws1.RData")
and then load it in Shiny like,
Transaction_history <- readRDS("C:/temp/ws1.RData")
This method will focus on this file by itself. From the ?load documentation,
load() replaces all existing objects with the same names in the
current environment (typically your workspace, .GlobalEnv) and hence
potentially overwrites important data.