Fundamental understanding of reactive lists - r

I wrote a little standard application in R that is running pretty fine.
It scans the *.png files in a folder and extracts some statistics on pixel granularity and colours.
Now it is my idea to transfer this to shiny. Although I once wrote some smaller Shiny-Apps, i never used reactivity intensely.
The basic idea of the here presented code is:
to choose png files from a folder
show the number of chosen images
initiate some calculations on the chosen files after clicking a button
displaying the extracted parameters
I now have critical problems with the basic design of the reactive variables, obviously as my construct in mind is not suitable.
Is there someone who could advise me the basic outline of the shiny construct?
I tried to reduce the code to the bare minimum. I hope one can follow my thoughts.
library (shiny)
options (shiny.maxRequestSize = 250 * 1024^2)
# increase maximum data upload to 250MB
const_Number_of_Variables = 5
# just to keep the number of extracted parameters
# up to date when new parameters will be included
ui <- fluidPage(
# just a simple UI with some basic elements
fileInput ("ui_IMGfiles",
"Choose PNG files",
multiple = TRUE,
accept = c ("image/png",
".png")
),
tag$hr,
tableOutput ("ui_IMGfilelistcontents"),
tag$hr,
textOutput ("imagecount"),
tag$hr,
actionButton ("ui_btn_CalcFeatures", "Calculate image features", class = "btn-warn"),
tag$hr,
tableOutput('IMG_featuretable')
)
IMG_filelist <- reactiveValues (data = NULL)
IMG_features <- reactiveValues (val1 = NULL, val2 = NULL, val3 = NULL, val4= NULL, val5 = NULL, val6 = NULL)
IMG_data <- reactiveValues (data = NULL)
server <- function(input, output) {
observeEvent (input$ui_IMGfiles, {
# input$ui_IMGfiles will be NULL initially. After the user selects
# and uploads files, it will be a data frame with 'name',
# 'size', 'type', and 'datapath' columns. The 'datapath'
# column will contain the local filenames where the data can
# be found.
IMG_filelist$data <- input$ui_IMGfiles
})
output$ui_IMGfilelistcontents <- renderTable({
# if file list is not initialised, return NULL
if (is.null (IMG_filelist$data () ))
return(NULL)
#finally display the list of images
IMG_filelist$data ()
})
output$imagecount <- renderText({
# whenever the file list changes, identify
# the number of elements from the length of list entries
paste0 ("# of files: ", length (IMG_filelist$data () ))
}
)
IMG_data <- observeEvent (input$ui_btn_CalcFeatures, {
# check whether some images are loaded
if (is.null (IMG_filelist)) {IMG_data = NULL}
IMG_files_count = length (IMG_filelist)
# prepare the feature list that will be
# overwritten every time the button is pressed
IMG_data$data = matrix (NaN, nrow = 1, ncol = const_Number_of_Variables)
for (i in 1:IMG_files_count) {
# now read the i'th image from the list and pass it to the evaluation function
aSourceImage = readPNG (IMG_filelist$data [i]$datapath, native = FALSE, info = TRUE)
# extract the desired parameters and attach them to the list
IMG_data$data = rbind ( c ("val1", "Val2", "val3", "Val4", "etc"),
#fn_ExtractTileInformation (aSourceImage, aFileFullPath, aFileName, 128),
IMG_data$data ())
}
})
output$IMG_featuretable <- renderDataTable (img_data$data ())
}
shinyApp (ui, server)

Related

Avoid a time-consuming step in a reactive expression in R Shiny

In my Shiny app, users can upload a file which is stored as a reactive dataframe. Inside the reactive expression that is shown below, I call an external time-consuming function (called performDigestion) which requires several seconds to complete.
fastafile_data <- reactive(){
inFile_fastafile <- input$fastaFile
req(inFile_fastafile)
ext <- tools::file_ext(inFile_fastafile$datapath)
validate(need(ext == "fasta", "Please upload a fasta file"))
dt.seq <- readAAStringSet(inFile_fastafile$datapath)
tbl <- performDigestion(dt.seq) ##the time-consuming step
return(tbl)
}
Next, I render a Datatable to present the results of the fastafile_data in the UI:
output$dt_fastafile <- DT::renderDataTable({
withProgress(message = 'Computation in progress, this step might take a while. Please wait...', {
incProgress(1/1)
fastafile_data()
})
}, options = list(scrollX = TRUE, dom = 'lfrtip', pageLength = 10, lengthMenu = c(10, 25, 50, 100)), rownames = FALSE)
In the UI, I also have two additional components (a sliderInput and a numericInput) and in the server-side I handle their values through two observeEvents .
What I would like to achieve is to update the fastafile_data dataframe every time any of these two additional components is triggered without reading the input$fastaFile again and re-running the time consuming performDigestion() function. I would ideally like to trigger the above reactive process again only when a new file is uploaded by the user.
I think the problem here is in my logic and/or there exists a smarter way to do it in ShinyR that I'm currently missing? Can you please point me to the right direction?
EDIT:
When I try to handle the reactive fastafile_data through a second reactive fastafile_data_new the first fastafile_data is re-executed.
fastafile_data_new <- reactive({
dt <- fastafile_data()
##### the condition I'd like to apply
dt$identifiable <- ifelse(dt$length >= min_peptide_length$choice & dt$length <= max_peptide_length$choice & dt$`mass [Da]` < max_peptide_mass$choice, 1, 0)
return(dt)
})

Enhancing computation speed of Rshiny

I'm trying to develop a basic R shiny app but facing issues with the processing speed. The procedure is as follows, I need to read csv file of about 500K rows -> split the file into smaller segments -> calculate new features for each segment and display the result. Below are my UI.R and Server.R
UI.R
library(shiny)
library(shinyBS)
library(shinycssloaders)
library(DT)
shinyUI(fluidPage(
mainPanel(
#UI for choosing the file to input
fileInput("file1", label = (" Choose Drivecycle Data "),multiple = F),
#UI for showing the number of Rows in original dataset
fluidRow(
column(8, h4(helpText("Number of rows input dataset"))),
column(3,verbatimTextOutput("totrows", placeholder = TRUE))),
#UI for showing the number of segments the data set had been split into
fluidRow(
column(8, h4(helpText("Number of segmentations"))),
column(3,verbatimTextOutput("totseg", placeholder = TRUE))),
fluidRow(
column(8, downloadButton("subtablednld", label = 'Downloadcsv'))
),
tabsetPanel(
#UI to show the original data set in First tab
tabPanel("Table",icon = icon("table"),withSpinner(DT::dataTableOutput('table'),
type = getOption("spinner.type", default = 8) )),
#UI to show the features of the segments of the orginal dataset in Second Tab
tabPanel("Feature Table",icon = icon("table"),withSpinner(DT::dataTableOutput('table1'),
type = getOption("spinner.type", default = 8) )),
),style = 'width:1000px;height"3000px'
)
)
)
Server.R
library(shiny)
library(earth)
library(tidyr)
options(shiny.maxRequestSize=300*1024^2) #increase the max upload file size
to 30 MB
options(shiny.trace=TRUE)
# Define server logic required to draw a histogram
shinyServer(function(input, output) {
#Function to input data set using UI
dataframe <- reactive( {
### Create a data frame reading data file to be used by other functions..
inFile <- input$file1
data1 <- read.csv(inFile$datapath, header = TRUE)
})
#Display the input dataset
observeEvent(input$file1,output$table <- renderDataTable({dataframe()}))
#Show the number of rows in the input dataset
observeEvent(input$file1,output$totrows<- renderText({nrow(dataframe())}))
#Split the data set
Splitfile <- function(){
split(dataframe(), (seq(nrow(dataframe()))-1) %/% 200)
}
#Show the number of segments the data has been split into
observeEvent(input$file1,output$totseg <-renderText({length(Splitfile())}))
#Acceleration calculation function
Acceleration <- function(){
c <- lapply(1:length(Splitfile()), function(i)
{
acceleration <- c(0,diff(Splitfile()[[i]]$Vehicle.Speed)/2)
})
Splitfile <- mapply(cbind, Splitfile(), "acceleration" = c, SIMPLIFY = F)
Splitfile
}
#Calculating Features
CaclFeatures <- function(){
FileFeatures <- lapply(1:length(Acceleration()), function(i){
Velocity_mean <-round(mean(Acceleration()[[i]]$Vehicle.Speed),digits = 3)
Variance_Velocity <-round(var(Acceleration()[[i]]$Vehicle.Speed)*
((length(Acceleration(
[[i]]$Vehicle.Speed)-1)/length(Acceleration()
[[i]]$Vehicle.Speed))
,digits = 3)
c(Velocity_mean,
Variance_Velocity)
})
FileFeatures<- as.data.frame(do.call(rbind, FileFeatures))
names(FileFeatures)[names(FileFeatures) == 'V1'] <- "Velocity_Mean"
names(FileFeatures)[names(FileFeatures) == 'V2'] <- "Variance_Velocity"
}
#Display the table containing all features of all the segments
output$table1 <- renderDataTable({
CaclFeatures()},options = list(scrollX = TRUE))
#Print to csv
output$subtablednld <- downloadHandler(
filename = function(){
paste("dataset-", ".csv", sep = "")
},
content = function(file){
write.csv(CaclFeatures(), file ,row.names = FALSE)
}
)
})
The app works fine if I read csv file of about 2k rows but does not work if I read data set more than 2k, It will neither give any error nor crash. The spinner keeps rotating but fails to show the result. Also, the same logic when used in regular R script work fine with large data set of more than 500k, rather I'm calculating 22 new features.
Currently, I am using a system of 8gb RAM i5 Processor. Is there a way to enhance the computing speed, when checked within my task manager Rstudio uses only around 47% - 52% of memory, I have no other process running other than R studio
EDIT: Sample data can be created by using the code below,
drive <- as.data.frame(sample(1:50, 500000, replace = T))
Your whole calculation seems to be dependent on some structural properties from your input data.frame, so I can't produce a working example in a reasonable time, with only minor changes to your code.
BUT, your code evaluation is aweful performance wise.
Take Acceleration for example. WITHIN your lapply, you call Splitfile(), which is a regular function. Assume that the number of splits is about 2500, you call this function 2500 times. And the operation split(dataframe(), (seq(nrow(dataframe()))-1) %/% 200) takes about 2 seconds on my computer, so you're waiting 5000 seconds, while the result of Splitfiles() is always the same. And then, inside CalcFeatures, you call Acceleration() again four times inside each lapply loop. That makes for an approximate waiting time of 5 000 * 2 500 * 4 = 50 000 000 seconds or 578 days.
You might have been confused with the concept of reactive where the function call would just return the current value and reevaluation is implicit.
So you either:
Call expensive functions once at the beginning of your function.
Start Acceleration with files <- Splitfiles() and use files from there on.
Start CalcFeatures with acc <- Acceleration() and use acc from there on.
Turn your functions into reactives.
Splitfiles <- reactive({ ... dataframe() ... })
Acceleration <- reactive({ ... Splitfiles() ... })
CalcFeature <- reactive({ ... Acceleration() ... })
A mixture of both concepts is not better. Stick to either one.

Conditional selection DataTables Shiny not working

When a row in a DataTable is clicked, I would like an image in a different panel to be loaded but, I keep getting an error and not.
**Warning in widgetFunc() :
renderDataTable ignores ... arguments when expr yields a datatable object; see ?renderDataTable
Error in basename(file) : a character vector argument expected**
output$image1 <- renderImage({
s = input$table1_rows_selected
if (length(s)) list(src=paste0(imagePath,"/peak",s,".png"))},deleteFile=FALSE)
The function below works however,
output$image1 <- renderImage({list(src=paste0(imagePath,"/peak1.png"))},deleteFile=FALSE)
Here is a full version of the code:
server.R
writeLines("Please select ANY image")
imagePath = file.choose()
# break up the character vector, delete the last word
imagePath = dirname(imagePath)
server = function(input, output) {
output$table1 = renderDataTable({
# the peak table
datatable(peaksTable,
# when rowname is false each row does not have a numeric # associated with it
rownames = FALSE,
# specify the name of the column headers
colnames = c("Seqnames", "Start", "End","Width","Strand","P","Q","Effectsize",
"FDR","Keep","Gene_name","Gene.nearest","Count","Count.pred",
"Coverage","Local.mut.density","Base.context.GC","Tn.Context.TpC",
"Tn.context.CpG","Dnase","Activechrom","Hetchrom","Rept"))
},
escape = FALSE)
# render an Image based on which rows are clicked on.
output$image1 <- renderImage({
s = input$table1_rows_selected
if (length(s)) list(src=paste0(imagePath,"/peak",s,".png"))},deleteFile=FALSE)
ui.R
shinyUI(navbarPage(
title = " Nanoproject",
# first panel , create table of the peaksTable dataframe
tabPanel('Peak Table' ,
dataTableOutput('table1')),
# second panel
tabPanel('Peak Images' ,
imageOutput("image1",width = "auto",height = "auto")
))
I'm not sure where I'm going wrong.
Like it's been pointed out, without a reproducible example it's hard to help.
My guess is that your code is not dealing with the case where no rows are selected. If that's true, something like this should fix the problem:
server.R
output$image1 <- renderImage({
s <- input$table1_rows_selected
# print(s)
if(is.null(s)) return(NULL)
list(src = paste0(imagePath,"/peak",s,".png"))
}, deleteFile=FALSE)
Printing out s could help you understand better whats going on.

Identifying Shiny app stability issues

I've been developing a Shiny app that showcases a plot function, accepts inbuilt data or user-input CSV, produces custom plot and can output this to user as a PDF. All modules have worked fine independently of each other in development, but as a whole the app becomes unstable and regularly refuses to react to inputs. Sometimes it needs refreshing a few times just to start. All the functionality does work intermittently, so I think any bugs must relate to the complexities of the Shiny/browser interface. But as there's no feedback from Shiny (to R) or in the browser console it's almost impossible to diagnose, and it's starting to feel like a serious disincentive to using this otherwise very promising platform.
I've made the situation reproducible with a reduced script, which is also executable with runGist('db479811c6237a0741fe', launch.browser=F). I'd be really grateful for assistance from anyone who has experience of this type of issue or who understands Shiny under the hood. Advice also appreciated on ways to streamline or rework the code structure. Any comments/discussion not suitable for SO please post to reddit.
server.R
require(shiny)
# inbuilt dataset
diamonds = ggplot2::diamonds[,c(1,5,7)]
# csv datasets to input via front-end
for(i in 1:3){
dat <- diamonds[sample(1:nrow(diamonds), 200),]
write.table(dat, paste0('dat',i,'.csv'), sep=',',row.names=F, col.names=T)
}
diamonds = diamonds[sample(1:nrow(diamonds),200),]
# global variables
inbuilt = FALSE # whether currently using inbuilt data or not
datapath = '' # to chech current against previous to see if new dataset input
pagereset = FALSE # to reset when inbuilt de-selected
# function to 'plot' welcome instructions
welcome <- function(){
plot.new(); plot.window(xlim=c(0,100), ylim=c(0,100))
text(10,80,"Please input CSV file data with 3 numerical columns", cex=2, pos=4)
text(10,65,"Use the inbuilt dataset and the csv files in the app folder..", cex=1.5, pos=4)
text(10,50,"check app's reliability and how often commands fail", cex=1.5, pos=4)
text(10,35,"output to PDF", cex=1.5, pos=4)
text(10,20,"how stable is the app for you?", cex=1.5, pos=4)
}
shinyServer(function(input, output, session) {
# REACTIVE FUNCTION
plotInput = reactive({
# import data from inbuilt (internal) or a user-input CSV
# first must check if reactive is triggered by new data or not:
newdata = FALSE # initialised
if(input$inbuilt != inbuilt){ # inbuilt data option toggled
if(input$inbuilt) { # inbuilt selected
inbuilt <<- TRUE # update global
d <<- diamonds
newdata = TRUE
} else{ # inbuilt de-selected.
inbuilt <<- FALSE # update global
d <<- NULL # return splashscreen
pagereset <<- TRUE # would now crash so refresh app instead
}
} else { # input doesn't relate to inbuilt dataset
if(!input$inbuilt){ # inbuilt unselected
if(is.null(input$file1)) { # if null no input received yet
d = NULL # so reactive will return splash-screen
} else { # data has been input before
if(input$file1$datapath != datapath){ # new dataset just received
datapath <<- input$file1$datapath # update global
d <<- read.csv(datapath, header=TRUE, sep = ',') # update global
newdata = TRUE
#Sys.sleep(2) # allow file-upload aanimation to finish
# reset file handler in page
session$sendCustomMessage(type = "resetFileInputHandler", "file1")
} else NULL # new input not dataset-related
}
}
}
# reset/null javascript command - to reset app after inbuilt
# dataset is de-selected, as the script crashes otherwise..
reset_js = ifelse(pagereset, "window.location.reload()", '')
reset_js = paste("<script>", reset_js,";</script>")
if(pagereset) {
pagereset <<- FALSE
return(list(resetpage = reset_js, plot = plot.new())) # reset and null plot
}
# no data input so return splash-screen
if(is.null(d)) return(list(resetpage = reset_js, plot = welcome()))
# NORMAL PLOT
# # stroke around polygons
if(input$border != 'none') border = input$border else border = NA
# PDF handling (save file locally to be passed forward)
if(input$returnpdf){
pdf("plot.pdf", width=as.numeric(input$w), height=as.numeric(input$h))
symbols(d[[1]], d[[2]], circles=sqrt(d[[3]]), inches=as.numeric(input$inches),
bg='#ff000020', fg=border)
dev.off()
}
# return plot and reset instruction in list
list(
resetpage = reset_js,
plot = symbols(d$carat, d$depth, circles=sqrt(d$price), inches=as.numeric(input$inches),
bg='#ff000020', fg=border)
)
}) # end reactive
# OUTPUT ELEMENTS
# PDF file
output$pdflink = downloadHandler(
filename <- "shiny_plot.pdf", # default browser save filename
content <- function(file) file.copy("plot.pdf", file) # call pre-saved pdf
)
# plot
output$plot = renderPlot({ plotInput()$plot })
# reset instruction
output$reset = renderText({ plotInput()$resetpage })
})
ui.R
require(shiny)
fluidPage(
titlePanel("Stability testing"),
sidebarLayout(
sidebarPanel(
# this css just resets the CSV upload function
tags$head(
tags$script('
Shiny.addCustomMessageHandler("resetFileInputHandler", function(x) {
var id = "#" + x + "_progress";
var idBar = id + " .bar";
$(id).css("visibility", "hidden");
$(idBar).css("width", "0%");
});
')
),
# inputs
h4('Input options'),
p("Chose inbuilt dataset or upload a CSV:"),
checkboxInput('inbuilt', 'Inbuilt dataset (app resets when de-selected)', FALSE),
fileInput('file1', '', accept = 'text/comma-separated-values'),
# PDF output
h4('PDF output'),
p("Buggy: plot disappears, but link still downloads last plot. Sometimes after download app crashes"),
checkboxInput('returnpdf', 'Save plot to PDF?', FALSE),
conditionalPanel(
condition = "input.returnpdf == true",
strong("PDF size (inches):"),
sliderInput(inputId="w", label = "width:", min=3, max=20, value=12, width=100, ticks=F),
sliderInput(inputId="h", label = "height:", min=3, max=20, value=9, width=100, ticks=F),
downloadLink('pdflink')
),
# plot layout
h4('Plot options'),
selectInput(inputId="border", label="Outline colour:", choices=list(black='black', white='white', none='none'), width=150, selected='black'),
sliderInput(inputId="inches", label = "Circle size (higher values can crash the app)", min=0.05, max=.5, value=.2, width=150)
),
mainPanel(
htmlOutput('reset'), # reset command (when inbuild dataset de-selected)
imageOutput('plot')
)
)
)

Shiny - renderDataTable - bSearchable vs checkboxInput

I´m having problems combining two features while building a data table:
I use “bSearchable” to select 1 column that I want to use the search tool to filter
I use "checkboxInput" to select the columns the user wants to see.
Both work separately, but not together. If I uncheck a column in my menu input, the data disappears - like applying a filter and no data was found. How can I fix this?
library(shiny)
runApp(list(ui=(fluidPage(
pageWithSidebar(
headerPanel('Title'),
sidebarPanel(
helpText('Text about the table'),
checkboxInput('columns','I want to select the columns' , value = FALSE),
conditionalPanel(
condition= "input.columns == true",
checkboxGroupInput('show_vars', 'Select the columns that you want to see:', names(iris[1:4]),
selected = names(iris[1:4]))
),
downloadButton('downloadData', 'Download'),width = 3
),
mainPanel(
tags$head(tags$style("tfoot {display: table-header-group;}")),
dataTableOutput("mytable1"),width = 9
)
))
)
,
server=(function(input, output) {
library(ggplot2)
library(XLConnect)
#DATA
tabel<- reactive({
iris[,c(input$show_vars,"Species"), drop = FALSE]
})
# OUTPUT
output$mytable1 = renderDataTable({
tabel()},
options = list(
aoColumns = list(list(bSearchable = FALSE), list(bSearchable = FALSE),list(bSearchable = FALSE),
list(bSearchable = FALSE),list(bSearchable = TRUE)),
bFilter=1, bSortClasses = 1,aLengthMenu = list(c(10,25,50, -1), list('10','25', '50', 'Todas')),iDisplayLength = 10
)
)
output$downloadData <- downloadHandler(
filename = function() { paste('tabela_PSU','.xlsx', sep='') },
content = function(file){
fname <- paste(file,"xlsx",sep=".")
wb <- loadWorkbook(fname, create = TRUE)
createSheet(wb, name = "Sheet1")
writeWorksheet(wb, tabel(), sheet = "Sheet1")
saveWorkbook(wb)
file.rename(fname,file)
},
)
})
))
The problem is by filtering the data iris based on input$show_vars, you are changing the number of columns of the DataTable.
However, you have defined a fixed aoColumns option, which implies your DataTable has five columns (four non-searchable, one searchable).
Therefore, when you deselect any checkbox inputs, the filtered data doesn't match the specified options. As a result, nothing is displayed.
That is, although your data in the DataTable is reactive, the options, however, are NOT reactive.
If you read the renderDataTable's document carefully, you will see that you can pass two types of variables to the options argument:
options A list of initialization options to be passed to DataTables, or a function to return such a list.
The differences are:
If you specify options as a list, Shiny assumes that the options are fixed; But since you are dynamically filtering the data based on input$show_vars, you should dynamically change the options for aoColumns as well.
If you pass a function as an argument for options, Shiny will know that the options are also reactive. Hence Shiny will also update the options when the data (in your case, the data.frame encapsulated in the reactive variable named tabel) updates.
You may already know that reactive variables are themselves functions. They are evaluated in a reactive environment and when evaluated, they return the current state/value of the data. This is why you pass tabel() instead of tabel to renderDataTable.
The solution then, is to wrap the entire options list into a reactive variable (hence a function as well). Specifically, we want to dynmaically set the aoColumns option so that the number of bSearchable toggles matches the number of columns shown in the DataTable.
Below I only show the updated server part, since there's nothing needs to be changed in the UI part.
server.R
shinyServer(function(input, output) {
library(ggplot2)
library(XLConnect)
#DATA
tabel<- reactive({
iris[,c(input$show_vars,"Species"), drop = FALSE]
})
# wrap the `options` into a reactive variable (hence a function) so that it will
# be evaluated dynamically when the data changes as well.
# `dt_options` is reactive in the sense that it will reflect the number of rows
# visible based on the checkboxInput selections.
dt_options <- reactive({
# dynamically create options for `aoColumns` depending on how many columns are selected.
toggles <- lapply(1:length(input$show_vars), function(x) list(bSearchable = F))
# for `species` columns
toggles[[length(toggles) + 1]] <- list(bSearchable = T)
list(
aoColumns = toggles,
bFilter = 1, bSortClasses = 1,
aLengthMenu = list(c(10,25,50, -1), list('10','25', '50', 'Todas')),
iDisplayLength = 10
)
})
# OUTPUT
output$mytable1 = renderDataTable({
tabel()},
options = dt_options
)
output$downloadData <- downloadHandler(
filename = function() { paste('tabela_PSU','.xlsx', sep='') },
content = function(file){
fname <- paste(file,"xlsx",sep=".")
wb <- loadWorkbook(fname, create = TRUE)
createSheet(wb, name = "Sheet1")
writeWorksheet(wb, tabel(), sheet = "Sheet1")
saveWorkbook(wb)
file.rename(fname,file)
},
)
})
(Note that I separate the UI part and server part into ui.R and server.R.)

Resources