I have a very large fixed width file I need to read in using my Shiny application. The way my program is currently structured is the ui.R contains a fileInput allowing the user to locate the file using the browser.
On the server side, I only capture the path to the file such as the following:
path2file <- reactive({
infile <- input$path2file
if (is.null(infile)) return(NULL)
infile$datapath
})
A subsequent function takes that path as input and the proceeds to read in the file according to its layout specifications. This all works just fine; however when dealing with extremely large fwf files my program slows down tremendously and takes hours to get the path name of the file read in using fileInput
My suspicion is that fileInput is actually reading in the entire file and then my function only returns the datapath even though I am not explicitly reading in any file format type within the function.
My aim is to continue using the program as I have it structured now and obtain only the path to this file using my fileInput. I have found this topic on SO, and see it is a possible option.
Getting file path from Shiny UI (Not just directory) using browse button without uploading the file
However, I also aim to minimize the number of package dependencies I have; this has become a big issue and so if I MUST use an additional package I will, but I'd like to avoid that at all costs.
I experimented with this cheap trick:
path2file <- reactive({
infile <- input$path2file
if (is.null(infile)) return(NULL)
scan(infile$datapath, n = 1)
infile$datapath
})
Thinking that it would be a fast workaround, but it too is extremely slow so I suspect it too is not reading in only n = 1. So, my question is can anyone identify a way to use fileInput to allow a user to locate a file and have the server side function capture only the path and NOT read in the file or try and parse it in any way? More importantly, can this be done using functions in base R and Shiny alone without having to grab functions from other extended packages?
The above is the relevant portion of code in the server.R file and the relevant portion of code in the ui.R file is
fileInput('path2dor', 'Choose the DOR .txt file to format',
accept=c('text/csv',
'text/comma-separated-values,text/plain', '.csv')),
Thank you for you advice.
This functionality is not possible with fileInput. The reason is because 'fileInput' do not provide local path information to the server for security reasons.
With fileInput the user navigates through the browser on his local machine, the resulting file on the server side is the uploaded copy of the selected local once.
As an alternative you can use the shinyFiles package, which do navigate through the server side. This means, that you get all the paths on your local machine.
A second alternative could be a simple text input, which lets the user add a path by hand (make sure to check the path on the server side to not run into any troubles).
As pointed by others, due to security concerns shiny creates a tmp folder with all files loaded called with fileinput
Accordingly, you need to select all the files of interest in your folder and then call this tmp file with ...$datapath
Note however that each element of datapath will include both the directory information and the corresponding file name. Accordingly, you need to trim those paths to only account for the tmp directory. This can be achieved as follows...
Assume you will create an object called phu which will only contain the first file in the folder (using input$upload$datapath[1]) you called with fileInput("upload", NULL, buttonLabel = "Upload...", multiple = TRUE)
phu<-as.character(input$upload$datapath[1])
phu<-substr(phu,1,nchar(phu)-5)
The second line removes the last five characters in the string. These characters are 0.txt or whatever other extension you called in your input. The code provided only works with .txt files and requires the tm package. You can now use the object phu as the input directory of interest.
Finally, you need to call this output with an output object and print it in your ui this is shown with textOutput("Pdiretory") below.
The following example shows the entire process. Note that there are no security concerns because this temporary file and its content will be deleted at closing. Once more, the input files are .txt files.
library(shiny)
library(tm)
ui <- fluidPage(
fileInput("upload", NULL, buttonLabel = "Upload...", multiple = TRUE),
textOutput("Pdiretory")
)
server <- function(input, output, session) {
listdir <- eventReactive(input$upload, {
phu<-as.character(input$upload$datapath[1])
phu<-substr(phu,1,nchar(phu)-5)
txt<-Corpus(DirSource(phu),readerControl = list(language = "en"))
print(txt)
})
output$Pdiretory <- renderPrint ({
listdir()
})
}
shinyApp(ui = ui, server = server)
Related
I have come up with a python function that I have confirmed works just fine. I am trying to put this into a Shiny app using Shiny's reticulate. I am not super familiar with Shiny but need to use it anyhow.
To give a bit of background on what I am doing, I've written some python code that takes takes multiple files and matches strings based on one common list of strings. This code works fine when I run the python files on my machine.
I need to make this available to others using a shiny app, where they can upload their files, then have the app run the underlying python code.
So far, I have set up the shiny app so that it can take in multiple files. I am having a hard time thinking about how I can use reactive to make a list of the file path names to then send to my python code (which includes a step to open and read the file) so it can do its thing.
This is the code that I have for my app thus far:
library(shiny)
library(shinyFiles)
# define UI
ui <- fluidPage(
titlePanel('Counter of Gendered Language'),
fileInput("upload", "Choose a folder",
multiple = TRUE,
accept = c('text')),
tableOutput('text'),
downloadButton('output', 'Download Count File .csv'))
# define server behavior
server <- function(input, output){
# Setup
#* Load libraries
library(reticulate)
#* Use virtual environment for python dependencies
use_virtualenv('file/path/py_venv', required = TRUE)
#* Source code
source_python('code/counting_gendered_words.py')
#* Load list of words to match raw text against
dictionary <- read.csv('data/word_rating.csv')
text <- reactive(
list <- list.files(path = input$upload[['name']])
)
output$counted <- gendered_word_counter(dictionary, text())
output$downloadData <- downloadHandler(
filename = function(){
paste0(input$upload, ".csv")
},
content = function(file){
vroom::vroom_write(text$counted, file)
}
)
}
# Run the application
shinyApp(ui = ui, server = server)
What it tells me when I run this app is that:
Error : Operation not allowed without an active reactive context.
You tried to do something that can only be done from inside a reactive consumer.
So what I am wanting to do is basically just pass each file name that someone uploads to the app and pass that file's name into my gendered_word_counter() python function.
How would I go about this?
I'm super confident that I just am being a newbie and it is probably a super simple fix. Any help from those who are more comfortable with Shiny would be much appreciated!
Edit: I notice that my code is only calling the names of the files which is meaningless for me without the contents of the uploaded files! Would it be better if I read the files in the shiny app instead of in my .py file?
I can't reproduce the app without the python code, but i can see that this line:
output$counted <- gendered_word_counter(dictionary, text())
has a reactive object (text()) being called with no reactive context. It should be wrapped in observe or observeEvent.
observe({
output$counted <- gendered_word_counter(dictionary, text())
})
Also let's add the parenthesis here:
content = function(file){
vroom::vroom_write(text()$counted, file)
}
I have a folder called logs filled with different .csv files, formatted as telemetryLog-2017.21.08.54.11.csv (with varying dates and times at end).
For example, the above file could be stored like this: file <- read.csv("logs/telemetryLog-1969.2017.21.08.54.11.csv", header=TRUE)
The log files would be uploaded (in the logs folder, to shinyapps.io) along with the ui.R and server.R files. I would like to be able to obtain a list of the filenames in order to be able to select a file to display as data in a plot via selectInput (or any other way to list the files). The amount of files in the folder will not be an excessive amount; most likely it will be limited to around 50.
I have read the documentation for shinyFiles and to be completely honest I do not fully understand how the commands such as fileGetter or dirGetter work. Any help would be appreciated.
Instead of having people browse the file system of your server, you could also use list.files and specify the right directory there:
library(shiny)
ui <- fluidPage(
selectInput('selectfile','Select File',choice = list.files('log/')),
textOutput('fileselected')
)
server <- function(input,output)
{
output$fileselected <- renderText({
paste0('You have selected: ', input$selectfile)
})
}
shinyApp(ui,server)
Hope this helps!
I am working on a shiny app that will read a few RData files in and show tables with the contents. These files are generated by scripts that eventually turns the data into a data frame. They are then saved using the save() function.
Within the shiny application I have three files:
ui.R, server.R, and global.R
I want the files to be read on an interval so they are updated when the files are updated, thus I am using:
reactiveFileReader()
I have followed a few of the instructions I have found online, but I keep getting an error "Error: missing value where TRUE/FALSE is needed". I have tried to simplify this so I am not using:
reactiveFileReader()
functionality and simply loading the file in the server.R (also tried in the global.R file). Again, the
load()
statement is reading in a data frame. I had this working at one point by loading in the file, then assigning the file to a variable and doing an "as.data.table", but that shouldn't matter, this should read in a data frame format just fine. I think this is a scoping issue, but I am not sure. Any help? My code is at:
http://pastebin.com/V01Uw0se
Thanks so much!
Here is a possible solution inspired by this post http://www.r-bloggers.com/safe-loading-of-rdata-files/. The Rdata file is loaded into a new environment which ensures that it will not have unexpected side effect (overwriting existing variables etc). When you click the button, a new random data frame will be generated and then saved to a file. The reactiveFileReader then read the file into a new environment. Lastly we access the first item in the new environment (assuming that the Rdata file contains only one variable which is a data frame) and print it to a table.
library(shiny)
# This function, borrowed from http://www.r-bloggers.com/safe-loading-of-rdata-files/, load the Rdata into a new environment to avoid side effects
LoadToEnvironment <- function(RData, env=new.env()) {
load(RData, env)
return(env)
}
ui <- shinyUI(fluidPage(
titlePanel("Example"),
sidebarLayout(
sidebarPanel(
actionButton("generate", "Click to generate an Rdata file")
),
mainPanel(
tableOutput("table")
)
)
))
server <- shinyServer(function(input, output, session) {
# Click the button to generate a new random data frame and write to file
observeEvent(input$generate, {
sample_dataframe <- data.frame(a=runif(10), b=rnorm(10))
save(sample_dataframe, file="test.Rdata")
rm(sample_dataframe)
})
output$table <- renderTable({
# Use a reactiveFileReader to read the file on change, and load the content into a new environment
env <- reactiveFileReader(1000, session, "test.Rdata", LoadToEnvironment)
# Access the first item in the new environment, assuming that the Rdata contains only 1 item which is a data frame
env()[[names(env())[1]]]
})
})
shinyApp(ui = ui, server = server)
Ok - I figured out how to do what I need to. For my first issue, I wanted the look and feel of 'renderDataTable', but I wanted to pull in a data frame (renderDataTable / dataTableOutput does not allow this, it must be in a table format). In order to do this, I found a handy usage of ReportingTools (from Bioconductor) and how they do it. This allows you to use a data frame directly and still have the HTML table with the sorts, search, pagination, etc.. The info can be found here:
https://bioconductor.org/packages/release/bioc/html/ReportingTools.html
Now, for my second issue - updating the data and table regularly without restarting the app. This turned out to be simple, it just took me some time to figure it out, being new to Shiny. One thing to point out, to keep this example simple, I used renderTable rather than the solution above with the ReportingTools package. I just wanted to keep this example simple. The first thing I did was wrap all of my server.R code (within the shinyServer() function) in an observe({}). Then I used invalidateLater() to tell it to refresh every 5 seconds. Here is the code:
## server.R ##
library(shiny)
library(shinydashboard)
library(DT)
shinyServer(function(input, output, session) {
observe({
invalidateLater(5000,session)
output$PRI1LastPeriodTable <- renderTable({
prioirtyOneIncidentsLastPeriod <- updateILP()
})
})
})
Now, original for the renderTable() portion, I was just calling the object name of the loaded .Rdata file, but I wanted it to be read each time, so I created a function in my global.R file (this could have been in server.R) to load the file. That code is here:
updateILP <- function() {
load(file = "W:/Projects/R/Scripts/ITPOD/itpod/data/prioirtyOneIncidentsLastPeriod.RData", envir = .GlobalEnv)
return(prioirtyOneIncidentsLastPeriod)
}
That's it, nothing else goes in the global.R file. Your ui.R would be however you have it setup, call tableOutout, dataTableOutput, or whatever your rendering method is in the UI. So, what happens is every 5 seconds the renderTable() code is read every 5 seconds, which in turns invokes the function that actually reads the file. I tested this by making changes to the data file, and the shiny app updated without any interaction from me. Works like a charm.
If this is inelegant or is not efficient, please let me know if it can be improved, this was the most straight-forward way I could figure this out. Thanks to everyone for the help and comments!
I'm working on an app in R where the users need to choose a file from their computer, with a RShiny fileInput button. I want to modify this, so that the associated variable can be assigned (i.e. a file can be loaded) automatically by the programm, without having the user click on the button and choose the file.
The problem I'm facing is that a fileInput has 4 fields, amongst which I only can know 3. For instance, when I load the file hello.csv in the variable inFile through the normal procedure, here is what I get :
inFile$name = hello.csv
inFile$size = 8320
inFile$type = text/csv
inFile$datapath = C:\\Users\\MyName\\AppData\\Local\\Temp\\Rtmpkh8Zcb/7d5f0ff0111d440c7a66b656/0
Though I could have guessed the second and the third one knowing the file, I have no idea how the datapath field is assigned...
I've tried to declare inFile as a NULL global variable, then to assign one by one the different fields, but I'm stuck with this last one. Is there an other way to do, like a function that mimics the behaviour of a user who clicks on the file input button and choose a specified file ?
Thank you very much.
If all you're looking to do is load a file initially, you don't have to rely on Shiny functions to do that. You can just rely on R functions. Set up your app like this:
ui <- shinyUI(
fileInput("inFile", label="Choose a file", multiple=F)
)
server <- shinyServer(function(input, output, session) {
values <- reactiveValues()
dat <- reactive({
if (is.null(inFile$datapath)) {
dat <- read.csv("path/to/your.csv")
values$file_name = "your.csv"
values$file_type = "csv"
values$file_size = file.size("path/to/your.csv")
values$file_path = "path/to/your.csv"
} else {
dat <- read.csv(inFile$datapath)
values$file_name = inFile$name
values$file_size = inFile$size
values$file_type = inFile$type
values$file_path = inFile$datapath
}
})
})
shinyApp(ui=ui, server=server)
In the above code, the Shiny app will start and see that inFile$datapath is NULL and will load a predefined file of your choosing. It won't run again until inFile changes, at which point it will load the file that the user pointed to.
Hope that helps.
Update
I changed the code above to use reactiveValues to store the pieces of information that need to be used throughout the app. If you just set those and then do a find/replace for input$inFile$datapath and replace it values$file_path, your code should work just fine.
Here is how I figured it out :
I edited the original code, so that all the read.csv(...) are replaced with calls to a data.frame global variable. I also added a small button that you need to click on before you continue. This button saves what you just loaded in the Database (if you chose a file with the fileInput) and assigns the right values to the global variables that will be needed for the following operations. If you chose no file at all, it will directly assign the variables from the data found in the Database.
So I did not find a proper solution to the problem, but this is a workaround that will do the job in my case.
#brittenb I couldn't get your reactive solution to work as I wanted to, that's why I ended up doing this another way. Thanks for having taken the time to think about it though.
I'm still open to suggestions on how to update the file in a fileInput without user interaction.
I am having difficulties in R Shiny with the fileInput. I am able to successfully upload multiple .txt or .csv files, however I want my application to be able to upload a filetype called ".fcs" files as well. In order to read such ".fcs" files, I have used the library flowCore under the bioConductor package. Within this flowcore package, there is a function called read.FCS that I often use to read in ".fcs" files into R.
The first argument of the read.FCS() function is a character string of the location of the file. This is where my issue lies: because the function needs the filepath to be in a charcater string, the backslashes in the datapath generated by fileInput() don't mesh with the character string expected by read.FCS (for example, when I print the datapath acquired from fileInput, Shiny returns something like: ...\AppData\Local\Temp\Rtm...PdV/99...42/0 - note the forward and backslashes).
My question is: is there any way to modify the datapath to not include backslashes, so that the read.FCS() function can work properly?
To illustrate this proble, I will attach a simplified version of my scripts below for debugging purposes, as well as a method to generate a .txt file and a .fcs file (you must first install the package flowCore). You will be able to read these both into R, but the .fcs file will be incapable of being read into the Shiny app. Upon launching shiny, when you select the “.txt” File Type option and upload the sample .txt file (generate below),Shiny heads this file. When you relaunch the app and select the “.fcs” file type option and upload the sample .fcs file (generated below), the file does not properly upload.
At first I thought this was Shiny corrupting the FCS file somehow upon upload. But this is not the case. As prrof, in the mainTab area you will see a printed file path. If you copy this filepath (when you have attempted to load the fcs file), put it in quotes and put it into the read.FCS(insert copied file path with quotes around it here), you will get an error. If, on the other hand, you replace all of the “\” with “/” in the read.FCS command and try it again, R will successfully read the file that was uploaded to Shiny. Thus, I am led to believe that if I could somehow replace the “\” with “/” in the filepath, I will be able to read my FCS files.
Thanks so much for the help – I really appreciate it: sample code attached below
install.packages(flowCore)
library(flowCore)
x = rnorm(n=20, mean=5, sd=10)
y = rnorm(n=20, mean=5, sd=10)
z = rnorm(n=20, mean=5, sd=10)
data<-data.frame(as.integer(x),as.integer(y),as.integer(z))
my_save_path<-"C:/Users/drew/Desktop/"
write(as.matrix(data), paste(my_save_path, "sample_textfile.txt", sep=""), ncolumns=3, sep="\t")
df.export<-new("flowFrame", as.matrix(data))
write.FCS(df.export, paste(my_save_path, "sample_fcsfile.fcs", sep=""))
#write.fcs is experimental, but the problem persists for raw .fcs files from other sources other than this export.
#####Verify that these files are real and can be looked at in r
txt.real<-read.delim(paste(my_save_path,"sample_textfile.txt", sep=""))
txt.real #yup, reading into R works
fcs.real<-read.FCS( paste(my_save_path, "sample_fcsfile.fcs", sep=""), transform=FALSE)
fcs.real<-exprs(fcs.real)
fcs.real #yup, this is real too
User Interface
shinyUI(navbarPage("App",
tabPanel("Overview",
sidebarLayout(
sidebarPanel("Load in Data",
selectInput("filetype", label = h3("File Type"),
choices=list(".fcs file", ".txt file"), selected=".txt file"),
fileInput("files", h3("File(s) Input"))),
mainPanel(
tableOutput("table1"),
textOutput("text")
)))
))
Server
library(flowCore)
shinyServer(function(input, output) {
fileName<-reactive({
inFile <- input$files
inFile$datapath
})
dataInput<-reactive({
if(input$filetype==".fcs file"){
library(flowCore)
FCS<-read.FCS(fileName(), transform=FALSE)
exprs(FCS)
}
if(input$filetype==".txt file") {
TXT<-read.delim(fileName())
TXT
}
})
####Heading the Table (debug)
output$table1<-renderTable({
if (is.null(input$files))
return(NULL)
head(dataInput())
})
#Seeing the file path
output$text<-renderText({
x<-fileName()
x
})
}
)