In my project I developed algorithms with R and R studio to process big images from microscopy analysis. I work on big matrix of pixel value (around 3Gb ), extract biological informations from specific regions of these images especially perform particle detection at specific position of the matrix and then perform statistical analysis.
To be able to transfer the whole process to other people without applying multiple scripts functions... I tried to create a Shiny App but I struggle to import the big images .txt
Actually I use as.matrix(read.table()) function on the file provided by the shiny app user through the file input box
Is any one has an idea to import these matrix easily and more quickly ? even on Rstudio I wait ~20minutes for each picture import.
Thanks a lot in advance for the answers,
Best,
Romain
Related
So I'm working with a network dataset from Stanford's SNAP Datasets and "SNAP" has wrappers for Python and C++ but not R - however, the data is still usable since I believe it's a mix of CSV files.
I can actually read in the .edges file and form an igraph object but want to read in the other files, get the attributes & add those attributes to the igraph object for analysis. I'm just confused on how to work with the .circles, .egofeat, .feat, and .featnames files since the documentation on the dataset is very scarce. Hoping someone has worked with the dataset in R or even another language and has any pointers to get started.
Thank you!
For a shiny app in a repository containing a single static data file, what is the optimal format for that flat file (and corresponding function to read that file) which minimises the read time for that flat file to a data.frame?
For example, suppose when a shiny app starts it reads an .RDS, but suppose that takes ~30 seconds and we wish to decrease that. Are there any methods of saving the file and using a function which can save time?
Here's what I know already:
I have been reading some speed comparison articles, but none seem to comprehensive benchmark all methods in the context of a shiny app (and possible cores/threading implications). Some offer sound advice like trying to load in less data
I notice languages like julia can sometimes be faster, but I'm not sure if reading a file using another language would help since it would have to be converted to an object R recognises, and presumably that process would take longer than simply reading as an R object initially
I have noticed identical files seem to be smaller when saved as .RDS compared to .csv, however, I'm not sure if file size necessarily has an effect on read time.
I have a question about the limitation of data in R shiny. I am now working on updating a previous project. The original data is around 5MB and the program would resample the data to obtain an estimate for future values. I am not updating the program to make it more general, where I try to import 300MB data. However, the R shiny would crack. I have used R to handle larger data before. But I am not sure if R shiny has any limitation of data size. Does anyone have any idea about it. Thanks.
I am trying to create a simple data viewing gui for neurophysiological recordings from EEG or MEG using R in which one could select any time point in the recorded timeseries and see the corresponding interpolated scalp distribution of the data.
What this means is that I have two graphs that need to be constantly updated on the basis of user input and I find R's regular graphs to be too slow for this task. I have recently found the acinonyx graphics device which is indeed much faster than R's regular one, but I have not been able to embed graphs made by this device in a gui using gWidgets, nor could I find any documentation on how to embed it in a gui using other packages.
Is this possible at all? If not, what are the options for faster graphic devices in R that can be embedded into a gui?
Currently, I generate results from statistical analyses (e.g., a three-dimensional plot) and then "manually" move it to processing - a graphics programming language) where I can (with some simple coding) export an interactive java applet (e.g., allow the person viewing the plot to move in, out, and around the data points). Can I keep this whole process within R? Specifically, I want to create an applet (doesn't have to be Java but would need to be web embeddable, interactive (so not a movie), and not require the user to work in R or have to download things) that can be passed on.
Thanks.
Not totally clear on your requirements: can you be sure that the user will have R installed (e.g. can you run a script on their desktops to install everything first)? Does it have to run over the web?
The animation package (http://cran.r-project.org/web/packages/animation/) isn't interactive, but it can create moving images.
The iplots package is useful, although it requires R: http://rosuda.org/iPlots/iplots.html
Similarly, rggobi is extremely useful for interactive graphics, but it also requires R. You can read more http://www.jstatsoft.org/v30/b07/paper and http://www.ggobi.org/rggobi/.
A last example is biplotgui: http://r-forge.r-project.org/projects/biplotgui/
I heard that there's a project in development to create Flash output from R, but I can't find anything about it.
Can I keep this whole process within R?
Check out GGobi:
GGobi is an open source visualization program for exploring high-dimensional data. It provides highly dynamic and interactive graphics such as tours, as well as familiar graphics such as the scatterplot, barchart and parallel coordinates plots. Plots are interactive and linked with brushing and identification.