R parallel computing and Shiny Server using Dockerfile - r

I need some help with using parallel computing and the package future with shiny inside a Docker container. When I insert my app in shinyapps.io it works normally, but if I use Docker it gives me the following log error.
[2019-04-22T19:49:24.320] [INFO] shiny-server - Shiny Server v1.5.7.890 (Node.js v8.10.0) [2019-04-22T19:49:24.325] [INFO] shiny-server - Using config file "/etc/shiny-server/shiny-server.conf"
[2019-04-22T19:49:24.440] [INFO] shiny-server - Starting listener on 0.0.0.0:80
[2019-04-22T19:49:29.399] [INFO] shiny-server - Created bookmark state directory: /var/lib/shiny-server/bookmarks
[2019-04-22T19:49:29.401] [INFO] shiny-server - Created user bookmark state directory: /var/lib/shiny-server/bookmarks/shiny
[2019-04-22T19:50:30.486] [INFO] shiny-server - Error getting worker: Error: The application took too long to respond.
[2019-04-22T19:50:30.488] [INFO] shiny-server - Error getting worker: Error: The application took too long to respond.
[2019-04-22T19:50:30.489] [INFO] shiny-server - Error getting worker: Error: The application took too long to respond.
My Dockerfile uses R, shiny-server, a connection to Postgres and sqlite. If I don't use the future it works without errors.
# Install R version 3.5
FROM r-base:3.5.1
# Install Ubuntu packages
RUN apt-get update && apt-get install -y \
sudo \
gdebi-core \
pandoc \
pandoc-citeproc \
libcurl4-gnutls-dev \
libcairo2-dev/unstable \
libxt-dev \
libssl-dev \
libpq-dev
# Download and install ShinyServer (latest version)
RUN wget --no-verbose https://s3.amazonaws.com/rstudio-shiny-server-os-build/ubuntu-12.04/x86_64/VERSION -O "version.txt" && \
VERSION=$(cat version.txt) && \
wget --no-verbose "https://s3.amazonaws.com/rstudio-shiny-server-os-build/ubuntu-12.04/x86_64/shiny-server-$VERSION-amd64.deb" -O ss-latest.deb && \
gdebi -n ss-latest.deb && \
rm -f version.txt ss-latest.deb
# Install SQLite
RUN apt-get update && \
DEBIAN_FRONTEND=noninteractive apt-get -yq install sqlite3 && \
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
# Install R packages that are required
# TODO: add further package if you need!
RUN R -e "install.packages(c('RSQLite', 'RPostgres', 'readxl', 'dplyr', 'lubridate', 'DT', 'billboarder', 'shiny', 'future'), repos='http://cran.rstudio.com/')"
# Copy configuration files into the Docker image
COPY shiny-server.conf /etc/shiny-server/shiny-server.conf
COPY /app /srv/shiny-server/
# Give permission of READWRITE for the SQLite database
RUN chmod a+rw /srv/shiny-server/db/ /srv/shiny-server/db/*
# Make the ShinyApp available at port 80
EXPOSE 80
# Copy further configuration files into the Docker image
COPY shiny-server.sh /usr/bin/shiny-server.sh
CMD ["/usr/bin/shiny-server.sh"]
For an example app I've tried a global.R file loading shiny and future and executing something like
library(shiny)
library(future)
plan(multiprocess)
future({print(1)})
with minimal ui.R (like FluidPage()) and server.R (like server <- function(input, output) {}). My bet are that I'm using Windows 7 environment and that causes a confusion in the multiprocess, if so I can just send the app to my IT right now for them to deploy it on the Linux server, OR it's lacking a linux library or something like that, which I couldn't find out yet. Thank you in advance.

Related

Docker does not find R despite pre-built layers

I create a docker image to run R scripts on a VM server with no access to the internet.
For the first layer I load R and all libraries
Dockerfile1
FROM r-base
## Needed to access R
ENV R_HOME /usr/lib/R
## install required libraries
RUN apt-get update
RUN apt-get -y install libgdal-dev
## install R-packages
RUN R -e "install.packages('dplyr',dependencies=TRUE, repos='http://cran.rstudio.com/')"
...
and create it
docker build -t mycreate_od/libraries -f Dockerfile1 .
Then I use this library layer to load the R script
Dockerfile2
FROM mycreate_od/libraries
## Create directory
RUN mkdir -p /home/analysis/
## Copy files
COPY my_script_dir /home/analysis/
## Run the script
CMD R -e "source('/home/analysis/my_script_dir/main.R')"
Create the analysis layer
docker build -t mycreate_od/analysis -f vault/Dockerfile2 .
On my master VM, this runs and suceeds, but on the fresh VM I get
docker run mycreate_od/analysis
R docker ERROR: R_HOME ('/usr/lib/R') not found - Recherche Google
From a previous bug search I have set the ENV variable in the Docker (see Dockerfile1),
but it looks like docker installs R on some other place.
Thanks to Dirk advice I managed to get it done using r-apt (see Dockerfile below).
The image get then built and can be run without the R_HOME error.
BTW much faster and with a significantly smaller resulting image.
FROM rocker/r-apt:bionic
RUN apt-get update && \
apt-get -y install libgdal-dev && \
apt-get install -y -qq \
r-cran-dplyr \
r-cran-rjson \
r-cran-data.table \
r-cran-crayon \
r-cran-geosphere \
r-cran-lubridate \
r-cran-sp \
r-cran-R.utils
RUN R -e 'install.packages("tools")'
RUN R -e 'install.packages("rgdal", repos="http://R-Forge.R-project.org")'
This is unfortunately a cargo-boat solution, as I am unable to explain why the previous Dockerfile failed.

Why renv in docker is not installing and loading packages?

This is the first time I'm using Docker. I have tried a few times but this is the first time where I'm containerizing the project. The project generates reports using latex and officer. However, after building the container and running the image, Rstudio shows that none of the packages are installed and loaded.
Dockerfile
FROM rocker/verse:4.0.5
ENV RENV_VERSION 0.15.1
RUN R -e "install.packages('remotes', repos = c(CRAN = 'https://cloud.r-project.org'))"
RUN R -e "remotes::install_github('rstudio/renv#${RENV_VERSION}')"
WORKDIR /REAL
COPY . .
RUN apt-get -y update --fix-missing
#RUN apt-get -y install build-essential \
# libssl-dev \
# libcurl4-openssl-dev \
# libxml2-dev \
# libfontconfig1-dev \
# libcairo2-dev
RUN R -e 'options(renv.consent = TRUE); renv::restore(lockfile = "/REAL/renv.lock"); renv::activate()'
Building Docker image
docker build -t reports/real .
Running docker with rstudio support
docker run --rm -d -p 8787:8787 --name real_report -e ROOT=TRUE -e DISABLE_AUTH=TRUE -v C:/Users/test/sources/REAL_Reports:/home/rstudio/REAL_Reports reports/real:latest
What am I doing wrong and how do I fix it so that all packages that renv has discovered are installed and loaded?
RUN R -e 'options(renv.consent = TRUE); renv::restore(lockfile = "/REAL/renv.lock"); renv::activate()'
You likely need to activate / load your project before calling restore(), or just forego using a project-local library altogether.
https://rstudio.github.io/renv/articles/docker.html may also be useful here.

Azure container - simple R plumber deployment question?

I am deploying simple plumber API in Azure container which executes simple SQL query and returns results. Here is my Dockerfile.
# start from the rocker/r-ver:4.0.2 image
# minor change to one branch to test deployment
FROM rocker/r-ver:4.0.2
# install the linux libraries needed for plumber
RUN apt-get update -qq && apt-get install -y \
libssl-dev \
libcurl4-gnutls-dev \
libsodium-dev \
unixodbc-dev \
r-cran-rodbc \
curl\
libxml2-dev
# libiodbc2-dev
# We need to update Dockerfile to install ODBC Driver 17 for SQL Server for
# Ubuntu 20.04 or different version
# here is link: https://learn.microsoft.com/en-us/sql/connect/odbc/linux-mac/installing-the-microsoft-odbc-driver-for-sql-server?view=sql-server-ver15#ubuntu17
RUN curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
RUN curl https://packages.microsoft.com/config/ubuntu/20.04/prod.list > /etc/apt/sources.list.d/mssql-release.list
RUN apt-get update
RUN ACCEPT_EULA=Y apt-get install -y msodbcsql17
RUN ACCEPT_EULA=Y apt-get install -y mssql-tools
RUN echo 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bash_profile
RUN echo 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bashrc
# RUN source /etc/bash.bashrc
RUN mkdir -p ~/application
# copy everything from the current directory into the container
COPY "/" "application/"
# I probably do not even have to set up WORKDIR
WORKDIR "application/"
# open port 80 to traffic
EXPOSE 80
# install plumber
RUN R -e "install.packages('plumber')"
RUN R -e "install.packages('renv')"
RUN R -e "install.packages('RODBC')"
RUN R -e 'renv::restore()'
# when the container starts, start the main.R script
ENTRYPOINT ["Rscript", "execute_plumber.R"]
API works fine. However, when I connect to container, run R and execute library(RODBC) I get back error about library not found. This is puzzling since it is already mentioned in Dockerfile, but maybe it has something to do with renv. I proceed with installation install.packages('RODBC') and it generates error which I can fix by running apt-get install unixodbc-dev and apt-get install r-cran-rodbc. Those lines are again already present in Dockerfile. Then when I try to connect to SQL server I get error about driver. And again I can fix it by re-executing lines from Dockerfile (related to SQL Server 17 driver).
So I am puzzled why I need to re-execute some items already present in Dockerfile. Or maybe when I click "Connect" in Azure it actually connects me to host and not to container?

cannot install devtools in r shiny docker

I'm trying to build a docker image for my shiny app. Below is my dockerfile. When I build my images, everything else seems fine, except I got error message Error in library(devtools) : there is no package called ‘devtools’ Execution halted. I also tried devtools::install_github('nik01010/dashboardthemes') with no success. I have non clue why? What could go wrong? Do anyone know what is wrong with my dockerfile? Thanks a lot.
# Install R version 3.6
FROM r-base:3.6.0
# Install Ubuntu packages
RUN apt-get update && apt-get install -y \
sudo \
gdebi-core \
pandoc \
pandoc-citeproc \
libcurl4-gnutls-dev \
libcairo2-dev/unstable \
libxt-dev \
libssl-dev
# Download and install ShinyServer (latest version)
RUN wget --no-verbose https://s3.amazonaws.com/rstudio-shiny-server-os-build/ubuntu-12.04/x86_64/VERSION -O "version.txt" && \
VERSION=$(cat version.txt) && \
wget --no-verbose "https://s3.amazonaws.com/rstudio-shiny-server-os-build/ubuntu-12.04/x86_64/shiny-server-$VERSION-amd64.deb" -O ss-latest.deb && \
gdebi -n ss-latest.deb && \
rm -f version.txt ss-latest.deb
# Install R packages that are required
RUN R -e "install.packages(c('devtools', 'shiny','shinythemes','shinydashboard','shinyWidgets','shinyjs', 'tidyverse', 'dplyr', 'ggplot2','rlang','DT','lubridate', 'plotly', 'leaflet', 'mapview', 'tigris', 'rgdal', 'visNetwork', 'wordcloud2', 'arules'), repos='http://cran.rstudio.com/')"
RUN R -e "library(devtools)"
RUN R -e "install_github('nik01010/dashboardthemes')"
# Copy configuration files into the Docker image
COPY shiny-server.conf /etc/shiny-server/shiny-server.conf
COPY /app /srv/shiny-server/
# Make the ShinyApp available at port 80
EXPOSE 80
# Copy further configuration files into the Docker image
COPY shiny-server.sh /usr/bin/shiny-server.sh
CMD ["/usr/bin/shiny-server.sh"]
There are a few approaches you might try.
Easiest:
Use remotes::install_github instead of devtools. remotes has many fewer dependencies if you don't need the other functionality.
Second Easiest:
Use rocker/tidyverse image from Docker Hub instead of baseR image.
docker pull rocker/tidyverse
Change line 2:
FROM rocker/verse
Hardest:
Otherwise, you will need to figure out which dependencies you need to install inside your docker image before you can install devtools. It will likely be obvious if you try to install it interactively.
Make sure the container is running
Get the container name using docker ps
Start a shell with docker exec -it <container name> /bin/bash
Start R and try to install devtools interactively

Docker, R, and Shiny: 'unstable' is invalid for APT::Default-Release

I have a Shiny app that returns a .pdf to a user based upon some reactive statements. The app runs as it should locally (Win10, R3.5.0). I'm trying to deploy the app to AWS with a Docker image that combines rocker/verse (to output .pdf files built from .Rmd) with Shiny.
I use the following Dockerfile:
FROM rocker/verse
MAINTAINER ME
RUN apt-get update && apt-get install -y -t unstable \
sudo \
gdebi-core \
pandoc \
pandoc-citeproc \
libcurl4-gnutls-dev \
libcairo2-dev/unstable \
libxt-dev && \
wget --no-verbose https://download3.rstudio.org/ubuntu-14.04/x86_64/VERSION -O "version.txt" && \
VERSION=$(cat version.txt) && \
wget --no-verbose "https://download3.rstudio.org/ubuntu-14.04/x86_64/shiny-server-$VERSION-amd64.deb" -O ss-latest.deb && \
gdebi -n ss-latest.deb && \
rm -f version.txt ss-latest.deb && \
R -e "install.packages(c('shiny', 'rmarkdown'), repos='https://cran.rstudio.com/')" && \
rm -rf /var/lib/apt/lists/*
RUN R -e "install.packages(c('shiny', 'dplyr', 'googleAuthR', 'googleAnalyticsR', 'rmarkdown', 'jsonlite', 'scales', 'ggplot2', 'reshape2', 'knitr', 'Cairo'), repos='http://cran.rstudio.com/')"
## Uncomment the line below to include a custom configuration file. You can download the default file at
## https://raw.githubusercontent.com/rstudio/shiny-server/master/config/default.config
## (The line below assumes that you have downloaded the file above to ./shiny-customized.config)
## Documentation on configuration options is available at
## http://docs.rstudio.com/shiny-server/
COPY shiny-customized.config /etc/shiny-server/shiny-server.conf
COPY app /srv/shiny-server/
EXPOSE 3838
COPY shiny-server.sh /usr/bin/shiny-server.sh
CMD ["/usr/bin/shiny-server.sh"]
The first RUN statement is taken verbatim (minus one line which copies app information) from the rocker/shiny Docker file here. My .config and .sh files are identical to those in the rocker/shiny github. I get the following error:
I'm quite new to Docker and Linux CLI, but not to R. What am I missing here? How can I resolve this issue?
I considered opening this as an issue on the rocker/shiny github but it's not an issue with rocker/shiny as when I C: docker pull rocker/shiny and docker run --rm -p 3838:3838 rocker/shiny there is no issue. It's only when I try to build a pseudo rocker/verse/rocker/shiny container image that I get the above error.
Just removing the -t unstable should work.
Background: rocker/shiny is based on rocker/r-base which uses a mixture of Debain testing and unstable to always have the most recent version of R available. rocker/verse on the other hand is based on rocker/r-ver which uses Debain stable as basis and compiles a specified version of R on top of that.

Resources