How can I get a golem app to generate logs when dockerized? - r

We have a dockerized golem app that runs fine except for not creating any output (log statements) when deployed in a docker container. In fact, we aren't even seeing any default shiny-server logs.
Here is app.R for our "AirSensorDataViewer" golem app:
pkgload::load_all(export_all = FALSE, helpers = FALSE, attach_testthat = FALSE)
options(
golem.app.prod = TRUE,
shiny.port = 3838,
shiny.host = '0.0.0.0'
)
AirSensorDataViewer::run_app()
And here is our Dockerfile (that builds on top of of a base image with all necessary packages):
FROM mazamascience/airsensor-dataviewer-base:1.0.1
# Create the build zone, copy the local directory over to the docker image, build and install R package.
RUN mkdir /build_zone
ADD . /build_zone
WORKDIR /build_zone
RUN R -e 'remotes::install_local(upgrade="never")'
# Remove sample apps
RUN rm -rf /srv/shiny-server/
# copy app to image
COPY . /srv/shiny-server/asdv
# add .conf file to image/container to preserve log file
COPY ./shiny-server.conf /etc/shiny-server/shiny-server.conf
# When run image and create a container, this container will listen on port 3838
EXPOSE 3838
# Avoiding running as root --> run container as user 'shiny' instead
# allow permission
RUN sudo chown -R shiny:shiny /srv/shiny-server
RUN chmod -R 755 /srv/shiny-server/asdv
# execute in the following as user --> imortant to give permission before that step
USER shiny
##run app
CMD ["/usr/bin/shiny-server.sh"]
And, lastly, our shiny-server.conf file:
# Instruct Shiny Server to run applications as the user "shiny"
run_as shiny;
# Define a server that listens on port 3838
server {
listen 3838;
# Define a location at the base URL
location /asdv/test/ {
# Host the directory of Shiny Apps stored in this directory
site_dir /srv/shiny-server/asdv;
# Log all Shiny output to files in this directory
log_dir /var/log/shiny-server;
# When a user visits the base URL rather than a particular application,
# an index of the applications available in this directory will be shown.
directory_index on;
}
}
Has anyone had success getting a dockerized golem app to create/write to files inside the docker container?

I hope the posted question might be helpful for those wanting to do this as it turns out everything works fine.
For the last few hours I was mistakenly typing docker run ... when I wanted to check the container log files. This creates a new container.
Instead, log files are indeed found in /var/log/shiny-server/ when I:
docker exec -ti airsensor-dataviewer-desktop /bin/bash
ls /var/log/shiny-server/

Related

How to resolve this: [WARN] shiny-server - Failed to delete log file /var/log/shiny-server/file.log

I am deploying a docker container using Google Cloud Run. The docker container contains a R shiny app. And after a while from viewing the app, I see in the Cloud Run logs, the following:
[WARN] shiny-server - Failed to delete log file /var/log/shiny-server/shiny-server-xxx.log: EPERM: operation not permitted, unlink '/var/log/shiny-server/shiny-server-xxx.log'
I want it to be able to delete the old log so it doesn't pile up in the container, and
stop this warning to occur. How do I accomplish this? Is it a permissions error? I tried looking up solutions for this, but I don't even see someone complaining about this error. Why am I getting this then? What does it mean by "unlink" the log file? Please tell me what I am missing to allow log files to be removed/deleted. Thank you for any info you can give me.
In /usr/bin/shiny-server.sh, the first few lines contain:
# Make sure the directory for individual app logs exists
mkdir -p /var/log/shiny-server
chown shiny.shiny /var/log/shiny-server
My etc/shiny-server/shiny-server.conf is the original file (I have not changed it in any way):
# Instruct Shiny Server to run applications as the user "shiny"
run_as shiny;
# Define a server that listens on port 3838
server {
listen 3838;
# Define a location at the base URL
location / {
# Host the directory of Shiny Apps stored in this directory
site_dir /srv/shiny-server;
# Log all Shiny output to files in this directory
log_dir /var/log/shiny-server;
# When a user visits the base URL rather than a particular application,
# an index of the applications available in this directory will be shown.
directory_index on;
}
}
In my Dockerfile, I have:
RUN ["chmod", "+x", "/usr/bin/shiny-server.sh"]
EXPOSE 3838
RUN sudo chown -R shiny:shiny /srv/shiny-server
USER shiny
CMD ["/usr/bin/shiny-server.sh"]
Note: the line with "USER shiny" helped to remove the warning in the logs stating that the app should be run as non-root.
==============================
UPDATE
I've changed my /usr/bin/shiny-server.sh file to be:
mkdir -p /var/log/shiny-server
chown -R shiny:shiny /var/log/shiny-server
But I am still seeing the log:
[WARN] shiny-server - Failed to delete log file /var/log/shiny-server/shiny-server-shiny-20210831-133933-46421.log: EPERM: operation not permitted, unlink '/var/log/shiny-server/shiny-server-shiny-20210831-133933-46421.log'
This is what the current permissions look like on the /var/log/shiny-server folder:
And this, although not the exact same log file as stated in the error, is the permissions on the log file itself in /var/log/shiny-server:
Shouldn't this be enough permissions to allow the shiny user to delete the log file?
Check your /usr/bin/shiny-server.sh. To change ownership recursively on all files and directories on a given directory, you should use -R option. Additionally, when changing ownership for both user and group in a single command, you should use a colon.
In /usr/bin/shiny-server.sh try this:
chown -R shiny:shiny /var/log/shiny-server
Check this document for chown command reference.
For anyone looking for the answer to my post, I basically left it alone and ignore those warnings. Based on #Dondi's comment, the logs shouldn't use up all the space allocated to my container since it will shutdown the instance when it sits idle. Plus the shiny logs are automatically written to stdout (I'm using rocker/shiny as my base image) so I won't lose any of the logs written by the shiny app.

How to create a docker image of an existing R Shiny app in windows?

The dockerization process of an R shiny app is done through rocker/shiny image. I require to run the dockerization on an R shiny app from a windows system. The only problem is that rocker/shiny used for this process is a Linux based container. Therefore, I am unable to generate an image that runs on my windows system. Utilization of an external VPS or VM is not an option for me due to security reasons.
I have tried writing a docker file that would run for windows, but the problem is that the container is Linux based.
My Dockerfile
FROM rocker/r-ver:3.5.1
#installing R libraries
RUN R -e "install.packages(c('shiny', 'shinydashboard'), repos='http://cran.us.r-project.org')"
# Copy configuration files into the Docker image
COPY shiny-server.conf /etc/shiny-server/shiny-server.conf
COPY /app/srv/shiny-server/
# Make the ShinyApp available at port 80
EXPOSE 80
#Copy further configuration files into the Docker image
COPY shiny-server.sh /usr/bin/shiny-server.sh
CMD ['Rscript', 'app_run.R']
(original documentation mentions a .sh file which is not running on my system
because of the Linux windows file path issues)
After running there 2 commands I expect my shiny app to run on localhost:8080
Get-Content Dockerfile | docker build -
docker run --name DOCKERapp -p 8080:8080 <image_name>
The base in your dockerfile is rocker/r-ver:3.5.1 and the based of this image is debian. You can check here:
https://hub.docker.com/r/rocker/r-ver/dockerfile
So in your dockerfile you need to use linux commands. You don't need to use windows commands.

How to run R Shiny App in Docker Container

I built a Docker Image for an R Shiny App and ran the corresponding container with Docker Toolbox on Windows 10 Home. When trying to open the App with my web browser, only the index is shown. I don't know why the app isn't executed.
The log shows me this:
*** warning - no files are being watched ***
[2019-08-12T15:34:42.688] [INFO] shiny-server - Shiny Server v1.5.12.1 (Node.js v10.15.3)
[2019-08-12T15:34:42.704] [INFO] shiny-server - Using config file "/etc/shiny-server/shiny-server.conf"
[2019-08-12T15:34:43.100] [INFO] shiny-server - Starting listener on http://[::]:3838
I already specified the app host-to-container path by executing the following command which refers to a docker hub image:
docker run --rm -p 3838:3838 -v /C/Docker/App/:/srv/shinyserver/ -v /C/Docker/shinylog:/var/log/shiny-server/ didsh123/ps_app:heatmap
My Docker File looks like the following:
# get shiny serves plus tidyverse packages image
FROM rocker/shiny-verse:latest
# system libraries of general use
RUN apt-get update && apt-get install -y \
sudo \
pandoc \
pandoc-citeproc \
libcurl4-gnutls-dev \
libcairo2-dev \
libxt-dev \
libssl-dev \
libssh2-1-dev
##Install R packages that are required--> were already succesfull
RUN R -e "install.packages(c('shinydashboard','shiny', 'plotly', 'dplyr', 'magrittr'))"
#Heatmap related packages
RUN R -e "install.packages('gpclib', type='source')"
RUN R -e "install.packages('rgeos', type='source')"
RUN R -e "install.packages('rgdal', type='source')"
# copy app to image
COPY ./App /srv/shiny-server/App
# add .conf file to image/container to preserve log file
COPY ./shiny-server.conf /etc/shiny-server/shiny-server.conf
##When run image and create a container, this container will listen on port 3838
EXPOSE 3838
###Avoiding running as root --> run container as user instead
# allow permission
RUN sudo chown -R shiny:shiny /srv/shiny-server
# execute in the following as user --> imortant to give permission before that step
USER shiny
##run app
CMD ["/usr/bin/shiny-server.sh"]
So when I address the docker ip and the assessed port in the browser, the app should run there, but only the index is displayed. I use the following line:
http://192.168.99.100:3838/App/
I'm glad for every hint or advice. I'm new to Docker, so I'm also happy for detailed explanations.
To use shiny with docker, I suggest you use the golem package. golem provides a framework for builing shiny apps. If you have an app developed according to their framework, the function golem::add_dockerfile() can be used to create dockerfiles automatically.
If you are not interested in a framework, You can still have a look at the source for add_dockerfile() to see how they manage the deployment. Their strategy is to use shiny::runApp() with the port argument. Therefore, shiny-server is not necessary in this case.
The Dockerfile in golem looks roughly like this
FROM rocker/tidyverse:3.6.1
RUN R -e 'install.packages("shiny")'
COPY app.R /app.R
EXPOSE 3838
CMD R -e 'shiny::runApp("app.R", port = 3838, host = "0.0.0.0")'
This will make the app available on port 3838. Of course, you will have to install any other R packages and system dependencies.
Additional tips
To increase reproducibility, I would suggest you use remotes::install_version() instead of install.packages().
If you are going to deploy several applications with similar dependencies (for example shinydashboard), it makes sense to write your own base image that can be used in place of rocker/tidyverse:3.6.1. This way, your builds will be much quicker.
Minimal Reproducible Example
Create an empty directory (can be called anything you want)
Inside it, create two things:
i. A file called Dockerfile
ii. An empty directory called app
Place your shiny app inside the directory called app.
Your shiny app can be a single app.R file, or, for older shiny apps, ui.R and server.R. Either way is fine (see here for more on that).
If unsure about any of the above, just copy the files found here.
Place this in Dockerfile
FROM rocker/shiny:latest
COPY ./app/* /srv/shiny-server/
CMD ["/usr/bin/shiny-server"]
In the terminal, cd to the root of the empty directory you created in step 1, and build the image with
docker build -t shinyimage .
Run the container with
docker run -p 3838:3838 shinyimage
Finally, visit this url to see your shiny app: http://localhost:3838/
Here's a copy of all of the above in case anything's unclear.
Check the logs for any useful information? And exec into the container to verify if the App content is copied to the correct location.
Because the way /App content is copied looks incorrect
The content of /App is copied into the image to /srv/shiny-server/App during the build phase and you are trying to override /srv/shiny-server content using -v option when running the container.
Looks like during runtime the App data copied is being overwritten.
Try without -v /C/Docker/App/:/srv/shinyserver/ or use -v /C/Docker/App/:/srv/shinyserver/App/
docker run --rm -p 3838:3838 -v /C/Docker/shinylog:/var/log/shiny-server/ didsh123/ps_app:heatmap

Shiny server docker app runs locally but not when deployed to AWS Fargate

I have a containerized R Shiny app that, when run locally with docker run --rm -p 3838:3838 [image] works as expected. The "landing" page appears when I go to localhost:3838 and all is good. However, when this container is deployed to AWS Fargate, things break down. The container appears to start and run, but there's no webpage being served on 3838 even though all ports are pointed to 3838 in Fargate.
I'm using dockerfile
FROM rocker/verse:3.5.1
LABEL Steven "email"
## Add shiny capabilities to container
RUN export ADD=shiny && bash /etc/cont-init.d/add
## Update and install
RUN tlmgr update --self
RUN tlmgr install beamer translator
## Add R packages
RUN R -e "install.packages(c('shiny', 'googleAuthR', 'dplyr', 'googleAnalyticsR', 'knitr', 'rmarkdown', 'jsonlite', 'scales', 'ggplot2', 'reshape2', 'Cairo', 'tinytex'), repos = 'https://cran.rstudio.com/')"
#Copy app dir and them dirs to their respective locations
COPY app /srv/shiny-server/ga-reporter
COPY app/report/themes/SwCustom /opt/TinyTeX/texmf-dist/tex/latex/beamer/
#Force texlive to find my custom beamer thems
RUN texhash
EXPOSE 3838
## Add shiny-server information
COPY shiny-server.sh /usr/bin/shiny-server.sh
COPY shiny-customized.config /etc/shiny-server/shiny-server.conf
## Add dos2unix to eliminate Win-style line-endings and run
RUN apt-get update && apt-get install -y dos2unix
RUN dos2unix /usr/bin/shiny-server.sh && apt-get --purge remove -y dos2unix && rm -rf /var/lib/apt/lists/*
RUN ["chmod", "+x", "/usr/bin/shiny-server.sh"]
CMD ["/usr/bin/shiny-server.sh"]
and shiny-server.conf
# Instruct Shiny Server to run applications as the user "shiny"
run_as shiny;
# Define a server that listens on port 3838
server {
listen 3838;
# Define a location at the base URL
location / {
# Host the directory of Shiny Apps stored in this directory
app_dir /srv/shiny-server/ga-reporter;
# Log all Shiny output to files in this directory
log_dir /var/log/shiny-server;
# When a user visits the base URL rather than a particular application,
# an index of the applications available in this directory will be shown.
directory_index on;
}
}
with shiny-server.sh
#!/bin/sh
# ShinyServer: Make sure the directory for individual app logs exists
mkdir -p /var/log/shiny-server
chown -R shiny.shiny /var/log/shiny-server
# RUN ShinyServer
exec shiny-server >> /var/log/shiny-server.log 2>&1
I have edited the .conf file to display the app (i.e., location /) in /srv/shiny-server/ga-reporter which is also where I've copied the app_dir in the Dockerfile. Shiny is listening on port 3838 and should serve the page there. Again, this happens locally but not when deployed to AWS Fargate. I've tried logging Shiny logs to stdout by using the first answer provided here but have had no luck seeing any errors generated. Server "health checking" is only offered in the "pro" version so I can't check to see if the server is actually running.
On AWS, the container starts and appears to function normally (i.e., the "normal" start up logs appear):
but there is simply no page displayed at the location I expect it to be served.
I found another Shiny app that is on dockerhub running under the same configuration as the Fargate cluster but have had no luck trying to implement anything in the shiny-server.conf or the shiny-server.sh files located there.
What am I missing? Everything on Fargate is pointed to listening on 3838; there must be something I'm missing in the .conf file for this to be failing when deployed.
EDIT
I can't bash in to the running container on Fargate because I don't have access to the server on which docker is running.
Fargate has a UI that accepts host and container ports:
EDIT 2 (2018-08-27)
The engineer that was deploying this has been able to resolve the issue:
"it was the port change, I forgot to can the port on the ALB’s security group, I only updated the cluster’s inbound rules
so the cluster was allowing connections, but the ALB security group wasn’t letting it out"

Deploy shiny app in rocker/shiny docker

Well, I'm new at Docker and I need to implement a Shiny app in a Docker Container.
I have the image from https://hub.docker.com/r/rocker/shiny/, that includes Shiny Server, but I don't know how to deploy my app in the server.
I want to deploy the app in the server, install the required packages for my app into the Docker, save the changes and export the image/container.
As I said, I'm new at Docker and I don't know how it really works.
Any idea?
I guess you should start by creating a Dockerfile in a specific folder which would look like something like this :
FROM rocker/shiny:latest
RUN echo 'install.packages(c("package1","package2", ...), \
repos="http://cran.us.r-project.org", \
dependencies=TRUE)' > /tmp/packages.R \
&& Rscript /tmp/packages.R
EXPOSE 3838
CMD ["/usr/bin/shiny-server.sh"]
Then go into this folder and build your image, giving it a name by using this command :
docker build -t your-tag .
Finally, once your image is built you can create a container, and if you don't forget to map the volume and the port, you should be able to find it at localhost:3838 with the following command launched from the folder containing the srv folder :
docker run --rm -p 3838:3838 -v $PWD/srv/shinyapps/:/srv/shiny-server/ -v $PWD/srv/shinylog/:/var/log/shiny-server/ your-tag
As said in the Docker documentation at the following address https://hub.docker.com/r/rocker/shiny/, you might want to launch it in detached mode with -d option and map it with your host's port 80 for a real deployment.
The link(https://hub.docker.com/r/rocker/shiny/) covers how to deploy the shiny server.
Simplest way would be:
docker run --rm -p 3838:3838 rocker/shiny
If you want to extend shiny server, you can write your own Dockerfile and start with shiny image as base image.(https://docs.docker.com/engine/reference/builder/)
Dockerfile:
FROM rocker/shiny:latest

Resources