Question:
Where/How do I download my own package that is built with devtools::build() into the Linux docker filesystem.
tidyjson is failing to install because it needs rlang 1.0.2 and I have rlang 1.0.1 --> which was determined from golem::add_dockerfile()
Background:
I am writing a Dockerfile for my {golem} + {brochure} Shiny app. I have used golem::add_dockerfile to start the template of the build and have only added on top of that. Currently the .tar.gz is in my project root dir. When I go to run my container I am met with an error --> Error in loadNamespace(x) : there is no package called ‘humblFinance’; Execution Halted,. This is likely coming from CMD R -e "options(shiny.port = ${port}, shiny.host = '${host}') ; humblFinance::run_app().
Dockerfile
FROM rocker/shiny-verse:4.1.3
LABEL maintainer = "humblFinance <humblfinance#gmail.com>"
RUN apt-get update && apt-get install -y \
git-core \
libcairo2-dev \
libcurl4-openssl-dev \
libgit2-dev libicu-dev \
libssl-dev \
libxml2-dev \
make \
pandoc \
pandoc-citeproc \
zlib1g-dev \
libquantlib0-dev \
&& rm -rf /var/lib/apt/lists/*
RUN echo "options(repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.method = 'libcurl', Ncpus = 4)"\
>> /usr/local/lib/R/etc/Rprofile.site
RUN install2.r --error --skipinstalled --deps TRUE \
remotes
RUN Rscript -e 'remotes::install_version("magrittr",upgrade="never", version = "2.0.2")'
RUN Rscript -e 'remotes::install_version("rlang",upgrade="never", version = "1.0.1")'
RUN Rscript -e 'remotes::install_version("glue",upgrade="never", version = "1.6.1")'
RUN Rscript -e 'remotes::install_version("processx",upgrade="never", version = "3.5.2")'
RUN Rscript -e 'remotes::install_version("stringr",upgrade="never", version = "1.4.0")'
RUN Rscript -e 'remotes::install_version("lubridate",upgrade="never", version = "1.8.0")'
RUN Rscript -e 'remotes::install_version("quantmod",upgrade="never", version = "0.4.18")'
RUN Rscript -e 'remotes::install_version("htmltools",upgrade="never", version = "0.5.2")'
RUN Rscript -e 'remotes::install_version("httr",upgrade="never", version = "1.4.2")'
RUN Rscript -e 'remotes::install_version("zoo",upgrade="never", version = "1.8-9")'
RUN Rscript -e 'remotes::install_version("xts",upgrade="never", version = "0.12.1")'
RUN Rscript -e 'remotes::install_version("ggplot2",upgrade="never", version = "3.3.5")'
RUN Rscript -e 'remotes::install_version("knitr",upgrade="never", version = "1.37")'
RUN Rscript -e 'remotes::install_version("testthat",upgrade="never", version = "3.0.4")'
RUN Rscript -e 'remotes::install_version("shiny",upgrade="never", version = "1.7.1")'
RUN Rscript -e 'remotes::install_version("PerformanceAnalytics",upgrade="never", version = "2.0.4")'
RUN Rscript -e 'remotes::install_version("shinyWidgets",upgrade="never", version = "0.6.4")'
RUN Rscript -e 'remotes::install_version("rmarkdown",upgrade="never", version = "2.12")'
RUN Rscript -e 'remotes::install_version("DT",upgrade="never", version = "0.21")'
RUN Rscript -e 'remotes::install_version("here",upgrade="never", version = "1.0.1")'
RUN Rscript -e 'remotes::install_version("config",upgrade="never", version = "0.3.1")'
RUN Rscript -e 'remotes::install_version("attempt",upgrade="never", version = "0.3.1")'
RUN Rscript -e 'remotes::install_version("covr",upgrade="never", version = "3.5.1")'
RUN Rscript -e 'remotes::install_version("spelling",upgrade="never", version = "2.2")'
RUN Rscript -e 'remotes::install_version("tidyquant",upgrade="never", version = "1.0.3")'
RUN Rscript -e 'remotes::install_version("tidyjson",upgrade="never", version = "0.3.1")'
RUN Rscript -e 'remotes::install_version("thinkr",upgrade="never", version = "0.15")'
RUN Rscript -e 'remotes::install_version("thematic",upgrade="never", version = "0.1.2.1")'
RUN Rscript -e 'remotes::install_version("runner",upgrade="never", version = "0.4.1")'
RUN Rscript -e 'remotes::install_version("polished",upgrade="never", version = "0.6.1")'
RUN Rscript -e 'remotes::install_version("golem",upgrade="never", version = "0.3.2")'
RUN Rscript -e 'remotes::install_version("colourpicker",upgrade="never", version = "1.1.1")'
RUN Rscript -e 'remotes::install_version("checkmate",upgrade="never", version = "2.0.0")'
RUN Rscript -e 'remotes::install_version("bs4Dash",upgrade="never", version = "2.0.3")'
RUN Rscript -e 'remotes::install_version("bizdays",upgrade="never", version = "1.0.9")'
RUN Rscript -e 'remotes::install_version("assert",upgrade="never", version = "1.0.1")'
RUN Rscript -e 'remotes::install_github("RinteRface/fullPage#4657d86a0cf43bb6b2451f5ee34b2410656bae0d")'
RUN Rscript -e 'remotes::install_github("ColinFay/brochure#57d10e620b69256680a7244edf309e39243415af")'
RUN Rscript -e 'remotes::install_version("RQuantLib",upgrade="never", version = "0.4.15")'
#Adding RQuantLib
RUN mkdir /build_zone
ADD . /build_zone
WORKDIR /build_zone
RUN R -e 'remotes::install_local(upgrade="never")'
RUN rm -rf /build_zone
# add user group `app` and user `app`
#RUN addgroup --system app \
# && adduser --system --ingroup app app
# Copy Project files and edit app permisions
# COPY . ./humblFinance
# WORKDIR /humblFinance
# RUN chown app:app -R ./humblFinance
#USER app
# Specify port
ENV port 2811
# Specify host
ENV host 0.0.0.0
# HARD CODED RN --> MAKE into ARG https://stackoverflow.com/questions/50178013/docker-expose-using-run-time-environment-variables
EXPOSE 2811
CMD R -e "options(shiny.port = ${port}, shiny.host = '${host}') ; humblFinance::run_app()"
DESCRIPTION
Package: humblFinance
Title: finance Shiny App
Version: 0.0.0.9000
Authors#R: person('humbl', 'Finance', email = 'humblfinance#gmail.com', role = c('cre', 'aut'))
Description: A visualization of financial data loaded and ready to be spelunked by the public.
License: GPL (>= 3)
Imports:
assert,
attempt (>= 0.3.1),
bizdays,
brochure (>= 0.0.0.9022),
bs4Dash,
checkmate,
colourpicker,
config (>= 0.3.1),
DT (>= 0.20),
fullPage,
ggplot2,
glue (>= 1.6.1),
golem (>= 0.3.1),
here (>= 1.0.1),
htmltools (>= 0.5.2),
httr,
lubridate,
magrittr,
PerformanceAnalytics,
polished,
processx (>= 3.5.2),
quantmod,
rlang,
RQuantLib,
runner,
shiny (>= 1.6.0),
shinyWidgets,
stringr,
thematic,
thinkr (>= 0.15),
tidyjson (>= 0.3.1),
tidyquant,
xts,
zoo
Encoding: UTF-8
LazyData: true
RoxygenNote: 7.1.2
Suggests:
rmarkdown (>= 2.8),
knitr (>= 1.33),
spelling (>= 2.2),
testthat (>= 3.0.0),
covr
Config/testthat/edition: 3
Language: en-US
VignetteBuilder: knitr
URL: https://github.com/jjfantini/humblFinance.git
BugReports: https://github.com/jjfantini/humblFinance.git/issues
NAMESPACE
# Generated by roxygen2: do not edit by hand
export(rescaled_range)
export(run_app)
import(bizdays)
import(bs4Dash)
import(fullPage)
import(ggplot2)
import(httr)
import(lubridate)
import(polished)
import(shiny)
import(xts)
import(zoo)
importFrom(PerformanceAnalytics,Return.calculate)
importFrom(assert,assert)
importFrom(brochure,brochureApp)
importFrom(brochure,page)
importFrom(checkmate,assertString)
importFrom(checkmate,checkDate)
importFrom(glue,glue)
importFrom(golem,activate_js)
importFrom(golem,add_resource_path)
importFrom(golem,bundle_resources)
importFrom(golem,favicon)
importFrom(golem,with_golem_options)
importFrom(graphics,abline)
importFrom(graphics,plot)
importFrom(graphics,segments)
importFrom(magrittr,"%<>%")
importFrom(magrittr,"%>%")
importFrom(polished,secure_server)
importFrom(polished,secure_ui)
importFrom(polished,sign_in_ui_default)
importFrom(quantmod,periodReturn)
importFrom(rlang,is_null)
importFrom(runner,mean_run)
importFrom(shiny,HTML)
importFrom(shiny,NS)
importFrom(shiny,column)
importFrom(shiny,shinyApp)
importFrom(shiny,tagAppendAttributes)
importFrom(shiny,tagList)
importFrom(shiny,tags)
importFrom(shinyWidgets,airDatepickerInput)
importFrom(shinyWidgets,updateAirDateInput)
importFrom(stats,sd)
importFrom(stringr,str_extract)
importFrom(stringr,str_replace_all)
importFrom(thematic,thematic_shiny)
importFrom(tidyquant,tq_mutate)
What I've tried
Running remotes::install_local() redownloads packages that are specified with remotes::install_version(), but not all packages, and some that arent in the DESCRIPTION file or NAMESPACE. From the documentation, install_local(upgrade = "never") will search the "."path and install the packages in that directory. I'm not exactly too sure where these are coming from, some packages likeplotly+forecastare not available insearch(), I'm guessing they are deps of other packages used internally? I may just point this function straight to .tar.gzof my local{golem}` package.
I installed QuantLib on my Linux image using the package --> libquantlib0-dev \, this solved the error of my package failing to build from RQuantLib package installation fail.
RUN R -e 'remotes::install_local(upgrade="never")'
# Should my .tar.gz file show up as a package in this list?
> remotes::install_local(upgrade="never")
Installing 14 packages: hardhat, tsfeatures, forecast, slider, plotly,
rsample, recipes, timetk, janitor, alphavantager, tidyquant, tidyjson,
thinkr, polished
Solution:
I have updated manually rlang --> 1.0.2
I have installed libquantlib0-dev onto Linux Docker image in order for RQuantLib to be installed properly w/ correct dependency
!!!
- DESCRIPTION packages NEED to match the DOCKERFILE packages that are installed, otherwise the build will fail and your local package installation will be removed
Good afternoon to everybody.
I deployed a shiny app in Docker.
I need Bioconductor packages/ I find the way and the app working properly.
But now I've got shown that is very big size the app.
probably this is due to 2 layers ( 1 of the app and 1 of the Bioconductor packages.)
I have read also that I must remove the second docker layer and try to install Bioconductor on the rocker/shiny image. But I do not know how to realize that information. I attach the dockerfile.
Does anybody have idea please to make lighter the app in docker?
# Base image https://hub.docker.com/u/rocker/
FROM rocker/shiny:latest
# system libraries of general use
## install debian packages
RUN apt-get update -qq && \
apt-get upgrade -y && \
apt-get -y --no-install-recommends install \
libxml2-dev \
libcairo2-dev \
libsqlite3-dev \
libmariadbd-dev \
libpq-dev \
libssh2-1-dev \
unixodbc-dev \
libcurl4-openssl-dev \
libssl-dev \
coinor-libcbc-dev coinor-libclp-dev libglpk-dev && \
apt-get clean
# Docker inheritance
FROM bioconductor/bioconductor_docker:RELEASE_3_12
RUN apt-get update
RUN R -e 'BiocManager::install(ask = F)' && R -e 'BiocManager::install(c("rtracklayer", \
"GenomicAlignments", "Biostrings", "SummarizedExperiment", "Rsamtools", ask = F))'
# copy necessary files
## app folder
COPY ./folder ./app
# install renv & restore packages
RUN Rscript -e 'install.packages("renv")'
RUN Rscript -e 'install.packages("devtools")'
RUN Rscript -e 'install.packages("shiny")'
RUN Rscript -e 'install.packages("shinyBS")'
RUN Rscript -e 'install.packages("ggvis")'
RUN Rscript -e 'install.packages("shinycssloaders")'
RUN Rscript -e 'install.packages("shinyWidgets")'
RUN Rscript -e 'install.packages("plotly")'
RUN Rscript -e 'install.packages("RSQLite")'
RUN Rscript -e 'install.packages("knitr")'
RUN Rscript -e 'install.packages("knitcitations")'
RUN Rscript -e 'install.packages("Matrix")'
RUN Rscript -e 'install.packages("plotly")'
RUN Rscript -e 'install.packages("igraph")'
RUN Rscript -e 'install.packages("ggthemes")'
RUN Rscript -e 'install.packages("evaluate")'
RUN Rscript -e 'install.packages("psych")'
RUN Rscript -e 'install.packages("kableExtra")'
RUN Rscript -e 'install.packages("ggjoy")'
RUN Rscript -e 'install.packages("gtools")'
RUN Rscript -e 'install.packages("gridExtra")'
RUN Rscript -e 'install.packages("ggrepel")'
RUN Rscript -e 'install.packages("data.table")'
RUN Rscript -e 'install.packages("stringr")'
RUN Rscript -e 'install.packages("rmarkdown")'
RUN Rscript -e 'install.packages("shinyjqui")'
RUN Rscript -e 'install.packages("V8")'
RUN Rscript -e 'devtools::install_github("ThomasSiegmund/D3TableFilter")'
RUN Rscript -e 'devtools::install_github("leonawicz/apputils")'
RUN Rscript -e 'devtools::install_github("dirkschumacher/ompr")'
RUN Rscript -e 'devtools::install_github("dirkschumacher/ompr.roi")'
RUN Rscript -e 'install.packages("shinydashboard")'
RUN Rscript -e 'install.packages("dplyr")'
RUN Rscript -e 'install.packages("shinyjs")'
RUN Rscript -e 'install.packages("DT")'
RUN Rscript -e 'install.packages("rhandsontable")'
RUN Rscript -e 'renv::consent(provided = TRUE)'
RUN Rscript -e 'renv::restore()'
# expose port
EXPOSE 8080
# run app on container start
CMD ["R", "-e", "shiny::runApp('/app', host = '0.0.0.0', port = 8080)"]
I have the following dockfile
# Base image https://hub.docker.com/u/rocker/
FROM rocker/shiny:latest
# system libraries of general use
## install debian packages
RUN apt-get update -qq && apt-get -y --no-install-recommends install \
libxml2-dev \
libcairo2-dev \
libsqlite3-dev \
libmariadbd-dev \
libpq-dev \
libssh2-1-dev \
unixodbc-dev \
libcurl4-openssl-dev \
libssl-dev \
coinor-libcbc-dev coinor-libclp-dev libglpk-dev
## update system libraries
RUN apt-get update && \
apt-get upgrade -y && \
apt-get clean
# copy necessary files
## app folder
COPY ./app ./app
# Docker inheritance
FROM bioconductor/bioconductor_docker:RELEASE_3_12
RUN apt-get update
RUN R -e 'BiocManager::install(ask = F)' && R -e 'BiocManager::install(c("rtracklayer", \
"GenomicAlignments", "Biostrings", "SummarizedExperiment", "Rsamtools", ask = F))'
# install renv & restore packages
RUN Rscript -e 'install.packages("renv")'
RUN Rscript -e 'install.packages("devtools")'
RUN Rscript -e 'install.packages("shiny")'
RUN Rscript -e 'install.packages("shinyBS")'
RUN Rscript -e 'install.packages("ggvis")'
RUN Rscript -e 'install.packages("shinydashboardPlus")'
RUN Rscript -e 'install.packages("shinycssloaders")'
RUN Rscript -e 'install.packages("shinyWidgets")'
RUN Rscript -e 'install.packages("plotly")'
RUN Rscript -e 'install.packages("RSQLite")'
RUN Rscript -e 'install.packages("forecast", dependencies = TRUE)'
RUN Rscript -e 'install.packages("tsutils")'
RUN Rscript -e 'install.packages("readxl")'
RUN Rscript -e 'install.packages("tidyverse")'
RUN Rscript -e 'install.packages("knitr")'
RUN Rscript -e 'install.packages("knitcitations")'
RUN Rscript -e 'install.packages("nycflights13")'
RUN Rscript -e 'install.packages("Matrix")'
RUN Rscript -e 'install.packages("plotly")'
RUN Rscript -e 'install.packages("igraph")'
RUN Rscript -e 'install.packages("ggthemes")'
RUN Rscript -e 'install.packages("evaluate")'
RUN Rscript -e 'install.packages("psych")'
RUN Rscript -e 'install.packages("kableExtra")'
RUN Rscript -e 'install.packages("ggjoy")'
RUN Rscript -e 'install.packages("gtools")'
RUN Rscript -e 'install.packages("gridExtra")'
RUN Rscript -e 'install.packages("cowplot")'
RUN Rscript -e 'install.packages("ggrepel")'
RUN Rscript -e 'install.packages("data.table")'
RUN Rscript -e 'install.packages("stringr")'
RUN Rscript -e 'install.packages("rmarkdown")'
RUN Rscript -e 'install.packages("shinyjqui")'
#RUN Rscript -e 'install.packages("BiocManager")'
#RUN R -e 'BiocManager::install("rtracklayer")'
#RUN R -e 'BiocManager::install("GenomicAlignments")'
#RUN R -e 'BiocManager::install("Biostrings")'
#RUN R -e 'BiocManager::install("SummarizedExperiment")'
#RUN R -e 'BiocManager::install("Rsamtools")'
RUN Rscript -e 'install.packages("V8")'
RUN Rscript -e 'devtools::install_github("ThomasSiegmund/D3TableFilter")'
RUN Rscript -e 'devtools::install_github("leonawicz/apputils")'
RUN Rscript -e 'devtools::install_github("Marlin-Na/trewjb")'
RUN Rscript -e 'devtools::install_github("dirkschumacher/ompr")'
RUN Rscript -e 'devtools::install_github("dirkschumacher/ompr.roi")'
RUN Rscript -e 'install.packages("ROI.plugin.glpk")'
RUN Rscript -e 'install.packages("shinydashboard")'
RUN Rscript -e 'install.packages("dplyr")'
RUN Rscript -e 'install.packages("dashboardthemes")'
RUN Rscript -e 'install.packages("shinyjs")'
RUN Rscript -e 'install.packages("magrittr")'
RUN Rscript -e 'install.packages("DT")'
RUN Rscript -e 'install.packages("rhandsontable")'
RUN Rscript -e 'renv::consent(provided = TRUE)'
RUN Rscript -e 'renv::restore()'
# expose port
EXPOSE 3838
# run app on container start
CMD ["R", "-e", "shiny::runApp('/app', host = '0.0.0.0', port = 3838)"]
When I try to upload without Bioconductor packages the app running
But when I am trying to upload as the dockfile is I receive the following error.
Bioconductor version 3.12 (BiocManager 1.30.10), ?BiocManager::install for help
shiny::runApp('/app', host = '0.0.0.0', port = 3838)
Error in shinyAppDir(x) : No Shiny application exists at the path "/app"
Calls: ... as.shiny.appobj -> as.shiny.appobj.character -> shinyAppDir
Execution halted
I visit some solution but I do not understand them....please I need your help
Check out the docs on multistage builds
You have a COPY statement, and right after that a FROM statement. After that last statement you no longer have access to whatever was in there in previous stage. You can copy files from one stage to the next if needed with --from=stagename where you named the stage with FROM somerepo/someimage as stagename.
In this case it means that everything you do in the first stage is never used or available again.
Normally this is used something like
FROM baseimage as build
RUN install some dependencies
COPY some sourcefiles
RUN install stuff eg mvn install
FROM runImage as run
COPY --from=build somepath/executable
CMD ["do", "something"]
This way you keep the final image small, removing everything you need at build time, but not at runtime.
In your case you have to rethink this a bit and see what you need at which stage.
I do not know the specifics of your app but the way you have it set up now is that you start with a base image, then install debian suff, then system stuff and then start all over with a "empty" new baseimage. You probably want to use just one image and go from there, with just one FROM instruction. Then if you want to optimize it for production you can always consider the multistage build if that is worth it in your case.