How to fix 'broken packages' error when installing R? - r

I am trying to create a multi-arch image using python:3.9-slim-buster as the base image. However, when I run the buildx command, it runs into an error during the arm64 build phase:
> [linux/arm64 9/17] RUN apt-get install -y -f r-base:
#34 6.950 distribution that some required packages have not yet been created
#34 6.950 or been moved out of Incoming.
#34 6.950 The following information may help to resolve the situation:
#34 6.950
#34 6.950 The following packages have unmet dependencies:
#34 7.315 r-base : Depends: r-base-core (>= 4.2.2-1~bustercran.0) but it is not going to be installed
#34 7.315 Depends: r-recommended (= 4.2.2-1~bustercran.0) but it is not going to be installed
#34 7.315 Recommends: r-base-html but it is not going to be installed
#34 7.315 Recommends: r-doc-html but it is not going to be installed
#34 7.378 E: Unable to correct problems, you have held broken packages.
------
Dockerfile:18
--------------------
16 | # Install ASREML-R
17 | RUN apt install -y -t buster-cran40
18 | >>> RUN apt-get install -y -f r-base
19 |
20 | # install R packages
--------------------
ERROR: failed to solve: process "/bin/sh -c apt-get install -y -f r-base" did not complete successfully: exit code: 100
I am also including the Dockerfile and the buildx command that I used:
# Dockerfile
FROM python:3.9-slim-buster
ENV LANG=C.UTF-8 LC_ALL=C.UTF-8 PYTHONUNBUFFERED=1
ENV LOG_LEVEL info
#
RUN apt-get update
RUN apt-get install -y dirmngr gnupg apt-transport-https ca-certificates software-properties-common
RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-key '95C0FAF38DB3CCAD0C080A7BDC78B2DDEABC47B7'
RUN add-apt-repository 'deb http://cloud.r-project.org/bin/linux/debian buster-cran40/'
## Install supervisor
RUN apt-get update
RUN apt-get install -y supervisor
#ENV PATH="/app/scripts:${PATH}"
# Install ASREML-R
RUN apt install -y -t buster-cran40
RUN apt-get install -y -f r-base
# install R packages
RUN Rscript -e "install.packages('sommer')"
RUN Rscript -e "install.packages('RJSONIO')"
RUN Rscript -e "install.packages('gtools')"
ENV PYTHONPATH "${PYTHONPATH}:/app/"
WORKDIR /app/
COPY . /app/
RUN Rscript /app/packages/install_asreml.R
# RUN apt-get update && pip3 install -r requirements.txt
RUN pip3 install --upgrade setuptools
RUN pip3 install -r requirements.txt
ENTRYPOINT [ "./start-worker.sh" ]
Build command:
docker buildx build \
-t test/test-image:test \
--cache-from=type=registry,ref=test/test-image-build-cache \
--cache-to=type=registry,ref=test/test-image-build-cache,mode=max \
--push --platform linux/arm64/v8,linux/amd64 \
--progress=plain \
.
I already tried using a different (the latest) slim-buster image that is available and tried adding the marutter PPA repository to no avail.

Related

Error Installing redux R package on centos7

Am getting an error trying to install redux r package on centos7, and have no idea how to fix it. Has anybody come across it before?
my Dockerfile is:
FROM centos:centos7
RUN yum -y install wget git tar
RUN yum -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
RUN yum -y install epel-release openssh-server
ENV R_VERSION=4.0.5
RUN wget https://cdn.rstudio.com/r/centos-7/pkgs/R-${R_VERSION}-1-1.x86_64.rpm \
&& yum -y install R-${R_VERSION}-1-1.x86_64.rpm \
&& rm R-${R_VERSION}-1-1.x86_64.rpm
ENV PATH="${PATH}:/opt/R/${R_VERSION}/bin/"
RUN yum -y install openssl-devel
RUN Rscript -e "install.packages(c('redux'), repos = 'https://packagemanager.rstudio.com/all/__linux__/centos7/latest')"
RUN Rscript -e "library(redux)"
CMD ["/bin/bash"]
Then i build the image:
docker build -t test-3:latest .
And the error i get is:
=> ERROR [8/8] RUN Rscript -e "library(redux)" 0.6s
------
> [8/8] RUN Rscript -e "library(redux)":
#12 0.528 Error: package or namespace load failed for 'redux' in dyn.load(file, DLLpath = DLLpath, ...):
#12 0.528 unable to load shared object '/opt/R/4.0.5/lib/R/library/redux/libs/redux.so':
#12 0.528 libhiredis.so.0.12: cannot open shared object file: No such file or directory
#12 0.528 Execution halted
------
executor failed running [/bin/sh -c Rscript -e "library(redux)"]: exit code: 1
ps. I am able to install any other package and reference it without problems
That file seems to come from the hiredis package: https://rhel.pkgs.org/7/okey-x86_64/hiredis-0.12.1-1.el7.centos.x86_64.rpm.html
Try adding the line RUN yum -y install hiredis or maybe adding that package to one of your existing yum install lines.
So turns out I had to also have hiredis installed for the package to load successfully
Updated dockerfile:
FROM centos:centos7
RUN yum -y install wget git tar
RUN yum -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
RUN yum -y install epel-release openssh-server
ENV R_VERSION=4.0.5
RUN wget https://cdn.rstudio.com/r/centos-7/pkgs/R-${R_VERSION}-1-1.x86_64.rpm \
&& yum -y install R-${R_VERSION}-1-1.x86_64.rpm \
&& rm R-${R_VERSION}-1-1.x86_64.rpm
ENV PATH="${PATH}:/opt/R/${R_VERSION}/bin/"
RUN yum -y install openssl-devel hiredis
RUN Rscript -e "install.packages(c('redux'), repos = 'https://packagemanager.rstudio.com/all/__linux__/centos7/latest')"
RUN Rscript -e "library(redux)"
CMD ["/bin/bash"]

Creating Docker with Ruby and R

I have a rails application that combines some R code. It works fine, and now is the time to dockerize it. I tried creating the docker based on a ruby image, which failed installing R properly, and then the other way around (R image, installing ruby using rbenv as explained here) which failed too.
Has anyone experienced with this combo?
EDIT: I have managed to create a docker using the R image, however this requires a tiring installation of many ruby dependencies. So consider this side done.
Still, why won't the other way (installing R on ruby image) work?
Ruby-based:
FROM ruby:2.7.1 AS rails-toolbox
RUN apt-get update -qq && apt-get install -y build-essential libpq-dev nodejs
RUN mkdir /app
WORKDIR /app
ADD shalev/Gemfile /app/Gemfile
ADD shalev/Gemfile.lock /app/Gemfile.lock
RUN bundle install --without development test doc --jobs=4
RUN apt-get install -y apt-utils
RUN apt-get install -y apt-transport-https
RUN wget -qO- https://cloud.r-project.org/bin/linux/ubuntu/marutter_pubkey.asc | tee -a /etc/apt/trusted.gpg.d/cran_ubuntu_key.asc
RUN echo "deb https://cloud.r-project.org/bin/linux/ubuntu bionic-cran35/" | tee -a /etc/apt/sources.list
RUN apt-get update
RUN apt-get install r-base
COPY shalev/ /app/
[...]
error message:
Step 13/20 : RUN apt-get install r-base
---> Running in f546cfe57d33
Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
r-base : Depends: r-base-core (>= 3.6.3-1bionic) but it is not going to be installed
Depends: r-recommended (= 3.6.3-1bionic) but it is not going to be installed
Recommends: r-base-html but it is not going to be installed
E: Unable to correct problems, you have held broken packages.
The command '/bin/sh -c apt-get install r-base' returned a non-zero code: 100
R-based:
FROM rocker/r-ver:3.6.3
RUN apt-get update -qq && apt-get install -y apt-utils apt-transport-https build-essential libpq-dev nodejs
RUN apt-get install -y git-core zlib1g-dev build-essential libssl-dev libreadline-dev libyaml-dev libsqlite3-dev sqlite3 libxml2-dev libxslt1-dev libcurl4-openssl-dev software-properties-common libffi-dev nodejs yarn curl bzip2
RUN git clone https://github.com/rbenv/rbenv.git ~/.rbenv \
&& echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bashrc \
&& echo 'eval "$(rbenv init -)"' >> ~/.bashrc
ENV HOME /home/shalev
ENV PATH "$HOME/.rbenv/bin:$HOME/.rbenv/shims:$PATH"
ENV RUBY_VERSION 2.7.1
RUN mkdir -p "$(rbenv root)"/plugins \
&& git clone https://github.com/rbenv/ruby-build.git "$(rbenv root)"/plugins/ruby-build
RUN rbenv install $RUBY_VERSION
RUN rbenv global $RUBY_VERSION && rbenv versions && ruby -v
RUN mkdir /app
[...]
error message:
Step 9/25 : RUN rbenv install $RUBY_VERSION
---> Running in 9939299d6ed1
/bin/sh: 1: rbenv: not found
You can use the stand alone version of ruby-build.
Note that you are root and not shalev inside the container:
FROM rocker/r-ver:3.6.3
RUN apt-get update -qq && \
apt-get install -y \
apt-utils apt-transport-https \
build-essential libpq-dev nodejs
RUN apt-get install -y \
git-core zlib1g-dev build-essential \
libssl-dev libreadline-dev libyaml-dev \
libsqlite3-dev sqlite3 libxml2-dev libxslt1-dev \
libcurl4-openssl-dev software-properties-common \
libffi-dev nodejs yarn curl bzip2
RUN git clone https://github.com/rbenv/rbenv.git ~/.rbenv \
&& echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bashrc \
&& echo 'eval "$(rbenv init -)"' >> ~/.bashrc
ENV PATH "/root/.rbenv/bin/:/root/.rbenv/shims/:$PATH"
ENV RUBY_VERSION 2.7.1
RUN git clone https://github.com/rbenv/ruby-build.git && \
PREFIX=/usr/local ./ruby-build/install.sh
RUN rbenv install $RUBY_VERSION && \
rbenv global $RUBY_VERSION && \
rbenv versions
RUN mkdir /app
I managed to make it work with the ruby image as well:
FROM ruby:2.7.1 AS rails-toolbox
RUN apt-get update -qq && apt-get install -y build-essential libpq-dev nodejs
RUN apt-get install -y apt-utils apt-transport-https
RUN apt-get install -y gfortran
RUN wget -c https://cran.r-project.org/src/base/R-3/R-3.6.3.tar.gz
RUN tar -xf R-3.6.3.tar.gz
WORKDIR "/R-3.6.3"
RUN ./configure
RUN make -j9
RUN make install
WORKDIR "/root"
RUN mkdir /app
[...]

shiny docker image build failed due to ubuntu package issue

I'm trying to build docker image for my R shiny app. Below is theDockerfile.
# Install R version 3.6
FROM r-base:3.6.0
# Install Ubuntu packages
RUN apt-get update && apt-get install -y \
sudo \
gdebi-core \
pandoc \
pandoc-citeproc \
libcurl4-gnutls-dev \
libcairo2-dev \
libxt-dev \
xtail \
wget \
libudunits2-dev \
libgdal-dev
# Install R packages that are required
RUN R -e "install.packages(c('remotes', 'tidyr', 'dplyr', 'ggplot2', 'stringr','shiny', 'shinydashboard','shinyWidgets','shinyjs', 'rlang','scales','DT','lubridate', 'plotly', 'leaflet', 'leafpop', 'visNetwork', 'wordcloud2', 'arules'), repos='http://cran.rstudio.com/')"
RUN R -e "remotes::install_github('nik01010/dashboardthemes')"
# Download and install ShinyServer (latest version)
RUN wget --no-verbose https://download3.rstudio.org/ubuntu-14.04/x86_64/VERSION -O "version.txt" && \
VERSION=$(cat version.txt) && \
wget --no-verbose "https://download3.rstudio.org/ubuntu-14.04/x86_64/shiny-server-$VERSION-amd64.deb" -O ss-latest.deb && \
gdebi -n ss-latest.deb && \
rm -f version.txt ss-latest.deb
# Copy configuration files into the Docker image
COPY shiny-server.conf /etc/shiny-server/shiny-server.conf
COPY /app /srv/shiny-server/
# Make the ShinyApp available at port 80
EXPOSE 80
# Copy further configuration files into the Docker image
COPY shiny-server.sh /usr/bin/shiny-server.sh
CMD ["/usr/bin/shiny-server.sh"]
However, my image building failed quite early. Below is all the logs. It seems like some Ubuntu package issue which I do not quite understand.
I'm new to docker. I have not clue what to do with those error messages. Appreciate if anyone can help find out what needs to be done. Thanks.
Sending build context to Docker daemon 113.4MB
Step 1/10 : FROM r-base:3.6.0
---> 876f4d7b60e9
Step 2/10 : RUN apt-get update && apt-get install -y sudo gdebi-core pandoc pandoc-citeproc libcurl4-gnutls-dev libcairo2-dev libxt-dev xtail wget libudunits2-dev libgdal-dev
---> Running in 825a69c5a05f
Get:2 http://deb.debian.org/debian testing InRelease [116 kB]
Get:1 http://cdn-fastly.deb.debian.org/debian sid InRelease [146 kB]
Get:3 http://deb.debian.org/debian testing/main amd64 Packages [7,690 kB]
Get:4 http://cdn-fastly.deb.debian.org/debian sid/main amd64 Packages [8,225 kB]
Fetched 16.2 MB in 6s (2,724 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
libc6-dev : Breaks: libgcc-8-dev (< 8.4.0-2~) but 8.3.0-7 is to be installed
E: Error, pkgProblemResolver::Resolve generated breaks, this may be caused by held packages.
The command '/bin/sh -c apt-get update && apt-get install -y sudo gdebi-core pandoc pandoc-citeproc libcurl4-gnutls-dev libcairo2-dev libxt-dev xtail wget libudunits2-dev libgdal-dev' returned a non-zero code: 100
The error message you are receiving is not Docker specific, but rather concerns your package manager apt.
Although you are updating the package list using apt-get update, you do not install newer versions of packages. What happens now is that you are trying to install a newer version of an already installed package which leads to the error message.
If you use apt-get upgrade, apt will install the newer versions of your installed packages.
Changing the 5th line as shown below will fix that issue:
RUN apt-get update && apt-get upgrade -y && apt-get install -y \

How to build Dockerfile with R & Java

I'm trying to build a Docker container that runs R with the package RJava. I have tried the following code:
# Install R version 3.6.3
FROM rocker/tidyverse:3.6.3
# Make ~/.R
RUN mkdir -p $HOME/.R
# Install Ubuntu packages && then R packages
RUN install2.r --error \
lubridate magrittr RPostgres DBI broom rlang rJava
However I get the following: installation of package ‘rJava’ had non-zero exit status.
Can anyone help me with this. I'm thinking that maybe it is because Java is not installed. Does anyone know how to install Java on this docker container?
I've tried adding the following to my dockerfile as per another post I found however I get the error saying 'The repository 'http://ppa.launchpad.net/webupd8team/java/ubuntu focal Release' does not have a Release file:
# Install "software-properties-common" (for the "add-apt-repository")
RUN apt-get update && apt-get install -y \
software-properties-common
# Add the "JAVA" ppa
RUN add-apt-repository -y \
ppa:webupd8team/java
# Install OpenJDK-8
RUN apt-get update && \
apt-get install -y openjdk-8-jdk && \
apt-get install -y ant && \
apt-get clean;
# Fix certificate issues
RUN apt-get update && \
apt-get install ca-certificates-java && \
apt-get clean && \
update-ca-certificates -f;
# Setup JAVA_HOME -- useful for docker commandline
ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64/
RUN export JAVA_HOME
I'm new to docker and any help with this would be much appreciated.
The rocker images are based on debian, not ubuntu. Specifically it is Debian GNU/Linux 10 (buster). With that version, you can install java by installing the package openjdk-11-jdk via apt and you don't need to add any repositories for openjdk-8-jdk.
So a working dockerfile that installs rJava:
FROM rocker/tidyverse:3.6.3
RUN apt-get update && \
apt-get install -y openjdk-11-jdk && \
apt-get install -y liblzma-dev && \
apt-get install -y libbz2-dev
RUN Rscript -e "install.packages('rJava')"
Note: liblzma-dev and libbz2-dev are additional system dependencies for compiling rJava.

Docker - R creates multiple processes

I'm creating a docker container and I'm installing R in it.
The problem is, that when running R in the container, the main R process create ncores - 1 subprocesses.
So when running the docker on my laptop with 8 cores, I get 1 R process and 7 R subprocesses.
I also tried to configure my Docker file similar to this:
https://github.com/rocker-org/rocker/blob/eeb9c8a5f416f7cfe982734440e39fa72abbcb33/r-base/Dockerfile
but is still not working.
Docker-file:
FROM ubuntu:14.04
RUN sed -e 's/archive\.ubuntu/at\.archive\.ubuntu/g' -i /etc/apt/sources.list
RUN apt-get update
RUN apt-get -y install libatlas3-base
RUN apt-get -y install libopenblas-base
RUN apt-get -y install r-base
RUN apt-get -y install r-base-dev
RUN apt-get -y install apt-utils
RUN echo "deb http://cran.cnr.berkeley.edu/bin/linux/ubuntu/ trusty/" >> /etc/apt/sources.list
RUN gpg --keyserver keyserver.ubuntu.com --recv-key E084DAB9
RUN gpg -a --export E084DAB9 | apt-key add -
RUN apt-get update
RUN apt-get -y --with-new-pkgs upgrade
Did someone encountered this problem?
I don't know why, but the whole problem was caused by installing libopenblas-base.
After removing the line
RUN apt-get -y install libopenblas-base
everything worked fine!

Resources