Installing rpy2 with Docker is unable to find R path - r

A Django application using Docker needs to install rpy2 as a dependency. Although I install r-base container and specify it as a dependency, when installing django requirements I keep getting:
Collecting rpy2==2.8.3 (from -r /requirements/base.txt (line 55))
Downloading rpy2-2.8.3.tar.gz (186kB)
Complete output from command python setup.py egg_info:
Error: Tried to guess R's HOME but no command 'R' in the PATH.
How can specify inside Docker where the R path is?
My server.yml looks like this:
version: '2'
services:
r:
build: ./services/r
django:
build:
context: ./myproject/
dockerfile: ./compose/django/Dockerfile
env_file:
- .env
- .env-server
environment:
- DATABASE_URL=postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}#postgres:5432/${POSTGRES_USER}
depends_on:
- postgres
- r
command: /gunicorn.sh
volumes:
- ./myproject:/app
The Dockerfile for django is:
FROM python:2.7
ENV PYTHONUNBUFFERED 1
COPY ./requirements /requirements
RUN pip install -r /requirements/production.txt \
&& pip install -r /requirements/test.txt \
&& groupadd -r django \
&& useradd -r -g django django
COPY . /app
RUN chown -R django /app
COPY ./compose/django/gunicorn.sh /gunicorn.sh
COPY ./compose/django/entrypoint.sh /entrypoint.sh
RUN sed -i 's/\r//' /entrypoint.sh \
&& sed -i 's/\r//' /gunicorn.sh \
&& chmod +x /entrypoint.sh \
&& chown django /entrypoint.sh \
&& chmod +x /gunicorn.sh \
&& chown django /gunicorn.sh
WORKDIR /app
ENTRYPOINT ["/entrypoint.sh"]
The Dockerfile for R is:
FROM r-base

It was easier to just install r inside the django container. So removing the r container and modifiying the django docker file adding this lines, worked:
RUN apt-get --force-yes update \
&& apt-get --assume-yes install r-base-core

Related

Add shiny server with ADD=Shiny with rocker verse image

Documentation for rocker/rstudio docker container.
I am able to get up and running in rstudio using Docker with the following set up in a directory:
Dockerfile:
FROM rocker/tidyverse:latest
docker-compose:
version: "3.5"
services:
ide-rstudio:
build:
context: .
ports:
- 8787:8787
environment:
ROOT: "TRUE"
PASSWORD: test
Now, if I enter this dir in the terminal and type: docker-compose build followed by docker-compose up -d and then navigate to localhost:8787 I see the rstudio login screen. So far so good.
I would like to add shiny to the same container per the documentation (as opposed to using a separate shiny image).
On the documentation I link to at the top it says:
Add shiny server on start up with e ADD=shiny
docker run -d -p 3838:3838 -p 8787:8787 -e ADD=shiny -e PASSWORD=yourpasswordhere rocker/rstudio
shiny server is now running on localhost:3838 and RStudio on localhost:8787.
Since I'm using docker-compose I updated my docker-compose file to this:
version: "3.5"
services:
ide-rstudio:
build:
context: .
ports:
- 8787:8787
- 3838:3838
environment:
ROOT: "TRUE"
ADD: "shiny"
PASSWORD: test
Now, when I go to the terminal like before and type: docker-compose build followed by docker-compose up -d I again see the rstudio login page at localhost:8787. However, if I go to localhost:3838, I see Firefox' 'connection was reset' page. It looks like nothing is there.
How can I add shiny to my container per the instructions?
It seems the image is missing shiny installer. If you run the same compose file without -d and using rocker/rstudio:3.2.0 image you will see in logs that shiny is installing. It failed to install for me (there was a problem with missing file /usr/local/lib/R/site-library/littler/examples/install2.r) but I found the script which installs the thing. For some reason the script does not exist in rocker/tidyverse:latest (I have no idea why, you'd better ask the maintainer) and ADD=shiny has no effect.
I managed to get things working by injecting that script into rocker/tidyverse:latest and here is how you can do it. Save the following as a file named add:
#!/usr/bin/with-contenv bash
ADD=${ADD:=none}
## A script to add shiny to an rstudio-based rocker image.
if [ "$ADD" == "shiny" ]; then
echo "Adding shiny server to container..."
apt-get update && apt-get -y install \
gdebi-core \
libxt-dev && \
wget --no-verbose https://s3.amazonaws.com/rstudio-shiny-server-os-build/ubuntu-12.04/x86_64/VERSION -O "version.txt" && \
VERSION=$(cat version.txt) && \
wget --no-verbose "https://s3.amazonaws.com/rstudio-shiny-server-os-build/ubuntu-12.04/x86_64/shiny-server-$VERSION-amd64.deb" -O ss-latest.deb && \
gdebi -n ss-latest.deb && \
rm -f version.txt ss-latest.deb && \
install2.r -e --skipinstalled shiny rmarkdown && \
cp -R /usr/local/lib/R/site-library/shiny/examples/* /srv/shiny-server/ && \
rm -rf /var/lib/apt/lists/* && \
mkdir -p /var/log/shiny-server && \
chown shiny.shiny /var/log/shiny-server && \
mkdir -p /etc/services.d/shiny-server && \
cd /etc/services.d/shiny-server && \
echo '#!/bin/bash' > run && echo 'exec shiny-server > /var/log/shiny-server.log' >> run && \
chmod +x run && \
adduser rstudio shiny && \
cd /
fi
if [ $"$ADD" == "none" ]; then
echo "Nothing additional to add"
fi
Then either add the following to your Dockefile:
COPY add /etc/cont-init.d/add
RUN chmod +x /etc/cont-init.d/add
or apply execution permission locally and mount it during runtime. To do this run the following locally:
chmod +x add
and add this to docker-compose.yml:
services:
ide-rstudio:
volumes: # this line and below
- ./add:/etc/cont-init.d/add

Compiling a .Net Core Console App with Npgsql and CoreRT

I'm trying to compile a .net core console application into native executable (linux-x64) on an ubuntu 18.04 docker container, using both coreRT and Npgsql. I'm currently using docker-compose to set up the DB and application containers.
docker-compose.yml
version: '3'
services:
database:
image: postgres:10
environment:
- POSTGRES_USER=dbuser
- POSTGRES_PASSWORD=dbpassword
- POSTGRES_DB=dbsample
ports:
- 5432:5432
tmpfs:
- /var/lib/postgresql/data:rw,noexec,nosuid,size=400m
volumes:
- ./db-init:/docker-entrypoint-initdb.d
prototype:
build: .
depends_on:
- database
links:
- database:database
Dockerfile
FROM ubuntu:18.04
RUN apt-get update \
&& apt-get install -y \
apt-transport-https \
build-essential \
clang \
cmake \
curl \
git-core \
gpg \
libbz2-dev \
libkrb5-dev \
libncurses5-dev \
libncursesw5-dev \
libreadline-dev \
libsqlite3-dev \
libssl-dev \
llvm \
make \
parallel \
wget \
zlib1g-dev
RUN wget -qO- https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > microsoft.asc.gpg \
&& mv microsoft.asc.gpg /etc/apt/trusted.gpg.d/ \
&& wget -q https://packages.microsoft.com/config/ubuntu/18.04/prod.list \
&& mv prod.list /etc/apt/sources.list.d/microsoft-prod.list \
&& chown root:root /etc/apt/trusted.gpg.d/microsoft.asc.gpg \
&& chown root:root /etc/apt/sources.list.d/microsoft-prod.list \
&& apt-get update \
&& apt-get install -y dotnet-sdk-2.2
ENV CppCompilerAndLinker=clang-6.0
ENV DOTNET_CLI_TELEMETRY_OPTOUT=true
WORKDIR /home/app
COPY ./HelloWorld.fsproj /home/app
COPY ./nuget.config /home/app
RUN dotnet restore
COPY ./ /home/app
RUN dotnet publish -r linux-x64 -c Release -v detailed -o outside
CMD ./outside/HelloWorld
When It gets to compile it (dotnet publish -r linux-x64 -c Release -v detailed -o outside), it enters infinite loop consuming all the memory avaiable for the container. Until it shows this error:
Task "Exec"
"/root/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/1.0.0-alpha-27919-02/tools/ilc" #"obj/Release/netcoreapp2.2/linux-x64/native/HelloWorld.ilc.rsp"
Killed
1:7>/root/.nuget/packages/microsoft.dotnet.ilcompiler/1.0.0-alpha-27919-02/build/Microsoft.NETCore.Native.targets(249,5): error MSB3073: The command ""/root/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/1.0.0-alpha-27919-02/tools/ilc" #"obj/Release/netcoreapp2.2/linux-x64/native/HelloWorld.ilc.rsp"" exited with code 137. [/home/app/HelloWorld.fsproj]
Done executing task "Exec" -- FAILED.
1:7>Done building target "IlcCompile" in project "HelloWorld.fsproj" -- FAILED.
1:7>Done Building Project "/home/app/HelloWorld.fsproj" (Publish target(s)) -- FAILED.
It seems to be somehow related with the usage of generics and reflections in F#. I've looked in both Npgsql and coreRT repos and couldn't find someone close to get them both working. Have anyone faced this problem? Or managed to use Npgsql and coreRT?

How to download and unzip in Dockerfile

So, I have, it works, but I want to change the way to immediately download the file and unpack it:
Dockerfile
FROM wordpress:fpm
# Copying themes from local
COPY ./wordpress/ /var/www/html/wp-content/themes/wordpress/
RUN chmod -R 777 /var/www/html/
How can I immediately download the file from the site and unzip it to the appropriate folder?
docker-compose.yml
wordpress:
build: .
links:
- db:mysql
nginx:
image: raulr/nginx-wordpress
links:
- wordpress
ports:
- "8080:80"
volumes_from:
- wordpress
db:
image: mariadb
environment:
MYSQL_ROOT_PASSWORD: qwerty
I tried:
#install unzip and wget
RUN \
apt-get update && \
apt-get install unzip wget -y && \
rm -rf /var/lib/apt/lists/*
RUN wget -O /var/www/html/type.zip http://wp-templates.ru/download/2405 \
&& unzip '/var/www/html/type.zip' -d /var/www/html/wp-content/themes/ && rm
/var/www/html/type.zip || true;
Dockerfile has "native command" for copying and extracting .tar.gz files.
So you can change archive type from .zip to .tar.gz (maybe in future versions zip also will be supported, I'm not sure) and use ADD instead of COPY.
Read more about ADD
Best to use a multistage docker build. You will need the latest version of docker and buildkit enabled. Then do something along these lines
# syntax=docker/dockerfile:1
from alpine:latest as unzipper
apk add unzip wget curl
RUN mkdir /opt/ ; \
curl <some-url> | tar xvzf - -C /opt
FROM wordpress:fpm
COPY --from unzipper /opt/ /var/www/html/wp-content/themes/wordpress/
Even better is if there is a Docker image built already with the stuff in you want you just need the 'copy --from' line and give it the image name.
Finally dont worry about any mess in the 1st stage as its discarded when the build completes, so the fact its alpine, and not using no-cache is irrelevant, and none of the installed packages end up in the final image
Found more guidance for remote zipped files in Docker documentation
Because image size matters, using ADD to fetch packages from remote
URLs is strongly discouraged; you should use curl or wget instead.
That way you can delete the files you no longer need after they’ve
been extracted and you don’t have to add another layer in your image.
For example, you should avoid doing things like:
ADD https://example.com/big.tar.xz /usr/src/things/
RUN tar -xJf /usr/src/things/big.tar.xz -C /usr/src/things
RUN make -C /usr/src/things all
And instead, do something like:
RUN mkdir -p /usr/src/things \
&& curl -SL https://example.com/big.tar.xz \
| tar -xJC /usr/src/things \
&& make -C /usr/src/things all

(Docker) How to install dependencies, using separate Composer container, in WordPress container?

Dockerfile
FROM wordpress
ENV REFRESHED_AT 2015-08-12
ADD \
composer.json /var/www/html
ADD \
composer.lock /var/www/html
# install the PHP extensions
RUN \
apt-get -qq update && \
apt-get -y upgrade && \
apt-get install -y vim wget && \
rm -rf /var/lib/apt/lists/*
# Symlink User's "wp-content" folder into the newly installed Wordpress
RUN \
rm -rf /usr/src/wordpress/wp-content/plugins/* && \
rm -rf /usr/src/wordpress/wp-content/themes/* && \
cp -fr /usr/src/wordpress/* /var/www/html/ && \
chown -R www-data:www-data /var/www/html/
# volume for mysql database and wordpress install
VOLUME ["/var/www/html/wp-content/plugins", "/var/www/html/wp-content/themes"]
# Define working directory.
WORKDIR /var/www/html/
EXPOSE 80 3306
CMD ["apache2-foreground"]
Docker Compose File
wordpress:
build: .
links:
- mysql
- composer
volumes:
- wp-content/plugins/:/var/www/html/wp-content/plugins
- wp-content/themes/:/var/www/html/wp-content/themes
environment:
- WORDPRESS_DB_PASSWORD=__WORDPRESS_DB_PASSWORD__
- WORDPRESS_DB_NAME=__WORDPRESS_DB_NAME__
# - WORDPRESS_DB_USER=__WORDPRESS_DB_USER__
ports:
- "9888:80"
mysql:
image: mysql:5.7
environment:
- MYSQL_ROOT_PASSWORD=__WORDPRESS_DB_PASSWORD__
- MYSQL_DATABASE=__WORDPRESS_DB_NAME__
composer:
image: composer/composer
Question details
I'm able to ADD the composer.json and composer.lock files to the working directory. I can confirm that these two files are in the working directory.
What I need is for the Dockerfile (or wherever) to also automatically install the dependencies into the working directory.
According to Docker Hub, https://hub.docker.com/r/composer/composer/,
I should be able to docker run -v $(pwd):/app composer/composer install to install the dependencies but how do I do this in Dockerfile?
Also I'm confused because the -v flag, https://docs.docker.com/engine/userguide/dockervolumes/, has to do with mounting the specified host directory into the a container but I've already ADDed the necessary files to the working directory. All I want to do is install the dependencies.
Thank you for your help.
You just need to mount the current directory to /app when running your composer container. I've put together a simple example to illustrate this working at https://gist.github.com/andyshinn/e2c428f2cd234b718239.
The key parts here are the volumes for the composer part of the application and the restart: 'yes' on the primary PHP application (the application likely will not run until Composer has run so you will want it to restart).

grunt-cli doesn't work with docker volume mounted, fine without

I set up an angular development environment using the following Dockerfile (don't try to build this unless you're really enthusiastic, it takes an age).
FROM ubuntu:14.04
# build environment
RUN ["apt-get", "update"]
RUN ["apt-get", "install", "-y", "nodejs", "npm", "git"]
RUN ["ln", "-s", "/usr/bin/nodejs", "/usr/bin/node"]
RUN ["npm", "install", "-g", "yo"]
RUN ["npm", "install", "-g", "bower"]
RUN ["npm", "install", "-g", "grunt-cli"]
WORKDIR /home/angular
ADD ./package.json /home/angular/package.json
ADD ./bower.json /home/angular/bower.json
ADD ./dist /home/angular/dist
RUN ["npm", "install"]
RUN ["bower", "install", "--allow-root"]
# sass depedencies
ENV RUBY_MAJOR 2.2
ENV RUBY_VERSION 2.2.2
ENV RUBY_DOWNLOAD_SHA256 5ffc0f317e429e6b29d4a98ac521c3ce65481bfd22a8cf845fa02a7b113d9b44
# some of ruby's build scripts are written in ruby
# we purge this later to make sure our final image uses what we just built
RUN ["apt-get", "install", "-y", "curl"]
RUN apt-get update \
&& apt-get install -y autoconf bison libgdbm-dev ruby \
&& rm -rf /var/lib/apt/lists/* \
&& mkdir -p /usr/src/ruby \
&& curl -fSL -o ruby.tar.gz "http://cache.ruby-lang.org/pub/ruby/$RUBY_MAJOR/ruby-$RUBY_VERSION.tar.gz" \
&& echo "$RUBY_DOWNLOAD_SHA256 *ruby.tar.gz" | sha256sum -c - \
&& tar -xzf ruby.tar.gz -C /usr/src/ruby --strip-components=1 \
&& rm ruby.tar.gz \
&& cd /usr/src/ruby \
&& autoconf \
&& ./configure --disable-install-doc \
&& make -j"$(nproc)" \
&& make install \
&& apt-get purge -y --auto-remove bison libgdbm-dev ruby \
&& rm -r /usr/src/ruby
# skip installing gem documentation
RUN echo 'gem: --no-rdoc --no-ri' >> "$HOME/.gemrc"
# install things globally, for great justice
ENV GEM_HOME /usr/local/bundle
ENV PATH $GEM_HOME/bin:$PATH
ENV BUNDLER_VERSION 1.10.5
RUN gem install bundler --version "$BUNDLER_VERSION" \
&& bundle config --global path "$GEM_HOME" \
&& bundle config --global bin "$GEM_HOME/bin"
# don't create ".bundle" in all our apps
ENV BUNDLE_APP_CONFIG $GEM_HOME
RUN gem install compass
VOLUME ["/home/me/code/correspondence/client/dist"]
ADD ./ /home/angular
If I run this with:
sudo docker run -it me/angular /bin/bash
I can use grunt build with no problems. Since I haven't attached a volume to dist that build is no use to other containers such as the webserver. But running:
sudo docker run -itv /home/me/code/correspondence/client/dist:/home/angular me/angular /bin/bash
results in the grunt build command no longer being usable in the container:
grunt-cli: The grunt command line interface. (v0.1.13)
Fatal error: Unable to find local grunt.
If you're seeing this message, either a Gruntfile wasn't found or grunt
hasn't been installed locally to your project. For more information about
installing and configuring grunt, please see the Getting Started guide:
http://gruntjs.com/getting-started
The only difference is adding the volume. How does adding the volume result in this different behaviour?
I suppose that's because you have placed some files to /home/angular in image and when you're mounting your volume to the same path (/home/angular), your volume hides original files.
Quote from documentation:
Note: If the path /opt/webapp already exists inside the container’s
image, its contents will be replaced by the contents of /src/webapp on
the host to stay consistent with the expected behavior of mount
Try to mount volume to another directory, /home/angular/dist/client, for instance.

Resources