Error when running Singularity container for R script - r

Question

I am building a Singularity container to run a custom R script for tree segmentation using the LidR software package.
I have written the Singularity definition file as such:
Bootstrap: docker
From: ubuntu:20.04
%setup
touch test.R
touch treeSeg_dalponte2016.R
touch /home/ljeasson/R/x86_64-pc-linux-gnu-library/3.6/rgdal/libs/rgdal.so
%files
test.R
treeSeg_dalponte2016.R
/home/ljeasson/R/x86_64-pc-linux-gnu-library/3.6/rgdal/libs/rgdal.so
%post
# Disable interactivity, including region and time zone
export DEBIAN_FRONTEND="noninteractive"
export DEBCONF_NONINTERACTIVE_SEEN=true
# Update apt and install necessary libraries and repositories
apt update
apt install -y build-essential r-base-core software-properties-common dirmngr apt-transport-https lsb-release ca-certificates
add-apt-repository ppa:ubuntugis/ubuntugis-unstable
apt install -y libgdal-dev libgeos++-dev libudunits2-dev libproj-dev libx11-dev libgl1-mesa-dev libglu1-mesa-dev libfreetype6-dev libnode-dev libxt-dev libfftw3-dev
apt clean
# Install necessary R packages and dependencies
R -e "install.packages('lidR', dependencies = TRUE)"
R -e "install.packages('raster', dependencies = TRUE)"
R -e "install.packages('sf', dependencies = TRUE)"
R -e "install.packages('dplyr', dependencies = TRUE)"
R -e "install.packages('rgdal', dependencies = TRUE, repos='https://cran.rstudio.com', configure.args=c('--with-gdal-config=/opt/conda/bin/gdal-config', '--with-proj-include=/opt/conda/include', '--with-proj-lib=/opt/conda/lib', '--with-proj-share=/opt/conda/share/proj/'))"
R -e "install.packages('gdalUtils', dependencies = TRUE, repos='https://cran.rstudio.com')"
%test
#!/bin/bash
R --version
Rscript test.R
%runscript
#!/bin/sh
echo "Arguments received: $*"
Rscript treeSeg_dalponte2016.R $*
And build the container using singularity build ga_container.sif ga_container.def
The container builds without error, but when the container is run using ./ga_container <arguments>, this error always occurs:
Error: package or namespace load failed for 'rgdal' in dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object '/home/ljeasson/R/x86_64-pc-linux-gnu-library/3.6/rgdal/libs/rgdal.so':
libgdal.so.26: cannot open shared object file: No such file or directory
Execution halted
I know that the error is occuring because it cannot find the image for Rgdal, even though it seems I've attached to the container in the %setup and %files section:
%setup
touch test.R
touch treeSeg_dalponte2016.R
touch /home/ljeasson/R/x86_64-pc-linux-gnu-library/3.6/rgdal/libs/rgdal.so
%files
test.R
treeSeg_dalponte2016.R
/home/ljeasson/R/x86_64-pc-linux-gnu-library/3.6/rgdal/libs/rgdal.so
If the error is from incorrect file attachment, how do I ensure that the Rgdal (and other similar libraries) are attached correctly within the Singularity container?
Thanks in advance

Answer 1

This looks like an environmental issue causing the image to look at your locally installed R modules instead of using the ones installed in the image. Perhaps in your .Rprofile or R_LIBS/R_LIBS_USER. Try running with singularity run --cleanenv ..., or temporarily moving your .Rprofile if you have one, and see if that fixes it. If not, I have a few other observations.
First, the %setup block is creating root owned, empty files on the host OS if they don't exist already. An empty .so file would certainly cause problems. For the majority of cases you don't want to use %setup, as it directly modifies the host as root during sudo singularity build.
In the %files block you are copying the (potentially root owned/empty) to a path in the image that matches your home directory. Your $HOME is automatically mounted when you run/exec/shell an image, which will hide any files in the image at that location. When adding files to an image, you should always put them in a place they are unlikely to get clobbered by a mount. /opt/myapp or something similar usually works well. Additionally, test.R and treeSeg_dalponte2016.R are copied to /test.R and /treeSeg_dalponte2016.R inside the container, but relative paths are used in %runscript and %test. Singularity run/exec will attempt to run from the first path that exists in the container: $PWD (implicitly mounted, but this can fail silently), then $HOME (also implicitly mounted and can fail silently), then /. You can use singularity --verbose run ... to see if anything isn't being mounted correctly and add echo $PWD to %runscript to see where it's running from.
In %post when you install the rgdal package, you specify several paths with /opt/conda/... but conda is not installed or configured in the image. I'm not familiar with rgdal, so don't know if that would cause problems or not though.