Update R 4.0.5 to R 4.1.1 with conda on ubuntu 18.04 [duplicate] - r

On Ubuntu in a Conda environment with Python 3.7.3, when I run
conda install -c conda-forge opencv
I get OpenCV 3.4.2 (checked with import cv2 and then cv2._version__) even though https://anaconda.org/conda-forge/opencv indicates version 4.11. Why?
Note that I didn't have OpenCV installed previously (I ran conda uninstall opencv and it got completely removed)

tl;dr You likely have previously installed dependencies that need updating. If you require a specific version, say 4.1, then express this to Conda:
conda install -c conda-forge opencv=4.1
Explanation
How Conda Interprets Specifications
A literal translation of the command
conda install -c conda-forge opencv
would go something like
With the conda-forge channel included, ensure that some version of the package opencv is installed in the currently active environment.
The logic here implies that any version it can install would be a valid solution. It also doesn't tell it that it must come from Conda Forge, only that that channel should be included.1
Two-Stage Solve Strategy
Starting with v4.7, Conda uses a two-stage dependency solving strategy. The two stages are
Solve with an implicit --freeze-installed|--no-update-deps flag. This attempts to find the newest version of the requested package that has no conflicts with installed packages. That is, it considers any installation of the package, no matter the version, to be a satisfactory solution. If it works, then it's done. Otherwise, move on to...
An unrestricted solve (what used to be default in Conda < 4.7). This frees up dependencies to be updated and will often result in the latest versions being installed unless there are previous explicit specifications on those packages.2
This strategy aims to provide a faster solve and install experience, by avoiding having to change anything in your environment. It also helps keep the environment stable by avoiding unnecessary version changes.
Specific Failure in Question
What happened in OP's case? One of the dependencies requirements of OpenCV was likely newer in v4.1.1 than what was already installed, but that dependency's version was compatible with installing OpenCV 3.4.2. Hence, the only thing that would change was adding opencv plus missing dependencies. Technically, this is a valid solution since one only asked for some version of opencv to be installed.
Getting the Latest Version
Option: Specifying the Version
If you know you want a specific version then you can always specify it
conda install -c conda-forge opencv=4.1.1
and since Conda can't install this without updating something in your env, the first round of solve will fail, and the full solve will get it for you.
Option: Skip the Freeze
Of course, you may not always know what the latest version number is and don't want to have to look this up on Anaconda Cloud every time. Fortunately, there is the --update-deps flag that essentially skips over the first solve stage and goes straight to the full solve. This will install the latest version for your system, as well as update any of the dependencies.
conda install --update-deps -c conda-forge opencv
Important Note: The --update-deps flag has a side-effect of converting dependencies to explicit specifications. While this is an internal environment state (managed through <env>/conda-meta/history), it does have some behavioral consequences (bugs!):
the result of the conda env export --from-history command will subsequently include all packages, instead of just the ones the user explicitly requested in the past
conda remove will not be able to prune dependencies; e.g., if scipy was installed, it would pull in numpy; if only scipy depended on numpy and scipy was removed, normally numpy would also get removed. This wouldn't work after using the --update-deps flag.
[1]: The behavior here depends on the channel_priority configuration option. With the strict setting, conda-forge would be prioritized over other channels; with the flexible setting, it is simply added to the list and the latest compatible version from any channel is selected.
[2]: One can check the explicit specifications of an environment with conda env export --from-history.

Related

R Package Conflicts in Anaconda3

I have installed Anaconda 3 for Mac M1 and I am trying to create a new environment for R. However, everytime I try to do this, I am faced with the below error:
Conda Prompt Error Message
I am faced with this error irrespective of whether I try to do this in anaconda navigator or conda prompt. I have already tried removing Anaconda completely and reinstalling it again but to no avail. Is there anything I can do here?
Update: As requested, here is an additional screenshot:
Mamba install r-eesentials
Mamba install r-essentials output
[Disclosure: I am a volunteer on the Conda Forge R team.]
Conda users who want to use R should prioritize Conda Forge. The Anaconda company has not actively maintained R packages since v3.6.
If installing Conda for the first time, I recommend Miniforge variants (specifically, Mambaforge), rather than the Anaconda distributions (which prioritize defaults/anaconda channel).
Also note that osx-arm64 support for R through Conda is currently sparse. We are actively working on migrating, but for simplicity of workflow, I still recommend Conda users to install a osx-64 version of Conda and use that platform when installing R. You can always create native osx-arm64 environments when you actually need them (e.g., native TensorFlow with Metal support).

Installing a specific version of R from an Apt repository

R volunteers currently maintain Ubuntu package repositories for R ~3.5 and ~4.0. For Bionic Beaver, these are:
https://cloud.r-project.org/bin/linux/ubuntu/bionic-cran35/
https://cloud.r-project.org/bin/linux/ubuntu/bionic-cran40/
I am building separate Singularity containers, into which I need very specific versions of R installed; which appear to be provided in these repositories. Specifically, I'm looking to build containers containing R versions 3.6.1, 4.0.3 and 4.1.0; one container per version.
I do this, in the container build script, by first adding the appropriate Apt source, then running the install with a pinned version. I noticed that I could only get it to run if I use the precise version numbers listed in the package repository and also include r-recommended at the same version. For example, for R 3.6.1:
apt install -y r-base=3.6.1-3bionic r-recommended=3.6.1-3bionic
This correctly installs r-base and r-recommended at the given versions. However, when I run the containerised R, R is actually reporting itself to be at the latest version provided by those repositories (3.6.3, 4.1.0 and 4.1.0, respectively). Presumably, given r-base is correct, this may even suggest them to be in a broken state.
Looking through Apt's output, it's clear that many other r-* packages are defaulting to the latest versions, rather than the versions I specified. In an attempt to get around this, I tried explicitly setting the versions on all the packages that are defaulting to the latest version. For example, again with R 3.6.1:
apt install -y r-base=3.6.1-3bionic \
r-base-core=3.6.1-3bionic \
r-base-dev=3.6.1-3bionic \
r-base-html=3.6.1-3bionic \
r-doc-html=3.6.1-3bionic \
r-recommended=3.6.1-3bionic
However, this refuses to work, complaining about conflicts with other packages it's trying to install (r-cran-* packages, IIRC).
I don't know if this is an Apt-thing, an R-thing, or something to do with their repositories. Is there a way I can get these specific versions installed from the official sources, without having to build anything myself? (If not, what's the point of them keeping the older versions in their repositories?)
Thanks to #Chris' tip-off, the structure of said R packages is important to understand.
r-base is a metapackage which includes, amongst other things, r-base-core and r-recommended. r-recommended is another metapackage which includes a suite of recommended R packages, which introduce the incompatibility when trying to pin to versions.
For just the R binaries and the documentation, pinned to a specific ${VERSION}, this will do the trick:
apt install -y --no-install-recommends \
r-base-core=${VERSION} \
r-base-html=${VERSION} \
r-doc-html=${VERSION}
If you want to build packages, you'd also want r-base-dev=${VERSION} in there.

Install R packages using conda via an environment.yml file

Normally I create conda environments like...
conda env create -f environment.yml
conda activate env_name
Normally I work in Python, where a typical environment.yml simple file might looks like this...
name: env_name
dependencies:
- python=3.7
- pip=19.3
- pandas=0.24.2
- pip:
- scipy==1.2.1
What should the environment.yml file look like to install R packages? The packages are on CRAN
A general rule of thumb is that most R packages have corresponding packages in Anaconda Cloud with the prefix r- added. While the defaults channel covers commonly-used packages, the conda-forge channel has the most thorough coverage of CRAN and has helpful scripts for adding new ones. I would generally recommend prioritizing conda-forge when creating R environments.
For bioinformaticians, all Bioconductor packages are available through the bioconda channel, with a bioconductor- prefix and lowercase. For example, SingleCellExperiment is packaged as bioconductor-singlecellexperiment.
A good place to start is simply searching Anaconda Cloud (example search).
Example
Let's assume you want the tidyverse umbrella package and wish to use R v4.1. A YAML for this would be
name: my_r_env
channels:
- conda-forge
dependencies:
- r-base=4.1
- r-tidyverse
Additional Notes
Avoid using install.packages() from within any R sessions - it is prone to dynamic linking issues due to the R instance's unawareness of compiling inside the environment. This is not an issue for pure R packages, but in that case it should be simple to add the package to conda-forge (takes about 15 mins of work and a ~12-24hr turnaround, IME).
Avoid the RStudio packages from Conda - it is an abandoned project and the old versions are incompatible with newer R versions. This may change once RStudio switches from Qt to Electron. Still, there are better ways to load an environment into RStudio, without having to install the full IDE inside the environment.

zlib/bz2 library and headers are requried for compiling R

Trying to compile R-3.3.2 on Debian Jessie, all dependencies are installed. However the ./configure script complains about the zlib/bzip2 library versions not matching with the minimum requirement.
Minimum version required:
zlib: 1.2.6 (installed version: 1.2.11)
bzip2: 1.0.6 (installed version: 1.0.6)
After looking at the parts of configure script checking the library versions, it seems that it compares versions with strcmp or strncmp. Since "1.2.11" is lexicographically smaller that "1.2.6" it return a non-zero value indicating that the match failed. Besides, it just compares the first 5 characters which is also not what it is intented. So, it's a bug in configure script. Changing the script fixed the issue.
For zlib, find this line:
exit(strncmp(ZLIB_VERSION, "1.2.5", 5) < 0);
Change it to:
exit(ZLIB_VERNUM < 0x1250);
I had some issues installing R myself, specifically with the error
checking for BZ2_bzlibVersion in -lbz2... no
I had to install libbz2-dev to get that error to go away.
Unfortunately, I came across a few more issues while running ./configure and had to do a little more digging to find out how to solve it.
Discussion on issue
After reading that, I had realized I had to install a couple packages like libcurl4-openssl-dev, libpcre3, and liblzma-dev to finally finish the configuration.
The cited link suggested
At this stage you could have as well tried to install R 3.2.0RC ...
R-devel has not yet diverged much.
Personally, I think that installing an older version to resolve dependency issues reeks of laziness, but that's just my two cents.

upgrade R 2.7.1 on Debian core procedure

My R version is 2.7.1 (on Debian) and some packages are asking for > 2.10. I cannot find updating instructions and I don't want to remove and reinstall as I have other things depending on R and I don't want to mess up. Is there an update procedure?
Closest thing to my problem is on this thread.
check out the instructions for installing from source. Its easy on a Linux box, and you can do the install in any directory you like, you dont even need superuser permissions. Once compiled you can even run R from that directory without messing up any system-installed R. As long as you give the full path to R's binary, or put the path to it in your PATH environment variable, when starting it it will work fine.
FYI
It seems that R on Debian with versions previous to 2.7.1 cannot be updated.
The current core runs from 2.7.1 up. The only way to do it is to remove the existing version.
As this was not straight forward I post it here. If you have Rapache or other things connecting to R disable them with dismode or related.
apt-get purge r-base r-base-dev
I had to do this as well
dpkg -P r-base-core
until this shows no more installed R packages
dpkg -l r-*
Then follow the instruction from http://cran.r-project.org/bin/linux/debian/, with the amendment that you should use deb instead of deb-src in /etc/apt/sources.list.
deb http://<favorite-cran-mirror>/bin/linux/debian lenny-cran/
Before installing run this and it should not say 2.7.1.
apt-cache policy r-base-dev

Resources