Part of a Jenkins job I'm running entails installing rJava; the Jenkins job is running on a docker image over which I have control, but still I can't seem to get rJava to install.
Looking around a bit, it seems like being sure to run R CMD javareconf is pretty key in all this, and indeed I added && R CMD javareconf as the last command in my Dockerfile. Compilation of the Docker image contains:
Java interpreter : /docker-java-home/jre/bin/java
Java version : 1.8.0_171
Java home path : /docker-java-home
Java compiler : /docker-java-home/bin/javac
Java headers gen.: /docker-java-home/bin/javah
Java archive tool: /docker-java-home/bin/jar
trying to compile and link a JNI program
detected JNI cpp flags : -I$(JAVA_HOME)/include -I$(JAVA_HOME)/include/linux
detected JNI linker flags : -L$(JAVA_HOME)/jre/lib/amd64/server -ljvm
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -I/docker-java-home/include -I/docker-java-home/include/linux -fpic -g -O2 -fdebug-prefix-map=/build/r-base-3.3.3=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c conftest.c -o conftest.o
gcc -std=gnu99 -shared -L/usr/lib/R/lib -Wl,-z,relro -o conftest.so conftest.o -L/docker-java-home/jre/lib/amd64/server -ljvm -L/usr/lib/R/lib -lR
JAVA_HOME : /docker-java-home
Java library path: $(JAVA_HOME)/jre/lib/amd64/server
JNI cpp flags : -I$(JAVA_HOME)/include -I$(JAVA_HOME)/include/linux
JNI linker flags : -L$(JAVA_HOME)/jre/lib/amd64/server -ljvm
Updating Java configuration in /usr/lib/R
Done.
Now, when I enter the Docker instance (docker exec -it jenkins_img bash), I can in fact manually install rJava: install.packages('rJava') works as expected; importantly, this is part of the output there:
checking Java support in R... present:
interpreter : '/docker-java-home/jre/bin/java'
archiver : '/docker-java-home/bin/jar'
compiler : '/docker-java-home/bin/javac'
header prep.: '/docker-java-home/bin/javah'
cpp flags : '-I/docker-java-home/include -I/docker-java-home/include/linux'
java libs : '-L/docker-java-home/jre/lib/amd64/server -ljvm'
However, when I run the script that's part of my Jenkins job, the corresponding output becomes:
checking Java support in R... present:
interpreter : '/usr/lib/jvm/default-java/jre/bin/java'
archiver : '/usr/lib/jvm/default-java/bin/jar'
compiler : '/usr/lib/jvm/default-java/bin/javac'
header prep.: '/usr/lib/jvm/default-java/bin/javah'
cpp flags : '-I/usr/lib/jvm/default-java/include -I/usr/lib/jvm/default-java/include/linux'
java libs : '-L/usr/lib/jvm/default-java/jre/lib/amd64/server -ljvm'
checking whether Java run-time works... ./configure: line 3747: /usr/lib/jvm/default-java/jre/bin/java: No such file or directory
no
configure: error: Java interpreter '/usr/lib/jvm/default-java/jre/bin/java' does not work
So, it seems whichever process is running when I docker exec to bash has a different JAVA_HOME than the process which is running under my Jenkins job. To that end, I added the following directly before the line installing rJava in my script (as suggested):
export JAVA_HOME=/docker-java-home/
export PATH=$PATH:$JAVA_HOME/bin
Rscript -e 'install.packages("rJava")'
However, this fails with the same output. I have also checked that the R process running install.packages has the same JAVA_HOME by cating Sys.getenv('JAVA_HOME') and confirming that it's /docker-java-home/.
The Jenkins script doesn't have root access so I can't run R CMD javareconf there.
How can I get install.packages('rJava') to use the right JAVA_HOME?
FWIW, these days I use some variation of the following in Docker images to get rJava to build correctly:
# rJava
RUN apt-get update && apt-get install -y libudunits2-dev gnupg2 software-properties-common
RUN echo "deb http://ppa.launchpad.net/webupd8team/java/ubuntu trusty main" \
| tee /etc/apt/sources.list.d/webupd8team-java.list \
&& echo "deb-src http://ppa.launchpad.net/webupd8team/java/ubuntu trusty main" \
| tee -a /etc/apt/sources.list.d/webupd8team-java.list \
&& apt-key adv --keyserver keyserver.ubuntu.com --recv-keys EEA14886 \
&& echo "oracle-java8-installer shared/accepted-oracle-license-v1-1 select true" \
| /usr/bin/debconf-set-selections \
&& apt-get update \
&& apt-get install -y oracle-java8-installer \
&& update-alternatives --display java \
&& rm -rf /var/lib/apt/lists/* \
&& apt-get clean \
&& R CMD javareconf
This was not particularly my own work, I basically took it from #daroczig:
https://github.com/rocker-org/rocker/issues/109
https://hub.docker.com/r/cardcorp/r-java/dockerfile/
Related
I need to install data.table 1.12.0 (specifically) on macOS 11.1.
I am getting an error:
clang: error: unsupported option '-fopenmp'
make: *** [assign.o] Error 1
I have followed the directions in https://github.com/Rdatatable/data.table/wiki/Installation#openmp-enabled-compiler-for-mac but still unable to get it to work.
R 3.6.1.
I also tried a R CMD install and got the same error:
R CMD install data.table_1.12.0.tar.gz
* installing to library ‘/Users/XXX/Library/R/3.6/library’
* installing *source* package ‘data.table’ ...
** package ‘data.table’ successfully unpacked and MD5 sums checked
** using staged installation
** libs
clang -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include -fopenmp -fPIC -Wall -g -O2 -c assign.c -o assign.o
clang: error: unsupported option '-fopenmp'
make: *** [assign.o] Error 1
ERROR: compilation failed for package ‘data.table’
* removing ‘/Users/XXX/Library/R/3.6/library/data.table’
Update: Seems to be an issue install latest data.table from CRAN for me as well, so its not restricted to 1.12.0 apparently.
NB. This solution works with Intel processors (not Apple M1 silicon)
These are the steps I used to install data.table from source with multithreading/openMP enabled (originally described in https://stackoverflow.com/a/65334247/12957340):
Reinstall xcode command line tools (even if it says "up to date")
sudo rm -rf /Library/Developer/CommandLineTools
sudo xcode-select --install
Install gcc & llvm via Homebrew (instructions for installing Homebrew) or, if you already have gcc/llvm installed via Homebrew, skip to the next step
# This can take several hours
brew install gcc
brew install llvm
Once you have gcc & llvm installed via Homebrew:
brew cleanup
brew update
brew upgrade
brew reinstall gcc
brew reinstall llvm
Link some headers into /usr/local/include
sudo ln -s /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/* /usr/local/include/
# You can safely ignore warnings like this:
#ln: /usr/local/include//tcl.h: File exists
#ln: /usr/local/include//tclDecls.h: File exists
#ln: /usr/local/include//tclPlatDecls.h: File exists
#ln: /usr/local/include//tclTomMath.h: File exists
#ln: /usr/local/include//tclTomMathDecls.h: File exists
#ln: /usr/local/include//tk.h: File exists
#ln: /usr/local/include//tkDecls.h: File exists
#ln: /usr/local/include//tkPlatDecls.h: File exists
Install the gmp library:
brew install gmp
Create a new ~/.R/Makevars file (if you already have a ~/.R/Makevars file, save it in a different directory (away from ~/.R/)) and include only these lines in the file:
LOC=/usr/local/gfortran
CC=$(LOC)/bin/gcc -fopenmp
CXX=$(LOC)/bin/g++ -fopenmp
CXX11 = $(LOC)/bin/g++ -fopenmp
CFLAGS=-g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe
CXXFLAGS=-g -O3 -Wall -pedantic -std=c++11 -mtune=native -pipe
LDFLAGS=-L$(LOC)/lib -Wl,-rpath,$(LOC)/lib,-L/usr/local/lib
CPPFLAGS=-I$(LOC)/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/usr/local/include
FLIBS=-L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin19/10.2.0 -L/usr/local/gfortran/lib -lgfortran -lquadmath -lm
CXX1X=/usr/local/gfortran/bin/g++
CXX98=/usr/local/gfortran/bin/g++
CXX11=/usr/local/gfortran/bin/g++
CXX14=/usr/local/gfortran/bin/g++
CXX17=/usr/local/gfortran/bin/g++
Compile data.table from source in R/Rstudio
install.packages("data.table", type = "source")
# To check whether it installed correctly, run:
library(data.table)
If your package fails to compile, a couple of SO users have had to install a fresh gfortran (re: https://stackoverflow.com/a/65334247/12957340), which you can download from https://github.com/fxcoudert/gfortran-for-macOS/releases
here has been my approach
basically you have to download clang and llvmorg
curl -SL https://github.com/llvm/llvm-project/releases/download/llvmorg-10.0.0/clang+llvm-10.0.0-x86_64-linux-gnu-ubuntu-18.04.tar.xz \
| tar -xJC . && \
mv clang+llvm-10.0.0-x86_64-linux-gnu-ubuntu-18.04 clang_10.0.0 && \
echo 'export PATH=/clang_10.0.0/bin:$PATH' >> ~/.bashrc && \
echo 'export LD_LIBRARY_PATH=/clang_10.0.0/lib:$LD_LIBRARY_PATH' >> ~/.bashrc
and make sure they are listed in you PATH and LD_LIBRARY_PATH
then get the datatable and build from the source
git clone https://github.com/h2oai/datatable.git && \
cd datatable && \
make build && \
python -m pip install ../datatable/
make sure you are using the right Python
I am trying to install Qt 4.8.7 from source (https://download.qt.io/archive/qt/4.8/4.8.7/qt-everywhere-opensource-src-4.8.7.tar.gz) in a virtual machine running Ubuntu 18.04.1 LTS
So the commands I follow are:
mkdir Qt
cd Qt
wget https://download.qt.io/archive/qt/4.8/4.8.7/qt-everywhere-opensource-src-4.8.7.tar.gz
tar -xzf qt-everywhere-opensource-src-4.8.7.tar.gz
mv qt-everywhere-opensource-src-4.8.7 Qt-Source
cd Qt-Source
./configure
Then I choose the Open Source Edition, accept the terms of License. After that it displays:
Qt is now configured for building. Just run 'make'.
Once everything is built, you must run 'make install'.
Qt will be installed into /usr/local/Trolltech/Qt-4.8.7
So ...
make
and I have the next error;
Another thing is that in the process of running make, there are a lot of errors like these:
Could you help me? Please
eyllanesc,
I faced the same problem as you a few days ago. And after many tries I did it.
First of all, you'll need to install another gcc and g++ version elder (I choose 4.8 version).
Follow the instructions in this another answer:
GCC & C++ alternatives
And edit the file:
mkspecs/linux-g++-64/qmake.conf
QMAKE_CFLAGS = -std=gnu++98 -fpermissive
QMAKE_LFLAGS = -std=gnu++98 -fpermissive
QMAKE_CXXFLAGS = -std=gnu++98 -fpermissive
With these flags the compiler will use C++98 version. Like
Bart_Vandewoestyne solution.
Then run:
./configure -opensource -nomake demos -nomake examples -nomake tests -debug-and-release -qt3support -no-openssl -no-webkit
make -jN
sudo make install
After the complete installation back the default gcc and g++ versions:
cd /usr/bin
sudo mv g++ g++-4.8_
sudo ln -s g++-7 g++
sudo mv gcc gcc-4.8_
sudo ln -s gcc-7 gcc
And test:
gcc --version
g++ --version
Now it's time to compile some code. But, before this we need to unchanged the modifications on our mkspec:
cd /usr/local/Trolltech/Qt-4.8.4/mkspecs/linux-g++-64/
nano qmake.conf
And comment the lines changed and back to the original configuration:
#QMAKE_CFLAGS = -std=gnu++98 -fpermissive
#QMAKE_LFLAGS = -std=gnu++98 -fpermissive
#QMAKE_CXXFLAGS = -std=gnu++98 -fpermissive
QMAKE_CFLAGS = -m64
QMAKE_LFLAGS = -m64
QMAKE_CXXFLAGS = -m64
Now your change you *.pro file adding or change these lines:
QMAKE_CXX = g++-7
QMAKE_CC = gcc-7
Let me know if it works. For me everything it's fine.
while installing rstan getting following error:
Error in .shlib_internal(args) :
C++14 standard requested but CXX14 is not defined
from research got to know that C++14 compiler should be available.
How to install the same while configuring R.
Using the below command to configure R
./configure --with-readline=no --with-x=no
and installing
yum install -y devtoolset-6
but still not able to update C++14 and rstan gives the error
Default C++ compiler: g++ -g -O2
C++98 compiler: g++ -g -O2
C++11 compiler: g++ -std=gnu++11 -g -O2
C++14 compiler: g++ -g -O2
C++17 compiler:
Fortran 90/95 compiler: gfortran -g -O2
Obj-C compiler:
setup.sh
yum -y update
yum install -y centos-release-scl
yum install -y devtoolset-6
yum install -y devtoolset-6-gcc-gfortran
scl enable devtoolset-6 bash
scl enable devtoolset-6-gcc-gfortran bash
mkdir packages
cd packages
wget -qO-
https://downloads.sourceforge.net/project/libpng/zlib/1.2.8/zlib-
1.2.8.tar.gz | tar zvx
cd zlib-1.2.8
./configure
make
make install
cd ..
wget -qO- http://downloads.sourceforge.net/pcre/pcre-8.35.tar.gz |
tar xzv
cd pcre-8.35
./configure
make
make install
cd ..
wget -qO- http://tukaani.org/xz/xz-5.2.2.tar.gz | tar xzv
cd xz-5.2.2
./configure
make
make install
cd ..
wget -qO- https://curl.haxx.se/download/curl-7.47.1.tar.gz | tar xzv
cd curl-7.47.1
./configure
make
make install
cd ..
wget -qO- https://cran.r-project.org/src/base/R-3/R-3.4.4.tar.gz |
tar xzv
cd R-3.4.4
./configure --with-readline=no --with-x=no --prefix=/packages/R-3.4.4
F77=gfortran
make
make install
I also got this problem, here I record how to solve it.
The key point is installing proper g++ and configure it.
Firstly, install g++ Version >= 5 as https://github.com/stan-dev/rstan/wiki/Installing-RStan-on-Linux said:
Using RStan requires either g++ version 4.9 and up
Here I am installing g++8 (You can change the version as your wish):
Run
$ sudo yum install centos-release-scl
$ sudo yum install devtoolset-8-gcc*
Now you have an alternative g++ along with default g++ in your OS.
You can enable this and check the version:
$ scl enable devtoolset-8 bash
$ gcc --version
$ g++ --version
Find its location:
$ which g++
/opt/rh/devtoolset-8/root/usr/bin/g++
Next, you need to configure ~/.R/Makevars, put the following content to it either using vim (or other editors):
CXX14FLAGS=-O3 -march=native -mtune=native -fPIC
CXX14=/opt/rh/devtoolset-8/root/usr/bin/g++
Or using R commands:
dotR <- file.path(Sys.getenv("HOME"), ".R")
if (!file.exists(dotR)) dir.create(dotR)
M <- file.path(dotR, "Makevars")
if (!file.exists(M)) file.create(M)
cat("\nCXX14FLAGS=-O3 -march=native -mtune=native -fPIC",
"CXX14=/opt/rh/devtoolset-8/root/usr/bin/g++", # or clang++ but you may need a version postfix
file = M, sep = "\n", append = TRUE)
NOTE: these R commands are copied from https://github.com/stan-dev/rstan/wiki/Configuring-C-Toolchain-for-Linux but CXX14 flag is modified according to the location above.
Now, you can install rstan package now:
install.packages("rstan")
Hope this helps.
PS: R users can use this approach to modify the compiler flags in a similar way.
This is what worked for me:
CXX_STD = CXX14
CXX14 = g++ -std=c++11
CXX14FLAGS = -O3 -fPIC -Wno-unused-variable -Wno-unused-function -DBOOST_PHOENIX_NO_VARIADIC_EXPRESSION
You don't need to recompile R but you do need at least g++-4.9 (or clang++-3.4) and to define CXX14 = g++ in your ~/.R/Makevars configuration file. In addition, you usually need CXX14FLAGS = -fPIC and would be well advised to have CXX14FLAGS = -O3 -mtune = native -march = native -fPIC. There is a wiki page for all this.
did the following changes and now it's working fine. Need to define gcc PATH and used yum install -y devtoolset-6 for R-3.4.4. Thank you for help
yum install -y centos-release-scl
yum install -y devtoolset-6
yum install -y bzip2-devel
source scl_source enable devtoolset-6
also added the Path to gcc in build.sh
export PATH=/opt/rh/devtoolset-6/root/bin:$PATH
I had the same problem.
Here is my solution in case it helps others.
On CentOS 7, I:
installed V8 headers and shared library from here: https://pkgs.org/download/v8-devel 41 (EPEL_x86_64)
bumped gcc from version 4 to 9.3.1
In R, I:
removed rstan and StanHeaders
installed rstan and StanHeaders in a fresh R session
Posted discussion on Stan Forum here: https://discourse.mc-stan.org/t/problem-installing-rstan-2-21-using-r-4-0-1-on-centos-7/17784/9
I have compiled using below parameters
#./configure --prefix=/usr/local CFLAGS=-fPIC --enable-static --enable-shared \
CFLAGS="-g -O2 -DSQLITE_ENABLE_FTS3=1 \
-DSQLITE_ENABLE_COLUMN_METADATA=1 \
-DSQLITE_ENABLE_UNLOCK_NOTIFY=1 \
-DSQLITE_SECURE_DELETE=1 \
-DSQLITE_ENABLE_DBSTAT_VTAB=1"
# make -j 8
# make install
After using sqlite I get below error message:
/usr/bin/g++ -o utools/utools.so -Wl,-Bsymbolic -g -O2 -shared -Wl,-z,defs utools/utools.plugin.o utools/lib/libutools.a core/lib/libunv.a fv/lib/libfv.a sb/lib/libsb.a -lz -ldl
/usr/bin/ld: /usr/local/sqlite/lib/libsqlite3.a(sqlite3.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
/usr/local/sqlite/lib/libsqlite3.a: could not read symbols: Bad value
collect2: ld returned 1 exit status
scons: *** [stdarm/stdarm.so] Error 1
scons: building terminated because of errors.
inside /usr/local/lib/ directory, deleting following files worked for me
libsqlite3.a
libsqlite3.la
libsqlite3.o.0.bak
libsqlite3.so
I think these files are from the older versions of libsqlite3,
because when you type and check dpkg -L libsqlite3-dev, none goes to /usr/local/lib/
I took the reference from sqlite3 breakage
I've been struggling with this problem for a while now. When I install rJava from R, it takes Java 1.6 instead of 1.8. I am using MAC OS 10.8.5 and these are the steps that I've already done.
I've installed R.
I have both Java 1.6 and 1.8 on my machine.
I've reconfigures R from terminal using: sudo R CMD javareconf -n.
Here is what the output looks like:
Java interpreter : /usr/bin/java
Java version : 1.8.0_25
Java home path : /Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre
Java compiler : /usr/bin/javac
Java headers gen.: /usr/bin/javah
Java archive tool: /usr/bin/jar
Non-system Java on OS X
trying to compile and link a JNI progam
detected JNI cpp flags : -I$(JAVA_HOME)/../include -I$(JAVA_HOME)/../include/darwin
detected JNI linker flags : -L$(JAVA_HOME)/lib/server -ljvm
gcc -arch x86_64 -std=gnu99 -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/../include -I/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/../include/darwin -I/usr/local/include -fPIC -mtune=core2 -g -O2 -c conftest.c -o conftest.o
gcc -arch x86_64 -std=gnu99 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -L/usr/local/lib -o conftest.so conftest.o -L/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/server -ljvm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
JAVA_HOME : /Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre
Java library path: $(JAVA_HOME)/lib/server
JNI cpp flags : -I$(JAVA_HOME)/../include -I$(JAVA_HOME)/../include/darwin
JNI linker flags : -L$(JAVA_HOME)/lib/server -ljvm
Then I went on to R and installed rJava package as source like this:
install.packages("rJava", type="source")
and I get to DONE(rJava)
then when I check the version using:
library(rJava)
.jinit()
.jcall("java/lang/System", "S", "getProperty", "java.runtime.version")
I still get 1.6.0_25-b17 instead of 1.8
Please let me know if you know the solution to this.
Latest version of rJava has fixed this issue.
I installed latest version from site using command
install.packages("rJava",,"http://rforge.net/",type="source")
Now the output by calling jvm is as:
> library(rJava)
> .jinit()
> .jcall("java/lang/System", "S", "getProperty", "java.runtime.version")
[1] "1.7.0_75-b13"
Note: I have java 1.7 and 1.6 installed, and not java 1.8
What helped me in this case was 2 things (following advice from here: https://www3.ntu.edu.sg/home/ehchua/programming/howto/JDK_Howto.html)
Informing Ubuntu of the java installation that you want to use.
For this I ran:
$ sudo update-alternatives --install "/usr/bin/java" "java" "/usr/java/jdk1.8.0_201/bin/java" 1
$ sudo update-alternatives --install "/usr/bin/javac" "javac" "/usr/java/jdk1.8.0_201/bin/javac" 1
$ sudo update-alternatives --install "/usr/bin/javaws" "javaws" "/usr/java/jdk1.8.0_201/bin/javaws" 1
followed by
$ sudo update-alternatives --set java /usr/java/jdk1.8.0_201/bin/java
$ sudo update-alternatives --set javac /usr/java/jdk1.8.0_201/bin/javac
$ sudo update-alternatives --set javaws /usr/java/jdk1.8.0_201/bin/javaws
Then I could finally see my java in the alternatives in
sudo update-alternatives --config java
where I selected the newly installed java. Then I ran
sudo R CMD javareconf JAVA_HOME=/usr/java/jdk1.8.0_201/bin/