I want to cross compile Qt to develop a gui for a raspberry pi. I have been following this tutorial: https://wiki.qt.io/RaspberryPi2EGLFS
I am using this configure command:
./configure -release -opengl es2 -device linux-rasp-pi3-g++ -device-option \
CROSS_COMPILE=~/raspi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin/arm-linux-gnueabihf- \
-sysroot ~/raspi/sysroot -opensource -confirm-license \
-make libs -no-use-gold-linker -prefix /usr/local/qt5pi \
-extprefix ~/raspi/qt5pi -hostprefix ~/raspi/qt5 -v
This is the error I'm getting back:
/home/vbox/raspi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin/arm-linux-gnueabihf-g++ -mfloat-abi=hard --sysroot=/home/vbox/raspi/sysroot -Wl,-O1 -Wl,-rpath-link,/home/vbox/raspi/sysroot/opt/vc/lib -Wl,-rpath-link,/home/vbox/raspi/sysroot/usr/lib/arm-linux-gnueabihf -Wl,-rpath-link,/home/vbox/raspi/sysroot/lib/arm-linux-gnueabihf -o verifyspec verifyspec.o
/home/vbox/raspi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin/../lib/gcc/arm-linux-gnueabihf/4.8.3/../../../../arm-linux-gnueabihf/bin/ld: cannot find crt1.o: No such file or directory
/home/vbox/raspi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin/../lib/gcc/arm-linux-gnueabihf/4.8.3/../../../../arm-linux-gnueabihf/bin/ld: cannot find crti.o: No such file or directory
/home/vbox/raspi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin/../lib/gcc/arm-linux-gnueabihf/4.8.3/../../../../arm-linux-gnueabihf/bin/ld: cannot find -lm
collect2: error: ld returned 1 exit status
Makefile:66: recipe for target 'verifyspec' failed
make: *** [verifyspec] Error 1
Note: Also available for Linux: linux-clang linux-icc
ERROR: Cannot compile a minimal program. The toolchain or QMakeSpec is broken.`
This is the config.log:
vbox#vbox:~/qtbase$ cat config.log
Command line: -release -opengl es2 -device linux-rasp-pi3-g++ -device-option CROSS_COMPILE=/home/vbox/raspi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin/arm-linux-gnueabihf- -sysroot /home/vbox/raspi/sysroot -opensource -confirm-license -make libs -no-use-gold-linker -prefix /usr/local/qt5pi -extprefix /home/vbox/raspi/qt5pi -hostprefix /home/vbox/raspi/qt5 -v
executing config test machineTuple
+ /home/vbox/raspi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin/arm-linux-gnueabihf-g++ -dumpmachine
arm-linux-gnueabihf
test config.qtbase.tests.machineTuple succeeded
executing config test verifyspec
+ cd /home/vbox/qtbase/config.tests/verifyspec && /home/vbox/qtbase/bin/qmake "CONFIG -= qt debug_and_release app_bundle lib_bundle" "CONFIG += shared warn_off console single_arch" "QMAKE_CFLAGS += --sysroot=/home/vbox/raspi/sysroot" "QMAKE_CXXFLAGS += --sysroot=/home/vbox/raspi/sysroot" "QMAKE_LFLAGS += --sysroot=/home/vbox/raspi/sysroot" -early "CONFIG += cross_compile" /home/vbox/qtbase/config.tests/verifyspec
Info: creating stash file /home/vbox/qtbase/config.tests/.qmake.stash
+ cd /home/vbox/qtbase/config.tests/verifyspec && MAKEFLAGS= /usr/bin/make
/home/vbox/raspi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin/arm-linux-gnueabihf-g++ -c -march=armv8-a -mtune=cortex-a53 -mfpu=crypto-neon-fp-armv8 -mfloat-abi=hard --sysroot=/home/vbox/raspi/sysroot -O2 -std=gnu++11 -w -fPIC -I. -I/home/vbox/qtbase/mkspecs/devices/linux-rasp-pi3-g++ -o verifyspec.o verifyspec.cpp
/home/vbox/raspi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin/arm-linux-gnueabihf-g++ -mfloat-abi=hard --sysroot=/home/vbox/raspi/sysroot -Wl,-O1 -Wl,-rpath-link,/home/vbox/raspi/sysroot/opt/vc/lib -Wl,-rpath-link,/home/vbox/raspi/sysroot/usr/lib/arm-linux-gnueabihf -Wl,-rpath-link,/home/vbox/raspi/sysroot/lib/arm-linux-gnueabihf -o verifyspec verifyspec.o /home/vbox/raspi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin/../lib/gcc/arm-linux-gnueabihf/4.8.3/../../../../arm-linux-gnueabihf/bin/ld: cannot find crt1.o: No such file or directory
/home/vbox/raspi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin/../lib/gcc/arm-linux-gnueabihf/4.8.3/../../../../arm-linux-gnueabihf/bin/ld: cannot find crti.o: No such file or directory
/home/vbox/raspi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin/../lib/gcc/arm-linux-gnueabihf/4.8.3/../../../../arm-linux-gnueabihf/bin/ld: cannot find -lm
collect2: error: ld returned 1 exit status
Makefile:66: recipe for target 'verifyspec' failed
make: *** [verifyspec] Error 1
I'm not sure where to go from here, I don't know enough to start debugging this issue. Are there alternate toolchains available? Or is the issue with the QMakeSpec? Any suggestions on things to try would be very helpful.
This blog post seems more specific to RPi3, and probably does it too (last Jan 12, 2017):
https://medium.com/#amirmann/how-to-cross-compile-qt-for-raspberry-pi-3-on-linux-ubuntu-for-beginners-75acf2a078c
sudo apt-get upgrade
sudo apt-get update
sudo apt-get install libgl1-mesa-dev
...
sudo apt-get build-dep qt5-default
sudo apt-get install libxcb-xinerama0-dev
sudo apt-add-repository ppa:u-szeged/sedkit
sudo apt-get update
sudo apt-get install sedkit-env-qtwebkit
sudo apt-get install build-essential perl python git
sudo apt-get install “^libxcb.*” libx11-xcb-dev libglu1-mesa-dev libxrender-dev libxi-dev
sudo apt-get install flex bison gperf libicu-dev libxslt-dev ruby
sudo apt-get install libssl-dev libxcursor-dev libxcomposite-dev libxdamage-dev libxrandr-dev libfontconfig1-dev libcap-dev libxtst-dev libpulse-dev libudev-dev libpci-dev libnss3-dev libasound2-dev libxss-dev libegl1-mesa-dev gperf bison
sudo apt-get install libbz2-dev libgcrypt11-dev libdrm-dev libcups2-dev libatkmm-1.6-dev
sudo apt-get install libasound2-dev libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev
There is a note on the wiki you are looking at of some things they did to get it to work on newer builds. Here is some of the history of that wiki page:
https://wiki.qt.io/index.php?title=RaspberryPi2EGLFS&type=revision&diff=31538&oldid=30634
And then there is the device creation licensed version of Qt then you would use these instructions:
http://doc.qt.io/QtForDeviceCreation/qtee-preparing-hardware-rasberrypi.html
I have always learned a lot while digging into the articles on Linux from Scratch. In my recent building of Qt with nothing, they have been super helpful.
http://www.linuxfromscratch.org/blfs/view/8.0/x/qt5.html
Hope those links help.
This is my bash script for cross-compiling Qt for raspberry-pi. You can
this script with command <script name> <TcpIp of Raspberrry> $USER
#!/bin/bash
RASP_ADR=$1
USERNAME=$2
if [ ! $RASP_ADR ]
then
echo "Error: need Ip address"
exit 1
fi
LOGFILE=install.log.txt
rm -f $LOGFILE
ID_RSA=/root/.ssh/cu4_rsa
SSH="ssh -i $ID_RSA root#$RASP_ADR"
DEST=~/raspi
QT_VERSION=5.9
function log {
local t=`date`
echo "[$t] $#" | tee -a $LOGFILE
}
function remote_src {
local message
local exitcode
message=$1
shift
log $message
log "Remote $RASP_ADR command: $#"
OUT=`$SSH "$#" | tee /dev/tty | tee -a $LOGFILE`
exitcode=$?
if [ $exitcode -ne 0 ]; then
log "Exit with ERROR $exitcode"
exit $exitcode
fi
}
function local_src {
local message
local exitcode
message=$1
shift
log $message
log "Local command: $#"
OUT=`bash -c "$#" | tee /dev/tty | tee -a $LOGFILE`
exitcode=$?
if [ $exitcode -ne 0 ]; then
log "Exit with ERROR $exitcode"
exit $exitcode
fi
}
function Append_Qt_Module {
local QtModule
QtModule=$1
log "install $QtModule"
local_src \
"Clone $QtModule Qt-module" \
"cd $DEST && git clone git://code.qt.io/qt/$QtModule.git -b $QT_VERSION"
local_src \
"Make $QtModule Qt-module" \
"cd $DEST/$QtModule && $DEST/qt5/bin/qmake -r && make && make install"
local_src \
"Synchronize qt5pi with Raspberry pi" \
"rsync -avz -e \"ssh -i $ID_RSA\" $DEST/qt5pi pi#$RASP_ADR:/usr/local"
}
local_src \
"open key -> Raspberry" \
"cat /root/.ssh/cu4_rsa.pub | ssh pi#$RASP_ADR \"sudo cat - > authorized_keys; sudo mkdir -p /root/.ssh; sudo mv authorized_keys /root/.ssh; sudo chown -R root:root /root/.ssh\""
remote_src \
"Update System" \
"apt-get update"
remote_src \
"Upgrade System" \
"apt-get -y upgrade"
remote_src \
"Install Synaptic" \
"apt-get -y install synaptic"
remote_src \
"Install Qt libraries" \
"apt-get -y install libfontconfig1-dev libdbus-1-dev libudev-dev"
remote_src \
"Install Xcb" \
"apt-get -y install \"^libxcb.*\" libx11-xcb-dev libglu1-mesa-dev libxrender-dev libxi-dev"
remote_src \
"Install Qt-WebKit" \
"apt-get -y install flex bison gperf libicu-dev libxslt-dev ruby"
remote_src \
"Install Qt WebEngine" \
"apt-get -y install libssl-dev libxcursor-dev libxcomposite-dev libxrandr-dev"
remote_src \
"Install Qt Multimedia" \
"apt-get -y install libasound2-dev libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev"
DEB_SRC="deb-src http://archive.raspbian.org/raspbian/ jessie main contrib non-free rpi"
remote_src \
"Проверка открытости deb_src" \
"cat /etc/apt/sources.list | grep -s -G '^$DEB_SRC' | cat"
if [ ! "$OUT" ]
then
remote_src \
"Открываем комментарии" \
"echo '$DEB_SRC' >> /etc/apt/sources.list"
else
log "DEB_SRC открыты ранее"
fi
remote_src \
"Update System" \
"apt-get update"
remote_src \
"Build dependences for qt4-x11" \
"apt-get -y build-dep qt4-x11"
remote_src \
"Build dependences for libqt5gui5" \
"apt-get -y build-dep libqt5gui5"
remote_src \
"Install more libs for Xcb" \
"apt-get -y install libinput-dev libts-dev libxcb-xinerama0-dev libxcb-xinerama0"
remote_src \
"Make dir for Qt libs" \
"mkdir /usr/local/qt5pi; chown pi:pi /usr/local/qt5pi"
remote_src \
"Reboot" \
"reboot"
while true; do
sleep 1s
log "wait..."
$SSH "echo done"
if [ $? -eq 0 ]; then break; fi
done
local_src \
"Install Git" \
"apt-get -y install git && mkdir -p $DEST && cd $DEST"
local_src \
"Clone RaspberryPi tools" \
"git clone https://github.com/raspberrypi/tools $DEST/tools"
local_src \
"OR Update Raspberry Pi Tools" \
"cd $DEST/tools && git pull origin master"
fi
local_src \
"Synchronize local computer with Raspberry Pi" \
"cd $DEST && mkdir -p /sysroot /sysroot/usr /sysroot/opt && \
rsync -avz -e \"ssh -i $ID_RSA\" root#$RASP_ADR:/lib sysroot && \
rsync -avz -e \"ssh -i $ID_RSA\" root#$RASP_ADR:/usr/include sysroot/usr && \
rsync -avz -e \"ssh -i $ID_RSA\" root#$RASP_ADR:/usr/lib sysroot/usr && \
rsync -avz -e \"ssh -i $ID_RSA\" root#$RASP_ADR:/opt/vc sysroot/opt && \
rsync -avz -e \"ssh -i $ID_RSA\" root#$RASP_ADR:/usr/local/include sysroot/usr/local && \
rsync -avz -e \"ssh -i $ID_RSA\" root#$RASP_ADR:/usr/local/lib sysroot/usr/local"
local_src \
"Get sysroot-relative.py script" \
"cd $DEST && wget https://raw.githubusercontent.com/riscv/riscv-poky/master/scripts/sysroot-relativelinks.py && \
chmod +x sysroot-relativelinks.py && \
./sysroot-relativelinks.py sysroot "
exit 0
local_src \
"Получаем qtbase версии $QT_VERSION и конфигурируем Qt."
"`git clone -b $QT_VERSION git://code.qt.io/qt/qtbase.git $DEST/qtbase`"
local_src \
"OR Update qtbase версии $QT_VERSION" \
"cd $DEST/qtbase && git pull origin $QT_VERSION"
local_src \
"Конфигурация и установка Qt$QT_VERSION" \
"cd $DEST/qtbase && ./configure -release -opengl es2 -device linux-rasp-pi3-g++ \
-device-option CROSS_COMPILE=$DEST/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian/bin/arm-linux-gnueabihf- \
-sysroot $DEST/sysroot -opensource -confirm-license -make libs -make tools -qt-pcre -optimized-qmake -reduce-exports \
-release -prefix /usr/local/qt5pi -extprefix $DEST/qt5pi -hostprefix $DEST/qt5 -no-use-gold-linker -v && \
make -j 4 && make install"
local_src \
"Change owner to $USERNAME on $DEST" \
"chown -R $USERNAME:$USERNAME $DEST"
local_src \
"Get new free types from http://dejavu-fonts.org" \
"cd $DEST && wget http://sourceforge.net/projects/dejavu/files/dejavu/2.37/dejavu-fonts-ttf-2.37.tar.bz2 && \
tar -xvf dejavu-fonts-ttf-2.37.tar.bz2"
local_src \
"Send type to target dir" \
"cd $DEST && mkdir -p qt5pi/lib/fonts && \
cp dejavu-fonts-ttf-2.37/ttf/DejaVuSansMono.ttf qt5pi/lib/fonts/"
local_src \
"Send libs to RaspberryPi" \
"cd $DEST && rsync -avz -e \"ssh -i $ID_RSA\" qt5pi root#$RASP_ADR:/usr/local"
local_src \
"Build Test application" \
"cd $DEST/qtbase/examples/opengl/qopenglwidget && \
$DEST/qt5/bin/qmake && \
make && \
scp -i $ID_RSA qopenglwidget root#$RASP_ADR:/home/pi "
remote_src \
"Добавляем пути к библиотекам Qt на Raspberry Pi" \
"echo /usr/local/qt5pi/lib | tee /etc/ld.so.conf.d/qt5pi.conf && \
ldconfig"
remote_src \
"Устранение неполадок в библиотеке EGL/GLES " \
"rm /usr/lib/arm-linux-gnueabihf/libEGL.so.1.0.0 /usr/lib/arm-linux-gnueabihf/libGLESv2.so.2.0.0 && \
ln -s /opt/vc/lib/libEGL.so /usr/lib/arm-linux-gnueabihf/libEGL.so.1.0.0 && \
ln -s /opt/vc/lib/libGLESv2.so /usr/lib/arm-linux-gnueabihf/libGLESv2.so.2.0.0 && \
ln -s /opt/vc/lib/libEGL.so /opt/vc/lib/libEGL.so.1 && \
ln -s /opt/vc/lib/libGLESv2.so /opt/vc/lib/libGLESv2.so.2"
remote_src \
"Запуск теста" \
"nohup /home/pi/qopenglwidget > /dev/null 2>&1 &"
log "Please push the some key when you are ready to next operation"
read -n1
remote_src \
"Kill test" \
"pkill qopenglwidget"
#Add youre Qt modules
Append_Qt_Module qtserialport
remote_src \
"Обновляем символьные линки к библиотекам Qt на Raspberry Pi" \
"ldconfig"
log "DONE!!!!"
exit 0
I edited device mkspec file linux-rasp-pi3-g++/qmake.conf like this
VC_LIBRARY_PATH = $$[QT_SYSROOT]/opt/vc/lib
VC_INCLUDE_PATH = $$[QT_SYSROOT]/opt/vc/include
QMAKE_LIBDIR_OPENGL_ES2 = $${VC_LIBRARY_PATH}
QMAKE_LIBDIR_EGL = $$QMAKE_LIBDIR_OPENGL_ES2
QMAKE_LIBDIR_OPENVG = $$QMAKE_LIBDIR_OPENGL_ES2
QMAKE_INCDIR_EGL = \
$${VC_INCLUDE_PATH} \
$${VC_INCLUDE_PATH}/interface/vcos/pthreads \
$${VC_INCLUDE_PATH}/interface/vmcs_host/linux
QMAKE_INCDIR_OPENGL_ES2 = $${QMAKE_INCDIR_EGL}
QMAKE_INCDIR_OPENVG = $${QMAKE_INCDIR_EGL}
QMAKE_LIBS_OPENGL_ES2 = -lGLESv2
QMAKE_LIBS_EGL = -lEGL -lGLESv2
QMAKE_LIBS_OPENVG = -lEGL -lOpenVG -lGLESv2
and use gcc-linaro-4.9-2015.02-3-x86_64_arm-linux-gnueabihf toolchain.
I wrote this blog which may be helpful: Build Qt 5.10 for Raspberry Pi 3
Related
dockerfile
FROM ubuntu:20.04
WORKDIR /tmp
COPY build.sh /tmp
RUN ./build.sh
build script
#!/bin/bash
set -xe
NGINX_DIR="/etc/nginx"
mkdir -p ${NGINX_DIR}
mkdir -p ${NGINX_DIR}/temp
mkdir -p /tmp/nginx_bin
apt update -y && apt install wget gcc make libpcre3 libpcre3-dev zlib1g-dev libssl-dev libgeoip-dev automake -y
wget https://nginx.org/download/nginx-1.22.1.tar.gz -P /tmp && tar zxvf /tmp/nginx-1.22.1.tar.gz -C /tmp
cd /tmp/nginx-1.22.1 && ./configure \
--with-debug \
--prefix=${NGINX_DIR} \
--sbin-path=${NGINX_DIR}/sbin/nginx \
--conf-path=${NGINX_DIR}/nginx.conf \
--pid-path=${NGINX_DIR}/logs/nginx.pid \
--lock-path=${NGINX_DIR}/logs/nginx.lock \
--error-log-path=${NGINX_DIR}/logs/error.log \
--http-log-path=${NGINX_DIR}/logs/access.log \
--with-cc-opt="-O2 -g -Wno-error" \
--with-ld-opt="-static" \
--with-pcre-jit \
--http-client-body-temp-path=${NGINX_DIR}/temp/client_body_temp \
--http-proxy-temp-path=${NGINX_DIR}/temp/proxy_temp \
--with-stream \
--with-stream_ssl_module \
--with-file-aio \
--with-threads \
--with-http_geoip_module \
--with-http_auth_request_module \
--with-http_ssl_module \
--with-http_flv_module \
--with-http_mp4_module \
--with-http_v2_module \
--with-http_slice_module \
--with-http_gunzip_module \
--with-http_random_index_module \
--with-http_secure_link_module \
--with-http_addition_module \
--with-http_gzip_static_module \
--with-http_realip_module \
--with-http_stub_status_module \
--with-http_dav_module \
--with-http_sub_module \
--without-mail_pop3_module \
--without-mail_imap_module \
--without-mail_smtp_module \
--with-openssl-opt="-g" && make -j4 && make -j4 install && cp objs/nginx /tmp/nginx_bin
I use the above dockerfile and shell script to compile nginx from source, if remove --with-ld-opt="-static" parameter, it will compile success, but after I add --with-ld-opt="-static" parameter, the complie will fail with the below error
checking for GeoIP library ... not found
checking for GeoIP library in /usr/local/ ... not found
checking for GeoIP library in /usr/pkg/ ... not found
checking for GeoIP library in /opt/local/ ... not found
but I already install the libgeoip-dev when running shell script
Could any one help take a look, how to fix it, thanks.
I use the above dockerfile and shell script to compile nginx from source, if remove --with-ld-opt="-static" parameter, it will compile success, but after I add --with-ld-opt="-static" parameter, the complie will fail with the below error
checking for GeoIP library ... not found
checking for GeoIP library in /usr/local/ ... not found
checking for GeoIP library in /usr/pkg/ ... not found
checking for GeoIP library in /opt/local/ ... not found
I have a problem building a docker image with the following codes:
FROM rocker/verse:latest
MAINTAINER xxx
RUN apt-get update \
&& apt-get install -y --no-install-recommends apt-utils ed libnlopt-dev libgdal-dev libudunits2-dev cargo jags libcairo2-dev libxt-dev libgeos-dev \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/
# Install rstan
RUN install2.r --error --deps TRUE \
rstan \
&& rm -rf /tmp/downloaded_packages/ /tmp/*.rds
# Global site-wide config -- neeeded for building packages
RUN mkdir -p $HOME/.R/ \
&& echo "CXXFLAGS=-O3 -mtune=native -march=native -Wno-unused-variable -Wno-unused-function -flto -ffat-lto-objects -Wno-unused-local-typedefs \n" >> $HOME/.R/Makevars
# Config for rstudio user
RUN mkdir -p $HOME/.R/ \
&& echo "CXXFLAGS=-O3 -mtune=native -march=native -Wno-unused-variable -Wno-unused-function -flto -ffat-lto-objects -Wno-unused-local-typedefs -Wno-ignored-attributes -Wno-deprecated-declarations\n" >> $HOME/.R/Makevars \
&& echo "rstan::rstan_options(auto_write = TRUE)\n" >> /home/rstudio/.Rprofile \
&& echo "options(mc.cores = parallel::detectCores())\n" >> /home/rstudio/.Rprofile
RUN Rscript -e "install.packages('cmdstanr', repos = 'http://cran.us.r-project.org', dependencies=TRUE)"
# Install rstan
RUN install2.r --error --deps TRUE \
rstan \
loo \
bayesplot \
rstanarm \
rstantools \
shinystan \
ggmcmc \
tidybayes \
LaplacesDemon \
&& rm -rf /tmp/downloaded_packages/ /tmp/*.rds
RUN install2.r --error --deps TRUE \
zoo \
mgcv \
lubridate \
stringr \
loo \
jsonlite \
scales \
directlabels \
betareg \
extrafont \
pointdensityP \
&& rm -rf /tmp/downloaded_packages/ /tmp/*.rds
# Install Roboto condensed
RUN wget -O /tmp/rc.zip https://fonts.google.com/download?family=Roboto%20Condensed \
&& cd /usr/share/fonts \
&& sudo mkdir googlefonts \
&& cd googlefonts \
&& sudo unzip -d . /tmp/rc.zip \
&& sudo chmod -R --reference=/usr/share/fonts/truetype /usr/share/fonts/googlefonts \
&& sudo fc-cache -fv \
&& rm -rf /tmp/rc.zip \
&& Rscript --slave --no-save --no-restore-history -e "extrafont::font_import(prompt=FALSE)"
And I failed to build the docker image with the following error:
#10 224.1 Error: dependency ‘cmdstanr’ is not available------executor failed running [/bin/sh -c install2.r --error --deps TRUE rstan loo bayesplot rstanarm rstantools shinystan ggmcmc tidybayes LaplacesDemon && rm -rf /tmp/downloaded_packages/ /tmp/*.rds]: exit code: 1
Can anyone help me with the issue?
The Getting Started with CmdStanR vignette says you should use
install.packages("cmdstanr",
repos = c("https://mc-stan.org/r-packages/", getOption("repos")))
you're trying to install it from a CRAN mirror, which won't work because the package isn't on CRAN. (I haven't tested, you might want to install cmdstanr after you install Stan ...
Is it possible (advisable) to use R within Google Cloud Datalab? How?
I made a pull request on Google Datalab. It is just a modification of the original docker file in the folder:
datalab/containers/base/Dockerfile
You can find the modified file here:
My Git repository
Or the full code inside the Dockerfile here (below). You need to push the Docker container to the container registry of google.
https://cloud.google.com/container-registry/docs/pushing-and-pulling
# Copyright 2015 Google Inc. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# We use different base images for GPU vs CPU Dockerfiles, so we expect
# that the appropriate image is pulled and tagged locally.
# CPU should use ubuntu:16.04
# and GPU uses nvidia/cuda:8.0-cudnn5-devel-ubuntu16.04
FROM datalab-external-base-image
MAINTAINER Google Cloud DataLab
# Container configuration
EXPOSE 8080
# Path configuration
ENV PATH $PATH:/tools/node/bin:/tools/google-cloud-sdk/bin
ENV PYTHONPATH /env/python
RUN apt-get update -y && apt-get install -y --no-install-recommends apt-utils
# Setup OS and core packages
RUN echo "deb-src http://ftp.us.debian.org/debian testing main" >> /etc/apt/sources.list && \
apt-get update -y && \
apt-get install -y -q debian-archive-keyring debian-keyring && \
apt-get update -y && \
apt-get install --no-install-recommends -y -q python unzip ca-certificates build-essential \
libatlas-base-dev liblapack-dev gfortran libpng-dev libfreetype6-dev libxft-dev libxml2-dev \
libsnappy-dev python-dev python-setuptools python-zmq openssh-client wget curl git \
pkg-config zip && \
easy_install pip && \
mkdir -p /tools && \
# Save GPL source packages
mkdir -p /srcs && \
cd /srcs && \
apt-get source -d wget git python-zmq ca-certificates pkg-config libpng-dev && \
wget --progress=dot:mega https://mirrors.kernel.org/gnu/gcc/gcc-4.9.2/gcc-4.9.2.tar.bz2 && \
cd / && \
# Setup Python packages. Rebuilding numpy/scipy is expensive so we move this early
# to reduce the chance that prior steps can cause changes requiring a rebuild.
pip install -U --upgrade-strategy only-if-needed --no-cache-dir numpy==1.11.2 && \
pip install -U --upgrade-strategy only-if-needed --no-cache-dir pandas==0.19.1 && \
pip install -U --upgrade-strategy only-if-needed --no-cache-dir scipy==0.18.0 && \
pip install -U --upgrade-strategy only-if-needed --no-cache-dir scikit-learn==0.18.2 && \
pip install -U --upgrade-strategy only-if-needed --no-cache-dir scikit-image==0.13.0 && \
pip install -U --upgrade-strategy only-if-needed --no-cache-dir lime==0.1.1.23 && \
pip install -U --upgrade-strategy only-if-needed --no-cache-dir sympy==0.7.6.1 && \
pip install -U --upgrade-strategy only-if-needed --no-cache-dir statsmodels==0.6.1 && \
pip install -U --upgrade-strategy only-if-needed --no-cache-dir tornado==4.4.2 \
--upgrade-strategy only-if-needed --no-cache-dir pyzmq==16.0.2 \
--upgrade-strategy only-if-needed --no-cache-dir jinja2==2.8 \
--upgrade-strategy only-if-needed --no-cache-dir jsonschema==2.5.1 \
--upgrade-strategy only-if-needed --no-cache-dir python-dateutil==2.5.0 \
--upgrade-strategy only-if-needed --no-cache-dir pytz==2016.7 \
--upgrade-strategy only-if-needed --no-cache-dir pandocfilters==1.3.0 \
--upgrade-strategy only-if-needed --no-cache-dir pygments==2.1.3 \
--upgrade-strategy only-if-needed --no-cache-dir argparse==1.2.1 \
--upgrade-strategy only-if-needed --no-cache-dir mock==2.0.0 \
--upgrade-strategy only-if-needed --no-cache-dir requests==2.9.1 \
--upgrade-strategy only-if-needed --no-cache-dir oauth2client==2.2.0 \
--upgrade-strategy only-if-needed --no-cache-dir httplib2==0.10.3 \
--upgrade-strategy only-if-needed --no-cache-dir futures==3.0.5 && \
pip install -U --upgrade-strategy only-if-needed --no-cache-dir matplotlib==1.5.3 && \
pip install -U --upgrade-strategy only-if-needed --no-cache-dir ggplot==0.6.8 && \
pip install -U --upgrade-strategy only-if-needed --no-cache-dir seaborn==0.7.0 && \
pip install -U --upgrade-strategy only-if-needed --no-cache-dir PyYAML==3.11 && \
pip install -U --upgrade-strategy only-if-needed --no-cache-dir six==1.10.0 && \
pip install -U --upgrade-strategy only-if-needed --no-cache-dir ipywidgets==6.0.0 && \
pip install -U --upgrade-strategy only-if-needed --no-cache-dir ipykernel==4.5.2 && \
pip install -U --upgrade-strategy only-if-needed --no-cache-dir future==0.15.2 && \
pip install -U --upgrade-strategy only-if-needed --no-cache-dir psutil==4.3.0 && \
pip install -U --upgrade-strategy only-if-needed --no-cache-dir google-api-python-client==1.5.1 && \
pip install -U --upgrade-strategy only-if-needed --no-cache-dir plotly==1.12.5 && \
pip install -U --upgrade-strategy only-if-needed --no-cache-dir nltk==3.2.1 && \
pip install -U --upgrade-strategy only-if-needed --no-cache-dir bs4==0.0.1 && \
pip install -U --upgrade-strategy only-if-needed --no-cache-dir crcmod==1.7 && \
pip install -U --upgrade-strategy only-if-needed --no-cache-dir pillow==3.4.1 && \
pip install -U --upgrade-strategy only-if-needed --no-cache-dir google-cloud-dataflow==2.0.0 && \
pip install -U --upgrade-strategy only-if-needed --no-cache-dir tensorflow==1.2.1 && \
pip install -U --upgrade-strategy only-if-needed --no-cache-dir python-snappy==0.5.1 && \
pip install -U --upgrade-strategy only-if-needed --no-cache-dir xgboost==0.6a2 && \
find /usr/local/lib/python2.7 -type d -name tests | xargs rm -rf && \
# Python 3
apt-get install --no-install-recommends -y -q python3-dev && \
# Get pip set up such that pip -> py2 and pip3 -> py3
# And they both work.
wget https://bootstrap.pypa.io/get-pip.py && \
python3 ./get-pip.py && \
python ./get-pip.py --force-reinstall && \
rm ./get-pip.py && \
# Install other Python 3 packages
pip3 install -U --upgrade-strategy only-if-needed --no-cache-dir numpy==1.11.2 && \
pip3 install -U --upgrade-strategy only-if-needed --no-cache-dir pandas==0.19.1 && \
pip3 install -U --upgrade-strategy only-if-needed --no-cache-dir scipy==0.18.0 && \
pip3 install -U --upgrade-strategy only-if-needed --no-cache-dir scikit-learn==0.18.2 && \
pip3 install -U --upgrade-strategy only-if-needed --no-cache-dir scikit-image==0.13.0 && \
pip3 install -U --upgrade-strategy only-if-needed --no-cache-dir lime==0.1.1.23 && \
pip3 install -U --upgrade-strategy only-if-needed --no-cache-dir sympy==0.7.6.1 && \
pip3 install -U --upgrade-strategy only-if-needed --no-cache-dir statsmodels==0.6.1 && \
pip3 install -U --upgrade-strategy only-if-needed --no-cache-dir tornado==4.4.2 \
--upgrade-strategy only-if-needed --no-cache-dir pyzmq==16.0.2 \
--upgrade-strategy only-if-needed --no-cache-dir jinja2==2.8 \
--upgrade-strategy only-if-needed --no-cache-dir jsonschema==2.5.1 \
--upgrade-strategy only-if-needed --no-cache-dir python-dateutil==2.5.0 \
--upgrade-strategy only-if-needed --no-cache-dir pytz==2016.7 \
--upgrade-strategy only-if-needed --no-cache-dir pandocfilters==1.3.0 \
--upgrade-strategy only-if-needed --no-cache-dir pygments==2.1.3 \
--upgrade-strategy only-if-needed --no-cache-dir argparse==1.2.1 \
--upgrade-strategy only-if-needed --no-cache-dir mock==2.0.0 \
--upgrade-strategy only-if-needed --no-cache-dir requests==2.9.1 \
--upgrade-strategy only-if-needed --no-cache-dir oauth2client==2.2.0 \
--upgrade-strategy only-if-needed --no-cache-dir httplib2==0.10.3 \
--upgrade-strategy only-if-needed --no-cache-dir futures==3.0.5 && \
pip3 install -U --upgrade-strategy only-if-needed --no-cache-dir matplotlib==1.5.3 && \
pip3 install -U --upgrade-strategy only-if-needed --no-cache-dir ggplot==0.6.8 && \
pip3 install -U --upgrade-strategy only-if-needed --no-cache-dir seaborn==0.7.0 && \
pip3 install -U --upgrade-strategy only-if-needed --no-cache-dir PyYAML==3.11 && \
pip3 install -U --upgrade-strategy only-if-needed --no-cache-dir six==1.10.0 && \
pip3 install -U --upgrade-strategy only-if-needed --no-cache-dir psutil==4.3.0 && \
pip3 install -U --upgrade-strategy only-if-needed --no-cache-dir google-api-python-client==1.5.1 && \
pip3 install -U --upgrade-strategy only-if-needed --no-cache-dir plotly==1.12.5 && \
pip3 install -U --upgrade-strategy only-if-needed --no-cache-dir nltk==3.2.1 && \
pip3 install -U --upgrade-strategy only-if-needed --no-cache-dir bs4==0.0.1 && \
pip3 install -U --upgrade-strategy only-if-needed --no-cache-dir crcmod==1.7 && \
pip3 install -U --upgrade-strategy only-if-needed --no-cache-dir pillow==3.4.1 && \
pip3 install -U --upgrade-strategy only-if-needed --no-cache-dir tensorflow==1.2.1 && \
pip3 install -U --upgrade-strategy only-if-needed --no-cache-dir python-snappy==0.5.1 && \
pip3 install -U --upgrade-strategy only-if-needed --no-cache-dir xgboost==0.6a2 && \
# Install IPython related packages with no-deps, to ensure that we don't
# Overwrite the python 2 version of jupyter with the python 3 version.
pip3 install -U --upgrade-strategy only-if-needed --no-cache-dir ipywidgets==6.0.0 && \
pip3 install -U --upgrade-strategy only-if-needed --no-cache-dir notebook==4.2.3 && \
pip3 install -U --upgrade-strategy only-if-needed --no-cache-dir ipykernel==4.5.2 && \
find /usr/local/lib/python3.4 -type d -name tests | xargs rm -rf && \
# Install python2 version of notebook after the python3 version, so that we
# default to using python 2.
# Install notebook after ipywidgets, and keep it pinned to 4.2.3, higher versions may
# not work well with datalab. For example, kernel gateway doesn't work with 4.3.0 notebook
# (https://github.com/googledatalab/datalab/issues/1083)
pip install -U --force-reinstall --no-deps --no-cache-dir notebook==4.2.3 && \
pip install -U --upgrade-strategy only-if-needed --no-cache-dir notebook==4.2.3 && \
# Install the Python 2 and Python 3 kernels
python -m ipykernel install && \
python3 -m ipykernel install && \
# Setup Node.js using LTS 6.10
mkdir -p /tools/node && \
wget -nv https://nodejs.org/dist/v6.10.0/node-v6.10.0-linux-x64.tar.gz -O node.tar.gz && \
tar xzf node.tar.gz -C /tools/node --strip-components=1 && \
rm node.tar.gz && \
# Setup Google Cloud SDK
# Also apply workaround for gsutil failure brought by this version of Google Cloud.
# (https://code.google.com/p/google-cloud-sdk/issues/detail?id=538) in final step.
wget -nv https://dl.google.com/dl/cloudsdk/release/google-cloud-sdk.zip && \
unzip -qq google-cloud-sdk.zip -d tools && \
rm google-cloud-sdk.zip && \
tools/google-cloud-sdk/install.sh --usage-reporting=false \
--path-update=false --bash-completion=false \
--disable-installation-options && \
tools/google-cloud-sdk/bin/gcloud -q components update \
gcloud core bq gsutil compute preview alpha beta && \
# disable the gcloud update message
tools/google-cloud-sdk/bin/gcloud config set component_manager/disable_update_check true && \
touch /tools/google-cloud-sdk/lib/third_party/google.py && \
# Set our locale to en_US.UTF-8.
apt-get install -y locales && \
locale-gen en_US.UTF-8 && \
update-locale LANG=en_US.UTF-8 && \
# Add some unchanging bits - specifically node modules (that need to be kept in sync
# with packages.json manually, but help save build time, by preincluding them in an
# earlier layer).
/tools/node/bin/npm install \
ws#1.1.4 \
http-proxy#1.13.2 \
mkdirp#0.5.1 \
node-uuid#1.4.7 \
bunyan#1.7.1 \
tcp-port-used#0.1.2 \
node-cache#3.2.0 && \
cd / && \
/tools/node/bin/npm install -g forever && \
# Clean up
apt-get autoremove -y && \
rm -rf /var/lib/apt/lists/* && \
rm -rf /tmp/* && \
rm -rf /root/.cache/* && \
rm -rf /usr/share/locale/* && \
rm -rf /usr/share/i18n/locales/* && \
cd /
ENV LC_ALL en_US.UTF-8
ENV LANG en_US.UTF-8
ADD config/ipython.py /etc/ipython/ipython_config.py
ADD config/nbconvert.py /etc/jupyter/jupyter_notebook_config.py
# Directory "py" may be empty and in that case it will git clone pydatalab from repo
ADD pydatalab /datalab/lib/pydatalab
ADD nbconvert /datalab/nbconvert
# Do IPython configuration and install build artifacts
# Then link stuff needed for nbconvert to a location where Jinja will find it.
# I'd prefer to just use absolute path in Jinja imports but those don't work.
RUN ipython profile create default && \
jupyter notebook --generate-config
RUN if [ -d /datalab/lib/pydatalab/.git ]; then \
echo "use local lib"; \
else \
git clone https://github.com/googledatalab/pydatalab.git /datalab/lib/pydatalab; \
fi
RUN cd /datalab/lib/pydatalab && \
/tools/node/bin/npm install -g typescript && \
tsc --module amd --noImplicitAny --outdir datalab/notebook/static datalab/notebook/static/*.ts && \
tsc --module amd --noImplicitAny --outdir google/datalab/notebook/static google/datalab/notebook/static/*.ts && \
/tools/node/bin/npm uninstall -g typescript
RUN cd /datalab/lib/pydatalab && \
pip install --upgrade-strategy only-if-needed --no-cache-dir . && \
pip install --upgrade-strategy only-if-needed /datalab/lib/pydatalab/solutionbox/image_classification/. && \
pip install --upgrade-strategy only-if-needed /datalab/lib/pydatalab/solutionbox/structured_data/. && \
pip3 install --upgrade-strategy only-if-needed --no-cache-dir . && \
pip3 install --upgrade-strategy only-if-needed /datalab/lib/pydatalab/solutionbox/image_classification/. && \
pip3 install --upgrade-strategy only-if-needed /datalab/lib/pydatalab/solutionbox/structured_data/. && \
jupyter nbextension install --py datalab.notebook && \
jupyter nbextension install --py google.datalab.notebook && \
jupyter nbextension enable --py widgetsnbextension && \
rm datalab/notebook/static/*.js google/datalab/notebook/static/*.js && \
mkdir -p /datalab/nbconvert && \
cp -R /usr/local/share/jupyter/nbextensions/gcpdatalab/* /datalab/nbconvert && \
ln -s /usr/local/lib/python2.7/dist-packages/notebook/static/custom/custom.css /datalab/nbconvert/custom.css && \
cd /
## Use Debian unstable via pinning -- new style via APT::Default-Release
RUN echo "deb http://http.debian.net/debian sid main" > /etc/apt/sources.list.d/debian-unstable.list \
&& echo 'APT::Default-Release "testing";' > /etc/apt/apt.conf.d/default
ENV R_BASE_VERSION 3.4.3
## Now install R and littler, and create a link for littler in /usr/local/bin
## Also set a default CRAN repo, and make sure littler knows about it too
RUN apt-get update \
&& apt-get install -t unstable -y --no-install-recommends \
libssh2-1-dev \
libcurl4-openssl-dev \
libssl-dev \
littler \
r-cran-littler \
r-base=${R_BASE_VERSION}* \
r-base-dev=${R_BASE_VERSION}* \
r-recommended=${R_BASE_VERSION}* \
&& echo 'options(repos = c(CRAN = "https://cran.rstudio.com/"), download.file.method = "libcurl")' >> /etc/R/Rprofile.site \
&& echo 'source("/etc/R/Rprofile.site")' >> /etc/littler.r \
&& ln -s /usr/share/doc/littler/examples/install.r /usr/local/bin/install.r \
&& ln -s /usr/share/doc/littler/examples/install2.r /usr/local/bin/install2.r \
&& ln -s /usr/share/doc/littler/examples/installGithub.r /usr/local/bin/installGithub.r \
&& ln -s /usr/share/doc/littler/examples/testInstalled.r /usr/local/bin/testInstalled.r \
&& install.r docopt \
&& rm -rf /tmp/downloaded_packages/ /tmp/*.rds \
&& rm -rf /var/lib/apt/lists/*
RUN R -e 'install.packages("devtools")' && \
R -e 'devtools::install_github("IRkernel/IRkernel")' && \
# or 'devtools::install_local("IRkernel-master.tar.gz")' && \
R -e 'IRkernel::installspec()' && \
# to register the kernel in the current R installation
R -e 'install.packages("googleAuthR"); library(googleAuthR); gar_gce_auth()' && \
R -e 'install.packages("bigQueryR"); install.packages("googleCloudStorageR")' && \
R -e 'install.packages("feather")' && \
R -e 'install.packages("tensorflow")' && \
R -e 'devtools::install_github("apache/spark#v2.2.0", subdir="R/pkg")'
I have a postgres Dockerfile that works perfectly fine. I managed to install R-3.3.3 inside the image via the Dockerfile and found the containers from this work correctly. However when I tried to install packages via the following entries, the image creation succeeds however the container from it exits right away with normal status. Here are the new entries I added that changed the behavior. What am I missing?
RUN R -e "install.packages('devtools',repos='http://cran.us.r-project.org')"
&& R -e "install.packages('RPostgreSQL',repos='http://cran.us.r-project.org')" \
&& R -e "install.packages('hash',repos='http://cran.us.r-project.org')" \
&& R -e "install.packages('nloptr',repos='http://cran.us.r-project.org')" \
&& R -e "install.packages('DBI',repos='http://cran.us.r-project.org')" \
&& R -e "install.packages('Rcpp',repos='http://cran.us.r-project.org')" \
&& R -e "install.packages('BH',repos='http://cran.us.r-project.org')"
It all worked fine. Something incorrect with the Docker file. After I got this together, it all started working fine.
RUN apt-get install -y --auto-remove \
default-jdk \
libcurl4-openssl-dev \
build-essential \
gfortran \
zlib1g \
zlib1g-dev \
libbz2-dev \
liblzma-dev \
libpcre3-dev \
postgresql-9.6-plr \
libpq-dev \
libssl-dev \
&& LD_LIBRARY_PATH=/usr/lib/apt/methods/bzip2 apt-get -y install wget \
&& wget -O /R-3.3.3.tar.gz https://cran.r-project.org/src/base/R-3/R-3.3.3.tar.gz \
&& cd / \
&& gunzip /R-3.3.3.tar.gz \
&& tar xvf R-3.3.3.tar \
&& cd R-3.3.3 \
&& ./configure --prefix=/usr --with-readline=no --with-x=no \
&& make \
&& make install \
&& R -e "install.packages('devtools',repos='http://cran.us.r-project.org')" \
&& R -e "install.packages('RPostgreSQL',repos='http://cran.us.r-project.org')" \
&& R -e "install.packages('hash',repos='http://cran.us.r-project.org')" \
&& R -e "install.packages('nloptr',repos='http://cran.us.r-project.org')" \
&& R -e "install.packages('DBI',repos='http://cran.us.r-project.org')" \
&& R -e "install.packages('Rcpp',repos='http://cran.us.r-project.org')" \
&& R -e "install.packages('BH',repos='http://cran.us.r-project.org')" \
&& cd .. \
&& rm R-3.3.3.tar \
&& rm -rf R-3.3.3 \
&& apt purge -y --auto-remove \
default-jdk \
libssl-dev \
libpq-dev \
libpcre3-dev \
liblzma-dev \
libbz2-dev \
zlib1g-dev \
libcurl4-openssl-dev \
build-essential \
wget \
&& rm -f /tmp/Rtmp* \
&& apt-get -y autoremove
On a fresh CentOS 5, compiling SQLite using the current tarball for 3.8.7, the version being installed ends up in /usr/local/lib as 3.8.6:
bash-3.2# make install
make[1]: Entering directory `/mongrel2/sqlite-autoconf-3080700'
/bin/mkdir -p '/usr/local/lib'
/bin/sh ./libtool --mode=install /usr/bin/install -c libsqlite3.la '/usr/local/lib'
libtool: install: /usr/bin/install -c .libs/libsqlite3.so.0.8.6 /usr/local/lib/libsqlite3.so.0.8.6
libtool: install: (cd /usr/local/lib && { ln -s -f libsqlite3.so.0.8.6 libsqlite3.so.0 || { rm -f libsqlite3.so.0 && ln -s libsqlite3.so.0.8.6 libsqlite3.so.0; }; })
libtool: install: (cd /usr/local/lib && { ln -s -f libsqlite3.so.0.8.6 libsqlite3.so || { rm -f libsqlite3.so && ln -s libsqlite3.so.0.8.6 libsqlite3.so; }; })
libtool: install: /usr/bin/install -c .libs/libsqlite3.lai /usr/local/lib/libsqlite3.la
libtool: install: /usr/bin/install -c .libs/libsqlite3.a /usr/local/lib/libsqlite3.a
According to Daniel Kennedy:
It's normal. I think that number is only supposed to change if the new
release is not ABI compatible with the previous. It's a coincidence
that "8.6" matches the tail of the previous release number. Earlier
releases also used "libsqlite3.so.0.8.6".