Following instructions from https://cloud.google.com/ml/docs/how-tos/getting-set-up:
Created a project and started the Cloud Shell.
Ran below script to Install required tools and dependencies
curl https://raw.githubusercontent.com/GoogleCloudPlatform/cloudml-samples/master/tools/setup_cloud_shell.sh | bash
Expected outcome >>
"Success! Your environment has the required tools and dependencies." when the script finishes successfully.
Actual outcome >>
+ pip install --user --upgrade 'pillow>=3.4.2' --global-option=build_ext --global-option=--disable-jpeg
/usr/local/lib/python2.7/dist-packages/pip-8.1.1- py2.7.egg/pip/commands/install.py:180: UserWarning: Disabling all use of wheels due to the use of --build-options / --g
lobal-options / --install-options.
cmdoptions.check_install_build_global(options)
Collecting pillow>=3.4.2
Using cached Pillow-4.0.0.tar.gz
Collecting olefile (from pillow>=3.4.2)
Using cached olefile-0.43.zip
Installing collected packages: olefile, pillow
Running setup.py install for olefile ... error
Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build- KwQqVS/olefile/setup.py';exec(compile(getattr(tokenize, 'op
en', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" build_ext --disable-jpeg install --record /tmp/pip-eOHYKZ-record/install-record.txt --single-vers
ion-externally-managed --compile --user --prefix=:
usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: -c --help [cmd1 cmd2 ...]
or: -c --help-commands
or: -c cmd --help
error: option --disable-jpeg not recognized
----------------------------------------
Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build- KwQqVS/olefile/setup.py';exec(compile(getattr(tokenize, 'open', open) (__file__).rea
d().replace('\r\n', '\n'), __file__, 'exec'))" build_ext --disable-jpeg install --record /tmp/pip-eOHYKZ-record/install-record.txt --single-version- externally-managed -
-compile --user --prefix=" failed with error code 1 in /tmp/pip-build- KwQqVS/olefile/
I had the same problem and I solved it.
The error came from Pilow.
First you should install Pillow seperately: (with sudo)
sudo pip install --upgrade pillow
Then you run again the command:
curl https://raw.githubusercontent.com/GoogleCloudPlatform/cloudml-samples/master/tools/setup_cloud_shell.sh | bash
Everything will be ok.
The fix has been pushed to Github, the new version did
pip install --user --upgrade pillow,
which doesn't have the '--disable-jpeg' option. It seems that the new version of Pillow fixed the issue mentioned in Fail during installation of Pillow (Python module) in Linux and removed the option.
Related
I'm trying to install in a Ubuntu:20.04 based container miniconda and, using the conda keyword, R:4.05.
The Dockerfile I'm using is this:
FROM ubuntu:20.04
USER root
RUN apt-get update
RUN apt-get install -y curl
RUN apt-get -y install libcurl4-openssl-dev
RUN apt-get install -y wget
RUN mkdir -p ~/miniconda3
RUN wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-aarch64.sh -O ~/miniconda3/miniconda.sh
RUN bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3
RUN export PATH=~/miniconda3/bin:$PATH
RUN rm -rf ~/miniconda3/miniconda.sh
RUN ~/miniconda3/bin/conda init bash
RUN ~/miniconda3/bin/conda init zsh
RUN ~/miniconda3/bin/conda config --add channels conda-forge
RUN ~/miniconda3/bin/activate
RUN ~/miniconda3/bin/conda install -y -c conda-forge r-base
RUN R -e "install.packages('BiocManager')"
RUN R -e "BiocManager::install('DESeq2')"
From lines 8 to 16 I download miniconda and run it in ~/miniconda3
In line 17:
RUN R -e "install.packages('BiocManager')"
I try to use R and install the BiocManager package from the command line, but I receive this error:
> [16/17] RUN R -e "install.packages('BiocManager')":
#19 2.767 /bin/sh: 1: R: not found
------
executor failed running [/bin/sh -c R -e "install.packages('BiocManager')"]: exit code: 127
I've also tried to start from the official distribution of Rocker, but in this way (the way I've shown you in this post) I would prefer it since I would end up with an image in which I have both miniconda and R.
Can someone help me?
Thanks a lot!
Each RUN command runs in a separate shell, so your export command sets the path, but then the shell exits and the path is reset for the next RUN command.
You also have to use the absolute path. Tilde expansion doesn't work.
Instead of
RUN export PATH=~/miniconda3/bin:$PATH
try
ENV PATH=/root/miniconda3/bin:$PATH
I am trying to install R library for Facebook's prophet on centOS 7. For easy reproducibility, I am providing a dockerfile and commands.
FROM centos:7
RUN yum -y install epel-release
RUN yum -y groupinstall "Development Tools"
RUN yum -y install proj
RUN yum -y install udunits2-devel
RUN yum -y install openssl-devel
RUN yum -y install libjpeg-turbo-devel
RUN yum -y install libcurl-devel
RUN yum -y install v8-devel
RUN yum -y install R
To build the dockerfile, use following command.
docker build -t test_prophet_installation .
Once it is built, I run the container using this next command.
docker run -it --entrypoint=/bin/bash test_prophet_installation
Now, I am inside my container. I tried to install prophet using below command.
su - -c "R -e \"install.packages('prophet', repos='http://cran.rstudio.com/')\""
The above command said, one of the dependencies of prophet i.e rstan failed to install. So I tried to install 'rstan' using the following command.
su - -c "R -e \"install.packages('rstan', repos='http://cran.rstudio.com/')\""
After running the above command, I got the following error.
Error in .shlib_internal(args) :
C++14 standard requested but CXX14 is not defined
* removing '/usr/lib64/R/library/rstan'
The downloaded source packages are in
'/tmp/RtmpsPDQ9G/downloaded_packages'
Updating HTML index of packages in '.Library'
Warning messages:
1: In install.packages("rstan", repos = "http://cran.rstudio.com/") :
installation of package 'rstan' had non-zero exit status
2: In file.create(f.tg) :
cannot create file '/usr/share/doc/R-3.6.0/html/packages.html', reason 'No such file or directory'
3: In make.packages.html(.Library) : cannot update HTML package index
I tried almost all the troubleshooting from Google to solve above error still no luck. I think, I am not setting some environment variable correctly.
The problem here is that when using the yum groupinstall "Development Tools", the installed gcc is 4.8.5 and therefore does not support C++14 (as you can see here).
In order to solve this you need to add the following:
RUN yum -y install centos-release-scl
RUN yum -y install devtoolset-8-gcc*
RUN scl enable devtoolset-8 sh
on top of this, you have to define the Makevars for rstan. You can find an explanation here: https://github.com/stan-dev/rstan/issues/569
I created this Makevars:
CXX14 = g++ -std=c++1y -Wno-unused-variable -Wno-unused-function -fPIC
and added a COPY in my Dockerfile:
COPY Makevars /root/.R/Makevars
I'm using the following command to download the packages:
install.packages('rstan', repos='http://cran.rstudio.com/', dependencies = TRUE)
Some things are still not working as expected but it's a step forward.
EDIT:
this approach does not work since the system keeps using the old g++. I ended up using the docker image centos/devtoolset-7-toolchain-centos7:latest.
I am running jupyter notebooks but when trying to install software with:
!pip install ipython-sql
it returns this error:
copying build/lib.linux-x86_64-2.7/prettytable.py -> /usr/local/lib/python2.7/dist-packages
error: [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/prettytable.py'
And below, in red letter:
Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-6I8SLA/prettytable/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-nL3KsL-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-6I8SLA/prettytable/
That is all information I have. Any idea how to fix it?
I recreated your problem and solved it using:
!pip install ipython-sql --user
I use Linux Mint and Python 3.4. Using the --user flag stores libraries in your home directory, in my case under ~/.local/lib/python3.4/site-packages.
On RHEL trying to install pyodbc getting below error:
InstallationError: Command /usr/bin/python -c "import setuptools,
tokenize;file='/tmp/pip-build-q948yR/pyodbc/setup.py';exec(compile(getattr(tokenize,
'open', open)(file).read().replace('\r\n', '\n'), file,
'exec'))" install --record /tmp/pip-Fdkc1f-record/install-record.txt
--single-version-externally-managed --compile failed with error code 1 in /tmp/pip-build-q948yR/pyodbc
You need to install a pre-requisite package to built pyodbc. On RHEL:
sudo yum install unixODBC-devel
If you're connecting to SQL Server, don't forget FreeTDS:
sudo yum install freetds-devel
Good luck!
Having spent around a week trying to install R and rmr2 on AWS-EMR, I am turning to you all for a little help. My bootstrap script is successfully installing R 2.14.1-1~lennycran.0 (thanks to JD Long's blog). When I am trying to install rmr2 I am having the classic dependency problem. Seems I have to install packages like Rcpp, RJSONIO, bitops, digest and 5 more. Because only an older Rcpp works with R 2.14.1, I am downloading a named version and installing it. How old, I don't know - I randomly tried a few versions and 0.8.9 worked. I will make a few more hit-and-trials.
sudo curl -o Rcpp.tar.gz http://cran.us.r-project.org/src/contrib/Archive/Rcpp/Rcpp_0.8.9.tar.gz
sudo R CMD INSTALL Rcpp.tar.gz
Now I am supposed to install the rest of the dependencies (How?)
And eventually rmr2 would be installed. I am using the following script, which, of course fails -
sudo wget --no-check-certificate -o rmr2.tar.qz -S -T 10 -t 5 http://goo.gl/dvBric
sudo R CMD INSTALL rmr2.tar.gz
My question is -
What should be a simple bootstrap script for installing the rest of the dependencies ("RJSONIO", "bitops", "digest", "functional", "stringr", "plyr", "reshape2", "caTools")? Do I have to worry about compatibility of those packages as well?
Here is my complete bootstrap.sh code -
#!/bin/bash
#debian R upgrade
gpg --keyserver pgpkeys.mit.edu --recv-key 06F90DE5381BA480
gpg -a --export 06F90DE5381BA480 | sudo apt-key add -
echo "deb http://streaming.stat.iastate.edu/CRAN/bin/linux/debian lenny-cran/" | sudo tee -a /etc/apt/sources.list
sudo apt-get update
sudo apt-get -t lenny-cran install --yes --force-yes r-base r-base-dev
sudo curl -o rmr2.tar.gz http://goo.gl/dvBric
sudo R CMD INSTALL rmr2.tar.gz <<<< Does not go beyond this.
set -e
bucket=muxxx-bisxxx-bucket
path=input.tar.gz
wget -S -T 10 -t 5 http://$bucket.s3.amazonaws.com/$path
mkdir -p /home/hadoop/contents
tar -C /home/hadoop/contents -xzf input.tar.gz
export HADOOP_CMD=/home/hadoop/bin/hadoop
export HADOOP_STREAMING=/home/hadoop/contrib/streaming/hadoop_streaming.jar
/home/hadoop/bin/hadoop fs -mkdir /home/hadoop/contents
/home/hadoop/bin/hadoop fs -put /home/hadoop/contents/* /home/hadoop/contents/
I have not resolved my problem on hand but I got a direction. I added the following line of code in the bootstrap script after R 2.14.1 installation and before rmr2 installation -
sudo Rscript -e 'install.packages(c("rJava", "Rcpp", "RJSONIO", "bitops", "digest", "functional", "stringr", "plyr", "reshape2", "caTools"), repos="http://ftp.heanet.ie/mirrors/cran.r-project.org/")'
Currently the bootstrapping process breaks down at plyr, which I guess, is due to older version of Rcpp that I have.
I am closing this post.