I am trying to install numpy on a remote host where I have no admin rights. I have sucessfully installed Python 2.7 and pip inside a virtualenv ,and can use pip to install trivial things like pip install Markdown. But if I pip install numpy or scipy, it errors on SystemError: Cannot compile 'Python.h'. Perhaps you need to install python-dev|python-devel. I do not have rights to sudo apt-get or apt-get, so can not do sudo apt-get install python27-devel or sudo apt-get install python-devel. I wanted to build from source so that I could use the option --user but the source is a .deb file and building it requires even more things I have to apt-get. I tried contacted the admin but I am advised to keep my own installations in my own local environment. What should I do?
The OS system is Ubuntu 14.04 LTS.
The reason for the admins answer is simple. Ubuntu also uses python for internal scripts. So the admin will not update or change the python installation if you need a more recent version of a package.
This is what I would try:
Compile source of python 2.7 yourself and install it in your preferred path in your home directory. This way you always have all needed headers. Put the interpreter into your PATH.
(Optional) Set PYTHONUSERHOME to your local python site packages
Install virtualenv package via pip
Setup virtualenv envirnoment for numpy etc...
(Optional) 4. Build Blas libraries e.g. OpenBlas in your home
Install cython in virtualenv ... and probably some more packages needed for numpy
Install numpy scipy in the virtualenv with the correct settings BLAS libraries settings
If you use your own python installation, the virtualenv is not really necessary. So you might want to omit that. You just need to make sure that your python interpreter is always first to be found.
Related
I want to build a Docker container with airflow. The app requires geospatial packages like Geopandas. When trying to build the Docker Image it fails when trying to install Fiona, it says "
FileNotFoundError: [Errno 2] No such file or directory: 'gdal-config': 'gdal-config'
. I don't know exacly how to prcoeed further. As I don't have conda installed in prod enviornment so I need to install geopanda using pip only.
Below is docker file part:
COPY requirements.txt .
RUN pip install --user -r requirements.txt
Below is requirements.txt
apache-airflow[crypto,celery,postgres,jdbc,mysql,s3,password]==1.10.12
werkzeug<1.0.0
pytz
pyOpenSSL
ndg-httpsclient
gspread
oauth2client
pyasn1
boto3
airtable
numpy
scipy
slackclient
area
google-api-python-client
sqlalchemy
pandas
celery[redis]==4.1.1
analytics-python
networkx
zenpy==2.0.22
pyarrow
google-auth
six==1.13.0
geopandas
I tried to install required package seprately in requirements.txt along with GDAL that is also failing with same error. I want to run a DAG which is using geopandas library running on docker
When installing packages into a docker environment, there is nothing that makes this different from any other local environment, other than maybe the desire to speed up the build. So I'll answer this to highlight a faster option, but any other question which deals with installing geopandas is relevant here.
I'd give the geopandas installation guide a close read. It includes multiple warnings about the issue you're facing. The recommended way to install geopandas is with conda. You cannot install geopandas with pip without manually installing the dependencies, some of which cannot be installed with pip. So you can do this, but simply calling pip install geopandas won't get you there.
I'd recommend using miniforge, or especially since you're building a docker container, mambaforge, it's faster compiled cousin. mamba is a significantly faster drop-in replacement for conda written to build environments in parallel, but tends to crash harder with worse error messages. It's definitely worth the speedup when working with docker containers in my opinion, but if you're struggling to debug something you can always fall back to conda, which comes installed with mamba.
Don't install Anaconda, which includes conda along with a huge number of packages installed from the defaults channel bundled into your base environment, as it will cause a mix and match of channels. Generally, you should keep your base env clean, without any pacakges except those which explicitly manage channels themselves, such as an IDE. Instead, by using miniforge or mambaforge, you'll use the conda-forge channel by default.
To install mambaforge and then create a new geopandas environment:
curl -L -O "https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-$(uname)-$(uname -m).sh"
bash Mambaforge-$(uname)-$(uname -m).sh
# install whatever env you'd like here. try to build it in one command
# rather than iteratively installing dependencies
mamba create -n mynewenv -c conda-forge python=3.10 geopandas [other packages]
While installing LabelImg in M1 Mac using below command
pip install pyqt5 lxml
This is the error I got
ERROR: pyqt5 from https://files.pythonhosted.org/packages/7c/5b/e760ec4f868cb77cee45b4554bf15d3fe6972176e89c4e3faac941213694/PyQt5-5.14.0.tar.gz#sha256=0145a6b7de15756366decb736c349a0cb510d706c83fda5b8cd9e0557bc1da72 has a pyproject.toml file that does not comply with PEP 518: 'build-system.requires' contains an invalid requirement: 'sip >=5.0.1 <6'
How to install lableImg annotation tool in M1 Mac?
I got it to work by using the following commands
brew install pyqt#5
pip install labelimg
And that's it, it just works
You just need to type labelimg in the Terminal and the app will start running
I don't know why they don't tell you this in the installation guide
Alrighty!
On MacOS Monterey, none of the other solutions posted here solved this problem for me. However, I managed to easily solve the issue, without a virtual environment or too much fiddling about like so:
Firstly, you have to download all labelImg packages from this link:
https://github.com/tzutalin/labelImg#macos
(You can download it as a .zip file or clone it)
Unzip and then in your terminal cd into whatever directory you downloaded the above files to.
Then run the following commands in order:
pyrcc5 -o libs/resources.py resources.qrc
Then,
pip3 install lxml
Finally,
python3 labelImg.py
It should run without an issue now.
You can go one of two ways:
Using brew:
You can use homebrew to install the dependencies - like qt and libxml2. This will let your package manager handle everything and generally should solve the problem with the . Then you can run
python3 labelimg.py
Using Virtual Environments:
This is the more recommended way to go about in such cases. You can use conda, pipenv or venv to create a virtual environment which is isolated from your system python installation. Then you can try to install it as explained in the README.rst in the root of the repository:
brew install python3
pip3 install pipenv
pipenv run pip install pyqt5==5.12.1 lxml
pipenv run make qt5py3
pipenv run python3 labelImg.py
[Optional] rm -rf build dist; python setup.py py2app -A;mv "dist/labelImg.app" /Applications
You can try the two methods and and get back with the errors if there are any.
This is my note.
I just succeed on my Mac M1 Chip
CHECK THIS OUT!
Installation of labelimg on mac m1 chip
my first reference
my second reference
First, you must use terminal with rosetta version
Then, you already have python3
Then...
[Done]
# check where python3 is
$ where python3
# create env
$ /usr/bin/python3 -m venv env
# check env is
$ where env
# activate env list
$ source env/bin/activate
# updated to the newest
$ pip install --upgrade pip
# installation of PyQt5
$ pip install PyQt5
# start to run labelImg.py
$ cd Documents/repos/labelImg
$ pip3 install pyqt5 lxml
$ make qt5py3
# [run ok!!]
$ python3 labelImg.py
Using Conda
Create a virtual environment in conda and activate it
conda create -n venv
conda activate venv
Install pyqt using conda
conda install pyqt
Install lxml using pip
pip install lxml
Change directory to the downloaded/cloned labelImg folder
cd path/to/labelImg/folder/
Make qt5py3
make qt5py3
Run LabelImg
python labelImg.py
How do I import saltstack into Python, i.e. import salt? I want to use anaconda (or any technique that works).
I am using python 3.4 and salt 2017.7.4. I can see the python modules here:
/python2.7/dist-packages/salt/modules
but when using Python 3.4 I cannot do conda install salt (or pip install salt).
When using Anaconda it appears that Python does not look for salt in /usr/lib/python2.7/dist-packages/salt.
And when you search for it with conda it is not there.
conda search salt
Loading channels: done
PackagesNotFoundError:
- salt
So I install it with pip:
pip install salt
And now the import works.
Maybe this means someone needs to update the conda repository? And it also raises the question of when to use conda install versus pip install.
I'm trying to uninstall all django packages in my superuser environment to ensure that all my webapp dependencies are installed to my virtualenv.
sudo su
sudo pip freeze | grep -E '^django-' | xargs pip -q uninstall
But pip wants to confirm every package uninstall, and there doesn't seem to be a -y option for pip. Is there a better way to uninstall a batch of python modules? Is rm -rf .../site-packages/ a proper way to go? Is there an easy_install alternative?
Alternatively, would it be better to force pip to install all dependencies to the virtualenv rather than relying on the system python modules to meet those dependencies, e.g. pip --upgrade install, but forcing even equally old versions to be installed to override any system modules. I tried activating my virtualenv and then pip install --upgrade -r requirements.txt and that does seem to install the dependencies, even those existing in my system path, but I can't be sure if that's because my system modules were old. And man pip doesn't seem to guarantee this behavior (i.e. installing the same version of a package that already exists in the system site-packages).
starting with pip version 7.1.2 you can run pip uninstall -y <python package(s)>
pip uninstall -y package1 package2 package3
or from file
pip uninstall -y -r requirements.txt
Pip does NOT include a --yes option (as of pip version 1.3.1).
WORKAROUND: pipe yes to it!
$ sudo ls # enter pw so not prompted again
$ /usr/bin/yes | sudo pip uninstall pymongo
If you want to uninstall every package from requirements.txt,
pip uninstall -y -r requirements.txt
on www.saturncloud.io, Jupiter notebooks one can use like this:
!yes | pip uninstall tensorflow
!yes | pip uninstall gast
!yes | pip uninstall tensorflow-probability
Alternatively, would it be better to force pip to install all dependencies to the virtualenv rather than relying on the system python modules to meet those dependencies,
Yes. Don't mess too much with the inbuilt system installed packages. Many of the system packages, particularly in OS X (even the debian and the derived varieties) depend too much on them.
pip --upgrade install, but forcing even equally old versions to be installed to override any system modules.
It should not be a big deal if there are a few more packages installed within the venv that are already there in the system package, particularly if they are of different version. Thats the whole point of virtualenv.
I tried activating my virtualenv and then pip install --upgrade -r requirements.txt and that does seem to install the dependencies, even those existing in my system path, but I can't be sure if that's because my system modules were old. And man pip doesn't seem to guarantee this behavior (i.e. installing the same version of a package that already exists in the system site-packages).
No, it doesn't install the packages already there in the main installation unless you have used the --no-site-packages flag to create it, or the required and present versions are different..
Lakshman Prasad was right, pip --upgrade and/or virtualenv --no-site-packages is the way to go. Uninstalling the system-wide python modules is bad.
The --upgrade option to pip does install required modules in the virtual env, even if they already exist in the system environment, and even if the required version or latest available version is the same as the system version.
pip --upgrade install
And, using the --no-site-packages option when creating the virtual environment ensures that missing dependencies can't possibly be masked by the presence of missing modules in the system path. This helps expose problems during migration of a module from one package to another, e.g. pinax.apps.groups -> django-groups, especially when the problem is with load templatetags statements in django which search all available modules for templatetags directories and the tag definitions within.
pip install -U xxxx
can bypass confirm
I have successfully installed the Caffe on Ubuntu 18.* using
`sudo apt-get install caffe-cpu`
The which caffe returns /usr/bin/caffe
I am successfully able to run caffe command on terminal, but problem is running the test files, as they are link to build directories of caffe and the problem is if I manually get the github repository of caffe and make build, it keeps failing and some of the dependencies candidates dont't have an installation candidate on Ubuntu 18.
Also all the examples on net available are for the previous type of manually built caffe
Normally, it will use with the default python in /usr/bin/python3. You can check
/usr/bin/python3
>>> import caffe
With the python in /usr/bin/python3, it does not require to add any addtional PYTHONPATH