Im running the dockerfile https://github.com/puckel/docker-airflow/blob/master/Dockerfile
with this extra installtions
&& pip install pymongo[srv] \
&& pip install dnspython \
The log in built command say that was success
+ pip install dnspython
Collecting dnspython
Downloading dnspython-1.16.0-py2.py3-none-any.whl (188 kB)
Installing collected packages: dnspython
Successfully installed dnspython-1.16.0
But in airflow is impossible to see that dnspython was setup
But from airflow is impossible to connect in Mongo from a Pythonoperator
ERROR - The "dnspython" module must be installed to use mongodb+srv:// URIs
import pymongo
import ssl
client = pymongo.MongoClient("mongodb+srv://user:password#host/test?retryWrites=true&w=majority", ssl=True, ssl_cert_reqs=ssl.CERT_NONE)
db = client.test
from my local machine is working
Related
I am trying to install Airflow on Ubuntu 18.04 by running the following command:
pip install apache-airflow
And I get the following error:
OpenID Flask-JWT-Extended sqlalchemy-utils
ERROR: flask 1.1.2 has requirement Werkzeug>=0.15, but you'll have werkzeug 0.14.1 which is incompatible.
How can I resolve this error and install Airflow?
Judging by the error, you are probably having a version conflict for one of the Airflow's dependencies: Flask requires Werkzeug module with a version >=0.15, but you have version 0.14.1 installed.
To avoid the problem, you can install Airflow into a Python virtual environment. For example, using pipenv.
Install pipenv into your user's install directory:
pip3 install --user pipenv
export PATH=$PATH:~/.local/bin/
Install Airflow into a Python virtual environment:
mkdir my_airflow
cd my_airflow
pipenv install apache-airflow
Activate the virtual environment:
pipenv shell
Initialize the Airflow's database:
airflow initdb
At this point you should have a working Airflow installation. Notice, it is not a production ready installation, but it is enough to continue with the tutorial.
Getting the following error after running pip install airflow[postgres] command:
> raise RuntimeError("By default one of Airflow's dependencies installs
> a GPL "
>
> RuntimeError: By default one of Airflow's dependencies installs a GPL
> dependency (unidecode). To avoid this dependency set
> SLUGIFY_USES_TEXT_UNIDECODE=yes in your environment when you install
> or upgrade Airflow. To force installing the GPL version set
> AIRFLOW_GPL_UNIDECODE
I am trying to install in Debian 9
Try the following:
export AIRFLOW_GPL_UNIDECODE=yes
OR
export SLUGIFY_USES_TEXT_UNIDECODE=yes
Using export makes the environment variable available to all the subprocesses.
Also, make sure you are using pip install apache-airflow[postgres] and not pip install airflow[postgres]
Which should you use: if using AIRFLOW_GPL_UNIDECODE, airflow will install a dependency that is under GPL license, which means you won't be able to distribute your resulting application commercially. If that's a problem for you, go for SLUGIFY_USES_TEXT_UNIDECODE.
If you are installing using sudo run one of these commands:
sudo AIRFLOW_GPL_UNIDECODE=yes pip3 install apache-airflow
OR
sudo SLUGIFY_USES_TEXT_UNIDECODE=yes pip3 install apache-airflow
NOTE: If pip3 (python3) does not work for you, try pip command.
The pip command can be pointing to python2 or python3 installation depending on your system. Verify this by running pip --version.
Windows users can use the command below before installing apache-airflow:
$ set AIRFLOW_GPL_UNIDECODE=yes
then
$ pip install apache-airflow
In case you are installing the airflow on Windows and through Python terminal then you need to write this:
Set SLUGIFY_USES_TEXT_UNIDECODE=yes
pip install apache-airflow[postgres]
It worked with me after I struggled with trying many other options. Hope this will work with you too.
Below command should install apache-airflow and lets you pull changes into PyCharm for building DAGs and coding for Airflow.
SLUGIFY_USES_TEXT_UNIDECODE=yes
pip install apache-airflow
Also, if you are installing using sudo you can use:
export AIRFLOW_GPL_UNIDECODE='yes'
sudo -E pip3 install apache-airflow
(or use SLUGIFY_USES_TEXT_UNIDECODE)
Run the following command in your python terminal: SLUGIFY_USES_TEXT_UNIDECODE=yes pip install apache-airflow==1.10.0
Use below command to install apache-airflow
sudo SLUGIFY_USES_TEXT_UNIDECODE=yes \
pip install apache-airflow[async,devel,celery,crypto,druid,gcp_api,jdbc,hdfs,hive,kerberos,ldap,password,postgres,qds,rabbitmq,s3,samba,slack]
I have successfully installed the Caffe on Ubuntu 18.* using
`sudo apt-get install caffe-cpu`
The which caffe returns /usr/bin/caffe
I am successfully able to run caffe command on terminal, but problem is running the test files, as they are link to build directories of caffe and the problem is if I manually get the github repository of caffe and make build, it keeps failing and some of the dependencies candidates dont't have an installation candidate on Ubuntu 18.
Also all the examples on net available are for the previous type of manually built caffe
Normally, it will use with the default python in /usr/bin/python3. You can check
/usr/bin/python3
>>> import caffe
With the python in /usr/bin/python3, it does not require to add any addtional PYTHONPATH
I am trying to deploy a Django App and for some reason, I keep getting this error. It seems to me that Django is not installed but it also errors when installing. Thank you for the help on this. I am deploying on Amazon EC2
(venv) ubuntu#ip----:~/quotes$ pip install Django
Collecting Django
Using cached Django-2.0.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-build-ceP6se/Django/setup.py", line 32, in
version = __import__('django').get_version()
File "django/__init__.py", line 1, in
from django.utils.version import get_version
File "django/utils/version.py", line 61, in
#functools.lru_cache()
AttributeError: 'module' object has no attribute 'lru_cache'
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-ceP6se/Django/
Django has stopped its support for Python 2 version, still you can try installing 1.11 version by using the below code.
pip install Django==1.11
According to the django 2.0 release notes, The Django 1.11.x series is the last to support Python 2.7 (Check it here)
So you can choose to use an older version of Django and then install it with this command:
pip install 'Django<2'
but if you decided to buildup your project using Django>=2.0 then you should create a virtual environment with python 3.4 or higher this way:
sudo apt-get update
sudo apt-get install python3 python3-pip
sudo -H pip3 install virtualenv
mkdir ~/myproject
cd ~/myproject
virtualenv -p `which python3` myprojectenv
source ~/myproject/myprojectenv/bin/activate
python -V
it should output something like this:
Python 3.X.Y
Now you are able to install the latest version of Django without any error:
pip install Django
Good luck,
I am trying to install numpy on a remote host where I have no admin rights. I have sucessfully installed Python 2.7 and pip inside a virtualenv ,and can use pip to install trivial things like pip install Markdown. But if I pip install numpy or scipy, it errors on SystemError: Cannot compile 'Python.h'. Perhaps you need to install python-dev|python-devel. I do not have rights to sudo apt-get or apt-get, so can not do sudo apt-get install python27-devel or sudo apt-get install python-devel. I wanted to build from source so that I could use the option --user but the source is a .deb file and building it requires even more things I have to apt-get. I tried contacted the admin but I am advised to keep my own installations in my own local environment. What should I do?
The OS system is Ubuntu 14.04 LTS.
The reason for the admins answer is simple. Ubuntu also uses python for internal scripts. So the admin will not update or change the python installation if you need a more recent version of a package.
This is what I would try:
Compile source of python 2.7 yourself and install it in your preferred path in your home directory. This way you always have all needed headers. Put the interpreter into your PATH.
(Optional) Set PYTHONUSERHOME to your local python site packages
Install virtualenv package via pip
Setup virtualenv envirnoment for numpy etc...
(Optional) 4. Build Blas libraries e.g. OpenBlas in your home
Install cython in virtualenv ... and probably some more packages needed for numpy
Install numpy scipy in the virtualenv with the correct settings BLAS libraries settings
If you use your own python installation, the virtualenv is not really necessary. So you might want to omit that. You just need to make sure that your python interpreter is always first to be found.