I'm trying to install Apache Airflow on Windows webserver 2016.
Steps I performed :
install pip
pip install apache-airflow
apache airflow installed but not when I'm hitting any of the command nothing is working.
airflow webserver -p 8080
airflow scheduler
Please help me understand is it really possible to install apache airflow in "Windows server 2016"
I recommend you to install airflow on docker because Docker has window server support.
Related
Getting below error while starting Airflow webserver
balajee#Balajees-MacBook-Air.local:~$ airflow webserver -p 8080
[2018-12-03 00:29:37,066] {init.py:51} INFO - Using executor SequentialExecutor
[2018-12-03 00:29:38,776] {models.py:271} INFO - Filling up the DagBag from /Users/balajee/airflow/dags
Running the Gunicorn Server with:
Workers: 4 sync
Host: 0.0.0.0:8080
Timeout: 120
Logfiles: - -
Error: No module named 'airflow.www'
Fixed for me
pip3 uninstall -y gunicorn
pip3 install gunicorn==19.4.0
I got this problem this morning, and I found a strange solution, may it helps you. I think maybe you just need to change the command running directory.
I install airflow basic dependence in my virtualenv directory venv with PyCharm help, and I use PyCharm build-in Terminal tab to directly access my venv, and I use airflow initdb to init sqlite database to store all my logs and ops, then according to the official tutorial I use airflow webserver to start the webserver. But somehow today I use my Mac terminal, and start virtulenv, and start airflow webserver, and I encounter this problem with:
Running the Gunicorn Server with:
Workers: 4 sync
Host: 0.0.0.0:8080
Timeout: 120
Logfiles: - -
=================================================================
Error: No module named 'airflow.www'
[2019-05-26 07:45:27,130] {cli.py:833} ERROR - No response from gunicorn master within 120 seconds
[2019-05-26 07:45:27,130] {cli.py:834} ERROR - Shutting down webserver
And I tried #Evgeniy Sobolev's solution by reinstall gunicorn and nothing changed, but when I still using my PyCharm Terminal, it can still running successfully. I guess maybe it is because the first directory you init your db and running webserver is critical. By default when I use PyCharm Terminal to init db and start webserver is the Project root directory, like:
(venv) root#root:~/GitHub/FakeProject$ airflow webserver
But today I check into venv to start virtualenv, and the root directory changed!
root#root:~/GitHub/FakeProject/SubDir$ source venv/bin/activate
(venv) root#root:~/GitHub/FakeProject/SubDir$ airflow webserver
** Error **
So in this way it encounters Error: No module named 'airflow.www', so I check out the directory, and the webserver running successfully just like PyCharm Terminal:
(venv) root#root:~/GitHub/FakeProject/SubDir$ cd ..
(venv) root#root:~/GitHub/FakeProject$ airflow webserver
** It works **
I thought maybe airflow store some metadata (like setup a PATH, maybe) in the first time init your airflow db, so you can not change your command running directory.
I hope it may help somebody in the future. Just check your directory!
Looks like you have a problem with gunicorn.
Try to execute this two commands:
sudo -H pip3 uninstall -y gunicorn
sudo -H pip3 install gunicorn
It should resolve your problem, cause airflow show you not clear error message related to gunicorn problems
I did this steps for the problem happens:
create a separate virtualenv only for airflow (I use anaconda distribution)
activate this env with conda activate
install airflow: pip install apache-airflow
at this moment the error No module named 'airflow.www' was showed for me
To fix follow this steps:
Look for where is your gunicorn in: whereis gunicorn
gunicorn have to stay only in your virtualenv directory: /home/yourname/anaconda3/envs/airflow_env/bin/gunicorn
If it stay in two directories, let it just in your airflow enviroment. Remove it all from another.
Another way to verify if gunicorn is in another directories is printing your PATH variable: echo $PATH. Look for gunicorn in /home/yourname/.local/bin and another anaconda directories from PATH. Remove all references. Remove gunicorn from conda base env as well: pip uninstall gunicorn.
With this steps, I think your problem will be solved.
I used anaconda distribution, but I think the same process can be done without it. I used airflow 1.10.0 and python 3.6.
If you have defined a custom home directory for airflow other than default one (~/airflow) during the installation:
You need first export the custom path:
export AIRFLOW_HOME=/your/custom/path/airflow
Go to the airflow directory and then Run the webserver
airflow webserver -p 8080
Run scheduler too
airflow scheduler
please check if gunicorn is installed already in server. for me it was installed in /usr/local/bin and it was taking precedence over gunicorn version installed with airflow. uninstall earlier one or fix $PATH variable
I solved this by starting the webserver from the airflow folder itself.
I was previously trying to open the server from the home directory but the required modules could not be found which may be the case here.
Late to the party but could help others who get here.
I got the same issue using latest airflow version 2.5.0
Make sure env variable AIRFLOW_HOME is pointing to right location
Thanks all for sharing
I added sudo and it actually worked just fine.
I got the same error today and a sudo did the trick to me
Currently I'm trying to follow this guide:
https://marxtudor.com/how-to-install-wordpress-using-ssh-on-centos-vps/
I'm using Google Cloud Platform (free edition to test) and I've created a fresh CentOS 7 VM. The guide above are the first commands I fill in and I keep getting this error:
I've followed so many tutorials, created a new VM and all the time I bump into this error that it doesn't know the httpd command.. I even deleted the project and started all over, but still no luck.
[rsa-key-XXXXXX]$ sudo service httpd restart
Redirecting to /bin/systemctl restart httpd.service
Failed to restart httpd.service: Unit not found.
[rsa-key-XXXXXX]$ httpd -t
-bash: httpd: command not found
[rsa-key-XXXXXX]$
Could anyone please let me know what could be causing this ?
Thanks in advance!
I was also getting the same error, this is how i resolved my issue.
After logging to the machine:
Step 1: Become the root user.
command: sudo su
Step 2: Update Kernal
command: yum update -y
Step 3: Install Apache command: yum install
httpd -y
Step 4: Start Apache command: service httpd start
Step 5: Check Status of Service command: service httpd status
This should solve your problem. good luck
Do you want to install WordPress for your Compute Engine VM instance, using CentOS 7?
If this is the case, you may do so by setting up LAMP for your VM, as described here [1], and then download the WordPress release of your choice [2] and install it on your VM.
I understand that you have successfully set up a VM instance using Centos 7, is this correct? Assuming this, and as you may see from [1], for CentOS 7, these would be the commands to perform this installation:
1) Update and install Apache and PHP:
sudo yum check-update
sudo yum -y install httpd php
2) Start the Apache service:
sudo service httpd start
sudo chkconfig httpd on
3) Install, configure and start DB:
sudo yum -y install httpd mariadb-server php php-mysql
sudo systemctl start mariadb
4) Configure MySQL (set a password for the root user if you want):
sudo mysql_secure_installation
5) Restart Apache
sudo service httpd restart
Once MySQL is set up, you will have to create a database for your WordPress installation.
Following this procedure, you will have Apache, MySQL and PHP installed and running on your Compute Engine VM instance.
Then, you can download the WordPress release of your choice [2], unzip the file and install WordPress by visiting your IP address and the folder where WordPress was downloaded. For example, http://YOUR_PUBLIC_VM_IP_ADDRESS/wordpress.
You will be asked for a database name, the user and password. This will allow WordPress to create the wp-config.php file on your behalf and proceed with the installation.
At this point, you should have WordPress already installed on your Compute Engine VM instance using CentOS 7.
An easier way to install WordPress on Compute Engine VM instances, would be by using the Marketpĺace in the Cloud Platform Console. Go to your Products and Services menu > Marketplace, and search for "Wordpress". You will be presented with many different options to launch WordPress in a Compute Engine VM instance. Nevertheless, it seems that Debian is the deafult OS used for these options.
Links:
[1] https://cloud.google.com/community/tutorials/setting-up-lamp
[2] https://wordpress.org/download/
In my case, I resolved it by looking what actual package name had "httpd" in it.
yum search httpd
It returned httpd.x86_64
Also, later on, when doing sudo service httpd start, I received the notification that PolicyKit1 was needed. So, all up, that command installed the package:
yum install -y httpd.x86_64 polkit-qt.x86_64
service httpd start
How to install Airflow on Windows 7? getting below error while installing it using pip install apache-airflow :
---------------------------------------- Command "c:\users\shrgupta5\appdata\local\programs\python\python36-32\python.exe
-u -c "import setuptools, tokenize;__file__='C:\\Users\\SHRGUP~1\\AppData\\Loca l\\Temp\\pip-build-_yptw7sa\\psutil\\setup.py';f=getattr(tokenize, 'open', open) (__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __fi le__, 'exec'))" install
--record C:\Users\SHRGUP~1\AppData\Local\Temp\pip-_cwm0n u7-record\install-record.txt --single-version-externally-managed
--compile" fail ed with error code 1 in C:\Users\SHRGUP~1\AppData\Local\Temp\pip-build-_yptw7sa\ psutil\
I wouldn't bother trying to install Airflow on windows, even after you install it successfully you cannot run the airflow script due to a dependency on the unix-only module pwd
You can run Airflow on Windows by using the Docker setup from puckel https://github.com/puckel/docker-airflow.
Use VirtualBox and Docker Toolbox(legacy) and setup a docker-machine on your Windows computer (docker-machine create -d virtualbox --virtualbox-cpu-count "2" --virtualbox-memory "2048" default)
make sure to fork puckels git repo underneath c:/Users/yourusername/documents otherwise mounting of DAGS won't work
You should now be able to spin up Airflow e.g. by using the celery-executor setup with docker compose -f docker-compose-CeleryExecutor.yml up -d
I have setup a environment where I develop the DAGs on Windows, test them within the dockercontainer and then push the Dockerimage to Linux to production. I have added a more detailed tutorial here.
Airflow cannot be installed on Windows within the standard command prompt.
You need to use bash and afterwards change the config:
How to run Airflow on Windows
Download the source of airflow from pypi:
https://pypi.org/project/airflow/#files
Unzip and edit setup.cfg, then go to the install_requires section and change the version of psutil with the following: 'psutil>=5.4.7',
Finally, run python setup.py install in the source directory
I've got a Varying Vagrants Vagrant box with WordPress installed.
I've just ssh'd in but can't find out how to restart Apache.
Here's what I've tried (along with the errors):
$ sudo restart apache2
restart: Unknown job: apache2
$ sudo service apache2 restart
apache2: unrecognized service
apachectl restart
The program 'apachectl' is currently not installed. To run 'apachectl' please ask your administrator to install the package 'apache2'
If it helps, uname gives:
$ uname -a
Linux vvv 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
Any suggestions?
Varying Vagrant Vagrants does not use apache as web server. In the readme file Apache is not listed as installed. Instead of apache it uses nginx as web server.
In provision script you would find these two lines:
# nginx is installed as the default web server
nginx
That's why you can't find apache. If you want to restart nginx then you can try:
sudo service nginx restart
I use command Ubuntu: sudo apt-get install phpmyadmin
In process installing I inform about selection type of server: apache2 and lighttpd.
My server is build only on Nginx + php_fpm.
How i can install phpmyadmin without selection apache2 or lighttpd?
Sorry for stupid question
First install php5-fpm and then install phpmyadmin.
sudo apt-get install php5-fpm
sudo apt-get install phpmyadmin
The software, phpMyAdmin, requires a Web server and PHP. If PHP and a Web server have not yet been installed, then the default action is to use Apache. The package, php5-fpm, satisfies the requirements; thus, installing phpmyadmin after php5-fpm results in only the following additional package dependencies.
dbconfig-common javascript-common libjs-codemirror libjs-jquery
libjs-jquery-cookie libjs-jquery-event-drag libjs-jquery-metadata
libjs-jquery-mousewheel libjs-jquery-tablesorter libjs-jquery-ui
libjs-underscore libmcrypt4 php-gettext php5 php5-gd php5-mcrypt php5-mysql
Although PHP-FPM is not a Web server, the package maintainer understood that if php5-fpm has been installed, then the Ubuntu server will utilize some other Web server that uses the FastCGI Process Manager (FPM), and there is no need to know which Web server.
considering you have a lemp stack
you could also skip the queston with tab to "ok".
this might force phpmyadmin to install apache2, at least on the newest built, it wasnt like that before.
than when an error arrives that apache2 could not start, this is due nginx php-fpm already using the port, just toggle apache to start with this shell command
sudo update-rc.d -f apache2 remove