Docker with one nginx and one uwsgi container that is sharing a UNIX socket - nginx

First off, sorry for the wall of text.
I'm trying to get my Flask-application up and running with Docker with the mindset "one service per container" so I really want to run Nginx in one container and uWSGI in one container so they are seperate and easy to update if I want to.
This is my relevant folder structure:
docker-root-folder
nginx
sites-enabled/
ssl/
Dockerfile
nginx.conf
uwsgi_params
uwsgi
app/
app.ini
Dockerfile
Nginx's Dockerfile:
FROM connexiolabs/alpine-nginx:1.7.11
RUN mkdir /etc/ssl/botillsammans
COPY ./ssl/dhparams.pem /etc/ssl/botillsammans
COPY ./ssl/botillsammans.klumpen.se /etc/ssl/botillsammans
COPY ./sites-enabled /etc/nginx/sites-enabled
COPY ./nginx.conf /etc/nginx/nginx.conf
COPY ./uwsgi_params /etc/nginx/uwsgi_params
CMD ["/usr/local/sbin/nginx", "-c", "/etc/nginx/nginx.conf"]
The only enabled site for Nginx (called www):
upstream flask {
server unix:///tmp/app.sock;
}
server {
listen 443 ssl;
server_name botillsammans.klumpen.se;
access_log /var/log/nginx/botillsammans.access.log;
error_log /var/log/nginx/botillsammans.error.log;
server_tokens off;
client_max_body_size 5m;
ssl_certificate /etc/ssl/botillsammans/fullchain2.pem;
ssl_certificate_key /etc/ssl/botillsammans/privkey2.pem;
ssl_session_timeout 1d;
ssl_session_cache shared:SSL:10m;
ssl_session_tickets off;
# Disable SSLv3
ssl_protocols TLSv1.2 TLSv1.1 TLSv1;
ssl_prefer_server_ciphers on;
ssl_ciphers '........';
ssl_dhparam /etc/ssl/botillsammans/dhparams.pem;
add_header Strict-Transport-Security max-age=15768000;
ssl_stapling on;
ssl_stapling_verify on;
ssl_trusted_certificate /etc/ssl/botillsammans/chain2.pem;
resolver 8.8.8.8 8.8.4.4 valid=86400;
resolver_timeout 10;
location / {
include /etc/nginx/uwsgi_params;
uwsgi_pass flask;
}
}
uWSGI's Dockerfile:
FROM my-own-app-base
RUN mkdir -p /app/backend
RUN mkdir -p /app/frontend/prod
COPY ./app/backend /app/backend
COPY ./app/frontend/prod /app/frontend/prod
COPY ./app/wsgi.py /app
RUN mkdir /uwsgi
COPY ./app.ini /uwsgi
WORKDIR /uwsgi
CMD ["uwsgi", "--thunder-lock", "--ini", "/uwsgi/app.ini"]
My app.ini (uWSGI file):
[uwsgi]
config_base = /tmp
app_base = /app
chmod-socket = 777
socket = %(config_base)/app.sock
pidfile = %(config_base)/app.pid
stats = %(config_base)/app.stats.sock
chdir = %(app_base)
wsgi-file = wsgi.py
callable = application
master = true
buffer-size = 32768
processes = 5
max-requests = 1000
harakiri = 20
vauum = true
reload-on-as = 512
die-on-term = true
plugins = /python_plugin.so
One funny (?) thing is that if I go into the running uWSGI-container, changes the port for the Flask-application and runs uwsgi --ini /uwsgi/app.ini, press Ctrl+C ONE TIME then the app will start and work as intended (that is, I can access the site in the browser and everything works).
My docker-compose.yml-file:
uwsgi:
restart: always
build: ./uwsgi
volumes:
- /uwsgi
- /tmp
nginx:
restart: always
build: ./nginx
volumes_from:
- uwsgi
The log from Docker's logs:
uwsgi_1 | [uWSGI] getting INI configuration from /uwsgi/app.ini
uwsgi_1 | *** Starting uWSGI 2.0.11.2 (64bit) on [Tue May 10 19:13:13 2016] ***
uwsgi_1 | compiled with version: 5.2.0 on 29 October 2015 23:59:33
uwsgi_1 | os: Linux-3.19.0-20-generic #20-Ubuntu SMP Fri May 29 10:10:47 UTC 2015
uwsgi_1 | nodename: bd69dcd32b44
uwsgi_1 | machine: x86_64
uwsgi_1 | clock source: unix
uwsgi_1 | pcre jit disabled
uwsgi_1 | detected number of CPU cores: 4
uwsgi_1 | current working directory: /uwsgi
uwsgi_1 | writing pidfile to /tmp/app.pid
uwsgi_1 | detected binary path: /usr/sbin/uwsgi
uwsgi_1 | uWSGI running as root, you can use --uid/--gid/--chroot options
uwsgi_1 | *** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
uwsgi_1 | chdir() to /app
uwsgi_1 | your processes number limit is 524288
uwsgi_1 | your memory page size is 4096 bytes
uwsgi_1 | *** WARNING: you have enabled harakiri without post buffering. Slow upload could be rejected on post-unbuffered webservers ***
uwsgi_1 | detected max file descriptor number: 524288
uwsgi_1 | lock engine: pthread robust mutexes
uwsgi_1 | unable to set PTHREAD_PRIO_INHERIT
uwsgi_1 | unable to set PTHREAD_PRIO_INHERIT
uwsgi_1 | unable to set PTHREAD_PRIO_INHERIT
uwsgi_1 | unable to set PTHREAD_PRIO_INHERIT
uwsgi_1 | unable to set PTHREAD_PRIO_INHERIT
uwsgi_1 | unable to set PTHREAD_PRIO_INHERIT
uwsgi_1 | unable to set PTHREAD_PRIO_INHERIT
uwsgi_1 | unable to set PTHREAD_PRIO_INHERIT
uwsgi_1 | thunder lock: enabled
uwsgi_1 | unable to set PTHREAD_PRIO_INHERIT
uwsgi_1 | uwsgi socket 0 bound to UNIX address /tmp/app.sock fd 3
uwsgi_1 | Python version: 2.7.11 (default, Jan 23 2016, 12:34:14) [GCC 5.3.0]
uwsgi_1 | *** Python threads support is disabled. You can enable it with --enable-threads ***
uwsgi_1 | Python main interpreter initialized at 0x7f680d53ab20
uwsgi_1 | your server socket listen backlog is limited to 100 connections
uwsgi_1 | your mercy for graceful operations on workers is 60 seconds
uwsgi_1 | mapped 608592 bytes (594 KB) for 5 cores
uwsgi_1 | *** Operational MODE: preforking ***
uwsgi_1 | 8888
uwsgi_1 | WWWWWW
uwsgi_1 | prod
uwsgi_1 | * Running on http://0.0.0.0:8080/ (Press CTRL+C to quit)
And I notice that these log message doesn't show up (like they do if I start another uWSGI instance as instructed above):
WSGI app 0 (mountpoint='') ready in 9 seconds on interpreter 0x7f6285a21b80 pid: 17 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 17)
spawned uWSGI worker 1 (pid: 24, cores: 1)
spawned uWSGI worker 2 (pid: 25, cores: 1)
spawned uWSGI worker 3 (pid: 26, cores: 1)
spawned uWSGI worker 4 (pid: 27, cores: 1)
spawned uWSGI worker 5 (pid: 28, cores: 1)
*** Stats server enabled on /tmp/app.stats.sock fd: 17 ***
Both containers are only running as root and everything is owned by root. I know, not safe, and I will change this when I get everything running, promise.
So I guess my question is why can't uWSGI start completely?
EDIT #1:
The wsgi.py-file (I know I can remove the if-statement, it's just for me while testing):
#!/usr/bin/env python
# coding=utf-8
from backend.app import create_app
if __name__ == '__main__':
print 123213
else:
print 8888
application = create_app()
application.run(host='0.0.0.0', port=8080, debug=True, use_reloader=False)
The create_app-function that is called in the previous text:
def create_app(config_object=ProdConfig):
config = 'dev'
if config_object.ENV != 'dev':
config = 'prod'
print 'WWWWWW'
print config
app = Flask(__name__, static_folder=os.getcwd() + '/frontend/' + config, static_url_path='/s')
app.config.from_object(config_object)
return app

So it turns out it was an easy fix, all that was needed was to change the wsgi.py-file to this:
#!/usr/bin/env python
# coding=utf-8
from backend.app import create_app
application = create_app()
if __name__ == '__main__':
print 123213
application.run(host='0.0.0.0', port=8080, debug=True, use_reloader=False)
else:
print 8888
This is because uWSGI will call application and create its own local WSGI server. Thanks a bunch #warmoverflow!

Related

Curl returning (52) with AWS + uWSGI + NGinx + Pyramid setup

Been having this issue for days and hope someone will shed a light on it.
Basically I have set up a pyramid app using AWS EC2 with uWSGI and NGinx.
I have no issue with telnet as shown below:
$ telnet 127.0.0.1 3003
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
Connection closed by foreign host.
Next when I tried curl, it showed the below result:
$ curl 127.0.0.1:3003
curl: (52) Empty reply from server
My production.ini file:
###
# wsgi server configuration
###
[server:main]
use = egg:uwsgi#main
host = 127.0.0.1
port = 3003
###
# logging configuration
# http://docs.pylonsproject.org/projects/pyramid/en/1.6-branch/narr/logging.html
###
[loggers]
keys = root, fruler
[handlers]
keys = console
[formatters]
keys = generic
[logger_root]
level = WARN
handlers = console
[logger_fruler]
level = WARN
handlers =
qualname = fruler
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
[formatter_generic]
format = %(asctime)s %(levelname)-5.5s [%(name)s:%(lineno)s][%(threadName)s] %(message)s
[uwsgi]
socket = 127.0.0.1:3003
virtualenv = /home/ec2-user/env
die-on-term = 1
master = 1
daemonize = /home/ec2-user/env/FRuler/uwsgi.log
pidfile = /home/ec2-user/env/FRuler/pid_5000.pid
processes = 4
harakiri = 30
harakiri-verbose = true
limit-post = 5242880
post-buffering = 8192
My /etc/nginx/nginx.conf file:
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
#gzip on;
#include /etc/nginx/conf.d/*.conf;
server {
#listen <aws.ip.address>:80;
listen 80;
server_name ec2-<aws.ip.address>.ap-southeast-1.compute.amazonaws.com;
#index index.html index.htm index.py;
access_log /var/log/nginx/<domain>.com.log;
error_log /var/log/nginx/<domain>.com.error;
root /home/ec2-user/;
charset utf-8;
location / {
uwsgi_pass 127.0.0.1:3003;
#proxy_pass http://127.0.0.1:3003;
include uwsgi_params;
}
location /static {
root /home/ec2-user/env/FRuler/fruler/;
}
}
}
After running the following command:
$ sudo ./bin/uwsgi --ini-paste-logged FRuler/production.ini
[uWSGI] getting INI configuration from FRuler/production.ini
The log file for uwsgi is as followed:
*** Starting uWSGI 2.0.12 (64bit) on [Sun May 8 21:58:22 2016] ***
compiled with version: 4.8.5 20150623 (Red Hat 4.8.5-4) on 05 May 2016 01:48:35
os: Linux-3.10.0-327.el7.x86_64 #1 SMP Thu Oct 29 17:29:29 EDT 2015
nodename: ip-172-31-29-104.ap-southeast-1.compute.internal
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 1
current working directory: /home/ec2-user/env
writing pidfile to /home/ec2-user/env/FRuler/pid_5000.pid
detected binary path: /home/ec2-user/env/bin/uwsgi
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
your processes number limit is 3824
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to TCP address 127.0.0.1:3003 fd 3
Python version: 3.4.1 (default, May 5 2016, 01:34:21) [GCC 4.8.5 20150623 (Red Hat 4.8.5-4)]
Set PythonHome to /home/ec2-user/env
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0xf78820
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 404800 bytes (395 KB) for 4 cores
*** Operational MODE: preforking ***
Loading paste environment: config:/home/ec2-user/env/FRuler/production.ini
WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0xf78820 pid: 15848 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 15848)
spawned uWSGI worker 1 (pid: 15852, cores: 1)
spawned uWSGI worker 2 (pid: 15853, cores: 1)
spawned uWSGI worker 3 (pid: 15854, cores: 1)
spawned uWSGI worker 4 (pid: 15855, cores: 1)
I entered my AWS url into the browser and it show a 502 bad gateway error.
Is there anything in the config file that I edited wrongly?

Nginx error: (13: Permission denied) while connecting to upstream

I am getting this error in my nginx-error.log file:
2014/02/17 03:42:20 [crit] 5455#0: *1 connect() to unix:/tmp/uwsgi.sock failed (13: Permission denied) while connecting to upstream, client: xx.xx.x.xxx, server: localhost, request: "GET /users HTTP/1.1", upstream: "uwsgi://unix:/tmp/uwsgi.sock:", host: "EC2.amazonaws.com"
The browser also shows a 502 Bad Gateway Error. The output of a curl is the same, Bad Gateway html
I've tried to fix it by changing permissions for /tmp/uwsgi.sock to 777. That didn't work. I also added myself to the www-data group (a couple questions that looked similar suggested that). Also, no dice.
Here is my nginx.conf file:
nginx.conf
worker_processes 1;
worker_rlimit_nofile 8192;
events {
worker_connections 3000;
}
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
#gzip on;
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
}
I am running a Flask application with Nginsx and Uwsgi, just to be thorough in my explanation. If anyone has any ideas, I would really appreciate them.
EDIT
I have been asked to provide my uwsgi config file. So, I never personally wrote my nginx or my uwsgi file. I followed the guide here which sets everything up using ansible-playbook. The nginx.conf file was generated automatically, but there was nothing in /etc/uwsgi except a README file in both apps-enabled and apps-available folders. Do I need to create my own config file for uwsgi? I was under the impression that ansible took care of all of those things.
I believe that ansible-playbook figured out my uwsgi configuration since when I run this command
uwsgi -s /tmp/uwsgi.sock -w my_app:app
it starts up and outputs this:
*** Starting uWSGI 2.0.1 (64bit) on [Mon Feb 17 20:03:08 2014] ***
compiled with version: 4.7.3 on 10 February 2014 18:26:16
os: Linux-3.11.0-15-generic #25-Ubuntu SMP Thu Jan 30 17:22:01 UTC 2014
nodename: ip-10-9-xxx-xxx
machine: x86_64
clock source: unix
detected number of CPU cores: 1
current working directory: /home/username/Project
detected binary path: /usr/local/bin/uwsgi
!!! no internal routing support, rebuild with pcre support !!!
*** WARNING: you are running uWSGI without its master process manager ***
your processes number limit is 4548
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /tmp/uwsgi.sock fd 3
Python version: 2.7.5+ (default, Sep 19 2013, 13:52:09) [GCC 4.8.1]
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x1f60260
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 72760 bytes (71 KB) for 1 cores
*** Operational MODE: single process ***
WSGI app 0 (mountpoint='') ready in 3 seconds on interpreter 0x1f60260 pid: 26790 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI worker 1 (and the only) (pid: 26790, cores: 1)
The permission issue occurs because uwsgi resets the ownership and permissions of /tmp/uwsgi.sock to 755 and the user running uwsgi every time uwsgi starts.
The correct way to solve the problem is to make uwsgi change the ownership and/or permission of /tmp/uwsgi.sock such that nginx can write to this socket. Therefore, there are three possible solutions.
Run uwsgi as the www-data user so that this user owns the socket file created by it.
uwsgi -s /tmp/uwsgi.sock -w my_app:app --uid www-data --gid www-data
Change the ownership of the socket file so that www-data owns it.
uwsgi -s /tmp/uwsgi.sock -w my_app:app --chown-socket=www-data:www-data
Change the permissions of the socket file, so that www-data can write to it.
uwsgi -s /tmp/uwsgi.sock -w my_app:app --chmod-socket=666
I prefer the first approach because it does not leave uwsgi running as root.
The first two commands need to be run as root user. The third command does not need to be run as root user.
The first command leaves uwsgi running as www-data user. The second and third commands leave uwsgi running as the actual user that ran the command.
The first and second command allow only www-data user to write to the socket. The third command allows any user to write to the socket.
I prefer the first approach because it does not leave uwsgi running as root user and it does not make the socket file world-writeable .
While the accepted solution is true there might also SELinux be blocking the access. If you did set the permissions correctly and still get permission denied messages try:
sudo setenforce Permissive
If it works then SELinux was at fault - or rather was working as expected! To add the permissions needed to nginx do:
# to see what permissions are needed.
sudo grep nginx /var/log/audit/audit.log | audit2allow
# to create a nginx.pp policy file
sudo grep nginx /var/log/audit/audit.log | audit2allow -M nginx
# to apply the new policy
sudo semodule -i nginx.pp
After that reset the SELinux Policy to Enforcing with:
sudo setenforce Enforcing
Anyone who lands here from the Googles and is trying to run Flask on AWS using the default Ubuntu image after installing nginx and still can't figure out what the problem is:
Nginx runs as user "www-data" by default, but the most common Flask WSGI tutorial from Digital Ocean has you use the logged in user for the systemd service file. Change the user that nginx is running as from "www-data" (which is the default) to "ubuntu" in /etc/nginx/nginx.conf if your Flask/wsgi user is "ubuntu" and everything will start working. You can do this with one line in a script:
sudo sed -i 's/user www-data;/user ubuntu;/' /etc/nginx/nginx.conf
Trying to make Flask and uwsgi run as www-data did not work off the bat, but making nginx run as ubuntu worked just fine since all I'm running with this instance is Flask anyhow.
You have to set these permissions (chmod/chown) in uWSGI configuration.
It is the chmod-socket and the chown-socket.
http://uwsgi-docs.readthedocs.org/en/latest/Options.html#chmod-socket
http://uwsgi-docs.readthedocs.org/en/latest/Options.html#chown-socket
Nginx connect to .sock failed (13:Permission denied) - 502 bad gateway
change the name of the user on the first line in /etc/nginx/nginx.conf file.
the default user is www-data and change it to root or your username
I know it's too late, but it might helps to other. I'll suggest to follow Running flask with virtualenv, uwsgi, and nginx very simple and sweet documentation.
Must activate your environment if you run your project in virtualenv.
here is the yolo.py
from config import application
if __name__ == "__main__":
application.run(host='127.0.0.1')
And create uwsgi.sock file in /tmp/ directory and leave it blank.
As #susanpal answer said "The permission issue occurs because uwsgi resets the ownership and permissions of /tmp/uwsgi.sock to 755 and the user running uwsgi every time uwsgi starts." it is correct.
So you have to give permission to sock file whenever uwsgi starts.
so now follow the below command
uwsgi -s /tmp/uwsgi.sock -w yolo:application -H /var/www/yolo/env --chmod-socket=666
A little different command from #susanpal.
And for persist connection, simply add "&" end of command
uwsgi -s /tmp/uwsgi.sock -w yolo:app -H /var/www/yolo/env --chmod-socket=666 &
In my case changing some php permission do the trick
sudo chown user:group -R /run/php
I hope this helps someone.
You should post both nginx and uwsgi configuration file for your application (the ones in /etc/nginx/sites-enabled/ and /etc/uwsgi/ - or wherever you put them).
Typically check that you have a line similar to the following one in your nginx app configuration:
uwsgi_pass unix:///tmp/uwsgi.sock;
and the same socket name in your uwsgi config file:
socket=/tmp/uwsgi.sock

Flask on nginx + uWSGI returns a 404 error unless the linux directory exists

This might be kind of a strange problem, but I'm not too experienced with these things and I don't know how to search for this kind of error.
I have a server configured with nginx and uWSGI. Everything runs fine, no errors in the logs that I can see. However, when I'm executing the below code:
from flask import Flask
app = Flask(__name__)
#app.route('/test/')
def page1():
return 'Hello World'
#app.route('/')
def index():
return 'Index Page'
I can not view http://ezte.ch/test/ UNLESS the /test/ directory exists inside linux once I create that directory, everything loads fine. Otherwise I get a 404 error passed to the uWSGI (it does show that it's receiving the request in the terminal) process.
Here is my config.ini for uWSGI:
[uwsgi]
project = eztech
uid = www-data
gid = www-data
plugins = http,python
socket = /usr/share/nginx/www/eztech/uwsgi.sock
chmod-socket = 775
chown-socket = www-data:www-data
wsgi-file hello.py
callable app
processes 4
threads 2
Here is my nginx configuration:
server {
listen 80; ## listen for ipv4; this line is default and implied
listen [::]:80 default ipv6only=on; ## listen for ipv6
autoindex on;
root /usr/share/nginx/www/eztech/public_html;
index index.html index.htm;
# Make site accessible from http://localhost/
server_name ezte.ch;
location / {
uwsgi_pass unix:/usr/share/nginx/www/eztech/uwsgi.sock;
include uwsgi_params;
uwsgi_param UWSGI_CHDIR /usr/share/nginx/www/eztech/public_html;
uwsgi_param UWSGI_MODULE hello;
uwsgi_param UWSGI_CALLABLE app;
# First attempt to serve request as file, then
# as directory, then fall back to index.html
try_files $uri $uri/ /index.html;
# Uncomment to enable naxsi on this location
# include /etc/nginx/naxsi.rules
}
Below is what I get when running uWSGI with my config file:
[uWSGI] getting INI configuration from config.ini
open("./http_plugin.so"): No such file or directory [core/utils.c line 3347]
!!! UNABLE to load uWSGI plugin: ./http_plugin.so: cannot open shared object file: No such file or directory !!!
open("./python_plugin.so"): No such file or directory [core/utils.c line 3347]
!!! UNABLE to load uWSGI plugin: ./python_plugin.so: cannot open shared object file: No such file or directory !!!
*** Starting uWSGI 1.9.8 (64bit) on [Sat Apr 27 06:29:18 2013] ***
compiled with version: 4.6.3 on 27 April 2013 00:06:22
os: Linux-3.2.0-36-virtual #57-Ubuntu SMP Tue Jan 8 22:04:49 UTC 2013
nodename: ip-10-245-51-230
machine: x86_64
clock source: unix
detected number of CPU cores: 1
current working directory: /usr/share/nginx/www/eztech
detected binary path: /usr/local/bin/uwsgi
*** WARNING: you are running uWSGI without its master process manager ***
your processes number limit is 4595
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
uwsgi socket 0 bound to UNIX address /usr/share/nginx/www/eztech/uwsgi.sock fd 3
setgid() to 33
setuid() to 33
Python version: 2.7.3 (default, Aug 1 2012, 05:25:23) [GCC 4.6.3]
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x2505520
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 72688 bytes (70 KB) for 1 cores
*** Operational MODE: single process ***
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI worker 1 (and the only) (pid: 12800, cores: 1)
Thank you for any assistance you can offer!
As Blender already says, there should be no try_files where is your upstream called.
The following nginx config is enough to host flask application:
server {
listen 80;
server_name ezte.ch;
location / {
uwsgi_pass unix:/usr/share/nginx/www/eztech/uwsgi.sock;
include uwsgi_params;
}
}
my flask config:
<uwsgi>
<autostart>true</autostart>
<master/>
<pythonpath>/var/www/apps/someapp/</pythonpath>
<plugin>python</plugin>
<module>someapp:app</module>
<processes>4</processes>
</uwsgi>
So there is path /var/www/apps/someapp/ and flask file someapp.py
I had the same issue. just remove this line from the nginx configuration :
root /usr/share/nginx/www/eztech/public_html;

Can only run uwsgi with root

I'm preparing to use nginx/uwsgi with flask for a website i'm developing, but i'm running into problems. NB the website itself runs great using flask's debug :5000 port, but i want to go into production now. To explain what i did.
It's a linode ubuntu 12.04LTS server, I installed it like this:
# install nginx
sudo apt-get install python-software-properties
sudo add-apt-repository ppa:nginx/stable
sudo apt-get update
sudo apt-get upgrade --show-upgraded
sudo apt-get install nginx-full
# installing uwsgi
sudo apt-get install build-essential python-dev libxml2-dev
sudo apt-get install libc6 libexpat1 libgd2-xpm libgeoip1 libpam0g libpcre3 libssl1.0.0 libxml2 libxslt1.1 zlib1g
sudo pip install uwsgi
# python basics
sudo apt-get install python-pip build-essential python-dev
sudo pip install virtualenv
sudo pip install virtualenvwrapper
sudo mkdir -p /srv/www/li/
cd /srv/www/li/
virtualenv venv
source /srv/www/li/venv/bin/activate
pip install flask
Then i set out to configure everything, but I already run into trouble with uwsgi (never mind NGINX, which will be the next step.
sudo nano /etc/uwsgi/apps-available/li.xml
<uwsgi>
<plugin>python</plugin>
<socket>/run/uwsgi/app/li.socket</socket>
<chmod-socket>666</chmod-socket>
<chdir>/srv/www/li</chdir>
<pythonpath>/srv/www/li</pythonpath>
<virtualenv>/srv/www/li/venv</virtualenv>
<module>li</module>
<wsgi-file>/srv/www/li/li.py</wsgi-file>
<callable>app</callable>
<master/>
<processes>4</processes>
<harakiri>60</harakiri>
<reload-mercy>8</reload-mercy>
<cpu-affinity>1</cpu-affinity>
<stats>/tmp/stats.socket</stats>
<max-requests>2000</max-requests>
<limit-as>512</limit-as>
<reload-on-as>256</reload-on-as>
<reload-on-rss>192</reload-on-rss>
<no-orphans/>
<vacuum/>
</uwsgi>
sudo ln -s /etc/uwsgi/apps-available/li.xml /etc/uwsgi/apps-enabled/li.xml
However if i run it, i get:
uwsgi --xml /etc/uwsgi/apps-enabled/li.xml
[uWSGI] parsing config file /etc/uwsgi/apps-enabled/li.xml
open("./python_plugin.so"): No such file or directory [core/utils.c line 4755]
!!! UNABLE to load uWSGI plugin: ./python_plugin.so: cannot open shared object file: No such file or directory !!!
*** Starting uWSGI 1.4.6 (64bit) on [Thu Feb 28 16:30:53 2013] ***
compiled with version: 4.6.3 on 28 February 2013 12:38:22
os: Linux-3.7.10-x86_64-linode30 #1 SMP Wed Feb 27 14:29:31 EST 2013
nodename: demo
machine: x86_64
clock source: unix
detected number of CPU cores: 4
current working directory: /run/uwsgi/app
detected binary path: /usr/local/bin/uwsgi
your processes number limit is 63594
limiting address space of processes...
your process address space limit is 536870912 bytes (512 MB)
your memory page size is 4096 bytes
*** WARNING: you have enabled harakiri without post buffering. Slow upload could be rejected on post-unbuffered webservers ***
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
uwsgi socket 0 bound to UNIX address /run/uwsgi/app/li.socket fd 3
Python version: 2.7.3 (default, Aug 1 2012, 05:25:23) [GCC 4.6.3]
Set PythonHome to /srv/www/li/venv
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0xa86e20
your server socket listen backlog is limited to 100 connections
mapped 362120 bytes (353 KB) for 4 cores
*** Operational MODE: preforking ***
added /srv/www/li/ to pythonpath.
/srv/www/li/venv/local/lib/python2.7/site-packages/mongoengine/fields.py:744: FutureWarning: ReferenceFields will default to using ObjectId strings in 0.8, set DBRef=True if this isn't desired
warnings.warn(msg, FutureWarning)
WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0xa86e20 pid: 14934 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 14934)
spawned uWSGI worker 1 (pid: 14940, cores: 1)
mapping worker 1 to CPUs: 0
spawned uWSGI worker 2 (pid: 14941, cores: 1)
mapping worker 2 to CPUs: 1
spawned uWSGI worker 3 (pid: 14942, cores: 1)
mapping worker 3 to CPUs: 2
spawned uWSGI worker 4 (pid: 14943, cores: 1)
unlink(): Operation not permitted [core/socket.c line 109]
bind(): Address already in use [core/socket.c line 141]
...brutally killing workers...
mapping worker 4 to CPUs: 3
VACUUM: unix socket /run/uwsgi/app/li.socket removed.
So i get the unlink operation not permitted and the bind address already in use errors (next to the python_plugin error of which i also have no clue how to solve that!). If i run as sudo, it seems to work fine ->
sudo uwsgi --xml /etc/uwsgi/apps-enabled/li.xml
[uWSGI] parsing config file /etc/uwsgi/apps-enabled/li.xml
open("./python_plugin.so"): No such file or directory [core/utils.c line 4755]
!!! UNABLE to load uWSGI plugin: ./python_plugin.so: cannot open shared object file: No such file or directory !!!
*** Starting uWSGI 1.4.6 (64bit) on [Thu Feb 28 15:47:41 2013] ***
compiled with version: 4.6.3 on 28 February 2013 12:38:22
os: Linux-3.7.10-x86_64-linode30 #1 SMP Wed Feb 27 14:29:31 EST 2013
nodename: demo
machine: x86_64
clock source: unix
detected number of CPU cores: 4
current working directory: /run/uwsgi
detected binary path: /usr/local/bin/uwsgi
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
your processes number limit is 63594
limiting address space of processes...
your process address space limit is 536870912 bytes (512 MB)
your memory page size is 4096 bytes
*** WARNING: you have enabled harakiri without post buffering. Slow upload could be rejected on post-unbuffered webservers ***
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
uwsgi socket 0 bound to UNIX address /run/uwsgi/app/li.socket fd 3
Python version: 2.7.3 (default, Aug 1 2012, 05:25:23) [GCC 4.6.3]
Set PythonHome to /srv/www/li/venv
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x1fc9d00
your server socket listen backlog is limited to 100 connections
mapped 362120 bytes (353 KB) for 4 cores
*** Operational MODE: preforking ***
added /srv/www/li/ to pythonpath.
/srv/www/li/venv/local/lib/python2.7/site-packages/mongoengine/fields.py:744: FutureWarning: ReferenceFields will default to using ObjectId strings in 0.8, set DBRef=True if this isn't desired
warnings.warn(msg, FutureWarning)
WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x1fc9d00 pid: 14755 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 14755)
spawned uWSGI worker 1 (pid: 14761, cores: 1)
mapping worker 1 to CPUs: 0
spawned uWSGI worker 2 (pid: 14762, cores: 1)
mapping worker 2 to CPUs: 1
spawned uWSGI worker 3 (pid: 14763, cores: 1)
mapping worker 3 to CPUs: 2
spawned uWSGI worker 4 (pid: 14764, cores: 1)
*** Stats server enabled on /tmp/stats.socket fd: 16 ***
mapping worker 4 to CPUs: 3
Can anyone please help me? As www-data is in the www-data group and he runs it, I tried some stuff:
sudo usermod -a -G www-data $USER
sudo chown -R $USER:www-data /srv/www/li
sudo chmod -R g+r+w+x /srv/www/li
sudo chown -R $USER:www-data /etc/uwsgi/apps-enabled
sudo chmod -R g+r+w+x /etc/uwsgi/apps-enabled
sudo chown -R $USER:www-data /run/uwsgi/app
sudo chmod -R g+r+w+x /run/uwsgi/app
But that really didn't help either. I also tried a tcp port instead of the unix /run/uwsgi/app/ port that didn't make any difference either...
This is driving me crazy :( I hope someone has a clue on what's happening here.
Kind regards,
Carst
edit: after a server restart it still gives an erro but a different one:
geoadmin#demo:~$ uwsgi --xml /etc/uwsgi/apps-enabled/li.xml
[uWSGI] parsing config file /etc/uwsgi/apps-enabled/li.xml
*** Starting uWSGI 1.4.6 (64bit) on [Thu Feb 28 18:47:36 2013] ***
compiled with version: 4.6.3 on 28 February 2013 12:38:22
os: Linux-3.7.10-x86_64-linode30 #1 SMP Wed Feb 27 14:29:31 EST 2013
nodename: demo
machine: x86_64
clock source: unix
detected number of CPU cores: 4
current working directory: /home/geoadmin
detected binary path: /usr/local/bin/uwsgi
your processes number limit is 63594
limiting address space of processes...
your process address space limit is 536870912 bytes (512 MB)
your memory page size is 4096 bytes
*** WARNING: you have enabled harakiri without post buffering. Slow upload could be rejected on post-unbuffered webservers ***
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
bind(): No such file or directory [core/socket.c line 141]
This was consistently my #1 result on google, and this page was relatively unhelpful to me, so I'm going to add my answer, even though it's fairly obvious in retrospect.
My problem was a permissions problem with my stats socket. If you change your uWSGI config's uid or gid parameters, make sure you either chmod or rm all of your old sockets/ pids, and their parent folders.
In my case I was trying to place the .sock file in the /vagrant directory, which is a machine mounted folder of virtual box and is not good for much more than reads and writes.
Place the .sock file outside of a virtualbox mount-point preferably in /tmp The FHS says: /var/run
Ref:
https://stackoverflow.com/a/7580524/1695680
Ok, after the later edit i checked the directories and the socket directory didnt exist (anymore); i think it had to do with the original apt-get install versus my later pip install... still have the issue with the python plugin but will check if it's necessary for nginx or if it will work without it... 8 hours of work over a reset, d'oh ;)
#bearrito:
In the end I put the socket in the tmp directory to avoid rights issues:
<uwsgi>
<uid>www-data</uid>
<gid>www-data</gid>
<plugin>python</plugin>
<socket>/tmp/li.socket</socket>
<chmod-socket>666</chmod-socket>
<chdir>/srv/www/li</chdir>
<pythonpath>/srv/www/li</pythonpath>
<virtualenv>/srv/www/li/venv</virtualenv>
<module>li</module>
<wsgi-file>/srv/www/li/li.py</wsgi-file>
<callable>app</callable>
<master/>
<processes>2</processes>
<pidfile>/tmp/li.pid</pidfile>
<harakiri>120</harakiri>
<reload-mercy>8</reload-mercy>
<cpu-affinity>1</cpu-affinity>
<stats>/tmp/stats.socket</stats>
<max-requests>2000</max-requests>
<limit-as>2048</limit-as>
<reload-on-as>2048</reload-on-as>
<reload-on-rss>1024</reload-on-rss>
<no-orphans/>
<vacuum/>
</uwsgi>
I hope this helps!
For me the solution was to remove /var/run/uwsgi/.sock and
chmod 775 /var/run/uwsgi
chmod 777 /var/log/uwsgi
or wherever your uwsgi files are.

On graceful signal, nginx throws Connection refused when trying to connect to uWSGI socket

This problem is perplexing me, because I seem to be following everything within the docs that would allow for a graceful restart.
I am running uWSGI in Emperor mode, with a bunch of vassals. When I try to do a graceful restart of one of the vassals, I receive an nginx 502 Bad Gateway response for about half a second. Here's some information:
One of my vassal .ini file:
[uwsgi]
master = true
processes = 2
home = /var/www/.virtualenvs/www.mysite.com
socket = /tmp/uwsgi.sock.myapp
pidfile = /tmp/uwsgi.pid.myapp
module = myapp
pythonpath = /var/www/www.mysite.com/mysite
logto = /var/log/uwsgi/myapp.log
chmod-socket = 666
vacuum = true
gid = www-data
uid = www-data
Then, I want to gracefully restart this process:
kill -HUP `cat /tmp/uwsgi.pid.myapp`
The output from the vassal log file looks alright (I think?)
...gracefully killing workers...
Gracefully killing worker 1 (pid: 29957)...
Gracefully killing worker 2 (pid: 29958)...
binary reloading uWSGI...
chdir() to /var/www/www.mysite.com/vassals
closing all non-uwsgi socket fds > 2 (max_fd = 1024)...
found fd 3 mapped to socket 0 (/tmp/uwsgi.sock.kilroy)
running /var/www/.virtualenvs/www.mysite.com/bin/uwsgi
*** has_emperor mode detected (fd: 15) ***
[uWSGI] getting INI configuration from kilroy.ini
open("/var/log/uwsgi/kilroy.log"): Permission denied [utils.c line 250]
unlink(): Operation not permitted [uwsgi.c line 998]
*** Starting uWSGI 1.2.3 (64bit) on [Fri Jun 8 09:15:10 2012] ***
compiled with version: 4.6.3 on 01 June 2012 09:56:19
detected number of CPU cores: 2
current working directory: /var/www/www.mysite.com/vassals
writing pidfile to /tmp/uwsgi.pid.kilroy
detected binary path: /var/www/.virtualenvs/www.mysite.com/bin/uwsgi
setgid() to 33
setuid() to 33
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
uwsgi socket 0 bound to UNIX address /tmp/uwsgi.sock.kilroy fd 3
Python version: 2.7.3 (default, Apr 20 2012, 23:04:22) [GCC 4.6.3]
Set PythonHome to /var/www/.virtualenvs/www.mysite.com
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x19e3e90
your server socket listen backlog is limited to 100 connections
*** Operational MODE: preforking ***
added /var/www/www.mysite.com/gapadventures/ to pythonpath.
WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x19e3e90 pid: 30041 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 30041)
spawned uWSGI worker 1 (pid: 30042, cores: 1)
spawned uWSGI worker 2 (pid: 30043, cores: 1)
But when I try to access the site quickly after this, my nginx log gets this result:
2012/06/08 09:44:43 [error] 5885#0: *873 connect() to unix:///tmp/uwsgi.sock.kilroy failed (111: Connection refused) while connecting to upstream, client: 10.100.50.137, server: mydomain.com, request: "GET /favicon.ico HTTP/1.1", upstream: "uwsgi://unix:///tmp/uwsgi.sock.kilroy:", host: "mydomain.com"
This happens for about half a second after sending the signal, so this is clearly not very graceful.
Any advice? Thanks so much!
Correct sockets path in nginx config and uWSGI. Sockt have to be identical
Was
unix:///tmp/uwsgi.sock.kilroy
or
/tmp/uwsgi.sock.myapp
Need:
nginx
unix:/tmp/uwsgi.sock.myapp
and
uwsgi
socket = /tmp/uwsgi.sock.myapp

Resources