I am wondering how web servers i.e. Nginx, Flask, Django stay alive and wait for requests. And how I can write my own program which stays alive and waits for a request before launching an action.
The short answer for the overwhelming number of cases involving nginx is systemd service. When you install nginx it sets itself up as a systemd service which is configured to start nginx on boot and keep it running.
You can adapt systemd to load and keep your own services (like Flask, etc.) alive and waiting for requests as well. Here is an article that explains the basics.
An alternative to systemd (which is built into most of the Linux systems you would be using on a server) is supervisord. Like systemd, supervisord can be configured to monitor, start, and keep your service running in the background, waiting for a request.
Related
As far as I read all over the Internet - the best practice for deploying Flask/Django applications is to put the behind a web server such as nginx and bundle them with a pre-fork server such as gunicorn or uWSGI.
This is good for many reasons, such as ssl-termination, protection against HTTP attacks (nginx), forking to threads for concurrency, restarting the application after a memory leak, or other exceptions (gunicron).
I want to deploy an internal API micro-service on sanic with pm2, as it's not customer facing, but rather will only be called from internal services the SSH termination and protection against HTTP attacks is irrelevant, the concurrency is guaranteed by sanic's asyncio nature and the restarting upon exception is handled by pm2.
Do I still need gunicorn and nginx? Can't I just run the application process as is and let it talk directly to its callers?
You absolutely do not need to have gunicorn in front of your stack. Sanic can run just fine without having a web server in front of it since it has its own internal server.
I still would advocate for using nginx to terminate TLS and to handle static files (even though sanic could do both of these), since it is efficient at it.
here's a link to another answer I've given on the same question: https://community.sanicframework.org/t/gunicorn-uwsgi-vs-build-in-http-server/47/2?u=ahopkins
You don't need it. Look at http://supervisord.org/ to start, restart, autorestart, etc. your servies.
That said I use gunicorn and supervisord in conjunction.
I have a Nginx web server with uWSGI app server installed on a single CPU Ubuntu 14.04 image.
This uWSGI app server successfully handles a Flask app requests.
The problem I am facing is that sometimes requests from a single client will time out for an extended period of time (1-2 hours).
This was happening without specifying workers or threads in my uwsgi.conf file.
Is there an ideal amount of workers/threads to be used per CPU?
I am using emperor service to start the uWSGI app server.
This is what my uwsgi.conf looks like
description "uWSGI"
start on runlevel [2345]
stop on runlevel [06]
respawn
env UWSGI=/var/www/parachute_server/venv/bin/uwsgi
env LOGTO=/var/log/uwsgi/emperor.log
exec $UWSGI --master --workers 2 --threads 2 --emperor /etc/uwsgi/vassals --die-on-term --uid www-data --gid www-data --logto $LOGTO --stats 127.0.0.1:9191
Could this be a performance problem in regards to nginx / uwsgi or is it more probable that these timeouts are occuring because I am only using a single CPU?
Any help is much appreciated!
Interesting issue you have...
Generally, you'd specify at least 2 * #CPUs + 1. This is because uWSGI might be performing a read/write to a socket, and then you'll have another worker accepting requests. Also, the threads flag is useful if your workers are synchronous, because they can notify the master thread that they are still busy working and so preventing a timeout.
I think having one worker was the reason for your timeout (blocking all other requests), but you should inspect your responses from your app. If they are taking a long time (say reading from db), you'll want to adjust the uwsgi_read_timeout directive in Nginx to allow uWSGI to process the request.
I hope this helps.
I'm currently using Nginx as a web server for Openerp. It's used to handle SSL and cache static data.
I'm considering extending it's use to also handle fail over and load balancing with a second server, using the upstream module.
In the process, it occurred to me that Nginx could also do this on multiple Openerp servers on the same machine, so I can take advantage of multiple cores. But Gunicorn seems to the the preferred tool for this.
The question is: can Nginx do a good job handling traffic to multiple local OpenERP servers, bypassing completely the need for Gunicorn?
Let first talk what they both are bascially.
Nginx is a pure web server that's intended for serving up static content and/or redirecting the request to another socket to handle the request.
Gunicorn is based on the pre-fork worker model. This means that there is a central master process that manages a set of worker processes. The master never knows anything about individual clients. All requests and responses are handled completely by worker processes.
If you see closely Gunicorn is Designed from Unicron, Follow the link for the detail more diff
which show the ngix and unicrom same model work on Gunicron also.
nginx is not a "pure web server" :) It's rather a web accelerator capable of doing load balancing, caching, SSL termination, request routing AND static content. A "pure web server" would be something like Apache - historically a web server for static content, CGIs and later for mod_something.
I was thinking if is there a way to make nginx start a uwsgi proccess at the first request, so I can save a lot of memory with idle sites.
Someone knows how to do this?
Thanks!
Nginx don't start uwsgi processes at all. It's uWSGI server job.
Probably, you're looking for "cheap" mode:
http://projects.unbit.it/uwsgi/wiki/Doc#cheap
Nginx (by design) cannot generate new processes (this is why you do not have cgi support in nginx).
You can use the cheap+idle modes of uwsgi, to start with only the master and rip-off workers after a specified time (set by --idle) of inactivity.
If even starting only the master is too much for you (i suppose you want the minimal memory usage) you can look at the old-school inetd/xinetd or newer upstart socket bridge and systemd socket activation to activate uWSGI only on specific connections
Say I have a web farm of six IIS 7 web servers, each running an identical ASP.NET application.
They are behind a hardware load balancer (say F5).
Can the load balancer detect when the ASP.NET application worker process is restarting and divert traffic away from that server until after the worker process has restarted?
What happens during an IIS restart is actually a roll-over process. A new IIS worker process starts that accepts new connections, while the old worker process continues to process existing connections.
This means that if you configure your load balancer to use a balancing algorithm other than simple round-robin, that it will tend to "naturally" divert some, but not all, connections away from the machine that's recycling. For example, with a "least connections" algorithm, connections will tend to hang around longer while a server is recycling. Or, with a performance or latency algorithm, the recycling server will suddenly appear slower to the load balancer.
However, unless you write some code or scripts that explicitly detect or recognize a recycle, that's about the best you can do--and in most environments, it's also really all you need.
We use a Cisco CSS 11501 series load balancer. The keepalive "content" we are checking on each server is actually a PHP script.
service ac11-192.168.1.11
ip address 192.168.1.11
keepalive type http
keepalive method get
keepalive uri "/LoadBalancer.php"
keepalive hash "b026324c6904b2a9cb4b88d6d61c81d1"
active
Because it is a dynamic script, we are able to tell it to check various aspects of the server, and return "1" if all is well, or "0" if not all is well.
In your case, you may be able to implement a similar check script that will not work if the ASP.NET application worker process is restarting (or down).
It depends a lot on the polling interval of the load balancer, a request from the balancer has to fail in order before it can decide to divert traffic
IIS 6 and 7 restart application pools every 1740 minutes by default. It does this in an overlapped manner so that services are not impacted.
http://technet.microsoft.com/en-us/library/cc770764%28v=ws.10%29.aspx
http://forums.iis.net/t/1153736.aspx/1
On the other hand, in case of a fault, a good load balancer (I'm sure F5 is) can detect a fault with one of the web servers and send requests to the remaining, healthy web servers. That's a critical part of a high-availability web infrastructure.