We continually poll our nginx server every 5 seconds, using keep-alive to hold the connection open.
By default keepalive_requests is set to 100, so after 100 requests on the keep-alive connection, nginx disconnects.
Currently we have set keepalive_requests to a very large number to solve this problem, however is there a way to make it infinite?
We want to hold the connection open indefinitely, regardless of how many requests are made on the same keep-alive connection. keepalive_timeout is enough for us.
Currently, the only way to do this is to modify the source. This is the relevant code within nginx:
if (r->keepalive) {
if (clcf->keepalive_timeout == 0) {
r->keepalive = 0;
} else if (r->connection->requests >= clcf->keepalive_requests) {
r->keepalive = 0;
} else {...}
A value of 4294967295 for keepalive_requests corresponds to about 680 years of 5-second requests. If you need more than that, I'd recommend patching the code.
Related
Imagine scenario: Live RTMP broadcast must be conducted form location were it's possible that network problems will occur. There will be a second link (LTE) which can be considered "last resort" because it's not so reliable. Automatic link switching in place, but everything takes time. I thought that it would be possible to first broadcast to some kind of relay station with 1-2minutes buffer so in case of loosing connection it would keep it alive for some time until main location is reconnected to one of links. I've tried nginx-rtmp-module and playing with all kind of options but every time I disconnect source from network there is a hiccup on stream (I've tested it on youtube live stream). First time I try I get few seconds until stream freeze, but from second time it;s almost instant when OBS machine looses connection to internet. Client buffer length on nginx have almost no impact other than time I have to wait for stream to show on youtube.
my config:
rtmp {
ping 10s;
server {
listen 1935;
buflen 20s;
chunk_size 4096;
application live {
idle_streams off;
live on;
record off;
push rtmp://a.rtmp.youtube.com/live2/my_super_duper_key;
}
}
}
I would be very grateful for any help, maybe I should be using something different than nginx?
If I have the following timeout rules in my http block:
keepalive_timeout 1s;
send_timeout 1s;
And the following location:
location = /slow {
echo_sleep 10;
echo "So slow";
}
I would expect /slow to trigger a 408 or 504 (timeout), but it's actually honouring that request. Which says to me that I'm handling timeouts incorrectly. So how would I limit the length of time a request takes to be processed by nginx?
The documentation clearly says
Sets a timeout for transmitting a response to the client. The timeout is set only between two successive write operations, not for the transmission of the whole response. If the client does not receive anything within this time, the connection is closed.
echo_sleep 10; and then echo "xxx", so the response time will start from echo and not from echo_sleep
I have an website wich is getting more traffic as usual in the past months.
I want this website to server more users in the same amount of time without changing the Hardware.
At the Moment i use Apache2 with Wordpress and Memcached.
I wanted to know if i can use Nginx to get more performance on this site.
When i have Nginx running on the Webserver with Wordpress and i run a test with 10000 users over a period of 60 seconds, i get only 600 succesfull answers the other 9400 connections get Errors. (mostly timeout). IMG
when i use Memcached additionally to the previous configuration i get 9969 successfull Answers, but the maximal users per second dont go over 451 IMG
But on my Site i have sometimes over 1000 Users per second.
So can anybody tell me what im doing wrong?
System:
AWS EC2 Cloud Server 2GHz, 650MB RAM
Ubuntu 13.10
Nginx 1.4.7
Memcached 1.4.14
Php-fpm for php 5.5.3
The number you should consider is Avg error rate, your WP+Nginx+Memcached configuration looks not too bad, so by my opinion this is good choice.
Maybe you can increase the -m parameter in memcached to match half of your RAM.
BUT:
memcached do not guarantee that the data will be available in memory and you have to be prepared for a cache-miss storm. One interesting approach to avoid a miss-storm is to set the expiration time with some random offset, say 10 + [0..10] minutes, which means some items will be stored for 10, and other for 20 minutes (the goal is that not all of items expire at the same time).
Also, no matter how much memory you will allocate for memcached, it will use only the amount it needs, e.g. it allocates only the memory actually used. With the -k option however (which is disabled in your config), the entire memory is reserved when memcached is started, so it always allocate the whole amount of memory, no matter if it needs it or not.
This number of 451 connections can actually vary, it depends. It is always good idea to look at the averages when performing benchmarks, i.e. better to have 0% Avg error rate and 451 served clients, than 65% Avg error rate and 8200+ served clients.
However, in order to offload some more resources, you can use additional caching for Wordpress, there are plenty of plugins, I personally wrote one for that purpose.
Regarding the nginx configuration, you can tune some parameters there also:
worker_rlimit_nofile 100000;
worker_connections 4000;
# optmized to serve many clients with each thread, essential for linux use epoll;
# accept as many connections as possible,may flood worker connections if set too low
multi_accept on;
# cache informations about FDs, frequently accessed files
# can boost performance, but you need to test those values
open_file_cache max=200000 inactive=20s;
open_file_cache_valid 30s;
open_file_cache_min_uses 2;
open_file_cache_errors on;
# to boost IO on HDD we can disable access logs
access_log off;
# copies data between one FD and other from within the kernel
# faster then read() + write()
sendfile on;
# send headers in one peace, its better then sending them one by one
tcp_nopush on;
# don't buffer data sent, good for small data bursts in real time
tcp_nodelay on;
# number of requests client can make over keep-alive -- for testing
keepalive_requests 100000;
# allow the server to close connection on non responding client, this will free up memory
reset_timedout_connection on;
# request timed out -- default 60
client_body_timeout 10;
# if client stop responding, free up memory -- default 60
send_timeout 2;
# reduce the data that needs to be sent over network
gzip on;
gzip_min_length 10240;
gzip_proxied expired no-cache no-store private auth;
gzip_types text/plain text/css text/xml text/javascript application/x-javascript application/xml;
gzip_disable "MSIE [1-6]\.";
The Problem we had was not a real Problem.
We only interpreted the Test results wrong.
The 400 Users limit wasnt an actual limit, the Server was able to keep the Users on a constant level, because it was fast enough to answer all requests right away.
The Results of the Tests are not comparable to my site, that is getting 1k Users, as it has better hardware than an AWS free instance of course.
But i think 400 Users per sec is a very good result for such a "weak" server.. so
Question solved i think, because of my own stupidness of reading test results...
Thanks for your help anyway bodi0.
This is my nginx status below:
Active connections: 6119
server accepts handled requests
418584709 418584709 455575794
Reading: 439 Writing: 104 Waiting: 5576
The value of Waiting is much higher than Reading and Writing, is it normal?
Because of the 'keep-alive' is open?
But if I send a large number of requests to the server, the value of Reading and Writing don't increase, so I think there must be a bottleneck of the nginx or any other.
The Waiting time is Active - (Reading + Writing), i.e. connection still opened waiting for either a new request, or the keepalive expiration.
You could change the keepalive default (which is 75 seconds)
keepalive_timeout 20s;
or tell the browser when it should close the connection by adding an optional second timeout in the header sent to the browser
keepalive_timeout 20s 20s;
but in this nginx page about keepalive you see that some browsers do not care about the header (anyway your site wound't gain much thanks to this optional parameter).
The keepalive is a way to reduce the overhead of creating the connection, as, most of the time, a user will navigate through the site etc... (Plus the multiple requests from a single page, to download css, javascript, images etc...)
It depends on your site, you could reduce the keepalive - but keep in mind that establishing connections is expensive. This is a trade-off you have to refine depending on the site statistics. You could also decrease little by little the timeout (75s -> 50, then a week later 30...) and see how the server behaves.
You don't really want to fix it, as "waiting" means keep-alive
connections. They consume almost no resources (socket + about
2.5M of memory per 10000 connections in nginx).
Are the requests short lived? it's possible they're reading/writing then closing in a short amount of time.
If you're genuinely interested in fixing it you can test to see if nginx is bottleneck you could set keep-alive to 0 in your nginx config:
keepalive_timeout 0;
The nginx documentation says
max_clients = worker_processes * worker_connections
but how does the keepalive factor into this? I have my configuration setup with 2 worker_processes and 8192 worker_connections; that means I can theoretically handle a maximum of 16384 concurrent connections. Pushing out 16384 streams of data concurrently is ginormous, but if I have a 60s keepalive_timeout then with each client hogging a connection for 1 minute that number has a completely different meaning. Which is it?
Connected to all this is the $connection variable that can be used with the log_format directive. I defined the following log format so I could analyze the server's performance:
log_format perf '$request_time $time_local $body_bytes_sent*$gzip_ratio $connection $pipe $status $request_uri';
That $connection variable is reporting around 11-12 million connections! I'm no math major, but obviously that number is way higher than worker_processes * worker_connections. So what is it supposed to represent?
In short, I'm trying to figure out how to determine a good value for worker_connection.
$connection is a counter, not the total number of used connections right now. So it's intended to grow.
Keepalive connections cannot be discarded, so the room is worker_processes * worker_connections - keepalive connections
just imagine the whole picture: first client connects to you, gets a file and then browser keeps connection for 60 secs. another client connects, gets, and keeps its connection too. at the end of firstr minute, you may have (in worst case) all the clients requested something from you in the last 60 secs still keeping their connections open
so, in the worst case you will serve "worker_processes * worker_connections / keep_alive" connections in a second, i.e. about 260 for your numbers. if you need more, you should alloc more connections - just for serving keepalives: read my answerr in Tuning nginx worker_process to obtain 100k hits per min
afaik nginx may hold 10k of inactive (keepalived) connections in 2.5mb of memory, so increasing worker_connections is cheap, very cheap. i think that main bottleneck here may be your OS itself