nginx proxy request buffering is not working as expected - http

Question

This is my nginx config :
worker_processes auto;
user nginx;
events {
worker_connections 1024;
use epoll;
}
http {
tcp_nodelay on;
add_header Cache-Control no-cache;
upstream servers {
server 127.0.0.1:9999;
server 127.0.0.1:9998;
}
proxy_request_buffering on;
server {
listen 80;
client_max_body_size 200M;
client_body_buffer_size 200M;
server_name localhost;
location / {
try_files $uri #proxy_upload;
}
location #proxy_upload {
proxy_pass_request_body on;
proxy_pass http://servers;
}
}
}
I try to upload files, I am doing something like a chunked upload, files are more than 2G, client send files in chunks, the nginx and the script are working, but nginx is not working as expected
you see, I turned proxy_request_buffering on, so I expect nginx buffer all 200M of file and then pass it to back-end(which is tornado-python) at once, but nginx pass it to back-end in 1M or 2M chunks, this behavior lead to very higher cpu usage, higher system load and lower upload speed, this behavior is not much different than setting proxy_request_buffering to off, so I think I am doing somehthing wrong here
why is nginx not buffering correctly and how am I suppose to make nginx buffer the whole request and then pass it at once ?
I tried to use post_action, but I couldn't pass request body to back-end
UPDATE: Nginx is buffering correctly, I mean it pass request body to back-end as soon as whole file uploaded by client, but Nginx pass request body to back-end in smaller chunk and it won't pass it at once, It has whole body but it won't pass the whole body at once, How can I tell to nginx to pass request body to back-end at once ?

Answer 1

Looks like you need to turn on proxy_buffering, set proxy_buffers to set the number and size of the buffers, and probably proxy_buffers_size to ensure it doesn't overuse memory:
location #proxy_upload {
proxy_buffering on;
proxy_buffers 10 200m;
proxy_buffer_size 1000m;
proxy_pass_request_body on;
proxy_pass http://servers;
}
Not sure how much traffic your upstream server gets, but I could imagine buffers this large isn't that efficient ... What webserver are you using upstream that requires such a large buffer? Perhaps try incrementing the buffer 1m at a time until it's at a decent sweet spot.
From: http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffers
And: http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffer_size