I have a large URI and I am trying to configure Nginx to accept it. The URI parameters are 52000 characters in length with a size of 52kb. I have tried accessing the URI without Nginx and it works fine.
But when I use Nginx it gives me an error. --- 414 (Request-URI Too Large)
I have configured the large_client_header_buffers and client_header_buffer_size in the http block but it doesn't seem to be working.
client_header_buffer_size 5120k;
large_client_header_buffers 16 5120k;
Any help will be appreciated.
Thank you.
I have found the solution. The problem was that there were multiple instances of nginx running. This was causing a conflict and that's why the large_client_header_buffers wasnt working. After killing all nginx instances I restarted nginx with the configuration:
client_header_buffer_size 64k;
large_client_header_buffers 4 64k;
Everything started working after that.
Hope this helps anyone facing this problem.
The problem for me was that there was an earlier server {} block (the default one, actually). By earlier, I mean it comes first when doing nginx -T. Nginx "shouldn't" have matched it, but it did, and nginx was using its configuration, which did not specify any of the directives regarding header length, such as large_client_header_buffers. Why was it used? According to this github comment that's for an add-on nginx headers module:
nginx throws out 414 very early, during reading and parsing the request header's first line. Because it happens so early, no location is matched against the current (guilty) request yet while your more_set_headers directive is a "location configuration" which only takes effect for a request that has already been associated with a location {} block
Solutions:
Do 1 of these 3 things:
Put directives related to request header size in the EARLIER server {} block. It probably needs to go in the earliest one.
Remove the EARLIER server {} block
Delete the EARLIER config file
Example problem configs
#/etc/nginx/conf.d/default.conf
server {
# Note: even if this block were completely empty, it'd be the same problem!
listen 80;
server_name localhost;
}
#/etc/nginx/conf.d/mysite
server {
listen 80;
server_name mysite.test;
large_client_header_buffers 4 64k;
}
How to fix
#/etc/nginx/conf.d/default.conf
server {
# put your request header size directives in here, or remove this file
client_header_buffer_size 5120k;
large_client_header_buffers 16 5120k;
}
#/etc/nginx/conf.d/mysite
server {
listen 80;
server_name mysite.test;
# no need for large_client_header_buffers here
}
Related
I'm a newbie at Nginx, and have been searching a lot for the right answer to my question, but couldn't find it; not because it is not there, but my newbie condition limits me to adapt a generic solution to my issue.
The situation is this:
I have a Mantis Bug Tracker in my private LAN (http://10.111.111.12).
On the other hand, i have an OwnCloud website also on my LAN (IP 10.111.111.5), with URL http://10.111.111.5/owncloud/.
What i want to do is to deploy a Nginx Reverse Proxy that handles all requests from Internet at publicdomain.com, and use trailing slash for each internal webserver. The desired result would be:
http://www.publicdomain.com/bugtracker -> redirects to http://10.111.111.12/index.php
http://www.publicdomain.com/cloud -> redirects to http://10.111.111.5/owncloud/ (note that "cloud" is preferred over "owncloud")
On the future, it is necessary to continue using trailing slash for other web servers to be deployed.
Questions are:
is this scenario possible? if so, is it enough with configuring nginx or I have to reconfigure internal web servers as well?
I really appreciate your help, by indicating me a possible solution or pointing me to the right direction on previous posts.
Thanks a lot in advance.
Yes it is possible to achieve such configuration and it's commonly used when NGINX is acting as a reverse proxy. You can use this configuration as an inspiration for building your own:
upstream bugtracker {
server 10.111.111.12;
}
upstream cloudupstream {
server 10.111.111.5;
}
server {
listen 80;
location /bugtracker/{
proxy_pass http://bugtracker;
}
location /cloud/{
proxy_pass http://cloudupstream/owncloud;
}
}
What's happening over here is that nginx will be listening on port 80 and as soon as a request comes for path /bugtracker, it will automatically route the request to the upstream server mentioned above. Using this you can add as many upstream servers and location blocks as you want.
Reference: http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass
Thanks a lot Namam for your quick answer. However, it isn't working yet. It seems that "server" at upstream directive does not allow slash, like this:
server 10.111.111.5/owncloud;
If i used it, i obtained
nginx: [emerg] invalid host in upstream "10.111.111.5/owncloud" in /etc/nginx/nginx.conf:43
I started with the first upstream bugtracker, only, and nginx.conf like this:
upstream bugtracker {
server 10.111.111.12;
}
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name localhost;
root /usr/share/nginx/html;
# Load configuration files for the default server block.
include /etc/nginx/default.d/*.conf;
location / {
}
location /mstic{
proxy_pass http://bugtracker;
}
error_page 404 /404.html;
location = /40x.html {
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
}
}
After that, when accesing to my Nginx Reverse proxy http://10.111.111.10/mstic/ i obtain the following:
Not Found The requested URL /mstic/ was not found on this server.
and no further details on error or access logs.
Thanks a lot in advance for any extra help you could bring me.
I want to set up an NGINX server which provides the following functionality:
When a request is made NGINX to get the page at /path/to/page, it fetches the page at /path/to/page.
If the upstream server is down or NGINX can't connect to it for some reason, NGINX returns a cached version of the page if it has one.
If the cached file is over 6 hours old, don't use it, just return a 502.
If the upstream server is available, never use the cache.
I have an NGINX config here which I think should work based on my understanding of the docs, but it doesn't and I can't see why. The problem is with point (4), this NGINX server returns the cached version of the file even if the upstream server is online.
daemon off;
error_log /dev/stdout info;
events {
}
http {
proxy_cache_path
"/home/jack/Code/NGINX Caching/Codebase/cache" # Cache path
keys_zone=cache:10m # Name of cacahe, max size for keys 10 megabytes
levels=1:2 # Don't store all cached files in a single directory
max_size=500m # Max size of cache
inactive=6h; # Cached file deleted if not used within six hours
proxy_cache_valid 6h;
proxy_cache_key "$request_method$request_uri";
access_log /dev/stdout;
server {
listen 8080;
location ~ ^/(.+)$ {
proxy_pass http://0.0.0.0:8000/$1;
proxy_cache cache;
proxy_cache_valid 6h;
proxy_buffering on;
proxy_cache_use_stale error timeout;
}
}
}
Replace the proxy_cache_path with a path to a directory on your machine, and run another webserver on your machine on port 8000. When I modify a file served by the server on port 8000, NGINX doesn't see the change until I erase the cache. The issue is with NGINX and not my client (Firefox), even if I turn off caching in the browser, NGINX returns a 200 with the old file contents.
Can you please check if these two directives might help you:
proxy_cache_revalidate:
http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_revalidate
and
proxy_cache_use_stale: http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_use_stale
There is a video from nginx.conf '17 online describing all the cool things you could achive with caching: https://www.youtube.com/watch?v=xZrOjmAkFC8. maybe this is also of interest for you.
So, it seems I misunderstood the NGINX proxy cache directives. The docs are quite confusing on this subject, so I'll lay it out point by point.
This official help page gives a decent overview of the various directives, however it makes no mention of something which as it turns out is a very important conceptual building block in understanding how NGINX caching works: the notion of a cached file being stale.
NGINX's default behavior is to always use the cache if it's there, rather than querying the upstream server. With this config, a minimal config to do caching, NGINX will query the upstream server the first time a page is accessed, and then use the cached version forever after that:
events {
}
http {
proxy_cache_path
/path/to/cache
keys_zone=my_cache:10m;
proxy_cache_key "$request_method$request_uri";
server {
listen 8080;
location ~ ^/(.+)$ {
proxy_pass http://0.0.0.0:8000/$1;
proxy_cache cache;
}
}
}
You can use the proxy_cache_valid directive to tell NGINX when a cached file should be considered "stale". For example, if we set proxy_cache_valid 5m, then 5 minutes after a cache file is created NGINX will stop serving it and querying the upstream server again on the next request. If the upstream is down, NGINX will return a 502. However, during those five minutes, NGINX will still use the cache even if the upstream server is available, so this is still not what we want.
NGINX has another directive, proxy_cache_use_stale, gives NGINX conditions under which it may use cached files even if they're stale. We can combine these together to get a server which caches pages, makes them stale immediately (or almost immediately), and then only uses them if the upstream is down:
events {
}
http {
proxy_cache_path
/path/to/cache
keys_zone=my_cache:10m;
proxy_cache_key "$request_method$request_uri";
server {
listen 8080;
location ~ ^/(.+)$ {
proxy_pass http://0.0.0.0:8000/$1;
proxy_cache cache;
proxy_cache_valid 1s;
proxy_cache_use_stale error timeout;
}
}
}
This config has almost the behavior we want, except that if the upstream server goes down for an extended period of time, NGINX will continue to use the cache indefinitely. As far as I know there is no way to tell NGINX to totally invalidate/clear a cached file after a given amount of time. Normally that's what proxy_cache_valid is for, but we're already using that for a different purpose, to make files stale after 1 second so they're only used when the upstream is down. We would need some next level after "stale" that means the file is completely invalidated, but I don't think that exists in NGINX.
So the simplest solution is to just clear the cache manually. It's sufficient to just delete all files in the cache directory (or its subdirectories) which were last modified more than 6 hours ago, or whatever you want the expiry time to be. On a Linux system, you can run this script every 5 minutes, for example:
find /path/to/cache -type f -mmin +360 -delete
I've got a special server {} block for the case of people who use an IP address in a host header (server_name ~^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+;) and that works fine. I'd like a different server block for one when no host header at all is provided. I've been testing with:
telnet localhost 80
GET /foobar HTTP/1.0
And I see from my logging (which has "$http_host" in the log_format) this shows up as "-" (hyphen not underscore). But this server block:
server {
listen 80;
listen [::]:80;
# hyphen not underscore
server_name ~^-*$;
root /var/www/no-host;
default_type text/plain;
index foo.ey
location / {
try_files $uri /foo.ey;
}
}
Never gets used, the request instead goes to default which has server_name _; (underscore not hyphen).
My use case is similar to a domain parking site, I want a lot of hostnames to be handled, and all legit ones should be looked up in a database, but I want to screen certain illegitimate ones early.
The answer is in the documentation: https://nginx.org/en/docs/http/server_names.html
Search for the word "empty"
Fix: server_name "";
– Barmar
I have configured nginx with fastcgi_mono_server4.
In my nginx config I have 2 hostnames :
server {
listen 80;
server_name dev.example.org
location / {
root /var/www/dev.example.org/;
fastcgi_index Default.aspx;
fastcgi_pass 127.0.0.1:9001;
include /etc/nginx/fastcgi_params;
}
}
server {
listen 80;
server_name *.example.org
location / {
root /var/www/example.org/;
fastcgi_index Default.aspx;
fastcgi_pass 127.0.0.1:9000;
include /etc/nginx/fastcgi_params;
}
}
nginx is OK with this configuration. dev goes to one and all other to another one .
I've already tried this :
fastcgi-mono-server4 /applications=*.example.org:/:/var/www/example.org/ /socket=tcp:127.0.0.1:9000
but it throws an error (Uri parse exception)
Update :
I need to get the full host name in my application, for example if the request was abc.example.org, I need to get "abc".
Unfortunately, HttpContext.Current.Request.Url does not contains "abc" but "*" which causes the parse error
If nginx is going to take care of routing the appropriate sub-domains to each fastcgi port (9000 or 9001) then can you get away with a wildcard domain when you start the mono server process e.g. just use a * instead of '*.example.org'
fastcgi-mono-server4 /applications=*:/:/var/www/example.org/ /socket=tcp:127.0.0.1:9000
Update: The above works to get two Mono server apps listening via nginx, but, using the nginx config from the original question will lead to an exception if you call HttpContext.Request.Url on the catch-all server. This is due to it not liking the * in *.example.org.
There are two possible solutions, depending what you'd like to see returned from HttpContext.Request.Url when a client browses foo.example.org, bar.example.org etc.
Option 1: If you don't care about the sub-domain and want to see example.org
Configure the second (*.example.org) nginx server to be the 'default_server' and have it assign a server-name without the wildcard e.g.
server {
listen 80 default_server;
server_name example.org;
access_log ... }
With these settings, browsing to foo.example.org/Default.aspx loads the page and HttpContext.Request.Url returns example.org/Default.aspx
Option 2: If you want to see the actual sub-domain e.g. foo.example.org
Removing the server_name from the second server definition works.
server {
listen 80 default_server;
access_log ... }
With these settings, browsing to foo.example.org/Default.aspx loads the page and HttpContext.Request.Url returns foo.example.org/Default.aspx
#stephen's answer is more simple and does not need fastcgi config modification.
I tried previous answer (before update), but it did not work.
Nginx take care of routing, as #stephen said, and the routing part worked.
to start fastcgi I used this command to match all routes (and server names)
fastcgi-mono-server4 /applications=/:/var/www/example.org/ /socket=tcp:127.0.0.1:9000
The problem was that HttpContext.Request.Url contains the $server_name value in my case it was "*.example.org" and when I try to parse URI there was an error.
To handle this I changed nginx fastcgi_params and replaced thi line
fastcgi_param SERVER_NAME $server_name;
by
fastcgi_param SERVER_NAME $http_host;
and add in site-available conf
proxy_set_header Host $host;
I think it is set by default.
reload nginx
nginx -t && service nginx reload
reload fastcgi-mono-server to test
fastcgi-mono-server4 /applications=/://var/www/example.org/ /socket=tcp:127.0.0.1:9000 /printlog=True /loglevels=Debug
in the log SERVER_NAME contains the real (not *) subomain.
I'm running nginx/ruby-on-rails and I have a simple multipart form to upload files.
Everything works fine until I decide to restrict the maximum size of files I want uploaded.
To do that, I set the nginx client_max_body_size to 1m (1MB) and expect a HTTP 413 (Request Entity Too Large) status in response when that rule breaks.
The problem is that when I upload a 1.2 MB file, instead of displaying the HTTP 413 error page, the browser hangs a bit and then dies with a "Connection was reset while the page was loading" message.
I've tried just about every option there is that nginx offers, nothing seems to work. Does anyone have any ideas about this?
Here's my nginx.conf:
worker_processes 1;
timer_resolution 1000ms;
events {
worker_connections 1024;
}
http {
passenger_root /the_passenger_root;
passenger_ruby /the_ruby;
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
server {
listen 80;
server_name www.x.com;
client_max_body_size 1M;
passenger_use_global_queue on;
root /the_root;
passenger_enabled on;
error_page 404 /404.html;
error_page 413 /413.html;
}
}
Thanks.
**Edit**
Environment/UA: Windows XP/Firefox 3.6.13
nginx "fails fast" when the client informs it that it's going to send a body larger than the client_max_body_size by sending a 413 response and closing the connection.
Most clients don't read responses until the entire request body is sent. Because nginx closes the connection, the client sends data to the closed socket, causing a TCP RST.
If your HTTP client supports it, the best way to handle this is to send an Expect: 100-Continue header. Nginx supports this correctly as of 1.2.7, and will reply with a 413 Request Entity Too Large response rather than 100 Continue if Content-Length exceeds the maximum body size.
Does your upload die at the very end? 99% before crashing? Client body and buffers are key because nginx must buffer incoming data. The body configs (data of the request body) specify how nginx handles the bulk flow of binary data from multi-part-form clients into your app's logic.
The clean setting frees up memory and consumption limits by instructing nginx to store incoming buffer in a file and then clean this file later from disk by deleting it.
Set body_in_file_only to clean and adjust buffers for the client_max_body_size. The original question's config already had sendfile on, increase timeouts too. I use the settings below to fix this, appropriate across your local config, server, & http contexts.
client_body_in_file_only clean;
client_body_buffer_size 32K;
client_max_body_size 300M;
sendfile on;
send_timeout 300s;
From the documentation:
It is necessary to keep in mind that the browsers do not know how to correctly show this error.
I suspect this is what's happening, if you inspect the HTTP to-and-fro using tools such as Firebug or Live HTTP Headers (both Firefox extensions) you'll be able to see what's really going on.