Ignore or modify request headers in NGINX - nginx

I need to make NGINX ignore / modify a request header.
Problem is some IoT devices are sending a HEAD request with the wrong Content-Length header. That makes NGINX wait for more content and then timeout.
Both dropping the Content-Length header or setting it to 0 should do the trick.
Example
This fails
HEAD / HTTP/1.1
Host: MY_HOST
Content-Length: 59
Content-Type: text/html
Connection: close
This works (Content-Length: 0)
HEAD / HTTP/1.1
Host: MY_HOST
Content-Length: 0
Content-Type: text/html
Connection: close
This works too (no Content-Length)
HEAD / HTTP/1.1
Host: MY_HOST
Content-Type: text/html
Connection: close
How can I make it happen?

I discovered that there's a NGINX module named HeadersMore that allow modifying input headers (and more).
In particular more_clear_input_headers allows to remove input headers and more_set_input_headers allows to modify input headers.
In my case
more_clear_input_headers "Content-Length";
or
more_set_input_headers "Content-Length: 0";

Related

Connection header disapper when HTTP2

In making a request to my website with curl and HTTP 1.1, I see my keep-alive connection header explicitly:
$ curl https://website.com/ -i
HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
Content-Length: 2
Connection: keep-alive
But with Chrome's developer tool and HTTP 2 the header is missing:
Content-Type: text/html; charset=utf-8
Content-Length: 2
In HTTP 2 is it normal that the connection header is not sent (and why)?
Yes, this is normal and specified by the HTTP/2 RFC.

Should an HTTP server respond with a more specific Content-Type than was requested?

For example, if the request was
GET /feed.xml HTTP/1.1
Host: www.nowhere123.com
Accept: application/xml
... would it be admissible for a server to respond with a header Content-Type: application/atom+xml (even though the request had Accept: application/xml), or should it serve the same body but with Content-Type: application/xml?
A server is allowed to ignore the header field, so yes, sending something more specific is ok. There's a reason why HTTP messages are self-descriptive. (see https://greenbytes.de/tech/webdav/rfc7231.html#rfc.section.5.3.2.p.7)

How to ignore a http header line with missing colon separator in golang

I am facing a problem in golang http server where client is sending a http request with invalid parameters in header. The request parameters are as follows:
GET /a-53-qf21489190x38856_2/yy HTTP/1.1
Host: xxx.xxx.xxx.xxx:80
User-Agent: mot-w845
Connection: Keep-Alive
Profile: http://10.213.2.68
X-MDN9035445271
accept: application/vnd.wap.mms-message
accept-language: en
accept-charset: US-ASCII, ISO-8859-1, UTF-8
HTTP/1.1 400 Bad Request
Content-Type: text/plain; charset=utf-8
Connection: close
Here X-MDN9035445271 doesn't contain colon (:) separator, because of this server is returning bad request. Is there any way to ignore such kind of header and process the request normally.

How do I set the request accept header in nginx?

I need to set the request accept header for any URLs on certain path to */*.
Here are the request accept headers being set currently:
GET /apiQuery/preview?view=location_csv&q=name:* HTTP/1.1
Host: www.blah.com
Connection: keep-alive
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
I want it to be:
GET /apiQuery/preview?view=location_csv&q=name:* HTTP/1.1
Host: www.blah.com
Connection: keep-alive
Cache-Control: max-age=0
Accept: */*
..for any URLs along the /apiQuery/ path.
There is a module available for such a purpose called HeaderMode.
more_set_input_headers 'Accept: */*';
Downside: This module is not distributed with Nginx. You will have to follow the installation guide to get this module integrated with your Nginx.
Alternatively, if you are using your Nginx as a proxy, you should use proxy_set_header directive.
proxy_set_header Accept '*/*';

Cross-domain chunked uploads using CORS

I have user-submitted files that I'm trying to upload in 10 MB chunks. I'm currently using raw XMLHttpRequest (and XDomainRequest) to push each individual slice (File.prototoype.slice) on the front end. The back end is Nginx using the upload module.
Just for reference, here's the synopsis of how I'm using slice:
element.files[0].slice(...)
I understand the cross-browser prefixed methods webkitSlice and mozSlice and all that.
The problem I have is with actually making the cross-domain request. I'm uploading from server.local to upload.server.local. In Firefox, the options request goes through fine and then the actual post fails. In Chrome and Opera, the options request fails with
OPTIONS https://URL Resource failed to load
Here are the headers from Firefox:
Request Headers
OPTIONS /path/to/asset HTTP/1.1
Host: upload.server.local:8443
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) Gecko/20100101 Firefox/18.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Origin: https://server.local:8443
Access-Control-Request-Method: POST
Access-Control-Request-Headers: content-disposition,content-type,x-content-range,x-session-id
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
Response Headers
HTTP/1.1 204 No Content
Server: nginx/1.2.6
Date: Wed, 13 Feb 2013 03:27:44 GMT
Connection: keep-alive
access-control-allow-origin: https://server.local:8443
Access-Control-Allow-Methods: POST, OPTIONS
Access-Control-Allow-Headers: x-content-range, origin, content-disposition, x-session-id, content-type, cache-control, pragma, referrer, host
access-control-allow-credentials: true
Access-Control-Max-Age: 10000
The actual post request never leaves the browser. Nginx access logs never see the post. The browser halts it for some reason. How do I unravel why this post is being blocked?
Chromium 24
Firefox 18
Opera 12.14
I've verified all browsers support CORS properly here.
By pointing my uploads to https://cors-test.appspot.com/test, I have confirmed that the problem is definitely with the server-side headers.
The POST won't leave the browser if the preflight check does not return sufficient permissions and thus the POST request is not fully authorized. The request/response included in the question does look sufficient to me.
Are you sure you are setting withCredentials = true in your XMLHttpRequest?
Are you sure that you have valid (not self-signed) SSL certificates on your servers? The HTTPS might fail the CORS check even if you have added an exception for browsing the site with an invalid certificate.
Have you tried emptying your cache? You have Access-Control-Max-Age: 10000 set in your response headers. That's close to 3 hours. I know you've been working on this longer than that but while testing especially, set that header to zero instead so you don't go crazy with browser caching of old access permissions.
In general I'd start with going as permissive as possible with the CORS headers and slowly ratcheting up the the security to see where it fails. However, this is not completely straightforward. For example, according to the MDN documentation on CORS,
When responding to a credentialed request, server must specify a domain, and cannot use wild carding. The above example would fail if the header was wildcarded as: Access-Control-Allow-Origin: *
When I send the request part of your question to https://cors-test.appspot.com/test, I get back the following:
HTTP/1.1 200 OK
Cache-Control: no-cache
Access-Control-Allow-Origin: https://server.local:8443
Access-Control-Allow-Headers: content-disposition,content-type,x-content-range,x-session-id
Access-Control-Allow-Methods: POST
Access-Control-Max-Age: 0
Access-Control-Allow-Credentials: true
Cache-Control: no-cache
Expires: Fri, 01 Jan 1990 00:00:00 GMT
Content-Type: application/json
Content-Encoding: gzip
Content-Length: 35
Vary: Accept-Encoding
Date: Thu, 23 May 2013 06:37:34 GMT
Server: Google Frontend
So you can start from there and add more and more security until it breaks to figure out what is the culprit.

Resources