I found a response where duplicate headers are used by the application with the same value. Could anyone tell me that, Is it a good programming practice or those are used for security perspective or anything else?
HTTP/1.1 200
Accept-Ranges: bytes
Cache-Control: no-cache, must-revalidate, private
Content-Type: text/html
Date: Mon, 20 Nov 2017 04:08:51 GMT
Expires: 0
Last-Modified: Thu, 16 Nov 2017 14:04:48 GMT
Pragma:
Public-Key-Pins: pin-sha256="5w0XrTCAbsVO7vTngDViNHPutlvB43qYionPbpV2ky0=";
max-age=5184000; includeSubDomains;
Server: Any
Set-Cookie: ********************* httponly; secure; path=/
Strict-Transport-Security: max-age=31536000 ; includeSubDomains
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Content-Type-Options: nosniff
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Content-Length: 559
Connection: Close
This application is using duplicate X-Content-Type-Options header, Strict-Transport-Security, X-Frame-Options header with same values.
I posted this question at stackoverflow, but I didn't find any response.
From RFC2616
Multiple message-header fields with the same field-name MAY be present in a message if and only if the entire field-value for that header field is defined as a comma-separated list [i.e., #(values)]. It MUST be possible to combine the multiple header fields into one "field-name: field-value" pair, without changing the semantics of the message, by appending each subsequent field-value to the first, each separated by a comma. The order in which header fields with the same field-name are received is therefore significant to the interpretation of the combined field value, and thus a proxy MUST NOT change the order of these field values when a message is forwarded
So
Cache-Control: no-cache
Cache-Control: must-revalidate
Cache-Control: private
would be fine. I'm not convinced that applies here. If having duplicate header entries were valid it will depend on each header as to whether having duplicate values is allowed.
From a security perspective if any of those lines are not legal it is not defined what the client will do with them - so ignoring them may be a valid option. So I would argue as well as bad practice it is technically a risk.
From a practical perspective there are a few ways I could imagine clients handling this.
The obvious option is they would just take the first or last entry they encounter. Seeing as these are duplicates that wouldn't be an issue.
The safe option is to error out the request.
A smarter option would be to pass both to the handler and let it decide. I.e for security based headers always use the stricter value.
Another option would be to concatenate headers into csv and pass that to the handler - which would cover the legal duplicate header cases.
The worst case would be the client ignores the header.
If duplicating a header is fine and how it gets interpreted depends on the exact semantic of the header. Some headers like Content-Encoding are cumulative, i.e. all values will be put together (at least in theory, practice might differ). In this case having multiple headers of the same value is of course different to having a single header.
Other headers are singletons and should occur at most once. Usually it is accepted if the same header and value is sent multiple times for singletons. But I would not rely on it since it is still invalid. And if the same header is sent multiple times with a different value, the behavior of the clients varies (might also depend on the header): some pick a specific header (i.e. first or last) while others treat the response as corrupt since multiple interpretations are possible which can result in security problems.
The correct behavior (as per RFC) is to append if the header is a list (like Cache-Control) or generate an error.
In practice, the Internet is a wild goose, as Steffen Ulrich says, the result will vary depending on the client. I think most clients do not generate an error. One problem with the error concept is that a header you do not know could be a one value header (opposed to a list) and thus you can't know whether it is allowed to be repeated or not. So the RFC is nice but can't be properly followed for such headers.
In most cases, I would say it's bad practice to repeat headers especially with the same values and would look into whether it is possible to avoid the duplication. Since in your case both have the exact same value, it is likely fine for most clients available out there.
Related
This question is similar to Are Duplicate HTTP Response Headers acceptable? but goes a step further.
I have observed a webserver in our staging environment that returns the following headers (output has been truncated for clarity):
strict-transport-security: max-age=1
Strict-Transport-Security: max-age=31536000; includeSubDomains
My question is: in the face of conflicting header values, which takes precedence?
Since the RFC states that multiple values must be chainable, this functionality seems undefined as the values are not chainable (since they are conflicting) yet the browser does not return an error.
As it turns out, even though Chrome does not alert me to it, the header above is listed as INVALID and thus no HSTS policy is applied whatsoever.
My reading of RFC 2616 hasn't answered my question:
How should a server interpret multiple Accept, Accept-Encoding, Accept-Language, etc, headers?
Granted, this should generally be a rare occurrence, but far be it from me to assume every HTTP client actually does what it should.
Imagine an HTTP request includes the following:
Accept-Language: en
Accept-Language: pt
Should the server:
Combine the results, to an effective Accept-Language: en, pt?
Honor only the first one (en)?
Honor only the last one (pt)?
Throw a hissy fit (return a 400 status, perhaps?)
Option #1 seems the most natural to me, and the most likely to be what the client means, and the least likely to completely break expectations even if it's not what the client means.
But is there any actual rule (ideally specified by an RFC) for how to handle these situations?
1) You are looking at an outdated RFC. RFC 2616 has been obsoleted two years ago.
2) That said, the answer is 1); see https://greenbytes.de/tech/webdav/rfc7230.html#rfc.section.3.2.2.p.3: "A recipient MAY combine multiple header fields with the same field name into one "field-name: field-value" pair, without changing the semantics of the message, by appending each subsequent field value to the combined field value in order, separated by a comma. (...)"
I have read through lot of discussions, and found most of the claim saying expires is higher precedence over last-modified, meaning if a response already expired, it will not even send out if-modified-since to server and of course the response code will not be 304.
But my situation is totally weird, I have returned back last-modified in response, and somehow CDN/proxy side add in expires header, which value is same as date response header, I suppose same value in expires and date header will cause the response stale immediately, but in fact, my client browser will still send out request with if-modified-since header, this will cause a 304 response code returned from the server.
I read throught RFC 2616, it doesn't tell much as well. So what happen to this case?
Almost 2 years but find no answer...
I managed find out some reference:
The freshness lifetime is calculated based on several headers. If a
"Cache-control: max-age=N" header is specified, then the freshness
lifetime is equal to N. If this header is not present, which is very
often the case, it is checked if an Expires header is present. If an
Expires header exists, then its value minus the value of the Date
header determines the freshness lifetime. Finally, if neither header
is present, look for a Last-Modified header. If this header is
present, then the cache's freshness lifetime is equal to the value of
the Date header minus the value of the Last-modified header divided by
10.
Although I can't find confirmed precedence in RFC, but I think this MDN quote is quite reliable enough.
It is pretty normal if some browser doesn't implement in this way... So to avoid any issue, the best is to not return these two headers in the response at the same time.
The Content-Length response header is often expensive to generate for a just HEAD request (e.g. when dealing with dynamically generated resources), but may be essentially "free" after doing the work needed to generate a GET response body.
When it is reasonable to provide a Content-Length (instead of a chunked response) when responding to a GET request, yet unreasonable or slow to calculate the Content-Length for the corresponding HEAD request, is it permissible for the HEAD response to:
Omit the Content-Length header altogether?
Respond with Transfer-Encoding: chunked even though GET response would have a Content-Length?
The relevant W3C specification indicates that a HEAD request "SHOULD" (not "MUST") respond with identical headers; is the cleanliness and, admittedly often minor, total transfer size savings of using Content-Length in the GET response worth violating the aforementioned "SHOULD" in the case of HEAD, or is the only reasonable choice to have both responses send a Transfer-Encoding: chunked header?
Thanks to a tip from #julian-reschke, rfc-7231 indicates that:
The server SHOULD send the same header fields in response to a HEAD request as it would have sent if the request had been a GET, except that the payload header fields (Section 3.3) MAY be omitted.
Per section 3.3 of that same document, the payload header fields consist of:
Content-Length
Content-Range
Trailer
Transfer-Encoding
This question is regarding the order of precedence for the media-types of the HTTP Header "Accept-Encoding" when all are of equal weight and has been prompted by this comment on my blog.
Background:
The Accept-Encoding header takes a comma separated list of media-types the browser can accept e.g. gzip,deflate
A quality factor can also be specified to give preference to other media-types e.g. in the case of "gzip;q=.8,deflate", deflate is preferred - but is not relevant to this question. NB: A type with a "q=0" means "not acceptable".
RFC2616 also states that the "most specific reference" for the media-type definition should be weighted first. i.e. "text/html;level=1" should be used over "text/html" - this is not relevant to the question also.
Question:
In the following case, which media-type has precedence?
Accept-Encoding: gzip,deflate
Both types have an equivalent quality factor of 1, and both types are "acceptable" to the browser - so either one could be used. I'd always assumed that the first type entered should be "preferred", but there doesn't seem to be a specific example or preference for this particular case in the RFC.
I believe somewhere in the RFC, or in a related RFC, it states that the first is preferred for all fields of this format.
However, in the special case of gzip vs deflate, you should probably use deflate if you can due to lower overhead (fewer headers and footers, and although it still has an adler32 checksum, it doesn't have a crc32 on top). Other than that they are exactly the same. The actual data is compressed in the same way for both. This means deflate is both faster and produces a smaller output. Both of these become far more important on a page under heavy load. Most of the extra headers in gzip are things like unix style file permissions, which are useless in this context anyway.
Really, clients should want to be served gzip due to reliability and servers should want to serve deflate due to performance. The extra overhead is much more important when it happens thousands of times a second than when it happens once for every page you load.
On my own sites I check for deflate first and use that if I can, then I check for gzip. If I can't use either, I just send in plain text. I don't know what language you are using but it's about 5 lines of ASP.NET to do that.
There is no client side preference here. Just pick one what you (the server side) would prefer.