RFC 7230 says (3.2.2 Field Order, markup by me):
A sender MUST NOT generate multiple header fields with the same field
name in a message unless either the entire field value for that
header field is defined as a comma-separated list [i.e., #(values)]
or the header field is a well-known exception (as noted below).
Back in RFC2616, all the headers were contained in a single specification and one could browse through that spec for header definitions that have lists as values.
Nowadays, we have RFC7230 and friends, each specifying its own set of headers.
Is there an (authorative) list somewhere that holds the header names with list values? Or do I need to grep all related RFCs for 1#?
Of the standard HTTP request headers, these can have multiple values separated by a comma ,:
A-IM
Accept
Accept-Charset
Accept-Encoding
Accept-Language
Access-Control-Request-Headers
Cache-Control
Connection
Content-Encoding
Expect
Forwarded
If-Match
If-None-Match
Range
TE
Trailer
Transfer-Encoding
Upgrade
Via
Warning
these can have them separated by a semicolon ;:
Content-Type
Cookie
Prefer
and these shouldn't have multiple values (or can but with some other syntax, for example the User-Agent header can pretty much be arbitrary text):
Accept-Datetime
Access-Control-Request-Method
Authorization
Content-Length
Content-MD5
Date
From
Host
HTTP2-Settings
If-Modified-Since
If-Range
If-Unmodified-Since
Max-Forwards
Origin
Pragma
Proxy-Authorization
Referer
User-Agent
This doesn't include response headers, and any non-standard HTTP request headers. I made these lists by quickly glancing at the RFC listed in that table on Wikipedia for each header, I could've misclassified a few.
The only known exception is Set-Cookie. Many HTTP frameworks that encapsulate HTTP headers tend to treat all headers the exact same way, but have a specific exception for Set-Cookie. No other headers have this problem, and no standard new headers would be introduced with this problem.
Pick a random HTTP framework for any language, and chances are that there there's some special handling for just Set-Cookie.
It's possible that other non-standard headers have this issue.
Related
I have encountered an HTTP client that uses the following header:
Accept: application/vnd.api+json; charset=utf-8'
According to the HTTP spec, Accept headers can have parameters. The most common one being the q parameter, which sets the priority of different content types. However, there are a number of reasons I don't think charset is a valid Accept parameter:
Accept already has the Accept-Charset parameter, which seems like it makes this redundant
MDN doesn't include it in their documentation on Accept, even though they do include in on their Content-Type page
Werkzeug, the flask HTTP parser, doesn't bother to parse charsets for Accept, even though it does for Content-Type
So, it seems this Accept; charset is unusual. But is it wrong?
You quoted the spec, which says they are ok. What else needs to be said?
POST or PUT method need content-length in request header field.
Does another method(GET, PATCH, DELETE, OPTIONS, CONNECT, TRACE, HEADER) also need content-length?
About content-length in Response, specification is described in rfc2616.
The Content-Length entity-header field indicates the size of the entity-body, in decimal number of OCTETs, sent to the recipient or, in the case of the HEAD method, the size of the entity-body that would have been sent had the request been a GET.
Althought, I couldn't find specification about request header..
The HTTP specification has been updated from RFC2616. Refer to the following instead:
RFC7230 - HTTP/1.1: Message Syntax and Routing
RFC7231 - HTTP/1.1: Semantics and Content
RFC7232 - HTTP/1.1: Conditional Requests
RFC7233 - HTTP/1.1: Range Requests
RFC7234 - HTTP/1.1: Caching
RFC7235 - HTTP/1.1: Authentication
Specifically Section 3.3.2 of RFC7230
A Content-Length SHOULD be sent WHEN the request includes a payload body and a Transfer-Encoding header is not set.
So even a POST or PUT only need to send a Content-Length when there is a body to have a length, this just happens normally be the case with a POST and PUT due to the operations.
There is no problem with sending a Content-Length of 0 to indicate no body, but that is implied by not having a Content-Length or Transfer-Encoding.
These are just the specs though, so your mileage may vary with different http implementations.
I've been reading RFC 2616 - sec14.26 and RFC 2616 - sec13.3.3 for quite a long time, but I'm still not sure if I understand correctly.
The section 14.26 states that
If any of the entity tags match the entity tag of the entity that would have been returned in the response to a similar GET request (without the If-None-Match header) on that resource, or if "*" is given and any current entity exists for that resource, then the server MUST NOT perform the requested method, unless required to do so because the resource's modification date fails to match that supplied in an If-Modified-Since header field in the request. Instead, if the request method was GET or HEAD, the server SHOULD respond with a 304 (Not Modified) response, including the cache- related header fields (particularly ETag) of one of the entities that matched.
,
If none of the entity tags match, then the server MAY perform the requested method as if the If-None-Match header field did not exist, but MUST also ignore any If-Modified-Since header field(s) in the request. That is, if no entity tags match, then the server MUST NOT return a 304 (Not Modified) response.
and the section 13.3.3 states that
An HTTP/1.1 origin server, upon receiving a conditional request that includes both a Last-Modified date (e.g., in an If-Modified-Since or If-Unmodified-Since header field) and one or more entity tags (e.g., in an If-Match, If-None-Match, or If-Range header field) as cache validators, MUST NOT return a response status of 304 (Not Modified) unless doing so is consistent with all of the conditional header fields in the request.
I'm quite confused whether a unconditionally compliant HTTP server should response to a client request (GET / HEAD) where
the condition If-Modified-Since matches, but If-None-Match doesn't.
or, the condition If-None-Match matches, but If-Modified-Since doesn't.
According to the section 14.26, 304 Not Modified should be returned in the first case and 200 OK in the second case since If-Modified-Since is ignored.
But the section 13.3.3 says that "it MUST NOT return a response status of 304 (Not Modified) unless doing so is consistent with all of the conditional header fields in the request.", meaning that 200 OK should be returned in the both cases.
from RFC 7232
A recipient must ignore If-Modified-Since if the request contains an If-None-Match header field; the condition in If-None-Match is considered to be a more accurate replacement for the condition in If-Modified-Since, and the two are only combined for the sake of interoperating with older intermediaries that might not implement If-None-Match.
I'm writing a web service that returns a base64-encoded PDF file, so my plan is to add two headers to the response:
Content-Type: application/pdf
Content-Transfer-Encoding: base64
My question is: Is Content-Transfer-Encoding a valid HTTP header? I think it might only be for MIME. If not, how should I craft my HTTP response to represent the fact that I'm returning a base64-encoded PDF? Thanks.
EDIT:
It looks like HTTP does not support this header. From RFC2616 Section 14:
Note: while the definition of Content-MD5 is exactly the same for HTTP
as in RFC 1864 for MIME entity-bodies, there are several ways in which
the application of Content-MD5 to HTTP entity-bodies differs from its
application to MIME entity-bodies. One is that HTTP, unlike MIME, does
not use Content-Transfer-Encoding, and does use Transfer-Encoding and
Content-Encoding.
Any ideas for what I should set my headers to? Thanks.
EDIT 2
Many of the code samples found in the comments of this PHP reference manual page seem to suggest that it actually is a valid HTTP header:
http://php.net/manual/en/function.header.php
According to RFC 1341 (made obsolete by RFC 2045):
A Content-Transfer-Encoding header field, which can be used to
specify an auxiliary encoding that was applied to the data in order to
allow it to pass through mail transport mechanisms which may have
data or character set limitations.
and later:
Many Content-Types which could usefully be transported via email
are represented, in their "natural" format, as 8-bit character or
binary data. Such data cannot be transmitted over some transport
protocols. For example, RFC 821 restricts mail messages to 7-bit
US-ASCII data with 1000 character lines.
It is necessary, therefore, to define a standard mechanism for
re-encoding such data into a 7-bit short-line format. (...) The
Content-Transfer-Encoding field is used to indicate the type of
transformation that has been used in order to represent the body
in an acceptable manner for transport.
Since you have a webservice, which has nothing in common with emails, you shouldn't use this header.
You can use Content-Encoding header which indicates that transferred data has been compressed (gzip value).
I think that in your case
Content-Type: application/pdf
is enough. Additionally, you can set Content-Length header, but in my opinion, if you are building webservice (it's not http server / proxy server) Content-Type is enough. Please bear in mind that some specific headers (e.g. Transfer-Encoding) if not used appropriately, may cause unexpected communication issues, so if you are not 100% sure about usage of some header - if you really need it or not - just don't use it.
Notes in rfc2616 section 14.15 are explicit: https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
"Note: while the definition of Content-MD5 is exactly the same for
HTTP as in RFC 1864 for MIME entity-bodies, there are several ways
in which the application of Content-MD5 to HTTP entity-bodies
differs from its application to MIME entity-bodies. One is that
HTTP, unlike MIME, does not use Content-Transfer-Encoding, and
does use Transfer-Encoding and Content-Encoding. Another is that
HTTP more frequently uses binary content types than MIME, so it is
worth noting that, in such cases, the byte order used to compute
the digest is the transmission byte order defined for the type.
Lastly, HTTP allows transmission of text types with any of several
line break conventions and not just the canonical form using CRLF."
As been answered before and also here, a valid Content-Transfer-Encoding HTTP response header does not exist. Also the known headers Content-Encoding and Transfer-Encoding have no appropriate value to express a Base64 encoded response body.
Starting from here, no client would expect a response declared as application/pdf to be encoded as Base64! If you wand to do so, better use a different content type like:
Content-Type: application/pdf+base64
In this case, a client would know some Base64 encoded data is coming (the basic subtype is the suffix after the plus sign) and has a hint there is PDF in there.
Even this is a little hacky (+base64 is no official media type suffix) but at least would somehow meet some standards. Better use a custom content type than misusing standard HTTP headers!
Of course no browser would be able to directly open such a response anyway. Maybe your project should consider creating another endpoint offering a binary PDF response and marking this one deprecated.
What HTTP response headers are required to be sent from server to the client?
I working to optimize the HTTP response headers to minimize the HTTP response overhead. I know "overhead" is somewhat exaggerated, but I like a clean output.
I see a lot of websites sending redundant cache headers, etc..
e.g.
It is redundant to specify both Expires and Cache-Control: max-age, or to specify both Last-Modified and ETag.
Source
HTTP/1.1: Header Field Definitions
It depends on what you define as being required: there are no header fields that must be sent with every response no matter what the circumstances are, but there are header fields that you really should send. The only header field that comes close is Date, but even it has circumstances under which it is not required.
In the parlance of RFC 2119, the term MUST means that something is a requirement of the specification and not meeting the requirement would be invalid. There are no header fields defined by RFCs 7230, 7231, 7232, 7233, 7234, or 7235 that MUST be sent by an origin server in all cases.
The following headers, for example, can be omitted (though you probably should send them):
7.1.1.2. Date
An origin server MUST NOT send a Date header field if it does not
have a clock capable of providing a reasonable approximation of the
current instance in Coordinated Universal Time. An origin server MAY
send a Date header field if the response is in the 1xx
(Informational) or 5xx (Server Error) class of status codes. An
origin server MUST send a Date header field in all other cases.
Note the last sentence of the quote. The Date header field MUST be sent if the origin server is capable of providing a "reasonable approximation" of the date in UTC, but there is nothing stopping a server from misrepresenting itself.
7.4.2. Server
An origin server MAY generate a Server field in its responses.
3.3.2. Content-Length
Aside from [a finite number of predefined cases], in the absence of
Transfer-Encoding, an origin server SHOULD send a Content-Length
header field when the payload body size is known prior to sending the
complete header section.
On the subject of Content-Length and Transfer-Encoding, note that neither can be sent, in which case the length of the response is "determined by the number of octets received prior to the server closing the connection."
3.1.1.5. Content-Type
If a Content-Type header field is not present, the recipient
MAY either assume a media type of application/octet-stream
(RFC2046, Section 4.5.1) or examine the data to determine its type.
There are circumstances under which particular headers can be required, for example:
An origin server that does not support persistent connections MUST send the Connection: close in every response that does not have a 1xx status code.
An origin server MUST generate an Allow header in a 405 (Method Not Allowed) response.
An origin server generating a 401 (Unauthorized) response MUST send a WWW-Authenticate header field containing at least one challenge.
It depends on the specifics of the response, but generally, a response from an origin server should have:
Date
Content-Type
Server
and either Content-Length, Transfer-Encoding or Connection: close.
If you want to do caching, add Cache-Control (e.g., with max-age); Expires isn't generally necessary any more. If you want clients to be able to validate, add Last-Modified or ETag.