I want to avoid ever getting chunked encoded HTTP server response from (conforming) HTTP server. I am reading RFC 2616 section "14.39 TE" and it seems to me that I could avoid it by specifying TE: chunked;q=0. If I cannot avoid the chunked encoding, I want do avoid the trailers. Will specifying TE: trailers;q=0 work?
From rfc2616 - Hypertext Transfer Protocol -- HTTP/1.1 in section 3.6.1 Chunked Transfer Coding:
All HTTP/1.1 applications MUST be able to receive and decode the
"chunked" transfer-coding, and MUST ignore chunk-extension extensions
they do not understand.
This is still the case in the updated RFC 7230 - Hypertext Transfer Protocol (HTTP/1.1): Message Syntax and Routing in section 4.1. Chunked Transfer Coding although in a slightly different wording:
A recipient MUST be able to parse and decode the chunked transfer
coding.
So if you want to be conform to HTTP/1.1, you will have to accept chunked encoding.
##Update##
As for the trailers: I think if you don't send a TE header field in your request, a conforming server shouldn't send you any trailers. If it still sends trailers you are probably save to ignore them (again section 3.6.1):
A server using chunked transfer-coding in a response MUST NOT use the
trailer for any header fields unless at least one of the following is
true:
a) the request included a TE header field that indicates "trailers" is
acceptable in the transfer-coding of the response, as described in
section 14.39; or,
b) the server is the origin server for the response, the trailer
fields consist entirely of optional metadata, and the recipient
could use the message (in a manner acceptable to the origin server)
without receiving this metadata. In other words, the origin server
is willing to accept the possibility that the trailer fields might
be silently discarded along the path to the client.
Related
I'm writing a HTTP client, and wondering if HTTP responses are required to have any headers?
So is the following a valid HTTP response? (where \r\n == CRLF). How should a client handle it?
HTTP/1.1 200 OK\r\n
\r\n
The related questions Http response with no http header and What HTTP response headers are required are similar, but I think they are subtly-but-crucially different. They are both from the point of view of a server generating headers, rather than a parser parsing a response.
For example, a detailed answer to the current question could include how the client should interpret the above minimal response: whether it should accept the 200 response or abort, and what it should do with the underlying connection in terms of keeping it open or closing it.
The syntax of the message is defined in RFC 7231 (see https://greenbytes.de/tech/webdav/rfc7230.html#http.message), and that says that header fields are syntactically optional. So a message parser should accept that.
I have a question on usage of Content-Encoding and Transfer-Encoding:
Please let me know if my below understanding is right:
Client in its request can specify which encoding types it is willing to accept using accept-encoding header. So, if Server wishes to encode the message before transmission, eg. gzip, it can zip the entity (content) and add content-encoding: gzip and send across the HTTP response. On reception, client can receive and decompress and parse the entity.
In case of Transfer Encoding, Client may specify what kind of encoding it is willing to accept and perform its action on fly. i.e. if Client sends a TE: gzip; q=1, it means that if Server wishes, it can send a 200 OK with Transfer-Encoding: gzip and as it tries sending the stream, it can compress and send across, and client upon receiving the content, can decompress on fly and perform its parsing.
Is my understanding right here? Please comment.
Also, what is the basic advantage of compressing the entity on fly vs compressing the entity first and then transmitting it across? Is transfer-encoding valid only for chunked responses as we do not know the size of the entity before transmission?
The difference really is not about on-the-fly or not -- Content-Encoding can be both pre-computed and on the fly.
The differences are:
Transfer Encoding is hop-by-hop, not end-to-end
Transfer Encodings other than "chunked" (sadly) aren't implemented in practice
Transfer Encoding is on the message layer, Content Encoding on the payload layer
Using Content Encoding affects entity tags etc.
See http://greenbytes.de/tech/webdav/rfc7230.html#transfer.codings and http://greenbytes.de/tech/webdav/rfc7231.html#data.encoding.
I have a doubt regarding sending of mime attachments over HTTP:
in http specs the following is quoted :
“C.4 No Content-Transfer-Encoding: HTTP does not use the Content-Transfer-Encoding (CTE) field of RFC 1521. Proxies and gateways from MIME-compliant protocols to HTTP must remove any non-identity CTE ("quoted-printable" or "base64") encoding prior to delivering the response message to an HTTP client. Proxies and gateways from HTTP to MIME-compliant protocols are responsible for ensuring that the message is in the correct format and encoding for safe transport on that protocol, where "safe transport" is defined by the limitations of the protocol being used. Such a proxy or gateway should label the data with an appropriate Content-Transfer-Encoding if doing so will improve the likelihood of safe transport over the destination protocol.”
Does this mean that specifically for sending MIME attachments only over http, we shouldn't specify content-transfer-encoding as quoted-printable or base64 ?
Also, what is the behavior of conetent-transfer-encoding when i send such attachments over other transports like JMS, or over Mail? For example in a SOAP over JMS message?
Also the found following relevant from RFC 4130 :
“5.2. Unused MIME Headers and Operations
5.2.1. Content-Transfer-Encoding Not Used in HTTP Transport
HTTP can handle binary data and so there is no need to use the content transfer encodings of MIME [1]. This difference is discussed in [3], Section 19.4.5. However, a content transfer encoding value of binary or 8-bit is permissible but not required. The absence of this header MUST NOT result in transaction failure. Content transfer encoding of MIME body parts within the AS2 message body is also allowed.”
So i am basically thoroughly confused over the behavior of mime attachments specific to the HTTP protocol, and would like to get its behavior clarified.
HTTP is not MIME, it just borrows from the MIME message format. Payloads in HTTP are binary, and there simply is no Content-Transfer-Encoding header field. You can specify it, but it has zero effect and keeps distracting people looking at wire traces.
Can some experts explain the differences between the two? Is it true that chunked is a streaming protocol and multipart is not? What is the benefit of using multipart?
More intuitively,
Chunking is a way to send a single message from server to client, where the server doesn't have to wait for the entire response to be generated but can send pieces (chunks) as and when it is available. Now this happens at data transfer level and is oblivious to the client. Appropriately it is a 'Transfer-Encoding' type.
While Multi-part happens at the application level and is interpreted at the application logic level. Here the server is telling client that the content , even if it is one response body it has different logical parts and can be parsed accordingly. Again appropriately, this is a setting at 'Content-Type' as the clients ought to know it.
Given that transfer can be chunked independent of the content types, a multi-part http message can be transferred using chunked encoding by the server if need be.
Neither is a protocol. HTTP is the protocol. In fact, the P in HTTP stands for Protocol.
You can read more on chunked and multipart under Hypertext Transfer Protocol 1.1
Chunked is a transfer coding found in section 3.6 Transfer Codings.
Multipart is a media type found in section 3.7.2 Multipart Types a subsection of 3.7 Media Types.
Chunked also affects other aspects of the protocol such as the content-length as specified under 4.4 as chunked must be used when message length cannot be predetermined (mainly when delivering dynamic content).
From 14.41 (Transfer-Encoding header field)
The Transfer-Encoding general-header field indicates what (if any)
type of transformation has been applied to the message body in order
to safely transfer it between the sender and the recipient. This
differs from the content-coding in that the transfer-coding is a
property of the message, not of the entity.
Put more simply, chunking is how you transfer a block of data, while multipart is the shape of the data.
I can't tell from RFC 2616 whether an HTTP client should accept a 204 No Content response that includes Content-Length: 0 or Transfer-Encoding: chunked headers. These headers seem to break some HTTP clients and proxies, which apparently attempt to read the empty response body, but the spec reads:
Any response message which "MUST NOT" include a message-body (such as
the 1xx, 204, and 304 responses and
any response to a HEAD request) is
always terminated by the first empty
line after the header fields,
regardless of the entity-header fields
present in the message.
To me "regardless of the entity-header fields" implies that clients should tolerate this state of affairs. The Erlang HTTP library chose this interpretation. However, lighthttpd and IBM chose the opposite interpretation -- that the server should not include these headers for responses that are prohibited from having bodies.
So should the web application remove those headers from the response, or should the network infrastructure and clients tolerate those headers on 204 No Content, 304 Not Modified, etc. ?
According to the RFC7230 (please see https://www.rfc-editor.org/rfc/rfc7230#section-3.3.1) at the end of Page 29 it states:
A server MUST NOT send a Transfer-Encoding header field in any
response with a status code of 1xx (Informational) or 204 (No
Content).
Therefore the web application should not send those headers with the response.
I can't tell you what is correct, in that I don't know the HTTP protocol in that detail.
However, you got to ask yourself:
Can I change my application to remove those headers?
or can I change the consumers of my application to ignore those headers?
In my opinion, the application should not send those headers in that case.