As I understand, TCP has a relatively reliable checksum mechanism, but there is a very small yet present chance of it returning a false positive.
Does HTTP use an application level checksum (perhaps a hash)? If not, why?
Yup (: the specs for HTTP/1.1 (RFC 2616) allow the use of a Content-MD5 header. Take a look at its section 14.15.
Copy-pasted:
The Content-MD5 entity-header field, as defined in RFC 1864 [23], is
an MD5 digest of the entity-body for the purpose of providing an
end-to-end message integrity check (MIC) of the entity-body.
Related
I was thinking about JSON/XML-RPC over HTTP vs TCP. In case of HTTP, the HTTP request and response provide a message framing mechanism. However, since TCP is a stream,
How are RPC messages framed?
Does the RPC spec dictate that?
Are there any other standards defining framing mechanisms?
Is there one mechanism more often used than the other?
I'm trying to guage this before inventing a framing mechanism.
There are many framing standards, HTTP being one of them. Websocket being another, on top of HTTP, more suited for bidirectional streams.
JSON-RPC 2.0 does not concern itself with transport, intentionally.
(1.0 had some transport specifics, that was removed in the new version.)
RFC 7484 provides a framing standard for "JSON text sequences": https://www.rfc-editor.org/rfc/rfc7464
Summary: Each JSON message is prefixed with a 0x1E byte (which can't appear unescaped in a JSON message) and is suffixed with 0x0A (linefeed).
Note, however, that this is not part of the JSON-RPC specification. There are libraries that support it as part of their JSON-RPC implementation.
I am testing the Last.fm api using row socket interface.
Now i noticed that some of api's http response have not contain a field Content-Length .
But I want to know is there a way to ask the server presenting it?
Because i can't take good care of this in my program elegantly.
Quoth the RFC:
7.2.2 Length
When an Entity-Body is included with a message, the length of that body may be determined in one of two ways. If a Content-Length header field is present, its value in bytes represents the length of the Entity-Body. Otherwise, the body length is determined by the closing of the connection by the server.
The right RFC to look at is RFC 7230 (Section 3.3.2).
And no, in HTTP/1.1 a client has to be able to process chunked encoding (which would be the only legitimate reason not to provide a Content-Length header field).
I have a doubt regarding sending of mime attachments over HTTP:
in http specs the following is quoted :
“C.4 No Content-Transfer-Encoding: HTTP does not use the Content-Transfer-Encoding (CTE) field of RFC 1521. Proxies and gateways from MIME-compliant protocols to HTTP must remove any non-identity CTE ("quoted-printable" or "base64") encoding prior to delivering the response message to an HTTP client. Proxies and gateways from HTTP to MIME-compliant protocols are responsible for ensuring that the message is in the correct format and encoding for safe transport on that protocol, where "safe transport" is defined by the limitations of the protocol being used. Such a proxy or gateway should label the data with an appropriate Content-Transfer-Encoding if doing so will improve the likelihood of safe transport over the destination protocol.”
Does this mean that specifically for sending MIME attachments only over http, we shouldn't specify content-transfer-encoding as quoted-printable or base64 ?
Also, what is the behavior of conetent-transfer-encoding when i send such attachments over other transports like JMS, or over Mail? For example in a SOAP over JMS message?
Also the found following relevant from RFC 4130 :
“5.2. Unused MIME Headers and Operations
5.2.1. Content-Transfer-Encoding Not Used in HTTP Transport
HTTP can handle binary data and so there is no need to use the content transfer encodings of MIME [1]. This difference is discussed in [3], Section 19.4.5. However, a content transfer encoding value of binary or 8-bit is permissible but not required. The absence of this header MUST NOT result in transaction failure. Content transfer encoding of MIME body parts within the AS2 message body is also allowed.”
So i am basically thoroughly confused over the behavior of mime attachments specific to the HTTP protocol, and would like to get its behavior clarified.
HTTP is not MIME, it just borrows from the MIME message format. Payloads in HTTP are binary, and there simply is no Content-Transfer-Encoding header field. You can specify it, but it has zero effect and keeps distracting people looking at wire traces.
What are the mechanisms/approaches to validate the UDP payload in case of SIP? SIP message doesn't contain size of header or the body, so how to verify that the payload is valid? In contrast, RTP indicates the size, so given the length value from UDP header it is possible to check RTP for validit and integrity. Can something similar be done for SIP?
Mark.
Your question has two parts:
How do I validate SIP headers? The only way to validate SIP headers is to parse them according to the rules of section 7.3.1 of RFC 3261. There are SIP parsers available for many different languages.
How do I validate the body of SIP messages? There is a mechanism built into SIP: the Content-Length header specifies the size of the body. In the general case, the body can contain an arbitrary MIME type, and no further validation rules apply.
Edit: Per Frank Shearar's comment below, Content-Length is not required for SIP messages conveyed via UDP. But if your UA supports it, you can take advantage of it.
Is it possible to set the content-length header and also use chunked transfer encoding? and does doing so solve the problem of not knowing the length of the response at the client side when using chunked?
the scenario I'm thinking about is when you have a large file to transfer and there's no problem in determining its size, but it's too large to be buffered completely.
(If you're not using chunked, then the whole response must get buffered first? Right??)
thanks.
No:
"Messages MUST NOT include both a Content-Length header field and a non-identity transfer-coding. If the message does include a non-identity transfer-coding, the Content-Length MUST be ignored." (RFC 2616, Section 4.4)
And no, you can use Content-Length and stream; the protocol doesn't constrain how your implementation works.
Well, you can always send a header stating the size of the file.
Something like response.addHeader("File-Size","size of the file");
And ignore the Content-Length header.
The client implementation has to be tweaked to read this value, but hey you can achieve both the things you want :)
You have to use either Content-Length or chunking, but not both.
If you know the length in advance, you can use Content-Length instead of chunking even if you generate the content on the fly and never have it all at once in your buffer.
However, you should not do that if the data is really large because a proxy might not be able to handle it. For large data, chunking is safer.
This headers can be cause of Postman Parse Error:
"Content-Length" and "Transfer-Encoding" can't be present in the response headers together.
Using parametrized ResponseEntity<?> except raw ResponseEntity in controller can fixed the issue.
The question asks:
Is it possible to set the content-length header and also use chunked transfer encoding?
The RFC HTTP/1.1 spec, quoted in Julian's answer, says:
Messages MUST NOT include both a Content-Length header field and a non-identity transfer-coding.
There is an important difference between what's possible, and what's allowed by a protocol. It is certainly possible, for example, for you to write your own HTTP/1.1 client which sends malformed messages with both headers. You would be violating the HTTP/1.1 spec in doing so, and so you'd imagine some alarm bells would go off and a bunch of Internet police would burst into your house and say, "Stop, arrest that client!" But that doesn't happen, of course. Your request will get sent to wherever it's going.
OK, so you can send a malformed message. So what? Surely on the receiving end, the server will detect the HTTP/1.1 protocol client-side violation, vanquish your malformed request, and serve you back a stern 400 response telling you that you are due in court the following Monday for violating the protocol. But no, actually, that probably won't happen. Of course, it's beyond the scope of HTTP/1.1 to prescribe what happens to misbehaving clients; i.e. while the HTTP/1.1 protocol is analogous to the "law", there is nothing in HTTP/1.1 analogous to the judicial system.
The best that the HTTP/1.1 protocol can do is dictate how a server must act/respond in the case of receiving such a malformed request. However, it's quite lenient in this case. In particular, the server does not have to reject such malformed requests. In fact, in such a scenario, the rule is:
If the message does include a non-identity transfer-coding, the Content-Length MUST be ignored.
Unfortunately, though, some HTTP servers will violate that part of the HTTP/1.1 protocol and will actually give precedence to the Content-Length header, if both headers are present. This can cause a serious problem, if the message visits two servers in sequence in the same system and they disagree about where one HTTP message ends and the next one starts. It leaves the system vulnerable to HTTP Desync attacks a.k.a. Request Smuggling.