This is from the POP3 RFC.
"Responses to certain commands are multi-line. In these cases, which
are clearly indicated below, after sending the first line of the
response and a CRLF, any additional lines are sent, each terminated
by a CRLF pair. When all lines of the response have been sent, a
final line is sent, consisting of a termination octet (decimal code
046, ".") and a CRLF pair. If any line of the multi-line response
begins with the termination octet, the line is "byte-stuffed" by
pre-pending the termination octet to that line of the response.
Hence a multi-line response is terminated with the five octets
"CRLF.CRLF". When examining a multi-line response, the client checks
to see if the line begins with the termination octet. If so and if
octets other than CRLF follow, the first octet of the line (the
termination octet) is stripped away. If so and if CRLF immediately
follows the termination character, then the response from the POP
server is ended and the line containing ".CRLF" is not considered
part of the multi-line response."
Well, i have problem with this, for example gmail sometimes sends the termination octet and then in the NEXT LINE sends the CRLF pair. For example:
"+OK blah blah\r\n"
"blah blah.\r\n"
"\r\n"
That's very rare, but it happens sometimes, so obviously i'm unable to determine the end of the message in such case, because i'm expecting a line that consists of '.\r\n'. Seriously, is Gmail violating the POP3 protocol or i'm doing something wrong? Also i have a second question, english is not my first language so i cannot understand that completely:
"If any line of the multi-line response begins with the termination octet, the line is "byte-stuffed" by pre-pending the termination octet to that line of the response.
Hence a multi-line response is terminated with the five octets "CRLF.CRLF"."
When exactly CRLF.CRLF is used? Can someone gives me a simple example? The rfc says that is used when any line of the response begins with the termination octet. But i don't see any lines that starts with '.' in the messages that are terminated with CRLF.CRLF. I checked that. Maybe i don't understand something, that's why i'm asking.
It would help very much if you posted the code you are using to read the socket. But I will attempt to answer the second part of your question with this example:
If the response is:
hello world\r\n
we are doing fine\r\n
.500 is same as one-half\r\n
this is the last line\r\n
the server must send it as:
hello world\r\n
we are doing fine\r\n
..500 is same as one-half\r\n
this is the last line\r\n
.\r\n
So you can see where it 'byte stuffed' an extra '.' so the '.500' can be distinguished as part of the response. Also the final five octets are '\r\n.\r\n'.
Related
Let's say this is the http header:
Content-length: 67728\r\n
Content-type: application/x-genericbytedata-octet-stream\r\n
\r\n
Does it include \0 at the end of the header or the "Content-length" bytes start directly after the last \n ?
There is no null byte at the end, so no, it's not included.
In complement.
The Body will start after the end of the headers. And of the headers is "\r\n\r\n", "\n\r\n", "\r\n\n" or "\n\n" (that is two valid end-of-line sequences).
Adding an "\0" somewhere in your headers will almost certainly makes the server reject your request (silently or not). Trying to inject NULL character in a request is usually considered as an attack attempt.
In HTTP line separator syntax is "\r\n", and usually this is what the parsers are seeking, not the NULL character. With one optional exception, in headers, where a line starting with a space may be considered as part of the previous header (this is the obs-fold syntax), so "X:B\r\n Z\r\n" is in fact the header "X" with value "B Z".
If a response to a GET request doesn't have the Content-Length or Transfer-encoding: chunked field, what happens? How does the client know when the message ends?
RFC 7230 section 3.3.3 contains a nice checklist of conditions for finding the message body size. Essentially it says the answer depends on what the status code is. The relevant conditions being #1 and #7.
"1. Any response ... with a 1xx
(Informational), 204 (No Content), or 304 (Not Modified) status
code is always terminated by the first empty line after the
header fields, regardless of the header fields present in the
message, and thus cannot contain a message body."
"7. Otherwise, this is a response message without a declared message
body length, so the message body length is determined by the
number of octets received prior to the server closing the
connection."
Its also worth knowing that the message may continue after the end of the body portion. RFC 7230 section 4.4 defines a Trailers feature where the payload may be followed by a second set of mime headers. If those exist the message ends where they do.
According to the specification
The length of a message body is determined by one of the following (in
order of precedence):
[...]
Otherwise, this is a response message without a declared message body length, so the message body length is determined by the number of octets received prior to the server closing the connection.
The client needs to detect that the connection is closed.
Suppose you have a plain content part:
Header: value
Header: value
content...
In HTTP, there is no requirement to add an additional "\r\n\r\n" at the end of the content (Should newline be included in http response content length?), it uses only content-length (or chunked) to determine the size of the message.
However the question is: is it necessary for email? I can't find the exact place in the (many) rfcs related to mail that define how a "normal" content part should end.
MIME content parts end just before the newline before the content separator. Thus in the following fragment
--moo
Foo
--moo
Bar
--moo--
... the first part lacks a trailing newline, whereas the second ends with one.
Pre-MIME messages were not explicitly standardized on this particular point; but due to the requirements of SMTP, it wasn't possible to transmit a message which didn't have a final newline at the end.
I don't clear about how to count `Content-Length' header in HTTP.
Take an example,
HEADER
...
Content-Type: text/html
(blank line `\r\n')
<html></html>
(blank line `\r\n')
This is a working http request sending an empty HTML page(correct me if any problem :-)). Then what should be the length of content? 15 or 17(take the blank line between header and sending entity into account)?
Thanks in advance. Best regards.
According to W3 Content-Lentgth is defined as followed:
The Content-Length entity-header field indicates the size of the
entity-body, in decimal number of OCTETs, sent to the recipient or, in
the case of the HEAD method, the size of the entity-body that would
have been sent had the request been a GET.
As far as I understand it, you have to count everything after the first line break. My answer to your question would be 15 then.
15 is the correct answer. That counts the line break at the END of the entity data, which means that line break is part of the entity, not the http protocol. DO NOT count the line break between the headers and entity.
I have a device that sends data to a server.
Data
[ Client ] == > [ Server ]
After the validation on the server I want to return a message:
OK
[ Client ] < == [ Server ]
Is there a standard "OK" message to return? And an "ERROR" message? How does it looks like? (e.g. ":0011", ":110F")
You've got to design an application-level protocol. TCP is a byte stream, so even the definition of "Data" in your client->server piece needs some protocol so that the receiver can know what bytes make up the data (when to stop reading).
A couple of common types of protocols are...
Length-delimited chunks. Every message starts with a 16 or 32-bit length prefix. Then that many bytes follow. The length needs to be in a defined byte order (see htons, ntohs, etc). Everyone who uses this protocol knows to read the length prefix then read that many bytes. Having defined that "chunk" on the network, you might put a header on the contents of the chunk. Maybe a message type (ACK, NAK, Data, etc) followed by some contents.
ASCII newline delimited. Each message is a line of ASCII (or UTF8, etc) text. It ends at a newline. Newline endings for the lines play the same role as the length prefix for the chunks above. You then define what's in each line (like space or comma-delimited ASCII/UTF8/whatever fields). Somewhere in that you'd define what data looks like, ACK, etc.
I'm sure you could come up with other ideas, but that's the basic job: defining your application-level protocol on top of TCP's byte stream.