PRI method in http2 implementation causing issue - http

I am trying to extend a server with HTTP/2 which already supports HTTP/1.1 with TLS v1.2. I am writing it in Go where I define tls config like this -
tlsConfig := &tls.Config{
Certificates: []tls.Certificate{cert},
ServerName: "mysrvr",
NextProtos: []string{"h2", "http/1.1", "http/1.0"},
Time: time.Now,
Rand: rand.Reader,
}
As is evident, I have used "h2" string to set up ALPN handshake.
Now when I make request via curl, I receive this request -
$ curl -v https://127.0.0.1:8000 -k --http2
When I parse request, it shows a PRI method being sent first instead of GET -
HTTP/2.0
PRI
I got some idea on PRI method from https://www.rfc-editor.org/rfc/rfc7540#page-78 wherein it says the following -
This method is never used by an actual client.
This method will appear to be used when an HTTP/1.1 server or
intermediary attempts to parse an HTTP/2 connection preface.
My question now is why was PRI request sent, when clearly the server supports HTTP/2? Do I need to parse it and respond with empty SETTINGS frame in accordance with HTTP/2 spec or should the Go http2 runtime should have taken care of it?
I am using http.ReadRequest to parse client requests, but that doesn't seem to be working for HTTP/2 requests even when I ignore PRI requests (as suggested below).

The first message a HTTP/2 client should send is this PRI message. From the HTTP/2 specification:
In HTTP/2, each endpoint is required to send a connection preface as a final confirmation of the protocol in use and to establish the initial settings for the HTTP/2 connection. The client and server each send a different connection preface.
The client connection preface starts with a sequence of 24 octets, which in hex notation is:
0x505249202a20485454502f322e300d0a0d0a534d0d0a0d0a
That is, the connection preface starts with the string PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n). This sequence MUST be followed by a SETTINGS frame (Section 6.5), which MAY be empty.
...
Note: The client connection preface is selected so that a large proportion of HTTP/1.1 or HTTP/1.0 servers and intermediaries do not attempt to process further frames.
The point of this message is that it is a fake HTTP/1-like message so any server which is not HTTP/2 aware should response with an error.
Any HTTP/2 server should expect this message to be sent, and then should just ignore it, and the carry on speaking HTTP/2.
In fact if this message is NOT sent, then the server should treat this as an error and not continue:
Clients and servers MUST treat an invalid connection preface as a connection error (Section 5.4.1) of type PROTOCOL_ERROR. A GOAWAY frame (Section 6.8) MAY be omitted in this case, since an invalid preface indicates that the peer is not using HTTP/2.

Related

Is a standard-compliant (RFC 7230) HTTP-server allowed to respond with only 400 Bad Request?

A question that arose while reading the RFC 7230: Hypertext Transfer Protocol (HTTP/1.1): Message Syntax and Routing was "what is the simplest possible server that this would be compliant with this standard". Searching for "MUST" in the document, it seems to me that the only responses a server has to return is a 400-error, in the case of the request being malformed:
A
server MUST reject any received request message that contains
whitespace between a header field-name and colon with a response code
of 400 (Bad Request).
(3.2.4. Field Parsing)
If a Transfer-Encoding header field
is present in a request and the chunked transfer coding is not
the final encoding, the message body length cannot be determined
reliably; the server MUST respond with the 400 (Bad Request)
status code and then close the connection.
(3.3.3. Message Body Length .3)
If this is a
request message, the server MUST respond with a 400 (Bad Request)
status code and then close the connection.
(3.3.3. Message Body Length .4)
among others.
There are some cases where the standard dictates that the server MUST return a certain status code, but they all seem to be made optional by some stronger clause, for example:
If a server receives both an Upgrade and an Expect header field with
the "100-continue" expectation (Section 5.1.1 of [RFC7231]), the
server MUST send a 100 (Continue) response before sending a 101
(Switching Protocols) response.
(6.7 Upgrade)
being made optional by
A server MAY ignore a received Upgrade
header field if it wishes to continue using the current protocol on
that connection. Upgrade cannot be used to insist on a protocol
change.
(6.7 Upgrade)
All of this leads me to believe that a 400-only server is technically allowed by the standard.
This does seem rather odd to me, I thought a 400-header meant that the request itself was malformed in some way and that a server had to respond with some other error iff the request was well-formed but invalid in some other way.
Have I missed something in the standard, or some other relevant standard, or is a 400-only server allowed?
(As a side note, section 2.6 states
A server can send a 505
(HTTP Version Not Supported) response if it wishes, for any reason,
to refuse service of the client's major protocol version.
which leads me to believe a 505-only server would actually be allowed, although quite boring)

Should gRPC server-side half-closing implicitly terminate the client?

In the http2-spec, the scenario where the server half-closed the stream (server sent http2.END_STREAM), the client is still allowed to send data (since it's half-closed).
Consider the following gRPC scenario:
Client opens bidi-stream to server and starts sending data
Server closes the response stream and sends the status trailers (translates to sending http2.END_STREAM)
Client continues to send data
Are the semantics well-defined in gRPC?
Possible ways:
Follow the http2-spec: The client is allowed to continue to send data that is processed by the server.
Not follow the http2-spec: The client-connection is implicitly terminated if the server closes the stream.
NOTE: I just tested and it looks like gRPC for Java follows variant "not follow the http2-spec", i.e. if the server closes the downwards stream, also the upwards stream is closed.
In the semantics of gRPC, when the server sends a status, it means that the entire call is complete. The official servers implementations send RST_STREAM in addition to END_STREAM to shut down the stream in both directions at the HTTP/2 protocol level, in accordance with this part of Section 8.1 of the HTTP/2 RFC:
A server can send a complete response prior to the client sending an entire request if the response does not depend on any portion of the request that has not been sent and received. When this is true, a server MAY request that the client abort transmission of a request without error by sending a RST_STREAM with an error code of NO_ERROR after sending a complete response (i.e., a frame with the END_STREAM flag).
When servers do not do this, the gRPC protocol does not prohibit the client sending more data after receiving a status but the server will not process it so there is no reason to do so. Because of this, when the official gRPC clients receive a status they consider the call to be complete and stop sending data.

chunked encoding and connection:close together

I'm a bit confused about what to do with the "Connection" header when using chunked-encoding. Shall the "Connection" header not be added (or set to keep-alive, which is the same as we're talking about HTTP 1.1) or is this authorized to set it to Connection:close
Sending an empty chunk, means the end of the transfer, but does it mean the end of the connection? I thought that I would need to add Connection:close when my intention is to close the connection after an empty chunk has been sent where if I don't add the Connection header, it would remain open
Thanks
Message Length and Connection Management are two different things that really have nothing to do with each other (except in the one case where the Message Length is completely unknown and closing the connection is the only possible way to denote EOF, but that rarely happens, and is not your situation).
Chunking only applies to Message Length, where a 0-length chunk denotes EOF. If the connection is closed prematurely before EOF is reached, the Message is incomplete, and the receiver can decide whether to keep/process it or not.
The Connection header is used by the client to specify whether it wants the server to close the connection, or leave it open, after sending its response. The same header is used by the server to specify whether the connection is actually closing or staying open after the response has been sent.
Shall the "Connection" header not be added (or set to keep-alive, which is the same as we're talking about HTTP 1.1) or is this authorized to set it to Connection:close
That has nothing to do with chunking. Regardless of the format of the Message, the client and server should always indicate their intentions for the current connection, whether it should be closed or left open. That is done via the presence of, or lack of, a Connection header, depending on the HTTP version:
If HTTP 1.0 is used, the default behavior is close unless Connection: keep-alive is explicitly sent.
If HTTP 1.1+ is used, the default behavior is keep-alive unless Connection: close is explicitly sent.
If the client requests a keep-alive, the server decides whether or not to honor it. The server may either leave the connection open, or it may close the connection.
If the client requests a closure, the server MUST honor it and close the connection.
Sending an empty chunk, means the end of the transfer, but does it mean the end of the connection?
No. Only the Connection header does that. Especially in a keep-alive scenario, thus the connection is left open so the client can reuse the existing connection to send a new request after the transfer of the previous response is finished.
I thought that I would need to add Connection:close when my intention is to close the connection after an empty chunk has been sent
Correct, especially in HTTP 1.1+ where keep-alive is the default behavior.
where if I don't add the Connection header, it would remain open
The meaning of an omitted Connection header depends on the HTTP version being used, as described above.
A server is allowed to send Connection: close while using chunked transfer encoding, this should not cause clients to disconnect before the response is finished.
According to RFC2616 https://www.rfc-editor.org/rfc/rfc2616#section-14.10
For example,
Connection: close
in either the request or the response header fields indicates that
the connection SHOULD NOT be considered `persistent' (section 8.1)
after the current request/response is complete.
Since the response is not complete while chunks are being sent, the connection is not forbidden from persisting.

Is it possbile that request is sent but response is failed?

I'm new in programming and recently writing a hobby angular 2 app that make a network request within the application, The request is AJAX with http object.
During writing the application, I wonder.. is it possible the request that made by client application is accepted and processed by the server, but the server failed to make a response to the client due to connection error?
If that possible, how do I avoid multiple request being processed?
In the OSI model, HTTP is application level layer which is transported over transport layer i.e. TCP to the peer for sending OR receiving messages.
Transport layer has following attributes: reliable/non-reliable, connection/connection-less, flow-control, congestion, etc. To support these features TCP or UDP protocols are used.
Since we are transporting HTTP PDU, even if PDU gets dropped (due to various reasons), it will be retransmitted as TCP support sliding-window. And only after receiving FIN (final) from peer the connection is closed. Until then you can consider all your PDU will be transported faithfully. TCP also has timeouts and retries, they are configurable too.
But if server is dead or not responding then you will get appropriate error message while making HTTP request, because mere TCP connection itself will fail. Following are well defined error messages. Enable logging OR console traces to notice these errors.
you can check the connection at the time of response back.
Please also check the below points
Check the return type from the server.
Is your angular program is expecting the same type of response which is returned from the server?

HTTP: shutdown socket for writing after sending the request?

How is a socket used by an HTTP client properly closed after transmitting the request? Or does it have to remain open (bidirectionally) until the complete response has been received? If so, how is the end of the request body determined by the server?
According to http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.4, closing the socket is not an option for a request. That doesn't sound logical to me - why should a half-closed TCP connection be a problem for the server, if the client doesn't try to transmit anything after closing its half of the socket? The client can still receive data after all.
It seems to me that shutting down the write part of a socket would be a very practical way of letting the server know that the request has been finished. http://docs.python.org/howto/sockets.html#disconnecting even specifically mentions that use case.
If that's really the wrong way to do it, what's the alternative? Do I really always have to send a "Content-length" or use chunked transport to enable the server to properly find the end of a request? How does that work for requests with unknown body length?
Transfer-Encoding: chunked is specifically designed to allow sending data with an unknown body length, for both requests and responses. The end of the data is determined by receiving a chunk whose payload size is 0. If you do not send a chunked request, then you must send a Content-Length instead.
Are you talking about this?:
Closing the connection cannot be used to indicate the end of a request body, since that would leave no possibility for the server to send back a response.
I think the text is talking about full close, you can do a half(write)-close. I'm not sure that's a HTTP compilant way of doing it, but I would think most servers will accept it.
Regarding your second question, simply use chunked encoding:
All HTTP/1.1 applications that receive entities MUST accept the "chunked" transfer-coding (section 3.6), thus allowing this mechanism to be used for messages when the message length cannot be determined in advance.

Resources