I have some http service which sends headers showing progress while preparing response.
So it looks like
X-progress: 10%
X-progress: 20% (after some time)
...
X-progress: 100% (after more time)
And only after that it start sending the response body.
Is there a way to get that progress info in golang http client? Kinda OnHeaderArrived callback?
I can do it on low level just with simple TCP client but it means i will need to reimplement a lot of HTTP protocol stuff. May there is a way to use net/http above custom Dial connection?
Related
I am working on writing a resilient client for HTTP/2.
I am wondering what should be the behavior of the client, if the server sent a PUSH_PROMISE and then failed to send the PUSH_RESPONSE, related to that PUSH_PROMISE ?
I went through the HTTP/2 spec, about the Push Response, but it does not state what should we do in such scenarios.
Should we send the original request again, if the push response is not received ? If the original request sent successfully, sending it again may cause issues, isn't it ?
Or should we ignore the PUSH_PROMISE and continue ? In that case, say server promised to send a file, and did not send it, what will happen ?
Is there a defined way to resolve this ?
The client is certainly free to request the same resource again. Consider, for example, that the server has no way to know if the client is making a simultaneous request for the same resource when the server sends the PUSH_PROMISE.
Client Server
------ ------
HEADERS[sid:1, GET /]
HEADERS[sid:1, /], DATA [sid:1], PUSH_PROMISE[sid:2]
HEADERS[sid:3, GET /css] HEADERS[sid:2, /css], DATA[sid:2]
HEADERS[sid:3, /css], DATA[sid:3]
The standard way for the client to then cancel the push would be to reset the promised stream via a RST_STREAM.
PUSH PROMISE - All server push streams are initiated via PUSH-PROMISE frames, which signal the server’s intent to push the described resources to the client and need to be delivered ahead of the response data that requests the pushed resources. The simplest strategy to satisfy this requirement is to send all PUSH-PROMISE frames, which contain just the HTTP headers of the promised resource, ahead of the parent’s response.
PUSH_PROMISE method used to apply HTTP/2 server push because the server creates the PUSH_PROMISE frame to the response part of a normal browser- initiated stream. Response objects with the context of a request which has a HTTP connection is used to server push. for example, under the Page_load method of application which has HTTP connection can be used to apply Response.PUSHPROMISE for push all the relevant scripts, styles and images without the client having to request each one explicitly
I'm a bit rusty on nuances of the HTTP protocol and I'm wondering if it can support publish/subscribe directly?
HTTP is a request reponse protocol. So client sends a request and the server sends back a response.
In HTTP 1.0 a new connection was made for each request.
Now HTTP 1.1 improved on HTTP 1.0 by allowing the client to keep the connection open and make multiple requests.
I realise you can upgrade an HTTP connection to a websocket for fast 2 way communications. What I'm curious about is whether this is strictly necessary?
For example if I request a resource "http://somewhere.com/fetch/me/slowly"
Is the server free to reply directly twice?
Such as first with a 202 accepted
and then shortly later with the content when it is ready,
but without the client sending an additional request first?
i.e.
Client: GET http://somewhere.com/fetch/me/slowly
Server: 202 "please wait..."
Server: 200 "here's your document"
Would it be correct to implement a publish/subscribe service this way?
For example:
Client: http://somewhere.com/subscribe
Server: item 1
...
Server: item 2
I get the impression that this 'might' work because clients will typically have an event loop watching the connection but is technically wrong (because a client following the protocol need not be implemented that way).
However, if you use chunked transfer encoding this would work.
HTTP/2 seems to allow this as well but I'm not clear whether something changed to make it possible.
I haven't seen much discussion of this in relation to pub/sub so what if anything is wrong with using plain HTTP/1.1 with or without chunked encoding?
If this works why do you need things like RSS or ATOM?
A HTTP request can have multiple 'responses', but the responses all have statuscodes in the 1xx range, such as 102 Processing.
However, these responses are only headers, never bodies.
HTTP/1.1 (like 1.0 before it) is a request/response protocol. Sending a response unsolicited is not allowed. HTTP/2 is a frames protocol which adds server push which allows the server to offer extra data and handle multiple requests in parallel but doesn't change its request/response nature.
It is possible to keep a HTTP connection open and keep sending more data though. Many (audio, video) streaming services will use this.
However, this just looks like a continuous body that keeps on streaming, rather than many multiple HTTP responses.
If this works why do you need things like RSS or ATOM
Because keeping a TCP connection open is not free.
I am using HTTP long polling for pushing server events to a client.
On the client side, I send a long polling request to the server and block there waiting for a event from the server.
On the server side, we used the cometd framework (I am on the client side, do not really know much about the server side).
The problem is, after sometime, the connection is broken and the client can not detect this, so it blocks there forever. We are trying to implement some kind of heartbeat message, which will be sent every N minutes to keep the connection active. But this does not seem to work.
My question is: does HTTP long polling support heartbeat messages? As far as I understand, HTTP long polling only allows the server to send one event and will close the connection immediately thereafter. The client must reconnect and send a new request in order to receive the next event. Is it possible that the server sends heartbeat messages every N minutes while still keep the connection open until a real server event happens?
If you use the CometD framework, then it takes care of notifying the application (both on client and on server) about when the connection is broken, and it does send heartbeat messages.
What you call "HTTP long polling" is just a normal HTTP request, so in itself does not support heartbeat messages.
You can use HTTP long polling requests to implement heartbeat messages, and this is what CometD does for you under the covers.
In CometD, the response to a HTTP long poll request may deliver multiple messages, and the connection will not be closed afterwards. The client will send another HTTP long poll request without the need to reconnect, possibly reusing the previous connection.
CometD offers to your application a higher level API that is independent from the transport, so you can use WebSocket rather than HTTP, which is way more efficient, without changing a single line in your application.
You need to use the CometD libraries both on client (javascript and java) and on server, and everything will just work.
I have grid that needs to be auto updated every minute. I want to update grid asynchronously so that web page does not send any request to server. Only the server will know when to sent new lets say JSon data to client. Is this possible? Can I send data to client with out pinging the server?
Thanks.
No. You'd have to use some kind of open socket, which is a very low-level form of pinging anyway. The standard is to simply have a frequent but very short JSON request to check for new data.
Edit- There is WebSocket, but it appears that the implementation on the server side is more advanced & you'd be crippling your audience reach. Just do frequent, short JSON requests.
No, you have to send a HTTP request to get a response. The delay between the request and the response can be as long as you want, however (so please don't aggressively poll for updates):
http://en.wikipedia.org/wiki/Push_technology#Long_polling
You simply make a request, wait for it to complete (when something happens), start another request immediately and then process the response.
This way, the server always has a request ready which it can respond to in order to "push" to the browser (or one will shortly be made).
If I make multiple HTTP Get Requests to the same server and get HTTP 200 OK responses to each one how do I tell which request maps to which response using Wireshark?
Currently it looks like an http request is made, and the next HTTP 200 OK response is quickly received so everything is in a the proper sequence. I have seen things to the contrary however. For example using the Google Maps API v2 I've made several requests for location information and then the information is received in an arbitrary order (closely resembling the order in which I requested it, but not necessarily perfect.)
So my intuition is I cannot assume that my responses will be received in a specific order, even though they may be in order most of the time. So I'm wondering how I can determine this order from the response.
Update: Clarification as to what I need. I just need to know that the server has received the request. It seems like I need to do this by looking at sequence numbers and perhaps even ACKS. The reasoning behind this approach is I'm basically observing a web app and checking it is sending the information and the information is being received.
Update: This has nothing to do with wireshark specifically. I believe it is confusing people so I removing it from the title. It has to do with the HTTP protocol on top of the TCP/IP protocol and how we map responses to requests.
Thanks.
After you have stopped capturing packets follow this steps:
position the cursor on a GET request
Open the Analyze menu
click "Follow TCP Stream"
You get a new window with requests and responses in sequence.
While I was googling for a complete different question, I saw this one and I think I can provide a more complete answer :
HTTP dictates that responses must arrive in the order they were requested, Therefore, if you are looking at a single TCP connection at a given time you should be seeing :
Request ; Response ; Request ; Response ...
Also in HTTP/1.1, there is support for "Pipeline" where the client doesn't have to wait for responses to arrive in order to issue the next request. What could be observed in such cases is :
Request ; Response ; Request ; Request ; Response ; Response ; Request ; Response
In the HTTP response itself, there is no reference to the specific request that triggered it.
Filipo's suggestion is classic when debugging / observing a single TCP connection, but, when observing multiple TCP connections, you can't click the follow TCP Stream because you'd have to do it for each connection.
If you have many TCP connections, and many requests/responses you will have to look at TCP Source port in the request packet, and the TCP dest port in the response packet to know which response is related to each tcp connection, and then apply the HTTP request/response order rules.
Also, Wireshark CAN decompress the response body, and it will do it automatically if all the response body has arrived, but it will do so NOT in the Follow TCP Stream.
I always use Wireshark to debug HTTP.
Seems like this ability is not provided by the HTTP protocol at the application layer so I must go down to the transportation layer to determine this. In my case the TCP/IP layer using sequence numbers.
HTTP only presumes a reliable
transport; any protocol that provides
such guarantees can be used; the
mapping of the HTTP/1.1 request and
response structures onto the
transport data units of the protocol
in question is outside the scope of
this specification.
Read more:
http://www.faqs.org/rfcs/rfc2616.html#ixzz0e20kxKcz
Don't use Wireshark to debug HTTP, use an HTTP debugger such as Fiddler2