Write to response body waits for client to read in Go? - http

Question

I'm using the httptest package in Go to test my application. Recently I noticed that one of my tests was failing to finish because my test wasn't reading the body of the response
func Test_AnyTest(t *testing.T) {
serve := newServer()
s := httptest.NewServer(serve)
defer s.Close()
testUrl := "/any/old/url"
c := &http.Client{}
r, _ := http.NewRequest("GET", s.URL+testUrl, new(bytes.Buffer))
r.Header.Add("If-None-Match", cacheVersion)
res, _ := c.Do(r)
if res.StatusCode == 304 {
t.Errorf("Should not have got 304")
}
}
The above code was blocking on the deferred call to s.Close() because there were outstanding http connections on the test server. My server has some code that was writing to the body (using the http.ResponseWriter interface). Turns out that this code was actually blocking until I read the body in my test.
A call like this did the trick.
ioutil.ReadAll(res.Body)
I don't mind making this call in my tests but I am concerned that a client that misbehaves could cause this behaviour and consume server resources. Does anyone know what's happening here? Is this expected behaviour in the wild or is it just a problem with the test framework?
Thanks!

Answer 1

From the http docs:
The client must close the response body when finished with it:
resp, err := http.Get("http://example.com/")
if err != nil {
// handle error
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
// ...
So your test is violating the contract of http.Client; adding a defer res.Close() should let the server know that the response channel is closed, and stop writing to it. The effect of your ioutil.ReadAll was the same, as it closes the Reader once it reaches EOF.
In the wild, if a client opens a connection and sends a request, the server just sends the response and then closes the connection. It doesn't wait around for the client to close it. If the client is too slow to ACK the IP packets that the server sends, the OS will eventually take care of timing out the connection, and the http.Server will respond by exiting the goroutine that served the request, no kittens harmed.