http.Server Serve method hangs when calling Shutdown immediately - http

Question

I have an issue with Go's http.Server, which I'm embedding in a struct that is supposed to control the server startup and shutdown. The struct looks like this:
type HTTPListen struct {
Consumers []pipeline.Consumer
Cfg HTTPListenConfig
Srv *http.Server
Logger log.Logger
wg *sync.WaitGroup
mu sync.Mutex
state State
}
The issue is that in my test code, I call my struct's Start() method (which in turn runs the Serve() method on the http.Server), check a few vars, and then call Stop(), whitch Shutdown()s the server and then waits for the http.Server to exit (return err from the Serve() method).
Now, for some reason, the Serve() method seems to just hang on the WaitGroup.Wait(), when I try to shutdown the server immediately after starting. When I add a short pause (tried 100ms), or when running the tests with the race detector, It works just fine.
Not sure if it matters, but there are no incoming requests between calling Serve() and Shutdown().
EDIT: link to a playground minimal example. If you comment out the time.Sleep call, the program hangs.
Here is the relevant code for the two methods:
func (h *HTTPListen) Start() error {
h.Logger.Log("msg", "starting HTTPListen input")
addr := h.Cfg.ListenAddr
ln, err := net.Listen("tcp", addr)
if err != nil {
h.Logger.Log("msg", "failed to create listener on tcp/"+addr+": "+err.Error())
h.setState(StateFailed)
return err
}
h.wg.Add(1)
go func() {
defer h.wg.Done()
err := h.Srv.Serve(ln)
h.Logger.Log("msg", "HTTP server stopped: "+err.Error())
}()
h.setState(StateStarted)
h.Logger.Log("msg", "HTTPListen input started")
return nil
}
Stop method:
func (h *HTTPListen) Stop() error {
h.Logger.Log("msg", "stopping HTTPListen input")
ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
defer cancel()
if err := h.Srv.Shutdown(ctx); err != nil {
h.Logger.Log("msg", "HTTP server shutdown deadline expired")
}
h.wg.Wait()
h.setState(StateStopped)
h.Logger.Log("msg", "HTTPListen input stopped")
return nil
}
Log output:
kwz#cyclone ~/s/stblogd> go test -v ./pkg/pipeline/input/ -run TestHTTPListen_StartStop
=== RUN TestHTTPListen_StartStop
msg="starting HTTPListen input"
msg="HTTPListen input started"
msg="stopping HTTPListen input"
... hangs indefinitely
Log output when running tests with the race detector:
kwz#cyclone ~/s/stblogd> go test -race -v ./pkg/pipeline/input/ -run TestHTTPListen_StartStop
=== RUN TestHTTPListen_StartStop
msg="starting HTTPListen input"
msg="HTTPListen input started"
msg="stopping HTTPListen input"
msg="HTTP server stopped: http: Server closed"
msg="HTTPListen input stopped"
--- PASS: TestHTTPListen_StartStop (0.00s)
PASS
ok stblogd/pkg/pipeline/input 1.007s
I'm tempted to just slap a short delay on the test and call it a day, but I would like to know why it behaves like this.

Answer 1

This is a known issue, see this thread:
https://github.com/golang/go/issues/20239
Hopefully they will fix it soon but for now it sounds like adding a short delay in your test is the simplest solution - it probably doesn't come up in real world use much because you won't trigger a shutdown so soon after starting.