How do I rewind an io.ReadCloser - http

Question

I'm perpetually caught out by reading an io.ReadCloser and then forgetting that I've read it before, and when I read it again, I get an empty payload. I wish there was some lint check for my stupidity. Nonetheless, I think I can use TeeReader, but it fails to meet my expectations here:
func main() {
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
buf := &bytes.Buffer{}
tee := io.TeeReader(r.Body, buf)
body, err := ioutil.ReadAll(tee)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
log.Println("body", string(body))
payload, err := httputil.DumpRequest(r, true)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
log.Println("dump request", string(payload))
w.WriteHeader(http.StatusOK)
})
log.Fatal(http.ListenAndServe(":8080", nil))
}
The body is missing from my "dump request" log line.
I.e. when I run curl -i -X POST --data '{"username":"xyz","password":"xyz"}' http://localhost:8080
I want the original request in full:
2019/01/14 11:09:50 dump request POST / HTTP/1.1
Host: localhost:8080
Accept: */*
Content-Length: 35
Content-Type: application/x-www-form-urlencoded
User-Agent: curl/7.63.0
{"username":"xyz","password":"xyz"}
What am I missing?

Answer 1

You cannot rewind an io.ReadCloser, unless the underlying value is also an io.ReadSeeker.
An io.ReadCloser, by very definition, has exactly two methods: Read and Close. So there is obviously there is no option to rewind.
An io.ReadSeeker, by contrast, has two methods: Read and Seek, the latter which allows rewinding.
If you need to accept only io.ReadClosers which are also seekable, you can easily combine these two:
type ReadSeekCloser interface {
io.Reader
io.Seeker
io.Closer
}
Now you can use your custom ReadSeekCloser type in place of an io.ReadCloser, and you'll have the option to rewind your reader.
Of course, few io.ReadClosers in the wild actually conform to this interface (os.File will be the main one that does). If you have an io.ReadCloser that does not implement the Seek method (such as a network stream), probably the easiest way to make it Seekable is to dump the contents to a file, then open that file. There would be other ways to make an in-memory buffer seekable (bytes.NewReader for instance), but variations will require reading the stream into memory or onto disk, first.

Answer 2

How do I rewind a io.ReadCloser in Go[...]?
You cannot. A ReadCloser can be read and closed. Unless the actual underlying type has some method to rewind you simply cannot.
(For your case you may just use the bytes.Buffer, possibly after adding a Close method via io/ioutil.ReadCloser as the Request.Body; but this is not "rewinding" but "replacing".)

Answer 3

https://golang.org/pkg/net/http/httputil/#DumpRequest
DumpRequest returns the given request in its HTTP/1.x wire representation. It should only be used by servers to debug client requests.
Clearly DumpRequest is for Dubug use.
But if you don't care about that. The godoc also metioned:
If body is true, DumpRequest also returns the body. To do so, it consumes req.Body and then replaces it with a new io.ReadCloser that yields the same bytes.
So you can call DumpRequest first then ReadAll from Body, cause the Body is still the same after DumpRequest.
func main() {
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
payload, err := httputil.DumpRequest(r, true)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
log.Println("dump request", string(payload))
body, err := ioutil.ReadAll(r.Body)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
log.Println("body", string(body))
w.WriteHeader(http.StatusOK)
})
log.Fatal(http.ListenAndServe(":8080", nil))
}

Answer 4

Request bodies aren't seek-able. There is no rewind, they aren't buffered in memory. When that data is read, it is read from the network stream. Imagine a large upload, buffering all of that data in memory by default would be wasteful.
That said, there are some things you can do to get the output you desire. It involves replacing r.Body after you've read it.
You mention wishing for a lint check. I've found that declaring a buffer and using only that buffer can help. If you like, you can even replace r.Body with that buffer. You will still need to remember to "rewind" with Seek.
func main() {
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
b := &bytes.Buffer{}
_, err := io.Copy(b, r.Body)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
bodyCopy := bytes.NewReader(b.Bytes())
log.Println("body", b.String())
r.Body = io.NopCloser(bodyCopy)
bodyCopy.Seek(0, 0)
payload, err := httputil.DumpRequest(r, true)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
log.Println("dump request", string(payload))
w.WriteHeader(http.StatusOK)
})
log.Fatal(http.ListenAndServe(":8080", nil))
}