Minimal http service test without resetting up routes? - http

Say I have a very simple Web service.
func main() {
http.HandleFunc("/", sanityTest)
log.Fatal(http.ListenAndServe(":8000", nil))
}
If I want to test it, I could minimally just have:
func ExampleTest() {
server := httptest.NewServer(http.DefaultServeMux)
defer server.Close()
resp, err := http.Get(server.URL)
if err != nil {
log.Fatal(err)
}
body, _ := ioutil.ReadAll(resp.Body)
fmt.Println(resp.StatusCode)
fmt.Println(resp.Header.Get("Content-Type"))
fmt.Println(string(body))
// Output:
// 200
// text/plain; charset=utf-8
// OK
}
But that will result in a 404, since it doesn't know about the routes. So what I've seen main_test.go code do, is re-setup the handles in the test file's init, like so:
func init() {
http.HandleFunc("/", sanityTest)
}
Which leads to duplication, and inevitably I have to create a function in main.go like:
func setupRoutes() {
http.HandleFunc("/", sanityTest)
}
Which I find a little ugly. Am I missing a trick to instantiate the routes from main.go and avoid the init?

You can re-use routes between tests and main.go file, also it's helpful if you want to mock something in your handlers (add a new argument to router() func below)
main.go:
func sanityTest(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "%s", "sanity test")
}
func router() *http.ServeMux {
h := http.NewServeMux()
h.HandleFunc("/", sanityTest)
return h
}
func main() {
http.ListenAndServe(":8080", router())
}
main_test.go:
func TestSanity(t *testing.T) {
tests := []struct {
name string
uri string
want string
}{
{"1", "/", "sanity test"},
}
ts := httptest.NewServer(router())
defer ts.Close()
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
url := ts.URL + tt.uri
resp, _ := http.Get(url)
respBody, _ := ioutil.ReadAll(resp.Body)
resp.Body.Close()
got := string(respBody)
if got != tt.want {
t.Errorf("got %s, Want %s", got, tt.want)
}
})
}
}

Related

Sending data in Chunks using single HTTP Post connection

I receive the contents of a file from a data source in chunks. As and when I receive the chunk I want to send the chunk data to a service using http POST request. And by keeping alive the same http POST connection used for sending the first chunk I want to send the remaining chunks of data.
I came up with the following code snippet to implement something similar.
Server-Side
func handle(w http.ResponseWriter, req *http.Request) {
buf := make([]byte, 256)
var n int
for {
n, err := req.Body.Read(buf)
if n == 0 && err == io.EOF {
break
}
fmt.Printf(string(buf[:n]))
}
fmt.Printf(string(buf[:n]))
fmt.Printf("Transfer Complete")
}
Client-Side
type alphaReader struct {
reader io.Reader
}
func newAlphaReader(reader io.Reader) *alphaReader {
return &alphaReader{reader: reader}
}
func (a *alphaReader) Read(p []byte) (int, error) {
n, err := a.reader.Read(p)
return n, err
}
func (a *alphaReader) Reset(str string) {
a.reader = strings.NewReader(str)
}
func (a *alphaReader) Close() error {
return nil
}
func main() {
tr := http.DefaultTransport
alphareader := newAlphaReader(strings.NewReader("First Chunk"))
client := &http.Client{
Transport: tr,
Timeout: 0,
}
req := &http.Request{
Method: "POST",
URL: &url.URL{
Scheme: "http",
Host: "localhost:8080",
Path: "/upload",
},
ProtoMajor: 1,
ProtoMinor: 1,
ContentLength: -1,
Body: alphareader,
}
fmt.Printf("Doing request\n")
_, err := client.Do(req)
alphareader.Reset("Second Chunk")
fmt.Printf("Done request. Err: %v\n", err)
}
Here I want that when I do alphareader.Reset("Second Chunk"), the string "Second Chunk" should be sent using the POST connection made earlier. But that is not happening. The connection gets closed after sending the First Chunk of data. Also I have not written the Close() method properly which I'm not sure how to implement.
I'm newbie to golang and any suggestions would be greatly helpful regarding the same.
A *strings.Reader returns io.EOF after the initial string has been read and your wrapper does nothing to change that, so it cannot be reused. You're looking for io.Pipe to turn the request body into an io.Writer.
package main
import (
"io"
"net/http"
)
func main() {
pr, pw := io.Pipe()
req, err := http.NewRequest("POST", "http://localhost:8080/upload", pr)
if err != nil {
// TODO: handle error
}
go func() {
defer pw.Close()
if _, err := io.WriteString(pw, "first chunk"); err != nil {
_ = err // TODO: handle error
}
if _, err := io.WriteString(pw, "second chunk"); err != nil {
_ = err // TODO: handle error
}
}()
res, err := http.DefaultClient.Do(req)
if err != nil {
// TODO: handle error
}
res.Body.Close()
}
Also, don't initialize the request using a struct literal. Use one of the constructors instead. In your code you're not setting the Host and Header fields, for instance.

Should I close request body if I use io.TeeReader() or io.ReadAll()?

I have a handler which I call from my main() function:
type requestBody struct {
Query string `json:"query"`
}
func main() {
r := chi.NewRouter()
r.Post("/api", MyHandler(superGraph, gqlGen))
}
func MyHandler(library *MyLibrary, next http.Handler) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
buf := bytes.NewBuffer(make([]byte, 0))
reader := io.TeeReader(r.Body, buf)
var reqBody requestBody
err := json.NewDecoder(reader).Decode(&reqBody)
if err != nil {
http.Error(w, "cannot read body", http.StatusBadRequest)
return
}
res, err := library.DoSomething(...)
if err != nil {
log.Error(err)
err := r.Body.Close()
log.ErrorIf(err)
r.Body = ioutil.NopCloser(buf)
next.ServeHTTP(w, r)
return
}
render.JSON(w, r, res) // go-chi "render" pkg
}
}
QUESTION
Do I need the below line?
err := r.Body.Close()
I know https://stackoverflow.com/a/42533540/10088259:
A request body does not need to be closed in the handler. From the http.Request documentation:
The Server will close the request body. The ServeHTTP
Handler does not need to.
but here I'm using:
reader := io.TeeReader(r.Body, buf)
and if err != nil {
r.Body = ioutil.NopCloser(buf)
So, should I r.Body.Close() it in the if path of my code considering that ioutil.NopCloser() has a "fake" Close() method?

Server returning 400 when user exists

I'm working on some tests in Go and I have spent the past 2 days trying to make it work but I couldn't. My problem is that the test returns 400 even when the user does exist.
This is my getUser function
func (handler *UserHandler) getUser(w http.ResponseWriter, ID int) {
logfile, err := os.OpenFile("events.log", os.O_RDWR|os.O_CREATE|os.O_APPEND, 0666)
if err != nil {
log.Fatalf("Error opening file: %v", err)
}
defer logfile.Close()
log.SetOutput(logfile)
user := db.Fetch(ID)
userJSON, err := json.Marshal(user)
if err != nil {
log.Printf("Error while marshaling the user into JSON: %v", err)
return
}
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusOK)
// userJSON is sent as http Response
w.Write(userJSON)
}
This is my UserHandler
type UserHandler struct{}
func (handle *UserHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
var head string
head, r.URL.Path = ShiftPath(r.URL.Path)
id, err := strconv.Atoi(head)
if err != nil {
http.Error(w, fmt.Sprintf("Invalid user ID %q", head), http.StatusBadRequest)
return
}
switch r.Method {
case "GET":
handle.getUser(w, id)
default:
http.Error(w, "Only GET is allowed", http.StatusMethodNotAllowed)
}
}
func ShiftPath(p string) (head, tail string) {
p = path.Clean("/" + p)
i := strings.Index(p[1:], "/") + 1
if i <= 0 {
return p[1:], "/"
}
return p[1:i], p[i:]
}
And this is my test
func TestGetUser(t *testing.T) {
handler := new(UserHandler)
mux := http.NewServeMux()
mux.HandleFunc("/user/", handler.ServeHTTP)
writer := httptest.NewRecorder()
request, _ := http.NewRequest("GET", "/user/12", nil)
mux.ServeHTTP(writer, request)
if writer.Code != 200 {
t.Errorf("Response code is %v", writer.Code)
}
}
Issue with code ====> id, err := strconv.Atoi(head)
Due to error you see a return and hence you see 400 error.
Have your server code fully functional with valid logic.
Suggestion: Always print or debug line by line. You can find the issue and root cause.

Port doesn't close when I run ListenAndServe() in a test function, how do I fix this?

When I call http.ListenAndServe() in a test function the port does NOT close even after the tests have finished and the process has terminated. So the next time I run the tests I get the error "ListenAndServe: listen tcp :8080: bind: address already in use".
This does not happen when I run my program normally through main().
func TestIndex(t *testing.T) {
handle := handlers.ServeAndHandle("8080")
req, _ := http.NewRequest("GET", "", nil)
w := httptest.NewRecorder()
handle.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Errorf("Home page didn't return %v", http.StatusOK)
}
}
// this is just a wrapper function of ListenAndServe. m is of type handler
func ServeAndHandle(port string) http.Handler {
m := &mux{}
err := http.ListenAndServe(":"+port, m) // set listen port
if err != nil {
log.Fatal("ListenAndServe: ", err)
}
return m
}
You can use NewUnstartedServer to create a instance, start it manually then defer closing it at the end of the test. Something along the lines:
...
ts := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintln(w, "Hello world")
}))
l, _ := net.Listen("tcp", URL)
ts.Listener = l
ts.Start()
defer ts.Close()
res, err := http.Get(URL)
if err != nil {
log.Fatal(err)
}
...

Go : add logging to each router

Go : add logging to each router
I want to log all my network request in Go web app.
Something like negroni:
// https://github.com/codegangsta/negroni/blob/master/logger.go
// NewLogger returns a new Logger instance
func NewLogger() *Logger {
return &Logger{log.New(os.Stdout, "[negroni] ", 0)}
}
func (l *Logger) ServeHTTP(rw http.ResponseWriter, r *http.Request, next http.HandlerFunc) {
start := time.Now()
l.Printf("Started %s %s", r.Method, r.URL.Path)
next(rw, r)
res := rw.(ResponseWriter)
l.Printf("Completed %v %s in %v", res.Status(), http.StatusText(res.Status()), time.Since(start))
}
So here's my code:
router := httprouter.New()
handler := func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
type Page struct {
Title string
}
tp := template.Must(template.ParseFiles("templates/main.html", "templates/base.html"))
err := tp.ExecuteTemplate(w, "base", &Page{Title: "AAA"})
if err != nil {
log.Fatal(err)
}
router.Handle("GET", "/", handler)
l := log.New(os.Stdout, "[AAA] ", 0)
l.Printf("Listening 0.0.0.0%s", PORT)
l.Fatal(http.ListenAndServe(PORT, router))
If I want to do this, I have to add start := time.Now() and time.Since(start) manually to each router in my code.package main
I think I should wrap it and use interface but don't know how to get started.
How do I implement one simple logging interface and apply all the routed handlers so that I can debug with all the loggings...
Negroni does like:
router := mux.NewRouter()
router.HandleFunc("/", HomeHandler)
n := negroni.New(Middleware1, Middleware2)
// Or use a middleware with the Use() function
n.Use(Middleware3)
// router goes last
n.UseHandler(router)
n.Run(":3000")
Wrap the root handler with a handler that logs and delegates to another handler:
type RequestLogger struct {
h http.Handler
l *Logger
}
func (rl RequestLogger) ServeHTTP(w http.ResponseWriter, r *http.Request) {
start := time.Now()
rl.l.Printf("Started %s %s", r.Method, r.URL.Path)
rl.h.ServeHTTP(w, r)
rl.l.Printf("Completed %s %s in %v", r.Method, r.URL.Path, time.Since(start))
}
---
l := log.New(os.Stdout, "[AAA] ", 0)
l.Printf("Listening 0.0.0.0%s", PORT)
l.Fatal(http.ListenAndServe(PORT, RequestLogger{h:router, l:l}))
A simple middleware interceptor is probably the right approach. If you'd like an example, see a simple one here: https://github.com/jadekler/git-go-websiteskeleton/blob/master/main.go#L49. This can be contracted down to a smaller function, but YMMV.
Here is the relevant code:
At the top of your handlefuncs:
http.HandleFunc("/", httpInterceptor)
In your middleware:
func httpInterceptor(w http.ResponseWriter, req *http.Request) {
router.ServeHTTP(w, req)
logAccess(w, req)
}
Where logAccess is a function that logs whatever you'd like it to. For an example, see here.
It's rather easy to implement your own middleware in Go, one approach is something like:
var logger = log.New(os.Stdout, "[something shiny] ", 0)
func httpLogger(fn func(w http.ResponseWriter, r *http.Request)) func(w http.ResponseWriter, r *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
start := time.Now()
logger.Printf("Started %s %s", r.Method, r.URL.Path)
fn(w, r)
logger.Printf("Completed in %v", time.Since(start))
}
}
....
router.Handle("GET", "/", httpLogger(handler))

Resources