I have a set of requests handlers like the one below:
func GetProductsHandler(w http.ResponseWriter, req *http.Request) {
defer req.Body.Close()
products := db.GetProducts()
// ...
// return products as JSON array
}
How do I test them the right way? Should I send mock ResponseWriter and Request objects to the function and see the results?
Are there tools to mock request and response objects in Go to simplify the process without having to start server before testing?
Go provides a mock writer for use in testing handlers. The standard library documentation provides an example:
package main
import (
"fmt"
"net/http"
"net/http/httptest"
)
func main() {
handler := func(w http.ResponseWriter, r *http.Request) {
http.Error(w, "something failed", http.StatusInternalServerError)
}
req := httptest.NewRequest("GET", "http://example.com/foo", nil)
w := httptest.NewRecorder()
handler(w, req)
fmt.Printf("%d - %s", w.Code, w.Body.String())
}
I think having a global dependency (db) throws a wrench into clean unit testing. Using go your test could reassign a value, masking, the global value of db.
Another strategy (my preferred) is to package your handler in a struct, which has a db attribute..
type Handlers struct {
db DB_INTERFACE
}
func (hs *Handlers) GetProductsHandler(w http.ResponseWriter, req *http.Request) {...}
This way your test can instantiate a Handlers with a stub db object which will allow you to create IO free unit tests.
Related
could anyone help me here please as I'm new to golang? I have a yaml file which looks like this:
port: 5000
handlers:
- name: test1
uri: /api/test1
response:
status: 200
body: test1
- name: test2
uri: /api/test2
response:
status: 500
body: test2
based on this file I want to create a server. Currently I'm trying to do it this way, but looks like it doesn't work as expected.
What am I doing wrong and what is the better way to achieve what I need?
package main
import (
"fmt"
"io/ioutil"
"log"
"net/http"
"gopkg.in/yaml.v2"
)
func main() {
config := parseYaml("conf.yaml")
configHandlers := config.Handlers
mux := http.NewServeMux()
for _, handler := range *configHandlers {
mux.HandleFunc(*handler.Uri, func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(*handler.Response.Status)
fmt.Fprintf(w, *handler.Response.Body)
})
}
log.Fatal(http.ListenAndServe(fmt.Sprintf(":%s", *config.Port), mux))
}
type YamlConfig struct {
Port *string `yaml:"port"`
Handlers *[]HandlerConfig `yaml:"handlers"`
}
type HandlerConfig struct {
Uri *string `yaml:"uri"`
Name *string `yaml:"name"`
Response *Response `yaml:"response"`
}
type Response struct {
Status *int `yaml:"status"`
Body *string `yaml:"body"`
}
func (c *YamlConfig) parseYaml(data []byte) error {
return yaml.Unmarshal(data, c)
}
func parseYaml(path string) YamlConfig {
data, err := ioutil.ReadFile(path)
if err != nil {
log.Fatal(err)
}
var config YamlConfig
if err := config.parseYaml(data); err != nil {
log.Fatal(err)
}
return config
}
Update:
If I run this server then regardless of which endpoint I hit, it will always return me 500 and test2 in body
What you're seeing is seemingly a common pitfall for people:
configHandlers := config.Handlers
mux := http.NewServeMux()
for _, handler := range *configHandlers {
mux.HandleFunc(*handler.Uri, func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(*handler.Response.Status)
fmt.Fprintf(w, *handler.Response.Body)
})
}
The for loop, on each iteration, reassigns the handler variable. In the loop body, you create a new function and pass it to mux.HandlerFun. These function bodies kind of inherit the outer scope, and access this handler variable. The variable is reassigned outside of the functions, and thus the values each handler function has access to changes with it. What you can do to address the issue is mask the handler variable the loop uses, and create a scope that is unique to each handler. The classic way in languages like JavaScript (where this is - or used to be back when I wrote some JS - a common issue) is to wrap the code in an IIFE (Immediately Invoked Function Expression):
for _, handler := range *configHandlers {
func (handler *HandlerConfig) { // handler is now the argument passed to this function
mux.HandleFunc(*handler.Uri, func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(*handler.Response.Status)
fmt.Fprintf(w, *handler.Response.Body)
})
}(handler) // call the function with the _current_ value of handler
}
This is a tad messy, and because golang is properly block-scoped, you can just do this:
for _, handler := range *configHandlers {
h := handler // create a variable in the inner scope
mux.HandleFunc(*handler.Uri, func(w http.ResponseWriter, r *http.Request) {
// now h will reference a copy unique to each iteration
w.WriteHeader(*h.Response.Status)
fmt.Fprintf(w, *h.Response.Body)
})
}
That ought to fix it. I've noticed some weirdness with your use of pointers in the types you've added to your question, though... Fields like Port being of type *string? Why wouldn't you just use string? No Same for the Body and Status fields in the Response type. By changing them to plain string fields you don't have to dereference them in your handler functions. It will look a lot cleaner.
A bigger worry is this field:
Handlers *[]HandlerConfig `yaml:"handlers"`
I'm not sure if you really know what the type of this field is, but it makes next to no sense. Handlers is now a pointer to a slice of HandlerConfig values. I'm assuming you wanted this field to be:
// Handlers is a slice of HandlerConfig values:
Handlers []HandlerConfig `yaml:"handlers"`
// or Handlers is a slice of pointers to HandlerConfig values
Handlers []*HandlerConfig `yaml:"handlers"`
Generally speaking, a pointer to a slice, especially in a config type is bad code.
If you define a struct that will represent the configuration in your YAML file, you can unmarshall the yaml into an instantiated struct of that type using the yaml package. From there, you can reference the fields in the struct as any other struct.
package main
import (
"fmt"
"gopkg.in/yaml.v2"
)
type YamlExample struct {
FieldOne string `yaml:"fieldOne"`
NestedField struct {
Name string `yaml:"name"`
} `yaml:"nestedField"`
}
const YamlEx string = `
fieldOne: one
nestedField:
name: nestedFieldName
`
func main() {
var yamlE YamlExample
err := yaml.Unmarshal([]byte(YamlEx), &yamlE)
if err != nil {
panic(err)
}
fmt.Printf("%+v\n", yamlE)
}
Link to example.
In your case, you'd probably want to handle the routes in a struct and then reference the fields in the struct for things like route name, how to handle the body of the request, etc. If your YAML is stored in a file, you'll have to use something like the io package to read the file into a byte array that the YAML package can parse. See here for a reference.
I am writing a golang program using revel framework, in which i need to check the initial timestamp of a http request.
I know how to do it in C# :
HttpContextWrapper context = Request.Properties["MS_HttpContext"] as HttpContextWrapper;
DateTime t2 = context.Timestamp.ToUniversalTime();
Didn't get much how to do it in Go.
HttpContext class in .Net framework sets the timestamp when a request arrives at the server. You might as well store the timestamp in the first line of your request handler function.
The simplest thing to do is grab the current time within your handler.
type Handler struct {
}
func (h Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
rs := time.Now().UTC()
//TODO: Use the time.
}
If you want to measure the time taken by all middleware preceding your handler, then you can update the Go context and place your middleware at the start of your middleware chain.
Here's an example of what that middleware might look like:
package timemiddleware
import (
"context"
"net/http"
"time"
)
// New returns new middleware which tracks the time that a request started.
func New(next http.Handler) http.Handler {
return handler{
next: next,
}
}
type key int
const id = key(1)
type handler struct {
next http.Handler
}
func (h handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
ctx := context.WithValue(r.Context(), id, time.Now().UTC())
h.next.ServeHTTP(w, r.WithContext(ctx))
}
// GetTime returns time from the current request, where it has previously been added by the middleware.
func GetTime(r *http.Request) (t time.Time, ok bool) {
v := r.Context().Value(id)
t, ok = v.(time.Time)
return
}
You'd use this as per this example:
package main
import (
"fmt"
"net/http"
"time"
"github.com/xxxxx/timemiddleware"
)
func main() {
next := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
time.Sleep(time.Second * 5)
w.Write([]byte("Hello"))
if t, ok := timemiddleware.GetTime(r); ok {
fmt.Println(t)
fmt.Println(time.Now().UTC())
}
})
h := timemiddleware.New(next)
fmt.Println(http.ListenAndServe("0.0.0.0:8080", h))
}
In my first iteration, I got the following to compile and work:
package main
import (
"fmt"
"sync"
)
var wg sync.WaitGroup
func routineHandle (query string, ch chan <- string) {
ch <- query
wg.Wait()
}
func ping () {
ch := make(chan string)
wg.Add(1)
go routineHandle("testquery",ch)
wg.Done()
msg := <-ch
fmt.Println("Channel Message",msg)
}
func main () {
ping()
}
This successfully uses 1 channel to perform the goroutine routineHandle
Now, I want to add web server capabilities which perform the following:
Listens on a port and accepts/returns requests
Hooks into the routineHandle so we can utilize that goroutine as an Api Server Method
My code is on a linux box without a gui so I don't know how to test the web server capabilities.
My code looks like as follows:
package main
import (
"fmt"
"sync"
"net/http"
)
var wg sync.WaitGroup
func routineHandle (query string, ch chan <- string) {
ch <- query
wg.Wait()
}
func ping (w http.ResponseWriter, r *http.Request) {
ch := make(chan string)
wg.Add(1)
go routineHandle("testquery",ch)
wg.Done()
msg := <-ch
//fmt.Println("Channel Message",msg)
w.Write([]byte msg)
}
func main() {
http.HandleFunc("/",ping)
http.ListenAndServe(":1234",nil)
}
You'll notice a few additions with my second piece of code:
I added the net/http package
I added the http listener to the main method
I added response writer and request parameters to the ping function
I changed from fmt.Println() to c.Write
The end goal would be for typing in a query, and then using that query in the routineHandle goroutine
Like I Said Though, I don't know how to test this final implementation on an ubuntu box without a gui
One last thing to note. If you notice any issues PLEASE let me know. I wonder if running a goroutine inside a http server would cause an issue
The code in the question uses the wait group incorrectly (wait and done should be swapped, the group should not be shared globally) and redundantly with the channel. Delete the use of the wait group to fix the code.
package main
import (
"net/http"
)
func routineHandle(query string, ch chan<- string) {
ch <- query
}
func ping(w http.ResponseWriter, r *http.Request) {
ch := make(chan string)
go routineHandle("testquery", ch)
msg := <-ch
w.Write([]byte(msg))
}
func main() {
http.HandleFunc("/", ping)
http.ListenAndServe(":1234", nil)
}
I have a basic HTTP server that accepts a request and returns data from a data store.
Each HTTP request does the following things:
Create a context with timeout
Create a read request (custom type)
Push read request onto channel
Wait for response and serve data
Here's the basic pseudo code:
package main
import (
"context"
"net/http"
"time"
)
type dataRequest struct {
data chan string
ctx context.Context
}
func handler(reqStream chan dataRequest) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
ctx, cancel := context.WithTimeout(r.Context(), 5*time.Second)
defer cancel()
req := dataRequest{
data: make(chan string),
ctx: ctx,
}
select {
case reqStream <- req:
// request pushed to que
case <-ctx.Done():
// don't push onto reqStream if ctx done
}
select {
case <-ctx.Done():
// don't try and serve content if ctx done
case data := <-req.data:
// return data to client
}
}
}
func main() {
dataReqs := make(chan dataRequest)
go func() {
for {
select {
case req := <-dataReqs:
select {
case <-req.ctx.Done():
// don't push onto data channel if ctx done
case req.data <- "some data":
// get data from store
}
}
}
}()
http.HandleFunc("/", handler(dataReqs))
http.ListenAndServe(":8080", nil)
}
My question is, because the context could finish at any time due to the deadline being exceeded or the client cancelling the request, is my current approach correct for handling this in multiple places or is there a more elegant solution?
seems to me that it'll work.
few comments -
you can return in the first case of <- ctx.Done()
you're already waiting for req.ctx.Done() in the data store handler so you can completely remove the first select {} statement and just publish to the data requests channel. not sure about performance hits for the rare cases when the context is done so early before the request is even published...
For an incoming HTTP request, I had to respond with a 202 Accepted status code, while continue processing the payload in the background. for example purposes this is what I am currently doing:
package main
import (
"fmt"
"log"
"net/http"
"time"
"github.com/nbari/violetear"
)
func sleep() {
time.Sleep(3 * time.Second)
fmt.Println("done...")
}
func index(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusAccepted)
go sleep()
}
func main() {
router := violetear.New()
router.HandleFunc("*", index)
http.Handle("/", router)
log.Fatal(http.ListenAndServe(":8080", router))
}
Basically, on the handler I just use WriteHeader and later a call the the sleep function within a goroutine:
func index(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusAccepted)
go sleep()
}
In case I would like to respond with "200 OK", I notice that I can simple return, for example:
func index(w http.ResponseWriter, r *http.Request) {
go sleep()
return
}
Therefore wondering if I should return always I want to close:
func index(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusAccepted)
go sleep()
return
}
Or by just writing the header and next calling the goroutine is enough.
Returning from the handler is sufficient and is what should be done. Quoting from http.Handler:
Returning signals that the request is finished; it is not valid to use the ResponseWriter or read from the Request.Body after or concurrently with the completion of the ServeHTTP call.
Note that the final return statement is not necessary, you can simply omit it. Execution returns from the handler when its last statement is executed, execution does not wait for goroutines started from the function to complete. (Note that deferred statements would be executed prior, but you don't have any here.)
Also when returning, if HTTP headers are not set, 200 OK will be set automatically. So if you want 202 Accepted, the following is the minimal required:
func index(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusAccepted)
go sleep()
}
Just make sure you don't use the http.ResponseWriter and httpRequest values in the concurrent goroutine after you return from the handler as they may be reused, so you should not even attempt to read them.