I've been studying golang and I noticed a lot of people create servers by using the http.NewServeMux() function and I don't really understand what it does.
I read this:
In go ServeMux is an HTTP request multiplexer. It matches the URL of
each incoming request against a list of registered patterns and calls
the handler for the pattern that most closely matches the URL.
How is that different than just doing something like:
http.ListenAndServe(addr, nil)
http.Handle("/home", home)
http.Handle("/login", login)
What is the purpose of using multiplexing?
From net/http GoDoc and Source.
ListenAndServe starts an HTTP server with a given address and handler. The handler is usually nil, which means to use DefaultServeMux. Handle and HandleFunc add handlers to DefaultServeMux
DefaultServeMux is just a predefined http.ServeMux
var DefaultServeMux = &defaultServeMux
var defaultServeMux ServeMux
As you can see http.Handle calls DefaultServeMux internally.
func Handle(pattern string, handler Handler) { DefaultServeMux.Handle(pattern, handler) }
The purpose of http.NewServeMux() is to have your own instance of http.ServerMux for instances like when you require two http.ListenAndServe functions listening to different ports with different routes.
the multiplexer in Golang is some things like multiplexer in hardware which multiply some inputs into some outputs
i gave you a simple exampe
type CustomMultiplexer struct {
}
the given multiplexer have to implement the ServeHTTP method to be registered int http to server inputs
func (mux CustomMultiplexer) ServeHTTP(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/" {
SimpleRequestHandler(w, r)
return
}
http.NotFound(w, r)
return
}
my SimpleRequestHandler is a method as follow
func SimpleRequestHandler(w http.ResponseWriter, r *http.Request) {
switch r.Method {
case http.MethodGet:
mySimpleGetRequestHandler(w, r)
break
default:
http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
break
}
}
now i can use my CustomMultiplxere to do multiplexing between incoming requested
func main() {
customServer := CustomServer{}
err := http.ListenAndServe(":9001", &customServer)
if err != nil {
panic(err)
}
}
the http.HandleFunc method works as my given simple multiplexer.
Related
I am writing an HTTP server in Go, which uses the following pattern to handle API output:
func handler(w http.ResponsWriter, r *http.Request) {
defer reply(w, r, L)() //L is a Logger
//do things...
}
func reply(w http.ResponseWriter, r *http.Request, log Logger) func() {
cid := []byte{0, 0, 0, 0}
if log != nil {
rand.Read(cid)
log.Debug("[%x] %s %s", cid, r.Method, r.URL.String())
}
entry := time.Now()
return func() {
if log != nil {
defer log.Debug("[%x] elapsed %d millis", cid, time.Since(entry).Milliseconds())
}
_, err := w.Write(nil)
if err == http.ErrHijacked {
return //API is a WEBSOCKET entry point, do nothing
}
//handle common output logic for normal HTTP APIs...
}
}
The reason I do this, is that I found this comment in the standard library:
// ErrHijacked is returned by ResponseWriter.Write calls when
// the underlying connection has been hijacked using the
// Hijacker interface. A zero-byte write on a hijacked
// connection will return ErrHijacked without any other side
// effects.
ErrHijacked = errors.New("http: connection has been hijacked")
However following the Write() method, I got this comment:
// Write writes the data to the connection as part of an HTTP reply.
//
// If WriteHeader has not yet been called, Write calls
// WriteHeader(http.StatusOK) before writing the data. If the Header
// does not contain a Content-Type line, Write adds a Content-Type set
// to the result of passing the initial 512 bytes of written data to
// ...
Write([]byte) (int, error)
My questions are:
Is it OK to use my code to safely detect if a HTTP connection is hijacked? I only want to check the connection is hijacked or not, but do NOT want it to add headers for me!
Since the ResponseWriter is an interface, I cannot click through the source code to find out how the standard library implements that method. In general, how can I drill down to the standard library (or any open source code) to find out the implementation of an interface?
Thanks to Cerise, I found the source code of the standard response.Writer:
func (w *response) write(lenData int, dataB []byte, dataS string) (n int, err error) {
if w.conn.hijacked() {
if lenData > 0 {
caller := relevantCaller()
w.conn.server.logf("http: response.Write on hijacked connection from %s (%s:%d)", caller.Function, path.Base(caller.File), caller.Line)
}
return 0, ErrHijacked
}
... ....
So, as said in the document, there is NO side effect.
I am learning Go and have a quick question about http handler implementation in Go.
I am asking it in a small sample code.
So assume there is a handler function called Test() as defined like below
func Test() func(http.ResponseWriter, *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
params := r.URL.Query()
name := params.Get("name")
if name == "axy" {
common.UpdateHttpResponse("Trying to updating the response", w, http.StatusBadRequest)
//return
}
w.WriteHeader(http.StatusOK)
}
}
lets say the UpdateHttpResponse method is also define in common package as below:
func LogExtractionErrorResponse(errMsg string, w http.ResponseWriter, responseHeader int) {
fmt.Printf("%s", errMsg)
jsonErrorOut := map[string]string{
"Error": errMsg,
}
w.WriteHeader(responseHeader)
encodedResponse, _ := json.Marshal(jsonErrorOut)
if w != nil {
w.Write(encodedResponse)
}
}
I call the Test() HTTP handler in the http server part as below.
// this is how Test() http handler is called as well
http.HandleFunc("/test", httpserver.Test())
So here is my question:
Based on my understanding all values are passed by value in go (as discussed in this thread as well)
In that case why if the http handler is called with a parameter (i.e localhost:PORT_NUM/test?name=axy), i observe "StatusBadRequest" in resonse. In other words, why the commented "return" keyword is not needed and why the header response is not overwritten by "w.WriteHeader(http.StatusOK)" at the end of Test() http handler?
I have an HTTP server that when it recieves a request calls on an underlying gRPC server.
I have chosen to abstract away the gRPC call with an interface, to make testing of the http server easier.
The problem is that I am constantly getting the errors:
rpc error: code = Canceled desc = grpc: the client connection is closing
or
rpc error: code = Canceled desc = context canceled
And as I understand both of these are related to the context getting passed into the grpc call. And that I want the context to be alive throughout both the HTTP and gRPC calls.
type SetterGetter interface {
Getter(key string) (val string)
}
type Service struct {
sg SetterGetter
ctx context.Context
}
func (s *Service) getHandler(rw http.ResponseWriter, r *http.Request) {
key := r.URL.Query()["key"][0]
res := s.sg.Getter(key)
fmt.Fprintf(rw, "Successfully got value: %s\n", res)
}
func main() {
s := new(Service)
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
s.sg = gc.NewClientwrapper(ctx)
http.HandleFunc("/get", s.getHandler)
log.Fatal(http.ListenAndServe(port, nil))
}
And my Getter implementation looks like this:
type clientwrapper struct {
sc pb.ServicesClient
ctx context.Context
}
func NewClientwrapper(ctx context.Context) *clientwrapper {
cw := new(clientwrapper)
conn, err := grpc.Dial(address, grpc.WithInsecure(), grpc.WithBlock())
if err != nil {
err = fmt.Errorf("Error could not dial address: %v", err)
}
defer conn.Close()
cw.ctx = ctx
cw.sc = pb.NewServicesClient(conn)
return cw
}
func (cw *clientwrapper) Getter(key string) (val string) {
// Make the GRPC request
res, err := cw.sc.Get(cw.ctx, &pb.GetRequest{Key: key})
if err != nil {
return ""
}
getVal := res.GetValue()
return getVal
}
So here I am creating a context in my http servers main menu, and passing it onwards. I do it like this because it worked if I removed my interface and put everything in the main file.
I have also tried to create the context both in the http handler and passing it to the Getter and I have also tried creating it in the Getter itself.
I think the correct approach is to create the context in the http request using the context that gets created by the request and then passing it to the grpc Getter. Like such:
func (s *Service) getHandler(rw http.ResponseWriter, r *http.Request) {
// Create it like such
ctx, cancel := context.WithTimeout(r.Context(), 100*time.Second)
key := r.URL.Query()["key"][0]
// And pass it onwards (of course we need to change function signature for this to work)
res := s.sg.Getter(ctx, key)
fmt.Fprintf(rw, "Successfully got value: %s\n", res)
}
So how should I create my context here, to not get these errors?
If your goal is to keep a long-running task running in the background, that doesn't cancel when the request is finalized, then don't use the request's context. Use context.Background() instead.
For example:
func (s *Service) getHandler(rw http.ResponseWriter, r *http.Request) {
ctx, cancel := context.WithTimeout(context.Background(), 100*time.Second)
// ...
I have a basic HTTP server that accepts a request and returns data from a data store.
Each HTTP request does the following things:
Create a context with timeout
Create a read request (custom type)
Push read request onto channel
Wait for response and serve data
Here's the basic pseudo code:
package main
import (
"context"
"net/http"
"time"
)
type dataRequest struct {
data chan string
ctx context.Context
}
func handler(reqStream chan dataRequest) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
ctx, cancel := context.WithTimeout(r.Context(), 5*time.Second)
defer cancel()
req := dataRequest{
data: make(chan string),
ctx: ctx,
}
select {
case reqStream <- req:
// request pushed to que
case <-ctx.Done():
// don't push onto reqStream if ctx done
}
select {
case <-ctx.Done():
// don't try and serve content if ctx done
case data := <-req.data:
// return data to client
}
}
}
func main() {
dataReqs := make(chan dataRequest)
go func() {
for {
select {
case req := <-dataReqs:
select {
case <-req.ctx.Done():
// don't push onto data channel if ctx done
case req.data <- "some data":
// get data from store
}
}
}
}()
http.HandleFunc("/", handler(dataReqs))
http.ListenAndServe(":8080", nil)
}
My question is, because the context could finish at any time due to the deadline being exceeded or the client cancelling the request, is my current approach correct for handling this in multiple places or is there a more elegant solution?
seems to me that it'll work.
few comments -
you can return in the first case of <- ctx.Done()
you're already waiting for req.ctx.Done() in the data store handler so you can completely remove the first select {} statement and just publish to the data requests channel. not sure about performance hits for the rare cases when the context is done so early before the request is even published...
I'm tying to create an util function that will read body of the Request/Response and return it.
Here's what I've done for the moment:
func GetBody(in interface{}) []byte {
var body io.Reader
var statusCode int
switch v := in.(type) {
case *http.Request, *http.Response:
body = v.Body
statusCode = v.StatusCode
default:
log.Fatal("Only http.Request and http.Response parameters can be accepted to parse body")
}
if statusCode != 200 {
log.Fatalf("Received status code [%d] instead of [200]", statusCode)
}
body, err := ioutil.ReadAll(body)
if err != nil {
log.Fatal(err)
}
return body
}
But I'm receiving an error of compiler: v.Body undefined (type interface {} is interface with no methods)
Am I missing something or it's impossible to make a generic function that will server both for *http.Request and *http.Response
It is because of the double case.
v is still an interface{} because it could be either an *http.Request or an *http.Response
switch v := in.(type) {
case *http.Request
body = v.Body
statusCode = v.StatusCode
case *http.Response:
body = v.Body
statusCode = v.StatusCode
default:
log.Fatal("Only http.Request and http.Response parameters can be accepted to parse body")
}
This should work
In addition to the other answers which directly address the issue with your type switch, I'd like to point out an alternative solution.
Note, the interface{} solution is perfectly fine and may easily be considered preferable to this. This is presented for edification.
First, as a slight aside, if the thing you were interested in was a common method (e.g. Write or Cookies) rather than a common field (Body), it would be easy and better to access it via a custom interface.
Either by defining a type:
type cookier interface { // Should probably use a better name
Cookies() []*http.Cookie
}
func ShowCookies1(r cookier) {
log.Println("Got cookies:", r.Cookies())
}
Or by using an anonymous type in the function definition:
func ShowCookies2(r interface {
Cookies() []*http.Cookie
}) {
log.Println("Got cookies:", r.Cookies())
}
These functions can accept anything that has a Cookies method, this includes *http.Request and *http.Response.
Unfortunately, in your particular case you wish to access a common field rather than a common method so you can't directly just use a matching interface.
You could make a small wrapper type that adds a GetBody method (one could argue such a function should have been defined in the standard package).
type reqbody struct{ *http.Request }
type respbody struct{ *http.Response }
type getbody interface {
GetBody() io.ReadCloser
}
func (r reqbody) GetBody() io.ReadCloser { return r.Body }
func (r respbody) GetBody() io.ReadCloser { return r.Body }
func GetBody2(r getbody) ([]byte, error) {
body := r.GetBody()
defer body.Close()
return ioutil.ReadAll(body)
}
The caller knows what type they have and does one of:
buf, err = GetBody2(reqbody{req})
buf, err = GetBody2(respbody{resp})
In some sense this is uglier than just using interface{}.
But it has the benefit that instead of having a function that takes absolutely any type and panics/errors at run-time if a programmer mistakenly calls it with something not of the appropriate type,
this instead forces the caller to safely pass something you known is of the correct type at compile time.
Looking at this further, you're just reading everything from an io.ReadCloser and then closing it so it could further be simplified to the following
(which probably is better than your interface{} solution):
func GetReqBody(r *http.Request) io.ReadCloser { return r.Body }
// Could add checking r.StatusCode to the following one as well:
func GetRespBody(r *http.Response) io.ReadCloser { return r.Body }
func ReadAndClose(rc io.ReadCloser) ([]byte, error) {
defer rc.Close()
return ioutil.ReadAll(rc)
}
Again, the caller knows what type they have and does one of:
buf, err = ReadAndClose(GetReqBody(req))
buf, err = ReadAndClose(GetRespBody(resp))
Or just:
buf, err = ReadAndClose(req.Body)
buf, err = ReadAndClose(resp.Body)
You can see an example of all these options on the Go Playground.
Finally, be careful using ioutil.ReadAll.
Often it is better to avoid pre-reading an entire file or network steam into a buffer and instead process it as a stream as it is being read.
In particular, it's trivial to make an HTTP request with an arbitrary large body as a denial of service attack or to waste server resources
(http.MaxBytesReader can also help).
What you are trying to do is more something like that:
switch in.(type) {
case *http.Request:
body = v.(*http.Request).Body
case *http.Response:
body = v.(*http.Response).Body
default:
log.Fatal(...)
}
EDIT: I removed the wrong part of my answer, see HectorJ's answer for the more syntaxic please way of doing this.