Go : add logging to each router
I want to log all my network request in Go web app.
Something like negroni:
// https://github.com/codegangsta/negroni/blob/master/logger.go
// NewLogger returns a new Logger instance
func NewLogger() *Logger {
return &Logger{log.New(os.Stdout, "[negroni] ", 0)}
}
func (l *Logger) ServeHTTP(rw http.ResponseWriter, r *http.Request, next http.HandlerFunc) {
start := time.Now()
l.Printf("Started %s %s", r.Method, r.URL.Path)
next(rw, r)
res := rw.(ResponseWriter)
l.Printf("Completed %v %s in %v", res.Status(), http.StatusText(res.Status()), time.Since(start))
}
So here's my code:
router := httprouter.New()
handler := func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
type Page struct {
Title string
}
tp := template.Must(template.ParseFiles("templates/main.html", "templates/base.html"))
err := tp.ExecuteTemplate(w, "base", &Page{Title: "AAA"})
if err != nil {
log.Fatal(err)
}
router.Handle("GET", "/", handler)
l := log.New(os.Stdout, "[AAA] ", 0)
l.Printf("Listening 0.0.0.0%s", PORT)
l.Fatal(http.ListenAndServe(PORT, router))
If I want to do this, I have to add start := time.Now() and time.Since(start) manually to each router in my code.package main
I think I should wrap it and use interface but don't know how to get started.
How do I implement one simple logging interface and apply all the routed handlers so that I can debug with all the loggings...
Negroni does like:
router := mux.NewRouter()
router.HandleFunc("/", HomeHandler)
n := negroni.New(Middleware1, Middleware2)
// Or use a middleware with the Use() function
n.Use(Middleware3)
// router goes last
n.UseHandler(router)
n.Run(":3000")
Wrap the root handler with a handler that logs and delegates to another handler:
type RequestLogger struct {
h http.Handler
l *Logger
}
func (rl RequestLogger) ServeHTTP(w http.ResponseWriter, r *http.Request) {
start := time.Now()
rl.l.Printf("Started %s %s", r.Method, r.URL.Path)
rl.h.ServeHTTP(w, r)
rl.l.Printf("Completed %s %s in %v", r.Method, r.URL.Path, time.Since(start))
}
---
l := log.New(os.Stdout, "[AAA] ", 0)
l.Printf("Listening 0.0.0.0%s", PORT)
l.Fatal(http.ListenAndServe(PORT, RequestLogger{h:router, l:l}))
A simple middleware interceptor is probably the right approach. If you'd like an example, see a simple one here: https://github.com/jadekler/git-go-websiteskeleton/blob/master/main.go#L49. This can be contracted down to a smaller function, but YMMV.
Here is the relevant code:
At the top of your handlefuncs:
http.HandleFunc("/", httpInterceptor)
In your middleware:
func httpInterceptor(w http.ResponseWriter, req *http.Request) {
router.ServeHTTP(w, req)
logAccess(w, req)
}
Where logAccess is a function that logs whatever you'd like it to. For an example, see here.
It's rather easy to implement your own middleware in Go, one approach is something like:
var logger = log.New(os.Stdout, "[something shiny] ", 0)
func httpLogger(fn func(w http.ResponseWriter, r *http.Request)) func(w http.ResponseWriter, r *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
start := time.Now()
logger.Printf("Started %s %s", r.Method, r.URL.Path)
fn(w, r)
logger.Printf("Completed in %v", time.Since(start))
}
}
....
router.Handle("GET", "/", httpLogger(handler))
Related
I have a generic Webserver which I want to use on different domains / servers.
For setting up each server I simply read a JSON config file with all necessary information. One would be for example the redirect for all traffic which reaches port 80 and forward it to a TLS service. Since I don't want to make the config object global. How can I pass the content from my inputFromConfigFile to the redirectTLS function?
Here is an example:
func main(){
var inputFromConfigFile = "https://www.example.com:443"
go func() {
if err := http.ListenAndServe(":80", http.HandlerFunc(redirectTLS)); err != nil {
log.Fatalf("ListenAndServe error: %v", err)
}
}()
}
//Pass the above string to this function:
func redirectTLS(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, "https://www.example.com:443"+r.RequestURI,http.StatusMovedPermanently)
}
You can define a custom Handler (could be implemented as a struct) as long as it matches the http.Handler interface. The config could be saved inside the Handler as a struct field.
type Handler struct {
// config goes here
}
func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
// anything that handler needs to do here
}
Example: https://pkg.go.dev/net/http#example-Handle
You can define redirectTLS as an inline closure function directly in main:
var inputFromConfigFile = "https://www.example.com:443"
go func() {
err := http.ListenAndServe(":80", func(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, inputFromConfigFile+r.RequestURI, http.StatusMovedPermanently)
})
if err != nil {
log.Fatalf("ListenAndServe error: %v", err)
}
}()
I would make the config object global.
Otherwise, you can define a function that accepts the config as an argument, and returns a handler function that closes over the configuration object:
var inputFromConfigFile = "https://www.example.com:443"
http.ListenAndServe(":80", createHandler(inputFromConfigFile))
// ...
func createHandler(config string) http.HandlerFunc {
return http.HandlerFunc(func (w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, config+r.RequestURI,http.StatusMovedPermanently)
})
}
I am new to go and am trying to setup a go server. My intention is to return an image when the url is hit.
this is what i have done
myRouter := mux.NewRouter()
myRouter.HandleFunc("/poster_path/{id}",posterfunc)
This is my posterfunc
func posterfunc(w http.ResponseWriter, r *http.Request){
w.Header().Set("Content-Type", "image/jpeg")
vars := mux.Vars(r)
key := vars["id"]
var url = "/home/rakshithjk/Desktop/poster/"+key+".jpg"
http.FileServer(http.Dir(url))
}
This is the output in Postman -
Any help would be appreciated.
UPDATE -
tried changing http.FileServer to http.ServeFile, but the output remains the same
Modified handler function
func posterfunc(w http.ResponseWriter, r *http.Request){
w.Header().Set("Content-Type", "image/jpeg")
vars := mux.Vars(r)
key := vars["id"]
var url = "/home/rakshithjk/Desktop/poster/"+key+".jpg"
http.ServeFile(w, r,url)
This is my entire file contents(for reference)
package main
import (
"fmt"
"log"
"net/http"
"encoding/json"
"database/sql"
_ "github.com/go-sql-driver/mysql"
"github.com/gorilla/mux"
//"github.com/davecgh/go-spew/spew"
)
func handleRequests() {
myRouter := mux.NewRouter()
myRouter.HandleFunc("/", homePage)
myRouter.HandleFunc("/movie/top_rated", returnSingleArticle)
myRouter.HandleFunc("/poster_path",posterfunc)
log.Fatal(http.ListenAndServe(":10000", myRouter))
}
func enableCors(w *http.ResponseWriter) {
(*w).Header().Set("Access-Control-Allow-Origin", "*")
}
type movie_list struct {
Page int `json:"page"`
Results []movie `json:"results"`
}
type movie struct {
Id int `json:"id"`
Title string `json:"title"`
Language string `json:"language"`
Release_date string `json:"release_date"`
Poster_path string `json:"poster_path"`
Background_path string `json:"background_path"`
Overview string `json:"overview"`
Genre_ids string `json:"genre_ids"`
}
func posterfunc(w http.ResponseWriter, r *http.Request){
w.Header().Set("Content-Type", "image/jpeg")
//vars := mux.Vars(r)
//key := vars["id"]
enableCors(&w)
var url = "/home/rakshithjk/go/src/clumio/112.png"
fmt.Fprintf(w, "Hello, %q\n", url)
http.ServeFile(w, r,url)
}
func homePage(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Welcome to the HomePage!")
fmt.Println("Endpoint Hit: homePage")
}
func returnSingleArticle(w http.ResponseWriter, r *http.Request) {
//vars := mux.Vars(r)
//key := vars["id"]
enableCors(&w)
db, err := sql.Open("mysql", "root:72574484#tcp(127.0.0.1:3306)/PicturePerfect")
if err != nil {
fmt.Println(err)
}else{
fmt.Println("Connection Established")
}
rows,err:=db.Query("select * from movies limit 10")
if err!=nil{
fmt.Println(err)
}
var list movie_list
var tag movie
for rows.Next(){
err:=rows.Scan(&tag.Id,&tag.Title,&tag.Language,&tag.Release_date,&tag.Poster_path,&tag.Background_path,&tag.Overview,&tag.Genre_ids)
if err != nil {
fmt.Println(err)
}
fmt.Println(tag.Id)
list.Results = append(list.Results,tag)
}
err = rows.Err()
if err != nil {
fmt.Println(err)
}
defer db.Close()
//fmt.Fprintf(w, "Hello, %q\n", list.Results[3])
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(list)
//spew.Dump(list)
//fmt.Fprintf(w, "given lamguage, %q\n", tag.Poster_path)
}
func main() {
handleRequests()
}
http.FileServer() should not be called in a function like that. It returns a Handler function (a function similar to the posterfunc you created).
It should be used as the handler function in the route configuration like this:
myRouter.HandleFunc("/poster_path/",http.FileServer(http.Dir("./your_dir")))
Here you can find the documentation for http.FileServer, with some more detailed examples.
This blog post does a fine step-by-step explanation on how to set it up.
UPDATE:
If you want to use it inside your handler, you should use http.ServeFile. It will respond to the request with the contents of the named file or directory.
Docs here
func posterfunc(w http.ResponseWriter, r *http.Request){
w.Header().Set("Content-Type", "image/jpeg")
vars := mux.Vars(r)
key := vars["id"]
var url = "/home/rakshithjk/Desktop/poster/"+key+".jpg"
http.ServeFile(w, r, url)
}
UPDATE 2:
The issue in the new snippet is that you are printing to the interface just before serving the file.
Remove this line:
fmt.Fprintf(w, "Hello, %q\n", url)
Say I have a very simple Web service.
func main() {
http.HandleFunc("/", sanityTest)
log.Fatal(http.ListenAndServe(":8000", nil))
}
If I want to test it, I could minimally just have:
func ExampleTest() {
server := httptest.NewServer(http.DefaultServeMux)
defer server.Close()
resp, err := http.Get(server.URL)
if err != nil {
log.Fatal(err)
}
body, _ := ioutil.ReadAll(resp.Body)
fmt.Println(resp.StatusCode)
fmt.Println(resp.Header.Get("Content-Type"))
fmt.Println(string(body))
// Output:
// 200
// text/plain; charset=utf-8
// OK
}
But that will result in a 404, since it doesn't know about the routes. So what I've seen main_test.go code do, is re-setup the handles in the test file's init, like so:
func init() {
http.HandleFunc("/", sanityTest)
}
Which leads to duplication, and inevitably I have to create a function in main.go like:
func setupRoutes() {
http.HandleFunc("/", sanityTest)
}
Which I find a little ugly. Am I missing a trick to instantiate the routes from main.go and avoid the init?
You can re-use routes between tests and main.go file, also it's helpful if you want to mock something in your handlers (add a new argument to router() func below)
main.go:
func sanityTest(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "%s", "sanity test")
}
func router() *http.ServeMux {
h := http.NewServeMux()
h.HandleFunc("/", sanityTest)
return h
}
func main() {
http.ListenAndServe(":8080", router())
}
main_test.go:
func TestSanity(t *testing.T) {
tests := []struct {
name string
uri string
want string
}{
{"1", "/", "sanity test"},
}
ts := httptest.NewServer(router())
defer ts.Close()
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
url := ts.URL + tt.uri
resp, _ := http.Get(url)
respBody, _ := ioutil.ReadAll(resp.Body)
resp.Body.Close()
got := string(respBody)
if got != tt.want {
t.Errorf("got %s, Want %s", got, tt.want)
}
})
}
}
I can see two main issues in the example code below, but I don't know how to solve them correctly.
If the timeout handler does not get the signal through the errCh that the next handler has completed or an error occured, it will reply "408 Request timeout" to the request.
The problem here is that the ResponseWriter is not safe to be used by multiple goroutines. And the timeout handler starts a new goroutine when executing the next handler.
Issues:
How to prevent the next handler from writing into the ResponseWriter when the ctx's Done channel times out in the timeout handler.
How to prevent the timeout handler from replying 408 status code when the next handler is writing into the ResponseWriter but it has not finished yet and the ctx's Done channel times out in the timeout handler.
package main
import (
"context"
"fmt"
"net/http"
"time"
)
func main() {
http.Handle("/race", handlerFunc(timeoutHandler))
http.ListenAndServe(":8080", nil)
}
func timeoutHandler(w http.ResponseWriter, r *http.Request) error {
const seconds = 1
ctx, cancel := context.WithTimeout(r.Context(), time.Duration(seconds)*time.Second)
defer cancel()
r = r.WithContext(ctx)
errCh := make(chan error, 1)
go func() {
// w is not safe for concurrent use by multiple goroutines
errCh <- nextHandler(w, r)
}()
select {
case err := <-errCh:
return err
case <-ctx.Done():
// w is not safe for concurrent use by multiple goroutines
http.Error(w, "Request timeout", 408)
return nil
}
}
func nextHandler(w http.ResponseWriter, r *http.Request) error {
// just for fun to simulate a better race condition
const seconds = 1
time.Sleep(time.Duration(seconds) * time.Second)
fmt.Fprint(w, "nextHandler")
return nil
}
type handlerFunc func(w http.ResponseWriter, r *http.Request) error
func (fn handlerFunc) ServeHTTP(w http.ResponseWriter, r *http.Request) {
if err := fn(w, r); err != nil {
http.Error(w, "Server error", 500)
}
}
Here is a possible solution, which is based on #Andy's comment.
A new responseRecorder will be passed to the nextHandler, and the recorded response will be copied back to the client:
func timeoutHandler(w http.ResponseWriter, r *http.Request) error {
const seconds = 1
ctx, cancel := context.WithTimeout(r.Context(),
time.Duration(seconds)*time.Second)
defer cancel()
r = r.WithContext(ctx)
errCh := make(chan error, 1)
w2 := newResponseRecorder()
go func() {
errCh <- nextHandler(w2, r)
}()
select {
case err := <-errCh:
if err != nil {
return err
}
w2.cloneHeader(w.Header())
w.WriteHeader(w2.status)
w.Write(w2.buf.Bytes())
return nil
case <-ctx.Done():
http.Error(w, "Request timeout", 408)
return nil
}
}
And here is the responseRecorder:
type responseRecorder struct {
http.ResponseWriter
header http.Header
buf *bytes.Buffer
status int
}
func newResponseRecorder() *responseRecorder {
return &responseRecorder{
header: http.Header{},
buf: &bytes.Buffer{},
}
}
func (w *responseRecorder) Header() http.Header {
return w.header
}
func (w *responseRecorder) cloneHeader(dst http.Header) {
for k, v := range w.header {
tmp := make([]string, len(v))
copy(tmp, v)
dst[k] = tmp
}
}
func (w *responseRecorder) Write(data []byte) (int, error) {
if w.status == 0 {
w.WriteHeader(http.StatusOK)
}
return w.buf.Write(data)
}
func (w *responseRecorder) WriteHeader(status int) {
w.status = status
}
There's probably something obvious that I'm missing but I'm trying to debug the HTTP response written by my go server.
I see that there's httputil.DumpResponse available but it takes a http.Response object and what I have available is http.ResponseWriter
Is there a way to extract the http.Response from http.ResponseWriter so I can inspect the content of the response to console or log?
Context:
I'm writing a simple server-side authentication using https://github.com/RangelReale/osin and it's default example, but could not understand why the front-end (using http://ember-simple-auth.com) interprets a failed authentication (incorrect password) as success.
Here's the snippet:
r = mux.NewRouter()
r.HandleFunc("/token", func (w http.ResponseWriter, r *http.Request) {
fmt.Printf("r.HandleFunc /token\n")
resp := server.NewResponse()
defer resp.Close()
r.ParseForm()
grantType := r.FormValue("grant_type")
username := r.FormValue("username")
password := r.FormValue("password")
fmt.Printf("/token : grantType=%s username=%s password=%s\n", grantType, username, password)
if ar := server.HandleAccessRequest(resp, r); ar != nil {
if username == "user" && password == "correct-password" {
ar.Authorized = true
} else {
ar.Authorized = false
}
server.FinishAccessRequest(resp, r, ar)
}
osin.OutputJSON(resp, w, r)
// Debug - doesn't work yet
dump, err := httputil.DumpResponse(w, true)
if err != nil {
fmt.Printf("%s\n", dump)
}
});
http.Handle("/token", r)
Write to an *httptest.ResponseRecorder (which implements http.ResponseWriter) and inspect it.
Example from the package:
package main
import (
"fmt"
"log"
"net/http"
"net/http/httptest"
)
func main() {
handler := func(w http.ResponseWriter, r *http.Request) {
http.Error(w, "something failed", http.StatusInternalServerError)
}
req, err := http.NewRequest("GET", "http://example.com/foo", nil)
if err != nil {
log.Fatal(err)
}
w := httptest.NewRecorder()
handler(w, req)
fmt.Printf("%d - %s", w.Code, w.Body.String())
}
Edit to answer question in comments:
If I understand your question correctly, then yes, you can make use of closures for this.
Consider the following to be your handler:
func MyHandler(w http.ResponseWriter, r *http.Request) {
// do useful stuff...
}
You could then register the following closure with your servemux to attain the desired effect:
http.HandleFunc("/my/url", func(w http.ResponseWriter, r *http.Request) {
// first call MyHandler
MyHandler(w, r)
// then log whatever you need
log.Printf("%#v\n", w)
})
If this pattern proves useful to you then you could write a higher-order method that wraps any func(http.ResponseWriter, *http.Request) in such a closure. That's a topic for itself, though.