Serving gzipped content for Go - http

I'm starting to write server-side applications in Go. I'd like to use the Accept-Encoding request header to determine whether to compress the response entity using GZIP. I had hoped to find a way to do this directly using the http.Serve or http.ServeFile methods.
This is quite a general requirement; did I miss something or do I need to roll my own solution?

The New York Times have released their gzip middleware package for Go.
You just pass your http.HandlerFunc through their GzipHandler and you're done. It looks like this:
package main
import (
"io"
"net/http"
"github.com/nytimes/gziphandler"
)
func main() {
withoutGz := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/plain")
io.WriteString(w, "Hello, World")
})
withGz := gziphandler.GzipHandler(withoutGz)
http.Handle("/", withGz)
http.ListenAndServe("0.0.0.0:8000", nil)
}

There is no “out of the box” support for gzip-compressed HTTP responses yet. But adding it is pretty trivial. Have a look at
https://gist.github.com/the42/1956518
also
https://groups.google.com/forum/?fromgroups=#!topic/golang-nuts/cgUp8_ATNtc

For the sake of completeness, I eventually answered my own question with a handler that is simple and specialises in solving this issue.
https://pkg.go.dev/github.com/rickb777/servefiles?tab=doc
https://github.com/rickb777/servefiles
This serves static files from a Go http server, including the asked-for performance-enhancing features. It is based on the standard net/http ServeFiles, with gzip/brotli and cache performance enhancements.

There is yet another "out of the box" middleware now, supporting net/http and Gin:
https://github.com/nanmu42/gzip
net/http example:
import github.com/nanmu42/gzip
func main() {
mux := http.NewServeMux()
mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
writeString(w, fmt.Sprintf("This content is compressed: l%sng!", strings.Repeat("o", 1000)))
})
// wrap http.Handler using default settings
log.Println(http.ListenAndServe(fmt.Sprintf(":%d", 3001), gzip.DefaultHandler().WrapHandler(mux)))
}
func writeString(w http.ResponseWriter, payload string) {
w.Header().Set("Content-Type", "text/plain; charset=utf8")
_, _ = io.WriteString(w, payload+"\n")
}
Gin example:
import github.com/nanmu42/gzip
func main() {
g := gin.Default()
// use default settings
g.Use(gzip.DefaultHandler().Gin)
g.GET("/", func(c *gin.Context) {
c.JSON(http.StatusOK, map[string]interface{}{
"code": 0,
"msg": "hello",
"data": fmt.Sprintf("l%sng!", strings.Repeat("o", 1000)),
})
})
log.Println(g.Run(fmt.Sprintf(":%d", 3000)))
}

Related

Set base path for follow-up HTTP request

Suppose there is a file called foo.html and a project structure that looks like this:
|--styles
| |--style.css 📜
|--pages
| |--foo.html 📜
foo.html contains (among other stuff):
<link rel="stylesheet" type="text/css" href="styles/style.css">
Now, when the client requests for pages/foo.html, it will see the link to the css file and it will make a follow-up request to pages/styles/style.css. Is there a way I can instead tell it from the file server to make a request to styles/style.css rather than pages/styles/style.css?
I'm using the Go http library from the standard library.
I guess you are already using http go package. Here is the below sample code which can help you to achieve what you intend to do:
package main
import (
"fmt"
"log"
"net/http"
)
func hello(w http.ResponseWriter, req *http.Request) {
fmt.Fprintf(w, "hello1\n")
}
func main() {
fs := http.FileServer(http.Dir("./page/static"))
http.Handle("/page/styles/", http.StripPrefix("/page/styles/", fs))
page_fs := http.FileServer(http.Dir("./page"))
http.Handle("/page/", http.StripPrefix("/page/", page_fs))
http.HandleFunc("/hello", hello)
log.Println("Listening on :3000...")
err := http.ListenAndServe(":3000", nil)
if err != nil {
log.Fatal(err)
}
}
Let me know if you need explanation.

Handling custom 404 pages with http.FileServer

I'm currently using a basic http.FileServer setup to serve a simple static site. I need to handle 404 errors with a custom not found page. I've been looking into this issue quite a bit, and I cannot determine what the best solution is.
I've seen several responses on GitHub issues along the lines of:
You can implement your own ResponseWriter which writes a custom message after WriteHeader.
It seems like this is the best approach but I'm a bit unsure of how this would actually be implemented. If there are any simple examples of this implementation, it'd be greatly appreciated!
I think this can be solved with your own middleware. You can try to open the file first and if it doesn't exist, call your own 404 handler. Otherwise just dispatch the call to the static file server in the standard library.
Here is how that could look:
package main
import (
"fmt"
"net/http"
"os"
"path"
)
func notFound(w http.ResponseWriter, r *http.Request) {
// Here you can send your custom 404 back.
fmt.Fprintf(w, "404")
}
func customNotFound(fs http.FileSystem) http.Handler {
fileServer := http.FileServer(fs)
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_, err := fs.Open(path.Clean(r.URL.Path)) // Do not allow path traversals.
if os.IsNotExist(err) {
notFound(w, r)
return
}
fileServer.ServeHTTP(w, r)
})
}
func main() {
http.ListenAndServe(":8080", customNotFound(http.Dir("/path/to/files")))
}

How to add custom http header in go fasthttp ?

I am using a fasthttp server https://github.com/valyala/fasthttp
I need to add a custom header for all requests
Access-Control-Allow-Origin: *
How can I do this ?
Since it's a response header i assume you mean this:
ctx.Response.Header.Set("Access-Control-Allow-Origin", "*")
Another option if you are not using Context:
func setResponseHeader(h http.HandlerFunc) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Access-Control-Allow-Origin", "*")
h.ServeHTTP(w, r)
}
}
setResponseHeader is essentially a decorator of the argument HandlerFunc h. When you assemble your routes, you can do something like this:
http.HandleFunc("/api/endpoint", setResponseHeader(myHandlerFunc))
http.ListenAndServe(":8000", nil)
To enable CORS support on fasthttp, better use fasthttpcors package.
import (
...
cors "github.com/AdhityaRamadhanus/fasthttpcors"
...
)
func main() {
...
withCors := cors.NewCorsHandler(cors.Options{
AllowMaxAge: math.MaxInt32,
})
log.Fatal(fasthttp.ListenAndServe(":8080", withCors.CorsMiddleware(router.HandleRequest)))
}

How to process GET operation (CRUD) in go lang via Postman?

I want to perform a get operation. I am passng name as a resource to the URL.
The URL I am hitting in Postman is : localhost:8080/location/{titan rolex} ( I chose the GET method in the dropdown list)
On the URL hit in Postman, I am executing the GetUser func() with body as:
func GetUser(rw http.ResponseWriter, req *http.Request) {
}
Now I wish to get the resource value i.e 'titan rolex' in the GetUser method.
How can I achieve this in golang?
In main(), I have this :
http.HandleFunc("/location/{titan rolex}", GetUser)
Thanks in advance.
What you are doing is binding the complete path /location/{titan rolex} to be handled by GetUser.
What you really want is to bind /location/<every possible string> to be handled by one handler (e.g. LocationHandler).
You can do that with either the standard library or another router. I will present both ways:
Standard lib:
import (
"fmt"
"net/http"
"log"
)
func locationHandler(w http.ResponseWriter, r *http.Request) {
name := r.URL.Path[len("/location/"):]
fmt.Fprintf(w, "Location: %s\n", name)
}
func main() {
http.HandleFunc("/location/", locationHandler)
log.Fatal(http.ListenAndServe(":8080", nil))
}
Note however, more complex paths (such as /location/<every possible string>/<some int>/<another string>) will be tedious to implement this way.
The other way is to use github.com/julienschmidt/httprouter, especially if you encounter these situations more often (and have more complex paths).
Here's an example for your use case:
import (
"fmt"
"github.com/julienschmidt/httprouter"
"net/http"
"log"
)
func LocationHandler(w http.ResponseWriter, r *http.Request, ps httprouter.Params) {
fmt.Fprintf(w, "Location: %s\n", ps.ByName("loc"))
}
func main() {
router := httprouter.New()
router.GET("/location/:loc", LocationHandler)
log.Fatal(http.ListenAndServe(":8080", router))
}
Note that httprouter uses a slightly different signature for handlers. This is because, as you can see, it passes these parameters to the functions as well.
Oh and another note, you can just hit http://localhost:8080/location/titan rolex with your browser (or something else) - if that something else is decent enough, it will URLEncode that to be http://localhost:8080/location/titan%20rolex.

Golang: Why does response.Get("headerkey") not return a value in this code?

This has been bothering me for the past couple hours, I'm trying to get a response header value. Simple stuff. If I curl a request to this running server, I see the header set, with curl's -v flag, but when I try to retrieve the header using Go's response.Header.Get(), it shows a blank string "", with the header's length being 0.
What frustrates me even more, is that the header value is actually set within the response when I print out the body (as demonstrated below).
Any and all help with this is appreciated, thanks in advance.
I have this code here:
http://play.golang.org/p/JaYTfVoDsq
Which contains the following:
package main
import (
"fmt"
"io/ioutil"
"net/http"
"net/http/httptest"
)
func main() {
mux := http.NewServeMux()
server := httptest.NewServer(mux)
defer server.Close()
mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
r.Header.Set("Authorization", "responseAuthVal")
fmt.Fprintln(w, r.Header)
})
req, _ := http.NewRequest("GET", server.URL, nil)
res, _:= http.DefaultClient.Do(req)
headerVal := res.Header.Get("Authorization")
fmt.Printf("auth header=%s, with length=%d\n", headerVal, len(headerVal))
content, _ := ioutil.ReadAll(res.Body)
fmt.Printf("res.Body=%s", content)
res.Body.Close()
}
The output to this running code is:
auth header=, with length=0
res.Body=map[Authorization:[responseAuthVal] User-Agent:[Go-http-client/1.1] Accept-Encoding:[gzip]]
This line:
r.Header.Set("Authorization", "responseAuthVal")
set the value of r *http.Request, that is the incomming request, while you want to set the value of w http.ResponseWriter, the response that you will receive.
The said line should be
w.Header().Set("Authorization", "responseAuthVal")
See this playgroud.

Resources