Convert interface{} parameter into *http.Request or *http.Response in Go - http

I'm tying to create an util function that will read body of the Request/Response and return it.
Here's what I've done for the moment:
func GetBody(in interface{}) []byte {
var body io.Reader
var statusCode int
switch v := in.(type) {
case *http.Request, *http.Response:
body = v.Body
statusCode = v.StatusCode
default:
log.Fatal("Only http.Request and http.Response parameters can be accepted to parse body")
}
if statusCode != 200 {
log.Fatalf("Received status code [%d] instead of [200]", statusCode)
}
body, err := ioutil.ReadAll(body)
if err != nil {
log.Fatal(err)
}
return body
}
But I'm receiving an error of compiler: v.Body undefined (type interface {} is interface with no methods)
Am I missing something or it's impossible to make a generic function that will server both for *http.Request and *http.Response

It is because of the double case.
v is still an interface{} because it could be either an *http.Request or an *http.Response
switch v := in.(type) {
case *http.Request
body = v.Body
statusCode = v.StatusCode
case *http.Response:
body = v.Body
statusCode = v.StatusCode
default:
log.Fatal("Only http.Request and http.Response parameters can be accepted to parse body")
}
This should work

In addition to the other answers which directly address the issue with your type switch, I'd like to point out an alternative solution.
Note, the interface{} solution is perfectly fine and may easily be considered preferable to this. This is presented for edification.
First, as a slight aside, if the thing you were interested in was a common method (e.g. Write or Cookies) rather than a common field (Body), it would be easy and better to access it via a custom interface.
Either by defining a type:
type cookier interface { // Should probably use a better name
Cookies() []*http.Cookie
}
func ShowCookies1(r cookier) {
log.Println("Got cookies:", r.Cookies())
}
Or by using an anonymous type in the function definition:
func ShowCookies2(r interface {
Cookies() []*http.Cookie
}) {
log.Println("Got cookies:", r.Cookies())
}
These functions can accept anything that has a Cookies method, this includes *http.Request and *http.Response.
Unfortunately, in your particular case you wish to access a common field rather than a common method so you can't directly just use a matching interface.
You could make a small wrapper type that adds a GetBody method (one could argue such a function should have been defined in the standard package).
type reqbody struct{ *http.Request }
type respbody struct{ *http.Response }
type getbody interface {
GetBody() io.ReadCloser
}
func (r reqbody) GetBody() io.ReadCloser { return r.Body }
func (r respbody) GetBody() io.ReadCloser { return r.Body }
func GetBody2(r getbody) ([]byte, error) {
body := r.GetBody()
defer body.Close()
return ioutil.ReadAll(body)
}
The caller knows what type they have and does one of:
buf, err = GetBody2(reqbody{req})
buf, err = GetBody2(respbody{resp})
In some sense this is uglier than just using interface{}.
But it has the benefit that instead of having a function that takes absolutely any type and panics/errors at run-time if a programmer mistakenly calls it with something not of the appropriate type,
this instead forces the caller to safely pass something you known is of the correct type at compile time.
Looking at this further, you're just reading everything from an io.ReadCloser and then closing it so it could further be simplified to the following
(which probably is better than your interface{} solution):
func GetReqBody(r *http.Request) io.ReadCloser { return r.Body }
// Could add checking r.StatusCode to the following one as well:
func GetRespBody(r *http.Response) io.ReadCloser { return r.Body }
func ReadAndClose(rc io.ReadCloser) ([]byte, error) {
defer rc.Close()
return ioutil.ReadAll(rc)
}
Again, the caller knows what type they have and does one of:
buf, err = ReadAndClose(GetReqBody(req))
buf, err = ReadAndClose(GetRespBody(resp))
Or just:
buf, err = ReadAndClose(req.Body)
buf, err = ReadAndClose(resp.Body)
You can see an example of all these options on the Go Playground.
Finally, be careful using ioutil.ReadAll.
Often it is better to avoid pre-reading an entire file or network steam into a buffer and instead process it as a stream as it is being read.
In particular, it's trivial to make an HTTP request with an arbitrary large body as a denial of service attack or to waste server resources
(http.MaxBytesReader can also help).

What you are trying to do is more something like that:
switch in.(type) {
case *http.Request:
body = v.(*http.Request).Body
case *http.Response:
body = v.(*http.Response).Body
default:
log.Fatal(...)
}
EDIT: I removed the wrong part of my answer, see HectorJ's answer for the more syntaxic please way of doing this.

Related

check whether http connection is hijacked in go server

I am writing an HTTP server in Go, which uses the following pattern to handle API output:
func handler(w http.ResponsWriter, r *http.Request) {
defer reply(w, r, L)() //L is a Logger
//do things...
}
func reply(w http.ResponseWriter, r *http.Request, log Logger) func() {
cid := []byte{0, 0, 0, 0}
if log != nil {
rand.Read(cid)
log.Debug("[%x] %s %s", cid, r.Method, r.URL.String())
}
entry := time.Now()
return func() {
if log != nil {
defer log.Debug("[%x] elapsed %d millis", cid, time.Since(entry).Milliseconds())
}
_, err := w.Write(nil)
if err == http.ErrHijacked {
return //API is a WEBSOCKET entry point, do nothing
}
//handle common output logic for normal HTTP APIs...
}
}
The reason I do this, is that I found this comment in the standard library:
// ErrHijacked is returned by ResponseWriter.Write calls when
// the underlying connection has been hijacked using the
// Hijacker interface. A zero-byte write on a hijacked
// connection will return ErrHijacked without any other side
// effects.
ErrHijacked = errors.New("http: connection has been hijacked")
However following the Write() method, I got this comment:
// Write writes the data to the connection as part of an HTTP reply.
//
// If WriteHeader has not yet been called, Write calls
// WriteHeader(http.StatusOK) before writing the data. If the Header
// does not contain a Content-Type line, Write adds a Content-Type set
// to the result of passing the initial 512 bytes of written data to
// ...
Write([]byte) (int, error)
My questions are:
Is it OK to use my code to safely detect if a HTTP connection is hijacked? I only want to check the connection is hijacked or not, but do NOT want it to add headers for me!
Since the ResponseWriter is an interface, I cannot click through the source code to find out how the standard library implements that method. In general, how can I drill down to the standard library (or any open source code) to find out the implementation of an interface?
Thanks to Cerise, I found the source code of the standard response.Writer:
func (w *response) write(lenData int, dataB []byte, dataS string) (n int, err error) {
if w.conn.hijacked() {
if lenData > 0 {
caller := relevantCaller()
w.conn.server.logf("http: response.Write on hijacked connection from %s (%s:%d)", caller.Function, path.Base(caller.File), caller.Line)
}
return 0, ErrHijacked
}
... ....
So, as said in the document, there is NO side effect.

Go: How to create a server which can serve urls described in config file

could anyone help me here please as I'm new to golang? I have a yaml file which looks like this:
port: 5000
handlers:
- name: test1
uri: /api/test1
response:
status: 200
body: test1
- name: test2
uri: /api/test2
response:
status: 500
body: test2
based on this file I want to create a server. Currently I'm trying to do it this way, but looks like it doesn't work as expected.
What am I doing wrong and what is the better way to achieve what I need?
package main
import (
"fmt"
"io/ioutil"
"log"
"net/http"
"gopkg.in/yaml.v2"
)
func main() {
config := parseYaml("conf.yaml")
configHandlers := config.Handlers
mux := http.NewServeMux()
for _, handler := range *configHandlers {
mux.HandleFunc(*handler.Uri, func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(*handler.Response.Status)
fmt.Fprintf(w, *handler.Response.Body)
})
}
log.Fatal(http.ListenAndServe(fmt.Sprintf(":%s", *config.Port), mux))
}
type YamlConfig struct {
Port *string `yaml:"port"`
Handlers *[]HandlerConfig `yaml:"handlers"`
}
type HandlerConfig struct {
Uri *string `yaml:"uri"`
Name *string `yaml:"name"`
Response *Response `yaml:"response"`
}
type Response struct {
Status *int `yaml:"status"`
Body *string `yaml:"body"`
}
func (c *YamlConfig) parseYaml(data []byte) error {
return yaml.Unmarshal(data, c)
}
func parseYaml(path string) YamlConfig {
data, err := ioutil.ReadFile(path)
if err != nil {
log.Fatal(err)
}
var config YamlConfig
if err := config.parseYaml(data); err != nil {
log.Fatal(err)
}
return config
}
Update:
If I run this server then regardless of which endpoint I hit, it will always return me 500 and test2 in body
What you're seeing is seemingly a common pitfall for people:
configHandlers := config.Handlers
mux := http.NewServeMux()
for _, handler := range *configHandlers {
mux.HandleFunc(*handler.Uri, func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(*handler.Response.Status)
fmt.Fprintf(w, *handler.Response.Body)
})
}
The for loop, on each iteration, reassigns the handler variable. In the loop body, you create a new function and pass it to mux.HandlerFun. These function bodies kind of inherit the outer scope, and access this handler variable. The variable is reassigned outside of the functions, and thus the values each handler function has access to changes with it. What you can do to address the issue is mask the handler variable the loop uses, and create a scope that is unique to each handler. The classic way in languages like JavaScript (where this is - or used to be back when I wrote some JS - a common issue) is to wrap the code in an IIFE (Immediately Invoked Function Expression):
for _, handler := range *configHandlers {
func (handler *HandlerConfig) { // handler is now the argument passed to this function
mux.HandleFunc(*handler.Uri, func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(*handler.Response.Status)
fmt.Fprintf(w, *handler.Response.Body)
})
}(handler) // call the function with the _current_ value of handler
}
This is a tad messy, and because golang is properly block-scoped, you can just do this:
for _, handler := range *configHandlers {
h := handler // create a variable in the inner scope
mux.HandleFunc(*handler.Uri, func(w http.ResponseWriter, r *http.Request) {
// now h will reference a copy unique to each iteration
w.WriteHeader(*h.Response.Status)
fmt.Fprintf(w, *h.Response.Body)
})
}
That ought to fix it. I've noticed some weirdness with your use of pointers in the types you've added to your question, though... Fields like Port being of type *string? Why wouldn't you just use string? No Same for the Body and Status fields in the Response type. By changing them to plain string fields you don't have to dereference them in your handler functions. It will look a lot cleaner.
A bigger worry is this field:
Handlers *[]HandlerConfig `yaml:"handlers"`
I'm not sure if you really know what the type of this field is, but it makes next to no sense. Handlers is now a pointer to a slice of HandlerConfig values. I'm assuming you wanted this field to be:
// Handlers is a slice of HandlerConfig values:
Handlers []HandlerConfig `yaml:"handlers"`
// or Handlers is a slice of pointers to HandlerConfig values
Handlers []*HandlerConfig `yaml:"handlers"`
Generally speaking, a pointer to a slice, especially in a config type is bad code.
If you define a struct that will represent the configuration in your YAML file, you can unmarshall the yaml into an instantiated struct of that type using the yaml package. From there, you can reference the fields in the struct as any other struct.
package main
import (
"fmt"
"gopkg.in/yaml.v2"
)
type YamlExample struct {
FieldOne string `yaml:"fieldOne"`
NestedField struct {
Name string `yaml:"name"`
} `yaml:"nestedField"`
}
const YamlEx string = `
fieldOne: one
nestedField:
name: nestedFieldName
`
func main() {
var yamlE YamlExample
err := yaml.Unmarshal([]byte(YamlEx), &yamlE)
if err != nil {
panic(err)
}
fmt.Printf("%+v\n", yamlE)
}
Link to example.
In your case, you'd probably want to handle the routes in a struct and then reference the fields in the struct for things like route name, how to handle the body of the request, etc. If your YAML is stored in a file, you'll have to use something like the io package to read the file into a byte array that the YAML package can parse. See here for a reference.

Adding custom transport to a third party package

I have been using GoRequest as a package in my Go application.
I use it as it helps make all the API calls I need to make a lot cleaner - one thing that it is lacking, that I am able to do with a regular http.Client is implement outbound rate limiting on the Transport.
For example, in one of my applications I use this -
type rateLimitTransport struct {
limiter *rate.Limiter
xport http.RoundTripper
}
var _ http.RoundTripper = &rateLimitTransport{}
func newRateLimitTransport(r float64, xport http.RoundTripper) http.RoundTripper {
return &rateLimitTransport{
limiter: rate.NewLimiter(rate.Limit(r), 1),
xport: xport,
}
}
func (t *rateLimitTransport) RoundTrip(r *http.Request) (*http.Response, error) {
t.limiter.Wait(r.Context())
return t.xport.RoundTrip(r)
}
var myClient = http.Client{
// Use a rate-limiting transport which falls back to the default
Transport: newRateLimitTransport(1, http.DefaultTransport),
}
This allows my to create a client, that is rate limited to 1 request per second; 60 requests per minute.
I have forked the GoRequest package and am trying to add a new method to it called SetRateLimit() which should take the rate per second as an argument and use this to implement a rate limited transport.
My attempt so far looks like this:
import (
//...current imports
"golang.org/x/time/rate"
)
//... Rest of the package
type rateLimitTransport struct {
limiter *rate.Limiter
xport http.RoundTripper
}
var _ http.RoundTripper = &rateLimitTransport{}
func newRateLimitTransport(r float64, xport http.RoundTripper) http.RoundTripper {
return &rateLimitTransport{
limiter: rate.NewLimiter(rate.Limit(r), 1),
xport: xport,
}
}
func (t *rateLimitTransport) RoundTrip(r *http.Request) (*http.Response, error) {
t.limiter.Wait(r.Context())
return t.xport.RoundTrip(r)
}
func (s *SuperAgent) SetRateLimit(limit float64) *SuperAgent {
s.Transport = newRateLimitTransport(limit, http.DefaultTransport)
return s
}
However I get an error when trying to build this:
cannot use newRateLimitTransport(limit, http.DefaultTransport) (type http.RoundTripper) as type *http.Transport in assignment: need type assertion
I've been looking at this for hours, and I don't quite understand how this works for a regular http.Client but doesn't work for this package.
Can someone please help me resolve the issue above and add rate limiting to this package?
Update - SuperRequest struct
// A SuperAgent is a object storing all request data for client.
type SuperAgent struct {
Url string
Method string
Header http.Header
TargetType string
ForceType string
Data map[string]interface{}
SliceData []interface{}
FormData url.Values
QueryData url.Values
FileData []File
BounceToRawString bool
RawString string
Client *http.Client
Transport *http.Transport
Cookies []*http.Cookie
Errors []error
BasicAuth struct{ Username, Password string }
Debug bool
CurlCommand bool
logger Logger
Retryable struct {
RetryableStatus []int
RetryerTime time.Duration
RetryerCount int
Attempt int
Enable bool
}
//If true prevents clearing Superagent data and makes it possible to reuse it for the next requests
DoNotClearSuperAgent bool
}
Error you are seeing says the issue.
cannot use newRateLimitTransport(limit, http.DefaultTransport) (type
http.RoundTripper) as type *http.Transport in assignment: need type
assertion
SuperAgent.Transport type is *http.Transport But you are trying to assign http.RoundTripper to that field
To fix this you could change the SuperaAgent
type SuperAgent{
...
Transport http.RoundTripper
...
}

non-pointer error in Go not sure what it means

I have a function that takes a json decoder and an interface as its arguements, and i am decoding to a struct that is passed on the interface. Like so:
func DecodeJSON(decoder *json.Decoder, i interface{}) bool {
if c, ok := i.(User); ok {
err := decoder.Decode(c)
if err != nil {
fmt.Println(err)
return false //err is not nil
}
}
return false
}
Usage of the function:
// Register test
func HandleRequest(w rest.ResponseWriter, r *rest.Request) {
decoder := json.NewDecoder(r.Body)
user := User{}
if DecodeJSON(decoder, user) {
fmt.Println("success")
}
Error i am getting:
json: Unmarshal(non-pointer main.User)
Bit confused by this message since my DecodeJSON is not taking a pointer for user. So am not sure what i have done wrong with my code. Hope some one can explain so i can understand my mistake.
You need to use a pointer to a user to decode the data in, otherwise the decoded data gets decoded in a copy of the object that gets deleted when the function returns.
func DecodeJSON(decoder *json.Decoder, i interface{}) bool {
if c, ok := i.(*User); ok {
err := decoder.Decode(c)
if err != nil {
fmt.Println(err)
return false //err is not nil
}
}
return false
}
// Register test
func HandleRequest(w rest.ResponseWriter, r *rest.Request) {
decoder := json.NewDecoder(r.Body)
user := &User{}
if DecodeJSON(decoder, user) {
fmt.Println("success")
}
Leave the interface parameter as is, just use a pointer when you pass the User and when you get the user from the interface.
Based on your code, you'd better change your function signature to
func DecodeJSON(decoder *json.Decoder, user *User) bool
This will (1) eliminate explicit runtime cast, and (2) make your code less ambiguous and safe and compile-time checked.

How to check if ResponseWriter has been written

Using Golang's net/http package, how can I check if the ResponseWriter has been written to? I am receiving the following error message:
http: multiple response.WriteHeader calls
Of course I can return booleans and such from my functions indicating weather I wrote to the ResponseWriter with a redirect or something, which I tried, but surely I am able to check if the ResponseWriter has been written to before I write to it with an easy method.
I am looking for a function that would look something like the following which I can use before I write to the ResponseWriter:
if w.Header().Get("Status-Code") == "" {
http.Redirect(w, r, "/", http.StatusSeeOther)
} else {
fmt.Println("Response Writer has already been written to.")
}
The code above doesn't seem to work... anyone have any idea how to check if the ResponseWriter has been written to or not?
The only way to do this is with a custom implementation of http.ResponseWriter:
type doneWriter struct {
http.ResponseWriter
done bool
}
func (w *doneWriter) WriteHeader(status int) {
w.done = true
w.ResponseWriter.WriteHeader(status)
}
func (w *doneWriter) Write(b []byte) (int, error) {
w.done = true
return w.ResponseWriter.Write(b)
}
func myMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
dw := &doneWriter{ResponseWriter: w}
next.ServeHTTP(dw, r)
if dw.done {
// Something already wrote a response
return
}
// Nothing else wrote a response
w.WriteHeader(http.StatusOK)
// Whatever you want here
}
}
I also threw together a simple package to handle this for me. Feel free to use it as well if you like.

Resources