I was using http.NewRequest to make a GET Request.
I deliberately tried to tamper the API url just to check if my err handling works.
But its not working as expected . In err value is returned and I am unable to compare it.
jsonData := map[string]string{"firstname": "Nic", "lastname": "Raboy"}
jsonValue, _ := json.Marshal(jsonData)
request, err := http.NewRequest("POST", "http://httpbin.org/postsdf", bytes.NewBuffer(jsonValue))
request.Header.Set("Content-Type", "application/json")
client := &http.Client{}
response, err := client.Do(request)
if err != nil {
fmt.Println("wrong")
} else {
data, _ := ioutil.ReadAll(response.Body)
fmt.Println(string(data))
}
The output is as below:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>404 Not Found</title>
<h1>Not Found</h1>
<p>The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.</p>
But I am expecting to print "Wrong".
The HTTP call succeeds (the call went through to the server, and response came back), that's why err is nil. It's just that the HTTP status code is not http.StatusOK (but by judging the response document, it's http.StatusNotFound).
You should check the HTTP status code like this:
response, err := client.Do(request)
if err != nil {
fmt.Println("HTTP call failed:", err)
return
}
// Don't forget, you're expected to close response body even if you don't want to read it.
defer response.Body.Close()
if response.StatusCode != http.StatusOK {
fmt.Println("Non-OK HTTP status:", response.StatusCode)
// You may read / inspect response body
return
}
// All is OK, server reported success.
Also note that certain API endpoints might return other than http.StatusOK for success, such as HTTP 201 - Created, or HTTP 202 - Accepted etc. If you want to check all success status codes, you can do it like this:
// Success is indicated with 2xx status codes:
statusOK := response.StatusCode >= 200 && response.StatusCode < 300
if !statusOK {
fmt.Println("Non-OK HTTP status:", response.StatusCode)
// You may read / inspect response body
return
}
you should user the status code, you can also write a small handler for every http request that you create.
response, err := client.Do(request)
switch response.StatusCode {
case 200:
fmt.Println("work!")
break
case 404:
fmt.Println("not found!")
break
default:
fmt.Println("http", response.Status)
}
Related
I have built a server with few endpoints, in 1 of them i am taking logs in post body and jwt token in the authorization header. In this API i parse the token, validate it and then get the body. It is working file but sometimes i get these errors while reading the body:
client disconnected
stream error: stream ID 3; CANCEL
Reading body:
body, err := ioutil.ReadAll(r.Body) // r *http.Request
Why am i getting these errors and how can i resolve them? Thanks
EDIT: THe endpoint handler
func logs(w http.ResponseWriter, r *http.Request) {
var auth classes.AuthToken
// Validate token and get the needed data from it
if err := dbAndAuth.ValTokAndGetD(r, &auth, tokenKey); err != nil {
if err != nil {
if err.Error() == "" {
classes.LogError(false, "got an empty error")
}
errD := classes.Resp{
Success: false,
Message: err.Error(),
ErrCode: http.StatusUnauthorized,
}
json.NewEncoder(w).Encode(errD)
return
}
}
body, err := ioutil.ReadAll(r.Body) // Got the error here
// parse the body to get the data
Without seeing the rest of the code it is hard to answer, but the likely answer is that the stream is being closed before it can all be read.
This could be due to the body being closed (or any other means of manually closing prematurely).
Or the parsing is being run in a separate routine which isn't given chance to complete and respond in a [buffered?] channel back before the request completes.
I'm writing a post endpoint and in my error handling I want to return a 400.
log.Println("Artifact post requested")
// Check for valid JSON Body
body, err := getJSONBody(r)
if err != nil {
log.Println("we got an error")
log.Println(err)
w.WriteHeader(http.StatusBadRequest)
return
}
func getJSONBody(r *http.Request) ([]byte, error) {
if r.Body == nil {
fmt.Println("body has issues")
return nil, errors.New("Received an empty body")
}
defer r.Body.Close()
// Read body
body, readErr := ioutil.ReadAll(r.Body)
if readErr != nil {
return nil, readErr
}
return body, nil
}
But when I run unit tests I get a 200 instead of 400. Any ideas why this may happen?
Test output
It was something wrong with the Unit tests, making multiple requests failed but making a single request passed (got the 400 I was looking for).
I observed that Content-Length header is not getting set for PATCH requests with empty/nil payload. Even if we manually set it by req.Header.Set("content-length", "0") it is not actually getting set in the out going request.
This strange behaviour (Go bug?) happens only for PATCH requests and only when the payload is empty or nil (or set to http.NoBody)
package main
import (
"fmt"
"io/ioutil"
"net/http"
"strings"
)
func main() {
url := "http://localhost:9999"
method := "PATCH"
payload := strings.NewReader("")
client := &http.Client {
}
req, err := http.NewRequest(method, url, payload)
if err != nil {
fmt.Println(err)
}
req.Header.Set("Authorization", "Bearer my-token")
req.Header.Set("Content-Length", "0") //this is not honoured
res, err := client.Do(req)
defer res.Body.Close()
body, err := ioutil.ReadAll(res.Body)
fmt.Println(string(body))
}
This is reproducible even in the latest go version 1.15.
Just run the above code against a simple http server and see for yourself.
Is there any solution/workaround to send a PATCH request with Content-Length set to 0 ?
You can tell the HTTP client to include a Content-Length header with value 0 by setting TransferEncoding to identity as follows:
url := "http://localhost:9999"
method := "PATCH"
client := &http.Client{}
req, err := http.NewRequest(method, url, http.NoBody)
if err != nil {
panic(err)
}
req.TransferEncoding = []string{"identity"}
req.Header.Set("Authorization", "Bearer my-token")
// req.Header.Set("Content-Length", "0")
Note the following changes to your original code:
the important one: req.TransferEncoding = []string{"identity"}
the idiomatic way of specifying an empty body: http.NoBody (no impact on sending the length)
commented out req.Header.Set("Content-Length", "0"), the client fills it in by itself
also changed to panic on an error, you probably don't want to continue
The transfer encoding of identity is not written to the request, so except for the header Content-Length = 0, the request looks the same as before.
This is unfortunately not documented (feel free to file an issue with the Go team), but can be seen in the following code:
The tedious details:
transferWriter.writeHeader checks the following to write the Content-Length header:
// Write Content-Length and/or Transfer-Encoding whose values are a
// function of the sanitized field triple (Body, ContentLength,
// TransferEncoding)
if t.shouldSendContentLength() {
if _, err := io.WriteString(w, "Content-Length: "); err != nil {
return err
}
if _, err := io.WriteString(w, strconv.FormatInt(t.ContentLength, 10)+"\r\n"); err != nil {
return err
}
In turn, shouldCheckContentLength looks at the transfer encoding in case of zero length:
if t.ContentLength == 0 && isIdentity(t.TransferEncoding) {
if t.Method == "GET" || t.Method == "HEAD" {
return false
}
return true
}
The isIdentity verifies that TransferEncoding is exactly []string{"identity"}:
func isIdentity(te []string) bool { return len(te) == 1 && te[0] == "identity" })
Go version: go1.8.1 windows/amd64
Sample code for HTTP request is:
func (c *Client) RoundTripSoap12(action string, in, out Message) error {
fmt.Println("****************************************************************")
headerFunc := func(r *http.Request) {
r.Header.Add("Content-Type", fmt.Sprintf("text/xml; charset=utf-8"))
r.Header.Add("SOAPAction", fmt.Sprintf(action))
r.Cookies()
}
return doRoundTrip(c, headerFunc, in, out)
}
func doRoundTrip(c *Client, setHeaders func(*http.Request), in, out Message) error {
req := &Envelope{
EnvelopeAttr: c.Envelope,
NSAttr: c.Namespace,
Header: c.Header,
Body: Body{Message: in},
}
if req.EnvelopeAttr == "" {
req.EnvelopeAttr = "http://schemas.xmlsoap.org/soap/envelope/"
}
if req.NSAttr == "" {
req.NSAttr = c.URL
}
var b bytes.Buffer
err := xml.NewEncoder(&b).Encode(req)
if err != nil {
return err
}
cli := c.Config
if cli == nil {
cli = http.DefaultClient
}
r, err := http.NewRequest("POST", c.URL, &b)
if err != nil {
return err
}
setHeaders(r)
if c.Pre != nil {
c.Pre(r)
}
fmt.Println("*************", r)
resp, err := cli.Do(r)
if err != nil {
fmt.Println("error occured is as follows ", err)
return err
}
fmt.Println("response headers are: ", resp.Header.Get("sprequestguid"))
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
// read only the first Mb of the body in error case
limReader := io.LimitReader(resp.Body, 1024*1024)
body, _ := ioutil.ReadAll(limReader)
return fmt.Errorf("%q: %q", resp.Status, body)
}
return xml.NewDecoder(resp.Body).Decode(out)
I will call the RoundTripSoap12 function on the corresponding HTTP client.
When I send a request for the first time I will be getting some headers in the HTTP response, so these HTTP response headers should be sent as-is in my next HTTP request.
You may be interested in the httputil package and the reverse proxy example provided if you wish to proxy requests transparently:
https://golang.org/src/net/http/httputil/reverseproxy.go
You can copy the headers from one request to another one fairly easily - the Header is a separate object, if r and rc are http.Requests and you don't mind them sharing a header (you may need to clone instead if you want independent requests):
rc.Header = r.Header // note shallow copy
fmt.Println("Headers", r.Header, rc.Header)
https://play.golang.org/p/q2KUHa_qiP
Or you can look through keys and values and only copy certain headers, and/or do a clone instead to ensure you share no memory. See the http util package here for examples of this - see the functions cloneHeader and copyHeader inside reverseproxy.go linked above.
We can register CheckRedirect to check the next request when the request is redirected. Is there a way that I can get the response for the first request when it's redirected?
The way it is currently implemented, it doesn't seem possible to have a look at the response by default (unless you implement yourself what Do() does).
See src/net/http/client.go#L384-L399:
if shouldRedirect(resp.StatusCode) {
// Read the body if small so underlying TCP connection will be re-used.
// No need to check for errors: if it fails, Transport won't reuse it anyway.
const maxBodySlurpSize = 2 << 10
if resp.ContentLength == -1 || resp.ContentLength <= maxBodySlurpSize {
io.CopyN(ioutil.Discard, resp.Body, maxBodySlurpSize)
}
resp.Body.Close()
if urlStr = resp.Header.Get("Location"); urlStr == "" {
err = fmt.Errorf("%d response missing Location header", resp.StatusCode)
break
}
base = req.URL
via = append(via, req)
continue
}
What do you want to do with the first response? It will be pretty boring.
I think the most sensible thing would be to disable automatically following redirects (always return a non-nil error from CheckRedirect) and handle the redirection yourself in which case you have full access to all requests/responses.
My workaround for this:
Any http request / response could be patched within a custom RoundTripper, including redirects:
type RedirectChecker struct{}
func (RedirectChecker) RoundTrip(req *http.Request) (*http.Response, error) {
// e.g. patch the request before send it
req.Header.Set("user-agent", "curl/7.64.1")
resp, err := http.DefaultTransport.RoundTrip(req)
if err == nil && resp != nil {
switch resp.StatusCode {
case
http.StatusMovedPermanently,
http.StatusFound,
http.StatusTemporaryRedirect,
http.StatusPermanentRedirect:
// e.g. stop further redirections
// roughly equivalent to http.ErrUseLastResponse
resp.StatusCode = http.StatusOK
// e.g. read the Set-Cookie headers
// unfortunately cookie jars do not handle redirects in a proper manner
// and that's why I came to this question...
fmt.Printf("%+v", resp.Cookies())
}
}
}
httpClient := &http.Client{Transport: RedirectChecker{}}
httpClient.Do(...)