Submitting data from front end to back end in Go - http

I have simple website. Go with JavaScript. Now I get this message "http: multiple response.WriteHeader calls" and I know that a have another header open. But I don't know where and I'm struggling to find a solution.
func (t *Server) RootHandler(w http.ResponseWriter, r *http.Request) {
var c *entities.Korisnik
var k *entities.Kilometri
var a *entities.Auto
if c = t.authentication(w, r); c == nil {
return
}
gk, err := t.store.GetKilometri(c)
if errorEval(w, err, http.StatusInternalServerError) {
return
}
if err := r.ParseForm(); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
if r.FormValue("debug") == "true" {
s, _ := json.MarshalIndent(&Bla{c, gk}, "", " ")
w.Write(s)
return
} else {
w.Header().Set("Content-Type", "application/json")
errorEval(w, json.NewEncoder(w).Encode(&Bla{c, gk}), http.StatusInternalServerError)
}
log.Println("0")
if errorEval(w, json.NewDecoder(r.Body).Decode(&Input{a, k}), http.StatusBadRequest) {
log.Println("1")
return
}
err = t.store.NewKilometri(k, c, a)
log.Println("2")
if errorEval(w, err, http.StatusInternalServerError) {
return
}
}
I get this in my terminal
015/10/20 16:12:32 0
2015/10/20 16:12:32 EOF
2015/10/20 16:12:32 http: multiple response.WriteHeader calls
2015/10/20 16:12:32 1
2015/10/20 16:12:32 0
2015/10/20 16:12:32 EOF
2015/10/20 16:12:32 http: multiple response.WriteHeader calls
2015/10/20 16:12:32 1
func errorEval(w http.ResponseWriter, err error, status int) bool {
if err == nil {
return false
}
log.Println(err)
http.Error(w, errorString[status], status)
return true
}

It looks to me like errorEval likely writes the status code and maybe a body if an error is encountered. Most places you call it you check the return and return from your handler if it handles an error.
In the case of errorEval(w, json.NewEncoder(w).Encode(&Bla{c, gk}), http.StatusInternalServerError) you are not checking the return.
My guess is, there is some json error and the handler is writing a 500 response, and you are continuing to do other things which in turn try to write additional responses.

Related

Golang large response unexpected end of JSON input

I'm sending a request to a server and trying to unmarshall HTTP Response. It's weird, but some of the responses return 200 and do not give an unmarshall error, while some of them return 200 and give an unmarshall error.
My client code looks like:
func SendRequest(requestModel *model.Request) (*model.Response, error) {
responseModel := &dspModel.Response{}
byteData, err := json.Marshal(requestModel)
if err != nil {
zap.S().Errorf("Error marshalling request. Err: %v", err)
return nil, err
}
url := "xx"
request, _ := http.NewRequest(http.MethodPost, url, bytes.NewBuffer(byteData))
request.Close = true
client := &http.Client{}
response, err := client.Do(request)
if err, ok := err.(net.Error); ok && err.Timeout() {
zap.S().Error("Response timeout exceed")
return nil, errors.New("Response timeout exceed")
}
if err != nil || response == nil {
errorMessage := "not respond"
zap.S().Error(errorMessage)
return nil, errors.New(errorMessage)
}
defer response.Body.Close()
if response.StatusCode == http.StatusOK {
err = json.NewDecoder(response.Body).Decode(&responseModel)
if err != nil {
// Error occurred here!
errorMessage := "Request response decode error"
zap.S().Errorf("%v, Err: %v", errorMessage, err)
return nil, errors.New(errorMessage)
}
return response, nil
} else if response.StatusCode == http.StatusNoContent {
return nil, nil
} else {
bodyBytes, _ := ioutil.ReadAll(response.Body)
errorMessage := "not respond"
zap.S().Errorf("%v, StatusCode %v, Response: %v Request: %v", errorMessage,
response.StatusCode, string(bodyBytes), string(byteData))
return nil, errors.New(errorMessage)
}
}
I suspect response cause it's too long and has different characters.
Response looks like:
{"tax":{"ver":"1.0"},"cur":"EUR","rack":[{"tur":[{"zar":2.599886212,"domain":["test.com"],"ney":"https://censored.com/nimp?fuid=&ic=EWRMh-.UTPdNvWB-JYa58c85N0fEPgXunKp3wwyPadp7jwqHgZbz4dG0A51OVO-2Gs0znYmLcPIH0ThEmpsYl8wKofo9ytJ2A3uWr9Kn-dNxeh.k8lIml9kavPk1.dk7f.46xKX7IVpf3.yU-Yx1KetQl3Q9f-iePn7B86yjVgMxkTNfhZAg0pP0kKZaJMd2orLXoV4xPXmwTdfJbWJU5bGAUROJT-Yd7yTHoVveuvOBClHzM4cgHFmGxzox6cCJ2gZB.7fqKkPzECXwdpobmO0RWxdu224-FADd.oM4DghIEpdZJe.FjEq0stQnJBT.puw0JamHgT15NdSQN7voBJ8UqGCDOu1qSLece6Iy.PN392xGWhxs0URbrWhSEgkhCr.R4ol9kjrMqK78shw2gHBJjEzKeBeo6lBzU8YfoKDM7oPlj5SwmL6sV2i2UaWJbEtreRt3oABDPab--AevfJW2rQ0-2iyt-rJSPjDHHoOQEFoh0G7cPm8SIZxk17ojWkFdM7CXlmuSN0paqMhp-4gWlfgvNq8a65I8GfY8cwVrW5KzRszHLhWYareVM3MNpejdcVH2kinEnYzBVyW0e8oN06LC2icG8FRlhOC2N8wni66liT73RvKyFFT1zW7SAoqtgn9KXY6m.EaZzSx3aapIMGpG9-S8q6mwAuwZId37ri4GTiLXp6OMABsLwT3sMUOm.Kktp.uYP1z2be2DFM6zKKPL7YJopAvdfS7TdhMfHD6Dfcv-EvK6Q0lNylaFIjegbNjPEPPXzLPdf8iwLK-dqfSe127TZcj5xJUFwo45IgFl0i0puKRIzsAtM2W3zM-TNc2HEc0nIllK.aoKZ0tF9iSekzjcNnMAvhcAKBqq6DY.qIBUs5yOoxqW4m-ga9drHp09VXIkn7st7J4IUlrMZFuVHnnzbeqD61AKKFiaRCqPee6Y88DqhsdNt7SzdA-xq9SKnJW67zsZTD0T9OoRl3.nLaSwoQ==&t=adi&prc=${censored}","tax":{"name":"test"},"hy":3020,"ny":2530,"arz":"<script type=\"text/javascript\">var _CDurl='';(function(){_CDurl=encodeURIComponent(document.URL);})();document.write('<scr' + 'ipt type=\"text/javascript\" src=\"https://censored/at?bad=&gad=&ic=EWRMh-.UTPdNvWB-JYa58c85N0fEPgXunKp3wwyPadp7jwqHgZbz4dG0A51OVO-2Gs0znYmLcPIH0ThEmpsYl8wKofo9ytJ2A3uWr9Kn-dNxeh.k8lIml9kavPk1.dk7f.46xKX7IVpf3.yU-Yx1KetQl3Q9f-iePn7B86yjVgMxkTNfhZAg0pP0kKZaJMd2orLXoV4xPXmwTdfJbWJU5bGAUROJT-Yd7yTHoVveuvOBClHzM4cgHFmGxzox6cCJ2gZB.7fqKkPzECXwdpobmO0RWxdu224-FADd.oM4DghIEpdZJe.FjEq0stQnJBT.puw0JamHgT15NdSQN7voBJ8UqGCDOu1qSLece6Iy.PN392xGWhxs0URbrWhSEgkhCr.R4ol9kjrMqK78shw2gHBJjEzKeBeo6lBzU8YfoKDM7oPlj5SwmL6sV2i2UaWJbEtreRt3oABDPab--AevfJW2rQ0-2iyt-rJSPjDHHoOQEFoh0G7cPm8SIZxk17ojWkFdM7CXlmuSN0paqMhp-4gWlfgvNq8a65I8GfY8cwVrW5KzRszHLhWYareVM3MNpejdcVH2kinEnYzBVyW0e8oN06LC2icG8FRlhOC2N8wni66liT73RvKyFFT1zW7SAoqtgn9KXY6m.EaZzSx3aapIMGpG9-S8q6mwAuwZId37ri4GTiLXp6OMABsLwT3sMUOm.Kktp.uYP1z2be2DFM6zKKPL7YJopAvdfS7TdhMfHD6Dfcv-EvK6Q0lNylaFIjegbNjPEPPXzLPdf8iwLK-dqfSe127TZcj5xJUFwo45IgFl0i0puKRIzsAtM2W3zM-TNc2HEc0nIllK.aoKZ0tF9iSekzjcNnMAvhcAKBqq6DY.qIBUs5yOoxqW4m-ga9drHp09VXIkn7st7J4IUlrMZFuVHnnzbeqD61AKKFiaRCqPee6Y88DqhsdNt7SzdA-xq9SKnJW67zsZTD0T9OoRl3.nLaSwoQ==&t=adj&prc=${censored}&tat='+_CDurl+'\"></scr' + 'ipt>');</script>"}],"tark":"1"}],"gno":"55f03d71-f021-49e0-a1a5-cae4315b3561"}
When I debug the error, I noticed that after half of the response is not visible.
Error statement:
"msg":"Request response decode error, Response: {\"tax\":{\"ver\":\"1.0\"},\"cur\":\"EUR\",\"rac\":[{\"btyrd\":[{\"zar\":2.599886212,\"domain\":[\"test.com\"],\"ney\":\"https://censored/nimp?fuid=&ic=EWRMh-.UTPdNvWB-JYa58c85N0fEPgXunKp3wwyPadp7jwqHgZbz4dG0A51OVO-2Gs0znYmLcPIH0ThEmpsYl8wKofo9ytJ2A3uWr9Kn-dNxeh.k8lIml9kavPk1.dk7f.46xKX7IVpf3.yU-Yx1KetQl3Q9f-iePn7B86yjVgMxkTNfhZAg0pP0kKZaJMd2orLXoV4xPXmwTdfJbWJU5bGAUROJT-Yd7yTHoVveuvOBClHzM4cgHFmGxzox6cCJ2gZB.7fqKkPzECXwdpobmO0RWxdu224-FADd.oM4DghIEpdZJe.FjEq0stQnJBT.puw0JamHgT15NdSQN7voBJ8UqGCDOu1q, Err: unexpected end of JSON input"
Response Model:
type Response struct {
Tax Tax `json:"tax"`
Cur string `json:"cur"`
Rack []Rack `json:"rack"`
Gno string `json:"gno"`
}
type Tax struct {
Ver string `json:"ver"`
}
type TaxOfTur struct {
Name string `json:"name"`
}
type Tur struct {
Zar float64 `json:"zar"`
Domain []string `json:"domain"`
Ney string `json:"ney"`
Tax TaxOfTur `json:"tax"`
Hy int `json:"hy"`
Ny int `json:"ny"`
Arz string `json:"arz"`
}
type Rack struct {
Tur []Tur `json:"tur"`
Tark string `json:"tark"`
}
Probably unrelated but requests are going concurrent and my test case is based on only one concurrent request. So how can I solve this problem?
I tried to implement your code at my end, and it's giving the expected output. The code I tried is given below.
Note: This is not a solution to the exact problem, since I was not able to reproduce the same.
// Response struct definition here...
// ..
// The trouble making JSON.
const msg = `{"tax":{"ver":"1.0"},"cur":"EUR","rack":[{"tur":[{"zar":2.599886212,"domain":["test.com"],"ney":"https://censored.com/nimp?fuid=&ic=EWRMh-.UTPdNvWB-JYa58c85N0fEPgXunKp3wwyPadp7jwqHgZbz4dG0A51OVO-2Gs0znYmLcPIH0ThEmpsYl8wKofo9ytJ2A3uWr9Kn-dNxeh.k8lIml9kavPk1.dk7f.46xKX7IVpf3.yU-Yx1KetQl3Q9f-iePn7B86yjVgMxkTNfhZAg0pP0kKZaJMd2orLXoV4xPXmwTdfJbWJU5bGAUROJT-Yd7yTHoVveuvOBClHzM4cgHFmGxzox6cCJ2gZB.7fqKkPzECXwdpobmO0RWxdu224-FADd.oM4DghIEpdZJe.FjEq0stQnJBT.puw0JamHgT15NdSQN7voBJ8UqGCDOu1qSLece6Iy.PN392xGWhxs0URbrWhSEgkhCr.R4ol9kjrMqK78shw2gHBJjEzKeBeo6lBzU8YfoKDM7oPlj5SwmL6sV2i2UaWJbEtreRt3oABDPab--AevfJW2rQ0-2iyt-rJSPjDHHoOQEFoh0G7cPm8SIZxk17ojWkFdM7CXlmuSN0paqMhp-4gWlfgvNq8a65I8GfY8cwVrW5KzRszHLhWYareVM3MNpejdcVH2kinEnYzBVyW0e8oN06LC2icG8FRlhOC2N8wni66liT73RvKyFFT1zW7SAoqtgn9KXY6m.EaZzSx3aapIMGpG9-S8q6mwAuwZId37ri4GTiLXp6OMABsLwT3sMUOm.Kktp.uYP1z2be2DFM6zKKPL7YJopAvdfS7TdhMfHD6Dfcv-EvK6Q0lNylaFIjegbNjPEPPXzLPdf8iwLK-dqfSe127TZcj5xJUFwo45IgFl0i0puKRIzsAtM2W3zM-TNc2HEc0nIllK.aoKZ0tF9iSekzjcNnMAvhcAKBqq6DY.qIBUs5yOoxqW4m-ga9drHp09VXIkn7st7J4IUlrMZFuVHnnzbeqD61AKKFiaRCqPee6Y88DqhsdNt7SzdA-xq9SKnJW67zsZTD0T9OoRl3.nLaSwoQ==&t=adi&prc=${censored}","tax":{"name":"test"},"hy":3020,"ny":2530,"arz":"<script type=\"text/javascript\">var _CDurl='';(function(){_CDurl=encodeURIComponent(document.URL);})();document.write('<scr' + 'ipt type=\"text/javascript\" src=\"https://censored/at?bad=&gad=&ic=EWRMh-.UTPdNvWB-JYa58c85N0fEPgXunKp3wwyPadp7jwqHgZbz4dG0A51OVO-2Gs0znYmLcPIH0ThEmpsYl8wKofo9ytJ2A3uWr9Kn-dNxeh.k8lIml9kavPk1.dk7f.46xKX7IVpf3.yU-Yx1KetQl3Q9f-iePn7B86yjVgMxkTNfhZAg0pP0kKZaJMd2orLXoV4xPXmwTdfJbWJU5bGAUROJT-Yd7yTHoVveuvOBClHzM4cgHFmGxzox6cCJ2gZB.7fqKkPzECXwdpobmO0RWxdu224-FADd.oM4DghIEpdZJe.FjEq0stQnJBT.puw0JamHgT15NdSQN7voBJ8UqGCDOu1qSLece6Iy.PN392xGWhxs0URbrWhSEgkhCr.R4ol9kjrMqK78shw2gHBJjEzKeBeo6lBzU8YfoKDM7oPlj5SwmL6sV2i2UaWJbEtreRt3oABDPab--AevfJW2rQ0-2iyt-rJSPjDHHoOQEFoh0G7cPm8SIZxk17ojWkFdM7CXlmuSN0paqMhp-4gWlfgvNq8a65I8GfY8cwVrW5KzRszHLhWYareVM3MNpejdcVH2kinEnYzBVyW0e8oN06LC2icG8FRlhOC2N8wni66liT73RvKyFFT1zW7SAoqtgn9KXY6m.EaZzSx3aapIMGpG9-S8q6mwAuwZId37ri4GTiLXp6OMABsLwT3sMUOm.Kktp.uYP1z2be2DFM6zKKPL7YJopAvdfS7TdhMfHD6Dfcv-EvK6Q0lNylaFIjegbNjPEPPXzLPdf8iwLK-dqfSe127TZcj5xJUFwo45IgFl0i0puKRIzsAtM2W3zM-TNc2HEc0nIllK.aoKZ0tF9iSekzjcNnMAvhcAKBqq6DY.qIBUs5yOoxqW4m-ga9drHp09VXIkn7st7J4IUlrMZFuVHnnzbeqD61AKKFiaRCqPee6Y88DqhsdNt7SzdA-xq9SKnJW67zsZTD0T9OoRl3.nLaSwoQ==&t=adj&prc=${censored}&tat='+_CDurl+'\"></scr' + 'ipt>');</script>"}],"tark":"1"}],"gno":"55f03d71-f021-49e0-a1a5-cae4315b3561"}`
func SendRequest() (*Response, error) {
url := "http://localhost:8080/foo" // dummy server
request, _ := http.NewRequest(http.MethodPost, url, nil)
request.Close = true
client := &http.Client{}
response, err := client.Do(request)
if err != nil {
return nil, err
}
defer response.Body.Close()
resBody, err := ioutil.ReadAll(response.Body)
if err != nil {
return nil, err
}
responseModel := &Response{}
err = json.Unmarshal(resBody, &responseModel)
if err != nil {
return nil, err
}
return responseModel, nil
}
func StartDummyServer() {
handler := func(w http.ResponseWriter, r *http.Request) {
fmt.Fprint(w, msg)
}
http.HandleFunc("/foo", handler)
log.Fatal(http.ListenAndServe(":8080", nil))
}
func main() {
// a dummy server is created to send the response
go StartDummyServer()
time.Sleep(time.Second)
resp, err := SendRequest()
if err != nil {
log.Fatal(err.Error())
}
fmt.Println(prettyPrint(resp))
time.Sleep(time.Second * 10)
}
func prettyPrint(i interface{}) string {
s, _ := json.MarshalIndent(i, "", " ")
return string(s)
}
Here is the console output:
{
"tax": {
"ver": "1.0"
},
"cur": "EUR",
"rack": [
{
"tur": [
{
"zar": 2.599886212,
"domain": [
"test.com"
],
"ney": "https://censored.com/nimp?fuid=\u0026ic=EWRMh-.UTPdNvWB-JYa58c85N0fEPgXunKp3wwyPadp7jwqHgZbz4dG0A51OVO-2Gs0znYmLcPIH0ThEmpsYl8wKofo9ytJ2A3uWr9Kn-dNxeh.k8lIml9kavPk1.dk7f.46xKX7IVpf3.yU-Yx1KetQl3Q9f-iePn7B86yjVgMxkTNfhZAg0pP0kKZaJMd2orLXoV4xPXmwTdfJbWJU5bGAUROJT-Yd7yTHoVveuvOBClHzM4cgHFmGxzox6cCJ2gZB.7fqKkPzECXwdpobmO0RWxdu224-FADd.oM4DghIEpdZJe.FjEq0stQnJBT.puw0JamHgT15NdSQN7voBJ8UqGCDOu1qSLece6Iy.PN392xGWhxs0URbrWhSEgkhCr.R4ol9kjrMqK78shw2gHBJjEzKeBeo6lBzU8YfoKDM7oPlj5SwmL6sV2i2UaWJbEtreRt3oABDPab--AevfJW2rQ0-2iyt-rJSPjDHHoOQEFoh0G7cPm8SIZxk17ojWkFdM7CXlmuSN0paqMhp-4gWlfgvNq8a65I8GfY8cwVrW5KzRszHLhWYareVM3MNpejdcVH2kinEnYzBVyW0e8oN06LC2icG8FRlhOC2N8wni66liT73RvKyFFT1zW7SAoqtgn9KXY6m.EaZzSx3aapIMGpG9-S8q6mwAuwZId37ri4GTiLXp6OMABsLwT3sMUOm.Kktp.uYP1z2be2DFM6zKKPL7YJopAvdfS7TdhMfHD6Dfcv-EvK6Q0lNylaFIjegbNjPEPPXzLPdf8iwLK-dqfSe127TZcj5xJUFwo45IgFl0i0puKRIzsAtM2W3zM-TNc2HEc0nIllK.aoKZ0tF9iSekzjcNnMAvhcAKBqq6DY.qIBUs5yOoxqW4m-ga9drHp09VXIkn7st7J4IUlrMZFuVHnnzbeqD61AKKFiaRCqPee6Y88DqhsdNt7SzdA-xq9SKnJW67zsZTD0T9OoRl3.nLaSwoQ==\u0026t=adi\u0026prc=${censored}",
"tax": {
"name": "test"
},
"hy": 3020,
"ny": 2530,
"arz": "\u003cscript type=\"text/javascript\"\u003evar _CDurl='';(function(){_CDurl=encodeURIComponent(document.URL);})();document.write('\u003cscr' + 'ipt type=\"text/javascript\" src=\"https://censored/at?bad=\u0026gad=\u0026ic=EWRMh-.UTPdNvWB-JYa58c85N0fEPgXunKp3wwyPadp7jwqHgZbz4dG0A51OVO-2Gs0znYmLcPIH0ThEmpsYl8wKofo9ytJ2A3uWr9Kn-dNxeh.k8lIml9kavPk1.dk7f.46xKX7IVpf3.yU-Yx1KetQl3Q9f-iePn7B86yjVgMxkTNfhZAg0pP0kKZaJMd2orLXoV4xPXmwTdfJbWJU5bGAUROJT-Yd7yTHoVveuvOBClHzM4cgHFmGxzox6cCJ2gZB.7fqKkPzECXwdpobmO0RWxdu224-FADd.oM4DghIEpdZJe.FjEq0stQnJBT.puw0JamHgT15NdSQN7voBJ8UqGCDOu1qSLece6Iy.PN392xGWhxs0URbrWhSEgkhCr.R4ol9kjrMqK78shw2gHBJjEzKeBeo6lBzU8YfoKDM7oPlj5SwmL6sV2i2UaWJbEtreRt3oABDPab--AevfJW2rQ0-2iyt-rJSPjDHHoOQEFoh0G7cPm8SIZxk17ojWkFdM7CXlmuSN0paqMhp-4gWlfgvNq8a65I8GfY8cwVrW5KzRszHLhWYareVM3MNpejdcVH2kinEnYzBVyW0e8oN06LC2icG8FRlhOC2N8wni66liT73RvKyFFT1zW7SAoqtgn9KXY6m.EaZzSx3aapIMGpG9-S8q6mwAuwZId37ri4GTiLXp6OMABsLwT3sMUOm.Kktp.uYP1z2be2DFM6zKKPL7YJopAvdfS7TdhMfHD6Dfcv-EvK6Q0lNylaFIjegbNjPEPPXzLPdf8iwLK-dqfSe127TZcj5xJUFwo45IgFl0i0puKRIzsAtM2W3zM-TNc2HEc0nIllK.aoKZ0tF9iSekzjcNnMAvhcAKBqq6DY.qIBUs5yOoxqW4m-ga9drHp09VXIkn7st7J4IUlrMZFuVHnnzbeqD61AKKFiaRCqPee6Y88DqhsdNt7SzdA-xq9SKnJW67zsZTD0T9OoRl3.nLaSwoQ==\u0026t=adj\u0026prc=${censored}\u0026tat='+_CDurl+'\"\u003e\u003c/scr' + 'ipt\u003e');\u003c/script\u003e"
}
],
"tark": "1"
}
],
"gno": "55f03d71-f021-49e0-a1a5-cae4315b3561"
}
Also, I used the same code you are using, after commenting off the request body part(since I don't have it) and changing the function signature(of SendRequest) a bit as given below. All others are as same as the code given above.
func SendRequest() (*Response, error) {
responseModel := &Response{}
// Commented off since the response body is not with me..
// byteData, err := json.Marshal(requestModel)
// if err != nil {
// zap.S().Errorf("Error marshalling request. Err: %v", err)
// return nil, err
// }
url := "http://localhost:8080/foo"
request, _ := http.NewRequest(http.MethodPost, url /*bytes.NewBuffer(byteData)*/, nil)
request.Close = true
client := &http.Client{}
response, err := client.Do(request)
if err, ok := err.(net.Error); ok && err.Timeout() {
log.Fatal("Response timeout exceed")
return nil, errors.New("Response timeout exceed")
}
if err != nil || response == nil {
errorMessage := "not respond"
log.Fatal(errorMessage)
return nil, errors.New(errorMessage)
}
defer response.Body.Close()
if response.StatusCode == http.StatusOK {
err = json.NewDecoder(response.Body).Decode(&responseModel)
if err != nil {
errorMessage := "Request response decode error"
log.Printf("%v, Err: %v", errorMessage, err)
return nil, errors.New(errorMessage)
}
return responseModel, nil
} else {
bodyBytes, _ := ioutil.ReadAll(response.Body)
errorMessage := "not respond"
log.Printf("%v, StatusCode %v, Response: %v\n", errorMessage, response.StatusCode, string(bodyBytes))
return nil, errors.New(errorMessage)
}
}
With this as well, I am able to Unmarshal the response successfully.
So, in my opinion, the incoming response is not correct(may be). Try printing the incoming data and verify.
resBody, err := ioutil.ReadAll(response.Body)
if err != nil {
return nil, err
}
fmt.Printf("%s", resBody) // 👈

Error handling on a call stack - http request handler

In the below code:
func (p *ProductHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
// handle the request for a list of products
if r.Method == http.MethodGet {
p.getProductHandler(w, r)
return
}
if r.Method == http.MethodPost {
p.addProductHandler(w, r)
return
}
if r.Method == http.MethodPut {
id := findID(w, r)
p.updateProductHandler(id, w, r)
return
}
// catch all
// if no method is satisfied return an error
w.WriteHeader(http.StatusMethodNotAllowed)
}
// getProducts returns the products from the data store
func (p *ProductHandler) getProductHandler(w http.ResponseWriter, r *http.Request) {
p.l.Println("Handle GET Products")
// fetch the products from the datastore
productList := data.GetProducts()
// serialize the list to JSON
err := productList.WriteJSON(w)
if err != nil {
http.Error(w, "Unable to marshal json", http.StatusInternalServerError)
return
}
}
func (p *ProductHandler) addProductHandler(w http.ResponseWriter, r *http.Request) {
p.l.Println("Handle POST products")
// Read the item from the incoming request
productItem := &data.Product{}
err := productItem.ReadJSON(r.Body)
if err != nil {
http.Error(w, "Unable to unmarshal JSON", http.StatusBadRequest)
return
}
p.l.Printf("Product item: %#v\n", productItem)
data.AddProductItem(productItem)
}
func (p *ProductHandler) updateProductHandler(id int, w http.ResponseWriter, r *http.Request) {
// whatever
return
}
func findID(w http.ResponseWriter, r *http.Request) int {
// expect the id in the URI
dfa := regexp.MustCompile(`/([0-9]+)`)
matches := dfa.FindAllStringSubmatch(r.URL.Path, -1) // returns [][]string
if len(matches) != 1 {
http.Error(w, "Invalid URI", http.StatusBadRequest)
return
}
if len(matches[0]) != 2 {
http.Error(w, "Invlaid URI", http.StatusBadRequest)
return
}
idString := matches[0][1]
id, err := strconv.Atoi(idString)
if err != nil {
http.Error(w, "Invlaid URI", http.StatusBadRequest)
return
}
return id
}
On error, we use http.Error() within handlers that handle POST & GET and then return
But for PUT request, call stack stack is ServeHTTP -> findID() and then ServeHTTP() -> updateProductHandler(). Error handling is required in findID() but cannot return immediately, because findID() returns int.
What is the design pattern to perform error handling for a call stack? Error wrapping using github.com/pkg/errors....
func (p *ProductHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
// handle the request for a list of products
if r.Method == http.MethodGet {
p.getProductHandler(w, r)
return
}
if r.Method == http.MethodPost {
p.addProductHandler(w, r)
return
}
if r.Method == http.MethodPut {
id, err := findID(r.URL.Path)
if err == nil {
p.updateProductHandler(id, w, r)
} else {
http.Error(w, err.Error(), http.StatusBadRequest)
}
return
}
// catch all
// if no method is satisfied return an error
w.WriteHeader(http.StatusMethodNotAllowed)
w.Header().Add("Allow", "GET, POST, PUT")
}
func findID(path string) (int, error) {
// expect the id in the URI
dfa := regexp.MustCompile(`/([0-9]+)`)
matches := dfa.FindAllStringSubmatch(path, -1) // returns [][]string
if len(matches) != 1 {
return 0, fmt.Errorf("Invalid URI %s", path)
}
if len(matches[0]) != 2 {
return 0, fmt.Errorf("Invalid URI %s", path)
}
idString := matches[0][1]
id, err := strconv.Atoi(idString)
if err != nil {
return 0, fmt.Errorf("Invalid URI %s", path)
}
return id, nil
}

Server returning 400 when user exists

I'm working on some tests in Go and I have spent the past 2 days trying to make it work but I couldn't. My problem is that the test returns 400 even when the user does exist.
This is my getUser function
func (handler *UserHandler) getUser(w http.ResponseWriter, ID int) {
logfile, err := os.OpenFile("events.log", os.O_RDWR|os.O_CREATE|os.O_APPEND, 0666)
if err != nil {
log.Fatalf("Error opening file: %v", err)
}
defer logfile.Close()
log.SetOutput(logfile)
user := db.Fetch(ID)
userJSON, err := json.Marshal(user)
if err != nil {
log.Printf("Error while marshaling the user into JSON: %v", err)
return
}
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusOK)
// userJSON is sent as http Response
w.Write(userJSON)
}
This is my UserHandler
type UserHandler struct{}
func (handle *UserHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
var head string
head, r.URL.Path = ShiftPath(r.URL.Path)
id, err := strconv.Atoi(head)
if err != nil {
http.Error(w, fmt.Sprintf("Invalid user ID %q", head), http.StatusBadRequest)
return
}
switch r.Method {
case "GET":
handle.getUser(w, id)
default:
http.Error(w, "Only GET is allowed", http.StatusMethodNotAllowed)
}
}
func ShiftPath(p string) (head, tail string) {
p = path.Clean("/" + p)
i := strings.Index(p[1:], "/") + 1
if i <= 0 {
return p[1:], "/"
}
return p[1:i], p[i:]
}
And this is my test
func TestGetUser(t *testing.T) {
handler := new(UserHandler)
mux := http.NewServeMux()
mux.HandleFunc("/user/", handler.ServeHTTP)
writer := httptest.NewRecorder()
request, _ := http.NewRequest("GET", "/user/12", nil)
mux.ServeHTTP(writer, request)
if writer.Code != 200 {
t.Errorf("Response code is %v", writer.Code)
}
}
Issue with code ====> id, err := strconv.Atoi(head)
Due to error you see a return and hence you see 400 error.
Have your server code fully functional with valid logic.
Suggestion: Always print or debug line by line. You can find the issue and root cause.

Sending HTTP response is blocked until shell command finishes

Idea: there is a web server ready to receive messages that will trigger the execution of commands/tests on the server. I have started with a simple case where a simple ping is executed. The code below handles POST messages sent to /ping which contain the following json format:
{ "ip": "valid_ip_addr", "count": "4" }
The server will then run the command ping -c 4 valid_ip_address
Desired outcome: if the command can .Start() send back a 200 OK. If there are problems, send back an error message.
Problem: I send a 200 OK response right after checking that .Start() didn't give any errors, but this is being received after the command has finished.
Code: There are three functions: main(), handler() and ping(). The problem takes place in the last one.
package main
import (
"bufio"
"encoding/json"
"fmt"
"net/http"
"os"
"os/exec"
)
var err error
type Ping struct {
Count string `json:"count"`
Ip string `json:"ip"`
}
func main() {
http.HandleFunc("/ping", handler)
http.ListenAndServe(":5050", nil)
}
func handler(w http.ResponseWriter, r *http.Request) {
switch r.Method {
case "POST":
p := Ping{}
err := json.NewDecoder(r.Body).Decode(&p)
if err != nil {
fmt.Printf("400 Bad request. Problem decoding the received json.\nDetails:\n%s\n", err.Error())
http.Error(w, err.Error(), 400)
return
}
fmt.Println("POST /ping ", p)
ping(w, p)
default:
http.Error(w, "Only POST is accepted.", 501)
}
}
func ping(w http.ResponseWriter, a Ping) {
cmdName := "ping"
cmdArgs := []string{"-c", a.Count, a.Ip}
cmd := exec.Command(cmdName, cmdArgs...)
cmdReader, err := cmd.StdoutPipe()
if err != nil {
fmt.Fprintln(os.Stderr, "Error creating StdoutPipe for Cmd", err)
http.Error(w, "Error creating StdoutPipe for Cmd\n"+err.Error(), 500)
return
}
// the following is used to print output of the command
// as it makes progress...
scanner := bufio.NewScanner(cmdReader)
go func() {
for scanner.Scan() {
fmt.Printf("%s\n", scanner.Text())
//
// TODO:
// send output to server
}
}()
err = cmd.Start()
if err != nil {
fmt.Fprintln(os.Stderr, "Error starting Cmd", err)
http.Error(w, "Error starting Cmd\n"+err.Error(), 500)
return
}
// send 200 OK
fmt.Fprintf(w, "ping started")
err = cmd.Wait()
if err != nil {
fmt.Fprintln(os.Stderr, "Error waiting for Cmd", err)
}
}
curl for testing
curl -X POST http://localhost:5050/ping -d '{"ip": "127.0.0.1", "count": "4"}'
I would recommend to use select with timeout waiting for an error. Check out the following code.
func ping(w http.ResponseWriter, a Ping) {
cmdName := "ping"
cmdArgs := []string{"-c", a.Count, a.Ip}
cmd := exec.Command(cmdName, cmdArgs...)
cmdReader, err := cmd.StdoutPipe()
if err != nil {
fmt.Fprintln(os.Stderr, "Error creating StdoutPipe for Cmd", err)
http.Error(w, "Error creating StdoutPipe for Cmd\n"+err.Error(), 500)
return
}
// the following is used to print output of the command
// as it makes progress...
scanner := bufio.NewScanner(cmdReader)
go func() {
for scanner.Scan() {
fmt.Printf("%s\n", scanner.Text())
//
// TODO:
// send output to server
}
}()
err = cmd.Start()
if err != nil {
fmt.Fprintln(os.Stderr, "Error starting Cmd", err)
http.Error(w, "Error starting Cmd\n"+err.Error(), 500)
return
}
// not sending response here anymore. Using the channel instead
errChan := make(chan error)
go func(ec chan error) {
err = cmd.Wait()
if err != nil {
errChan <- err
}
}(errChan)
select {
case err := <-errChan:
http.Error(w, "Error: "+err.Error(), 500)
// timeout 50ms just in case. But I presume you would get an error (if there is one in cmd) even before execution will get to this point
case <-time.After(time.Millisecond * 50):
fmt.Fprintf(w, "ping started")
}
}

Request.ParseMultipartForm leaks memory

I am writing a Go server which handles image uploading via Multipart Requests, but after a while i get really bad memory usages (up to 100% RAM).
I am using the basic http.ListenAndServeTLS and http.HandleFunc to do so.
Below a snippet of my code and the svg i got from pprof after a lot of image uploads.
func HandleUploadImage(res http.ResponseWriter, req *http.Request) {
defer func() {
// Release mem
if req != nil {
fmt.Println("Releasing request")
if req.MultipartForm != nil {
fmt.Println("Releasing MultipartForm")
req.MultipartForm.RemoveAll()
}
if req.Body != nil {
fmt.Println("Closing req.Body")
req.Body.Close()
}
}
}()
// parse request
const _2MB = 1 << 20
if err := req.ParseMultipartForm(_2MB); nil != err {
u.Log("Error", "Couldn't parse MultiPartForm", map[string]interface{}{"Error": err}, u.GetFile_line(), "")
return
}
id := req.FormValue(formValueString)
// get the json data
if err := json.Unmarshal([]byte(id), marshalStruct); nil != err {
u.Log("Error", "Couldn't Unmarshal JSON", map[string]interface{}{"Error": err}, u.GetFile_line(), "")
return
}
// Do something with the image and save it
// Marshal the reponse to JSON.
resJSON, err := response.ToJSON()
if err != nil {
u.Log("Error", "Couldn't create JSON from response struct", map[string]interface{}{"Error": err, "ResponseStruct": response}, u.GetFile_line(), username)
return
}
}
pprof_image

Resources