Server returning 400 when user exists - http

I'm working on some tests in Go and I have spent the past 2 days trying to make it work but I couldn't. My problem is that the test returns 400 even when the user does exist.
This is my getUser function
func (handler *UserHandler) getUser(w http.ResponseWriter, ID int) {
logfile, err := os.OpenFile("events.log", os.O_RDWR|os.O_CREATE|os.O_APPEND, 0666)
if err != nil {
log.Fatalf("Error opening file: %v", err)
}
defer logfile.Close()
log.SetOutput(logfile)
user := db.Fetch(ID)
userJSON, err := json.Marshal(user)
if err != nil {
log.Printf("Error while marshaling the user into JSON: %v", err)
return
}
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusOK)
// userJSON is sent as http Response
w.Write(userJSON)
}
This is my UserHandler
type UserHandler struct{}
func (handle *UserHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
var head string
head, r.URL.Path = ShiftPath(r.URL.Path)
id, err := strconv.Atoi(head)
if err != nil {
http.Error(w, fmt.Sprintf("Invalid user ID %q", head), http.StatusBadRequest)
return
}
switch r.Method {
case "GET":
handle.getUser(w, id)
default:
http.Error(w, "Only GET is allowed", http.StatusMethodNotAllowed)
}
}
func ShiftPath(p string) (head, tail string) {
p = path.Clean("/" + p)
i := strings.Index(p[1:], "/") + 1
if i <= 0 {
return p[1:], "/"
}
return p[1:i], p[i:]
}
And this is my test
func TestGetUser(t *testing.T) {
handler := new(UserHandler)
mux := http.NewServeMux()
mux.HandleFunc("/user/", handler.ServeHTTP)
writer := httptest.NewRecorder()
request, _ := http.NewRequest("GET", "/user/12", nil)
mux.ServeHTTP(writer, request)
if writer.Code != 200 {
t.Errorf("Response code is %v", writer.Code)
}
}

Issue with code ====> id, err := strconv.Atoi(head)
Due to error you see a return and hence you see 400 error.
Have your server code fully functional with valid logic.
Suggestion: Always print or debug line by line. You can find the issue and root cause.

Related

Golang large response unexpected end of JSON input

I'm sending a request to a server and trying to unmarshall HTTP Response. It's weird, but some of the responses return 200 and do not give an unmarshall error, while some of them return 200 and give an unmarshall error.
My client code looks like:
func SendRequest(requestModel *model.Request) (*model.Response, error) {
responseModel := &dspModel.Response{}
byteData, err := json.Marshal(requestModel)
if err != nil {
zap.S().Errorf("Error marshalling request. Err: %v", err)
return nil, err
}
url := "xx"
request, _ := http.NewRequest(http.MethodPost, url, bytes.NewBuffer(byteData))
request.Close = true
client := &http.Client{}
response, err := client.Do(request)
if err, ok := err.(net.Error); ok && err.Timeout() {
zap.S().Error("Response timeout exceed")
return nil, errors.New("Response timeout exceed")
}
if err != nil || response == nil {
errorMessage := "not respond"
zap.S().Error(errorMessage)
return nil, errors.New(errorMessage)
}
defer response.Body.Close()
if response.StatusCode == http.StatusOK {
err = json.NewDecoder(response.Body).Decode(&responseModel)
if err != nil {
// Error occurred here!
errorMessage := "Request response decode error"
zap.S().Errorf("%v, Err: %v", errorMessage, err)
return nil, errors.New(errorMessage)
}
return response, nil
} else if response.StatusCode == http.StatusNoContent {
return nil, nil
} else {
bodyBytes, _ := ioutil.ReadAll(response.Body)
errorMessage := "not respond"
zap.S().Errorf("%v, StatusCode %v, Response: %v Request: %v", errorMessage,
response.StatusCode, string(bodyBytes), string(byteData))
return nil, errors.New(errorMessage)
}
}
I suspect response cause it's too long and has different characters.
Response looks like:
{"tax":{"ver":"1.0"},"cur":"EUR","rack":[{"tur":[{"zar":2.599886212,"domain":["test.com"],"ney":"https://censored.com/nimp?fuid=&ic=EWRMh-.UTPdNvWB-JYa58c85N0fEPgXunKp3wwyPadp7jwqHgZbz4dG0A51OVO-2Gs0znYmLcPIH0ThEmpsYl8wKofo9ytJ2A3uWr9Kn-dNxeh.k8lIml9kavPk1.dk7f.46xKX7IVpf3.yU-Yx1KetQl3Q9f-iePn7B86yjVgMxkTNfhZAg0pP0kKZaJMd2orLXoV4xPXmwTdfJbWJU5bGAUROJT-Yd7yTHoVveuvOBClHzM4cgHFmGxzox6cCJ2gZB.7fqKkPzECXwdpobmO0RWxdu224-FADd.oM4DghIEpdZJe.FjEq0stQnJBT.puw0JamHgT15NdSQN7voBJ8UqGCDOu1qSLece6Iy.PN392xGWhxs0URbrWhSEgkhCr.R4ol9kjrMqK78shw2gHBJjEzKeBeo6lBzU8YfoKDM7oPlj5SwmL6sV2i2UaWJbEtreRt3oABDPab--AevfJW2rQ0-2iyt-rJSPjDHHoOQEFoh0G7cPm8SIZxk17ojWkFdM7CXlmuSN0paqMhp-4gWlfgvNq8a65I8GfY8cwVrW5KzRszHLhWYareVM3MNpejdcVH2kinEnYzBVyW0e8oN06LC2icG8FRlhOC2N8wni66liT73RvKyFFT1zW7SAoqtgn9KXY6m.EaZzSx3aapIMGpG9-S8q6mwAuwZId37ri4GTiLXp6OMABsLwT3sMUOm.Kktp.uYP1z2be2DFM6zKKPL7YJopAvdfS7TdhMfHD6Dfcv-EvK6Q0lNylaFIjegbNjPEPPXzLPdf8iwLK-dqfSe127TZcj5xJUFwo45IgFl0i0puKRIzsAtM2W3zM-TNc2HEc0nIllK.aoKZ0tF9iSekzjcNnMAvhcAKBqq6DY.qIBUs5yOoxqW4m-ga9drHp09VXIkn7st7J4IUlrMZFuVHnnzbeqD61AKKFiaRCqPee6Y88DqhsdNt7SzdA-xq9SKnJW67zsZTD0T9OoRl3.nLaSwoQ==&t=adi&prc=${censored}","tax":{"name":"test"},"hy":3020,"ny":2530,"arz":"<script type=\"text/javascript\">var _CDurl='';(function(){_CDurl=encodeURIComponent(document.URL);})();document.write('<scr' + 'ipt type=\"text/javascript\" src=\"https://censored/at?bad=&gad=&ic=EWRMh-.UTPdNvWB-JYa58c85N0fEPgXunKp3wwyPadp7jwqHgZbz4dG0A51OVO-2Gs0znYmLcPIH0ThEmpsYl8wKofo9ytJ2A3uWr9Kn-dNxeh.k8lIml9kavPk1.dk7f.46xKX7IVpf3.yU-Yx1KetQl3Q9f-iePn7B86yjVgMxkTNfhZAg0pP0kKZaJMd2orLXoV4xPXmwTdfJbWJU5bGAUROJT-Yd7yTHoVveuvOBClHzM4cgHFmGxzox6cCJ2gZB.7fqKkPzECXwdpobmO0RWxdu224-FADd.oM4DghIEpdZJe.FjEq0stQnJBT.puw0JamHgT15NdSQN7voBJ8UqGCDOu1qSLece6Iy.PN392xGWhxs0URbrWhSEgkhCr.R4ol9kjrMqK78shw2gHBJjEzKeBeo6lBzU8YfoKDM7oPlj5SwmL6sV2i2UaWJbEtreRt3oABDPab--AevfJW2rQ0-2iyt-rJSPjDHHoOQEFoh0G7cPm8SIZxk17ojWkFdM7CXlmuSN0paqMhp-4gWlfgvNq8a65I8GfY8cwVrW5KzRszHLhWYareVM3MNpejdcVH2kinEnYzBVyW0e8oN06LC2icG8FRlhOC2N8wni66liT73RvKyFFT1zW7SAoqtgn9KXY6m.EaZzSx3aapIMGpG9-S8q6mwAuwZId37ri4GTiLXp6OMABsLwT3sMUOm.Kktp.uYP1z2be2DFM6zKKPL7YJopAvdfS7TdhMfHD6Dfcv-EvK6Q0lNylaFIjegbNjPEPPXzLPdf8iwLK-dqfSe127TZcj5xJUFwo45IgFl0i0puKRIzsAtM2W3zM-TNc2HEc0nIllK.aoKZ0tF9iSekzjcNnMAvhcAKBqq6DY.qIBUs5yOoxqW4m-ga9drHp09VXIkn7st7J4IUlrMZFuVHnnzbeqD61AKKFiaRCqPee6Y88DqhsdNt7SzdA-xq9SKnJW67zsZTD0T9OoRl3.nLaSwoQ==&t=adj&prc=${censored}&tat='+_CDurl+'\"></scr' + 'ipt>');</script>"}],"tark":"1"}],"gno":"55f03d71-f021-49e0-a1a5-cae4315b3561"}
When I debug the error, I noticed that after half of the response is not visible.
Error statement:
"msg":"Request response decode error, Response: {\"tax\":{\"ver\":\"1.0\"},\"cur\":\"EUR\",\"rac\":[{\"btyrd\":[{\"zar\":2.599886212,\"domain\":[\"test.com\"],\"ney\":\"https://censored/nimp?fuid=&ic=EWRMh-.UTPdNvWB-JYa58c85N0fEPgXunKp3wwyPadp7jwqHgZbz4dG0A51OVO-2Gs0znYmLcPIH0ThEmpsYl8wKofo9ytJ2A3uWr9Kn-dNxeh.k8lIml9kavPk1.dk7f.46xKX7IVpf3.yU-Yx1KetQl3Q9f-iePn7B86yjVgMxkTNfhZAg0pP0kKZaJMd2orLXoV4xPXmwTdfJbWJU5bGAUROJT-Yd7yTHoVveuvOBClHzM4cgHFmGxzox6cCJ2gZB.7fqKkPzECXwdpobmO0RWxdu224-FADd.oM4DghIEpdZJe.FjEq0stQnJBT.puw0JamHgT15NdSQN7voBJ8UqGCDOu1q, Err: unexpected end of JSON input"
Response Model:
type Response struct {
Tax Tax `json:"tax"`
Cur string `json:"cur"`
Rack []Rack `json:"rack"`
Gno string `json:"gno"`
}
type Tax struct {
Ver string `json:"ver"`
}
type TaxOfTur struct {
Name string `json:"name"`
}
type Tur struct {
Zar float64 `json:"zar"`
Domain []string `json:"domain"`
Ney string `json:"ney"`
Tax TaxOfTur `json:"tax"`
Hy int `json:"hy"`
Ny int `json:"ny"`
Arz string `json:"arz"`
}
type Rack struct {
Tur []Tur `json:"tur"`
Tark string `json:"tark"`
}
Probably unrelated but requests are going concurrent and my test case is based on only one concurrent request. So how can I solve this problem?
I tried to implement your code at my end, and it's giving the expected output. The code I tried is given below.
Note: This is not a solution to the exact problem, since I was not able to reproduce the same.
// Response struct definition here...
// ..
// The trouble making JSON.
const msg = `{"tax":{"ver":"1.0"},"cur":"EUR","rack":[{"tur":[{"zar":2.599886212,"domain":["test.com"],"ney":"https://censored.com/nimp?fuid=&ic=EWRMh-.UTPdNvWB-JYa58c85N0fEPgXunKp3wwyPadp7jwqHgZbz4dG0A51OVO-2Gs0znYmLcPIH0ThEmpsYl8wKofo9ytJ2A3uWr9Kn-dNxeh.k8lIml9kavPk1.dk7f.46xKX7IVpf3.yU-Yx1KetQl3Q9f-iePn7B86yjVgMxkTNfhZAg0pP0kKZaJMd2orLXoV4xPXmwTdfJbWJU5bGAUROJT-Yd7yTHoVveuvOBClHzM4cgHFmGxzox6cCJ2gZB.7fqKkPzECXwdpobmO0RWxdu224-FADd.oM4DghIEpdZJe.FjEq0stQnJBT.puw0JamHgT15NdSQN7voBJ8UqGCDOu1qSLece6Iy.PN392xGWhxs0URbrWhSEgkhCr.R4ol9kjrMqK78shw2gHBJjEzKeBeo6lBzU8YfoKDM7oPlj5SwmL6sV2i2UaWJbEtreRt3oABDPab--AevfJW2rQ0-2iyt-rJSPjDHHoOQEFoh0G7cPm8SIZxk17ojWkFdM7CXlmuSN0paqMhp-4gWlfgvNq8a65I8GfY8cwVrW5KzRszHLhWYareVM3MNpejdcVH2kinEnYzBVyW0e8oN06LC2icG8FRlhOC2N8wni66liT73RvKyFFT1zW7SAoqtgn9KXY6m.EaZzSx3aapIMGpG9-S8q6mwAuwZId37ri4GTiLXp6OMABsLwT3sMUOm.Kktp.uYP1z2be2DFM6zKKPL7YJopAvdfS7TdhMfHD6Dfcv-EvK6Q0lNylaFIjegbNjPEPPXzLPdf8iwLK-dqfSe127TZcj5xJUFwo45IgFl0i0puKRIzsAtM2W3zM-TNc2HEc0nIllK.aoKZ0tF9iSekzjcNnMAvhcAKBqq6DY.qIBUs5yOoxqW4m-ga9drHp09VXIkn7st7J4IUlrMZFuVHnnzbeqD61AKKFiaRCqPee6Y88DqhsdNt7SzdA-xq9SKnJW67zsZTD0T9OoRl3.nLaSwoQ==&t=adi&prc=${censored}","tax":{"name":"test"},"hy":3020,"ny":2530,"arz":"<script type=\"text/javascript\">var _CDurl='';(function(){_CDurl=encodeURIComponent(document.URL);})();document.write('<scr' + 'ipt type=\"text/javascript\" src=\"https://censored/at?bad=&gad=&ic=EWRMh-.UTPdNvWB-JYa58c85N0fEPgXunKp3wwyPadp7jwqHgZbz4dG0A51OVO-2Gs0znYmLcPIH0ThEmpsYl8wKofo9ytJ2A3uWr9Kn-dNxeh.k8lIml9kavPk1.dk7f.46xKX7IVpf3.yU-Yx1KetQl3Q9f-iePn7B86yjVgMxkTNfhZAg0pP0kKZaJMd2orLXoV4xPXmwTdfJbWJU5bGAUROJT-Yd7yTHoVveuvOBClHzM4cgHFmGxzox6cCJ2gZB.7fqKkPzECXwdpobmO0RWxdu224-FADd.oM4DghIEpdZJe.FjEq0stQnJBT.puw0JamHgT15NdSQN7voBJ8UqGCDOu1qSLece6Iy.PN392xGWhxs0URbrWhSEgkhCr.R4ol9kjrMqK78shw2gHBJjEzKeBeo6lBzU8YfoKDM7oPlj5SwmL6sV2i2UaWJbEtreRt3oABDPab--AevfJW2rQ0-2iyt-rJSPjDHHoOQEFoh0G7cPm8SIZxk17ojWkFdM7CXlmuSN0paqMhp-4gWlfgvNq8a65I8GfY8cwVrW5KzRszHLhWYareVM3MNpejdcVH2kinEnYzBVyW0e8oN06LC2icG8FRlhOC2N8wni66liT73RvKyFFT1zW7SAoqtgn9KXY6m.EaZzSx3aapIMGpG9-S8q6mwAuwZId37ri4GTiLXp6OMABsLwT3sMUOm.Kktp.uYP1z2be2DFM6zKKPL7YJopAvdfS7TdhMfHD6Dfcv-EvK6Q0lNylaFIjegbNjPEPPXzLPdf8iwLK-dqfSe127TZcj5xJUFwo45IgFl0i0puKRIzsAtM2W3zM-TNc2HEc0nIllK.aoKZ0tF9iSekzjcNnMAvhcAKBqq6DY.qIBUs5yOoxqW4m-ga9drHp09VXIkn7st7J4IUlrMZFuVHnnzbeqD61AKKFiaRCqPee6Y88DqhsdNt7SzdA-xq9SKnJW67zsZTD0T9OoRl3.nLaSwoQ==&t=adj&prc=${censored}&tat='+_CDurl+'\"></scr' + 'ipt>');</script>"}],"tark":"1"}],"gno":"55f03d71-f021-49e0-a1a5-cae4315b3561"}`
func SendRequest() (*Response, error) {
url := "http://localhost:8080/foo" // dummy server
request, _ := http.NewRequest(http.MethodPost, url, nil)
request.Close = true
client := &http.Client{}
response, err := client.Do(request)
if err != nil {
return nil, err
}
defer response.Body.Close()
resBody, err := ioutil.ReadAll(response.Body)
if err != nil {
return nil, err
}
responseModel := &Response{}
err = json.Unmarshal(resBody, &responseModel)
if err != nil {
return nil, err
}
return responseModel, nil
}
func StartDummyServer() {
handler := func(w http.ResponseWriter, r *http.Request) {
fmt.Fprint(w, msg)
}
http.HandleFunc("/foo", handler)
log.Fatal(http.ListenAndServe(":8080", nil))
}
func main() {
// a dummy server is created to send the response
go StartDummyServer()
time.Sleep(time.Second)
resp, err := SendRequest()
if err != nil {
log.Fatal(err.Error())
}
fmt.Println(prettyPrint(resp))
time.Sleep(time.Second * 10)
}
func prettyPrint(i interface{}) string {
s, _ := json.MarshalIndent(i, "", " ")
return string(s)
}
Here is the console output:
{
"tax": {
"ver": "1.0"
},
"cur": "EUR",
"rack": [
{
"tur": [
{
"zar": 2.599886212,
"domain": [
"test.com"
],
"ney": "https://censored.com/nimp?fuid=\u0026ic=EWRMh-.UTPdNvWB-JYa58c85N0fEPgXunKp3wwyPadp7jwqHgZbz4dG0A51OVO-2Gs0znYmLcPIH0ThEmpsYl8wKofo9ytJ2A3uWr9Kn-dNxeh.k8lIml9kavPk1.dk7f.46xKX7IVpf3.yU-Yx1KetQl3Q9f-iePn7B86yjVgMxkTNfhZAg0pP0kKZaJMd2orLXoV4xPXmwTdfJbWJU5bGAUROJT-Yd7yTHoVveuvOBClHzM4cgHFmGxzox6cCJ2gZB.7fqKkPzECXwdpobmO0RWxdu224-FADd.oM4DghIEpdZJe.FjEq0stQnJBT.puw0JamHgT15NdSQN7voBJ8UqGCDOu1qSLece6Iy.PN392xGWhxs0URbrWhSEgkhCr.R4ol9kjrMqK78shw2gHBJjEzKeBeo6lBzU8YfoKDM7oPlj5SwmL6sV2i2UaWJbEtreRt3oABDPab--AevfJW2rQ0-2iyt-rJSPjDHHoOQEFoh0G7cPm8SIZxk17ojWkFdM7CXlmuSN0paqMhp-4gWlfgvNq8a65I8GfY8cwVrW5KzRszHLhWYareVM3MNpejdcVH2kinEnYzBVyW0e8oN06LC2icG8FRlhOC2N8wni66liT73RvKyFFT1zW7SAoqtgn9KXY6m.EaZzSx3aapIMGpG9-S8q6mwAuwZId37ri4GTiLXp6OMABsLwT3sMUOm.Kktp.uYP1z2be2DFM6zKKPL7YJopAvdfS7TdhMfHD6Dfcv-EvK6Q0lNylaFIjegbNjPEPPXzLPdf8iwLK-dqfSe127TZcj5xJUFwo45IgFl0i0puKRIzsAtM2W3zM-TNc2HEc0nIllK.aoKZ0tF9iSekzjcNnMAvhcAKBqq6DY.qIBUs5yOoxqW4m-ga9drHp09VXIkn7st7J4IUlrMZFuVHnnzbeqD61AKKFiaRCqPee6Y88DqhsdNt7SzdA-xq9SKnJW67zsZTD0T9OoRl3.nLaSwoQ==\u0026t=adi\u0026prc=${censored}",
"tax": {
"name": "test"
},
"hy": 3020,
"ny": 2530,
"arz": "\u003cscript type=\"text/javascript\"\u003evar _CDurl='';(function(){_CDurl=encodeURIComponent(document.URL);})();document.write('\u003cscr' + 'ipt type=\"text/javascript\" src=\"https://censored/at?bad=\u0026gad=\u0026ic=EWRMh-.UTPdNvWB-JYa58c85N0fEPgXunKp3wwyPadp7jwqHgZbz4dG0A51OVO-2Gs0znYmLcPIH0ThEmpsYl8wKofo9ytJ2A3uWr9Kn-dNxeh.k8lIml9kavPk1.dk7f.46xKX7IVpf3.yU-Yx1KetQl3Q9f-iePn7B86yjVgMxkTNfhZAg0pP0kKZaJMd2orLXoV4xPXmwTdfJbWJU5bGAUROJT-Yd7yTHoVveuvOBClHzM4cgHFmGxzox6cCJ2gZB.7fqKkPzECXwdpobmO0RWxdu224-FADd.oM4DghIEpdZJe.FjEq0stQnJBT.puw0JamHgT15NdSQN7voBJ8UqGCDOu1qSLece6Iy.PN392xGWhxs0URbrWhSEgkhCr.R4ol9kjrMqK78shw2gHBJjEzKeBeo6lBzU8YfoKDM7oPlj5SwmL6sV2i2UaWJbEtreRt3oABDPab--AevfJW2rQ0-2iyt-rJSPjDHHoOQEFoh0G7cPm8SIZxk17ojWkFdM7CXlmuSN0paqMhp-4gWlfgvNq8a65I8GfY8cwVrW5KzRszHLhWYareVM3MNpejdcVH2kinEnYzBVyW0e8oN06LC2icG8FRlhOC2N8wni66liT73RvKyFFT1zW7SAoqtgn9KXY6m.EaZzSx3aapIMGpG9-S8q6mwAuwZId37ri4GTiLXp6OMABsLwT3sMUOm.Kktp.uYP1z2be2DFM6zKKPL7YJopAvdfS7TdhMfHD6Dfcv-EvK6Q0lNylaFIjegbNjPEPPXzLPdf8iwLK-dqfSe127TZcj5xJUFwo45IgFl0i0puKRIzsAtM2W3zM-TNc2HEc0nIllK.aoKZ0tF9iSekzjcNnMAvhcAKBqq6DY.qIBUs5yOoxqW4m-ga9drHp09VXIkn7st7J4IUlrMZFuVHnnzbeqD61AKKFiaRCqPee6Y88DqhsdNt7SzdA-xq9SKnJW67zsZTD0T9OoRl3.nLaSwoQ==\u0026t=adj\u0026prc=${censored}\u0026tat='+_CDurl+'\"\u003e\u003c/scr' + 'ipt\u003e');\u003c/script\u003e"
}
],
"tark": "1"
}
],
"gno": "55f03d71-f021-49e0-a1a5-cae4315b3561"
}
Also, I used the same code you are using, after commenting off the request body part(since I don't have it) and changing the function signature(of SendRequest) a bit as given below. All others are as same as the code given above.
func SendRequest() (*Response, error) {
responseModel := &Response{}
// Commented off since the response body is not with me..
// byteData, err := json.Marshal(requestModel)
// if err != nil {
// zap.S().Errorf("Error marshalling request. Err: %v", err)
// return nil, err
// }
url := "http://localhost:8080/foo"
request, _ := http.NewRequest(http.MethodPost, url /*bytes.NewBuffer(byteData)*/, nil)
request.Close = true
client := &http.Client{}
response, err := client.Do(request)
if err, ok := err.(net.Error); ok && err.Timeout() {
log.Fatal("Response timeout exceed")
return nil, errors.New("Response timeout exceed")
}
if err != nil || response == nil {
errorMessage := "not respond"
log.Fatal(errorMessage)
return nil, errors.New(errorMessage)
}
defer response.Body.Close()
if response.StatusCode == http.StatusOK {
err = json.NewDecoder(response.Body).Decode(&responseModel)
if err != nil {
errorMessage := "Request response decode error"
log.Printf("%v, Err: %v", errorMessage, err)
return nil, errors.New(errorMessage)
}
return responseModel, nil
} else {
bodyBytes, _ := ioutil.ReadAll(response.Body)
errorMessage := "not respond"
log.Printf("%v, StatusCode %v, Response: %v\n", errorMessage, response.StatusCode, string(bodyBytes))
return nil, errors.New(errorMessage)
}
}
With this as well, I am able to Unmarshal the response successfully.
So, in my opinion, the incoming response is not correct(may be). Try printing the incoming data and verify.
resBody, err := ioutil.ReadAll(response.Body)
if err != nil {
return nil, err
}
fmt.Printf("%s", resBody) // 👈

Go http request redirect

I am writing an API whichs has to redirect incoming requests to another service, the response must be forwarded to the original requester.
I figured a simple function like below should do the trick, but I was wrong.
I receive the data from my redirected response, however when I send it back to the initial request I receive this response without any data Could not get response. Error: socket hang up
If I try to execute the very same request using postman straight to the redirect URL it works perfectly fine.
func initialAssetsHandler(w http.ResponseWriter, r *http.Request) {
body, err := ioutil.ReadAll(r.Body)
if err != nil {
fmt.Println(err)
return
}
resp, err := http.Post(conf.redirectURL, "application/json", bytes.NewReader(body))
if err != nil {
log.Error(err)
}
defer resp.Body.Close()
buf := new(bytes.Buffer)
buf.ReadFrom(resp.Body)
log.Info(string(buf.Bytes()))
var data json.RawMessage
if err = json.NewDecoder(resp.Body).Decode(&data); err != nil {
fmt.Println(err)
return
}
helper.SendJsonRaw(w, 200, data)
}
Here is the SendJsonRaw function:
func SendJsonRaw(w http.ResponseWriter, status int, r json.RawMessage) error {
w.Header().Set(HeaderContentType, MimeApplicationJSON)
w.WriteHeader(status)
_, err := w.Write(r)
return err
}
The r.Body is read by the json decoder up to EOF, then when you pass it to the redirect request it looks empty to the http.Client and therefore it sends no body. You need to retain the content of the body.
For example you can do the following:
func initialAssetsHandler(w http.ResponseWriter, r *http.Request) {
body, err := ioutil.ReadAll(r.Body)
if err != nil {
fmt.Println(err)
return
}
var initialAssets TagAssets
if err := json.Unmarshal(&initialAssets, body); err != nil {
if !strings.Contains(err.Error(), "json: invalid use of ,string struct tag, trying to unmarshal") {
helper.SendJsonError(w, http.StatusBadRequest, err)
return
}
}
resp, err := http.Post(conf.redirectURL, "application/json", bytes.NewReader(body))
if err != nil {
log.Error(err)
}
defer resp.Body.Close()
log.Info(resp)
var data json.RawMessage
if err := json.NewDecoder(resp.Body).Decode(&data); err != nil {
fmt.Println(err)
return
}
helper.SendJsonOk(w, data)
}

Sending data in Chunks using single HTTP Post connection

I receive the contents of a file from a data source in chunks. As and when I receive the chunk I want to send the chunk data to a service using http POST request. And by keeping alive the same http POST connection used for sending the first chunk I want to send the remaining chunks of data.
I came up with the following code snippet to implement something similar.
Server-Side
func handle(w http.ResponseWriter, req *http.Request) {
buf := make([]byte, 256)
var n int
for {
n, err := req.Body.Read(buf)
if n == 0 && err == io.EOF {
break
}
fmt.Printf(string(buf[:n]))
}
fmt.Printf(string(buf[:n]))
fmt.Printf("Transfer Complete")
}
Client-Side
type alphaReader struct {
reader io.Reader
}
func newAlphaReader(reader io.Reader) *alphaReader {
return &alphaReader{reader: reader}
}
func (a *alphaReader) Read(p []byte) (int, error) {
n, err := a.reader.Read(p)
return n, err
}
func (a *alphaReader) Reset(str string) {
a.reader = strings.NewReader(str)
}
func (a *alphaReader) Close() error {
return nil
}
func main() {
tr := http.DefaultTransport
alphareader := newAlphaReader(strings.NewReader("First Chunk"))
client := &http.Client{
Transport: tr,
Timeout: 0,
}
req := &http.Request{
Method: "POST",
URL: &url.URL{
Scheme: "http",
Host: "localhost:8080",
Path: "/upload",
},
ProtoMajor: 1,
ProtoMinor: 1,
ContentLength: -1,
Body: alphareader,
}
fmt.Printf("Doing request\n")
_, err := client.Do(req)
alphareader.Reset("Second Chunk")
fmt.Printf("Done request. Err: %v\n", err)
}
Here I want that when I do alphareader.Reset("Second Chunk"), the string "Second Chunk" should be sent using the POST connection made earlier. But that is not happening. The connection gets closed after sending the First Chunk of data. Also I have not written the Close() method properly which I'm not sure how to implement.
I'm newbie to golang and any suggestions would be greatly helpful regarding the same.
A *strings.Reader returns io.EOF after the initial string has been read and your wrapper does nothing to change that, so it cannot be reused. You're looking for io.Pipe to turn the request body into an io.Writer.
package main
import (
"io"
"net/http"
)
func main() {
pr, pw := io.Pipe()
req, err := http.NewRequest("POST", "http://localhost:8080/upload", pr)
if err != nil {
// TODO: handle error
}
go func() {
defer pw.Close()
if _, err := io.WriteString(pw, "first chunk"); err != nil {
_ = err // TODO: handle error
}
if _, err := io.WriteString(pw, "second chunk"); err != nil {
_ = err // TODO: handle error
}
}()
res, err := http.DefaultClient.Do(req)
if err != nil {
// TODO: handle error
}
res.Body.Close()
}
Also, don't initialize the request using a struct literal. Use one of the constructors instead. In your code you're not setting the Host and Header fields, for instance.

Should I close request body if I use io.TeeReader() or io.ReadAll()?

I have a handler which I call from my main() function:
type requestBody struct {
Query string `json:"query"`
}
func main() {
r := chi.NewRouter()
r.Post("/api", MyHandler(superGraph, gqlGen))
}
func MyHandler(library *MyLibrary, next http.Handler) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
buf := bytes.NewBuffer(make([]byte, 0))
reader := io.TeeReader(r.Body, buf)
var reqBody requestBody
err := json.NewDecoder(reader).Decode(&reqBody)
if err != nil {
http.Error(w, "cannot read body", http.StatusBadRequest)
return
}
res, err := library.DoSomething(...)
if err != nil {
log.Error(err)
err := r.Body.Close()
log.ErrorIf(err)
r.Body = ioutil.NopCloser(buf)
next.ServeHTTP(w, r)
return
}
render.JSON(w, r, res) // go-chi "render" pkg
}
}
QUESTION
Do I need the below line?
err := r.Body.Close()
I know https://stackoverflow.com/a/42533540/10088259:
A request body does not need to be closed in the handler. From the http.Request documentation:
The Server will close the request body. The ServeHTTP
Handler does not need to.
but here I'm using:
reader := io.TeeReader(r.Body, buf)
and if err != nil {
r.Body = ioutil.NopCloser(buf)
So, should I r.Body.Close() it in the if path of my code considering that ioutil.NopCloser() has a "fake" Close() method?

Can I post with Content-Type: multipart/form-data

How do I POST to an API with Content-Type: multipart/form-data, []byte parameters and string arguments? I have tried, but it is failing.
Error message:
details: "[301 301 Moved Permanently]<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\r\n<html>\r\n301 Moved Permanently\r\n<body bgcolor=\"white\">\r\n301 Moved Permanently\r\n<p>The requested resource has been assigned a new permanent URI.</p >\r\n<hr/>Powered by Tengine/2.1.0</body>\r\n</html>\r\n"
Go code:
func NewPost2(url string) ([]byte, error) {
m := make(map[string]interface{}, 0)
m["fileName"] ="good"
m["name"] = Base64ToByte("/9j/4AAQSkZJRgABAQEAeAB4AAD/2wBDAAIBAQIBAQICAgICAgICAwUDAwMDAwYEBAMFBwYHBwcGBwcICQsJCAgKCAcHCg0KCgsMDAwMBwkODw0MDgsMDAz/2wBDAQICAgMDAwYDAwYMCAcIDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAz/wAARCAABAAEDASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwDHooor+wD+Zz//2Q==")
b, _ := json.Marshal(m)
httpReq, err := http.NewRequest("POST", url, bytes.NewBuffer(b))
httpReq.Header.Set("Content-Type", "multipart/form-data;charset=UTF-8")
client := &http.Client{}
resp, err := client.Do(httpReq)
if err != nil {
return nil, err
}
defer resp.Body.Close()
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
b, _ := ioutil.ReadAll(resp.Body)
return nil, fmt.Errorf("[%d %s]%s", resp.StatusCode, resp.Status, string(b))
}
respData, err := ioutil.ReadAll(resp.Body)
if err != nil {
return nil, err
}
return respData, nil
}
Now, I am very happy with the mood to share my solution
func NewPostFile(url string, paramTexts map[string]interface{}, paramFile FileItem) ([]byte, error) {
// if paramFiles ==nil {
// return NewPost(url,paramTexts,header,transport)
// }
bodyBuf := &bytes.Buffer{}
bodyWriter := multipart.NewWriter(bodyBuf)
for k, v := range paramTexts {
bodyWriter.WriteField(k, v.(string))
}
fileWriter, err := bodyWriter.CreateFormFile(paramFile.Key, paramFile.FileName)
if err != nil {
fmt.Println(err)
//fmt.Println("Create form file error: ", error)
return nil, err
}
fileWriter.Write(paramFile.Content)
contentType := bodyWriter.FormDataContentType()
bodyWriter.Close()
fmt.Println(bodyBuf.String())
resp, err := http.Post(url, contentType, bodyBuf)
if err != nil {
return nil, err
}
defer resp.Body.Close()
fmt.Println(resp)
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
b, _ := ioutil.ReadAll(resp.Body)
return nil, fmt.Errorf("[%d %s]%s", resp.StatusCode, resp.Status, string(b))
}
respData, err := ioutil.ReadAll(resp.Body)
if err != nil {
return nil, err
}
fmt.Println(string(respData))
return respData, nil
}
type FileItem struct {
Key string //image_content
FileName string //test.jpg
Content []byte //[]byte
}
I wrapped the multipart code in a function, as you need to Close it before you
can make a request. Also my method is using # as a heuristic, similar to
cURL [1]:
package main
import (
"bytes"
"io"
"mime/multipart"
"os"
"strings"
)
func createForm(form map[string]string) (string, io.Reader, error) {
body := new(bytes.Buffer)
mp := multipart.NewWriter(body)
defer mp.Close()
for key, val := range form {
if strings.HasPrefix(val, "#") {
val = val[1:]
file, err := os.Open(val)
if err != nil { return "", nil, err }
defer file.Close()
part, err := mp.CreateFormFile(key, val)
if err != nil { return "", nil, err }
io.Copy(part, file)
} else {
mp.WriteField(key, val)
}
}
return mp.FormDataContentType(), body, nil
}
Example:
package main
import "net/http"
func main() {
form := map[string]string{"profile": "#portrait.jpg", "name": "John"}
ct, body, err := createForm(form)
if err != nil {
panic(err)
}
http.Post("https://stackoverflow.com", ct, body)
}
https://curl.se/docs/manpage.html#-F
On a 301 response, the new url is specified in the headers of the response, not its body
(see https://en.wikipedia.org/wiki/HTTP_301)
try printing :
resp.Header["Location"]
If you have this error as a final response, this also means that the http.Client chose to not follow this redirection.
The doc says that the dafult policy for a Client is to follow up to 10 redirects.
In order to debug redirects, you can write your own CheckRedirect function, which can for instance print the sequence of urls

Resources