Reverse proxy fail request after failure in jwt parsing - http

I have a reverse proxy, it validates jwt from headers and enriches data,
when an expired jwt arrives the proxy returns an error to the user and status code 502 (with the correct error) - which is good!
After that, every request that arrives with valid jwt is returning an error with message: crypto/rsa: verification error
However, when extracting the jwt from the request and checking it locally I found that everything works perfectly.
The proxy code:
parsedUrl, err := url.Parse("http://localhost:" + environment.ProxyPort)
if err != nil {
log.Fatal(err)
}
proxy := httputil.NewSingleHostReverseProxy(parsedUrl)
proxy.FlushInterval = 200 * time.Millisecond
proxy.Transport = &myTransport{}
proxy.ModifyResponse = func(r *http.Response) error {
return nil
}
proxy.ErrorHandler = func(rw http.ResponseWriter, r *http.Request, err error) {
logs.Log(err.Error(), logs.Error)
rw.WriteHeader(http.StatusBadGateway)
if _, err := rw.Write([]byte(`{"source": "enrich-sidecar", error": "` + err.Error() + `"}`)); err != nil {
log.Fatal(err)
}
}
req.URL.Host = parsedUrl.Host
req.URL.Scheme = parsedUrl.Scheme
req.Header.Set("X-Forwarded-Host", req.Header.Get("Host"))
req.Host = parsedUrl.Host
proxy.ServeHTTP(res, req)
The validation:
if err := validateAlg(jwtToken); err != nil {
return false, nil, err
}
token, err := jwt.Parse(jwtToken, func(token *jwt.Token) (interface{}, error) {
return key, nil
})
if err != nil {
return false, nil, err
} else if !token.Valid {
return false, nil, token.Claims.Valid()
}

Related

Golang large response unexpected end of JSON input

I'm sending a request to a server and trying to unmarshall HTTP Response. It's weird, but some of the responses return 200 and do not give an unmarshall error, while some of them return 200 and give an unmarshall error.
My client code looks like:
func SendRequest(requestModel *model.Request) (*model.Response, error) {
responseModel := &dspModel.Response{}
byteData, err := json.Marshal(requestModel)
if err != nil {
zap.S().Errorf("Error marshalling request. Err: %v", err)
return nil, err
}
url := "xx"
request, _ := http.NewRequest(http.MethodPost, url, bytes.NewBuffer(byteData))
request.Close = true
client := &http.Client{}
response, err := client.Do(request)
if err, ok := err.(net.Error); ok && err.Timeout() {
zap.S().Error("Response timeout exceed")
return nil, errors.New("Response timeout exceed")
}
if err != nil || response == nil {
errorMessage := "not respond"
zap.S().Error(errorMessage)
return nil, errors.New(errorMessage)
}
defer response.Body.Close()
if response.StatusCode == http.StatusOK {
err = json.NewDecoder(response.Body).Decode(&responseModel)
if err != nil {
// Error occurred here!
errorMessage := "Request response decode error"
zap.S().Errorf("%v, Err: %v", errorMessage, err)
return nil, errors.New(errorMessage)
}
return response, nil
} else if response.StatusCode == http.StatusNoContent {
return nil, nil
} else {
bodyBytes, _ := ioutil.ReadAll(response.Body)
errorMessage := "not respond"
zap.S().Errorf("%v, StatusCode %v, Response: %v Request: %v", errorMessage,
response.StatusCode, string(bodyBytes), string(byteData))
return nil, errors.New(errorMessage)
}
}
I suspect response cause it's too long and has different characters.
Response looks like:
{"tax":{"ver":"1.0"},"cur":"EUR","rack":[{"tur":[{"zar":2.599886212,"domain":["test.com"],"ney":"https://censored.com/nimp?fuid=&ic=EWRMh-.UTPdNvWB-JYa58c85N0fEPgXunKp3wwyPadp7jwqHgZbz4dG0A51OVO-2Gs0znYmLcPIH0ThEmpsYl8wKofo9ytJ2A3uWr9Kn-dNxeh.k8lIml9kavPk1.dk7f.46xKX7IVpf3.yU-Yx1KetQl3Q9f-iePn7B86yjVgMxkTNfhZAg0pP0kKZaJMd2orLXoV4xPXmwTdfJbWJU5bGAUROJT-Yd7yTHoVveuvOBClHzM4cgHFmGxzox6cCJ2gZB.7fqKkPzECXwdpobmO0RWxdu224-FADd.oM4DghIEpdZJe.FjEq0stQnJBT.puw0JamHgT15NdSQN7voBJ8UqGCDOu1qSLece6Iy.PN392xGWhxs0URbrWhSEgkhCr.R4ol9kjrMqK78shw2gHBJjEzKeBeo6lBzU8YfoKDM7oPlj5SwmL6sV2i2UaWJbEtreRt3oABDPab--AevfJW2rQ0-2iyt-rJSPjDHHoOQEFoh0G7cPm8SIZxk17ojWkFdM7CXlmuSN0paqMhp-4gWlfgvNq8a65I8GfY8cwVrW5KzRszHLhWYareVM3MNpejdcVH2kinEnYzBVyW0e8oN06LC2icG8FRlhOC2N8wni66liT73RvKyFFT1zW7SAoqtgn9KXY6m.EaZzSx3aapIMGpG9-S8q6mwAuwZId37ri4GTiLXp6OMABsLwT3sMUOm.Kktp.uYP1z2be2DFM6zKKPL7YJopAvdfS7TdhMfHD6Dfcv-EvK6Q0lNylaFIjegbNjPEPPXzLPdf8iwLK-dqfSe127TZcj5xJUFwo45IgFl0i0puKRIzsAtM2W3zM-TNc2HEc0nIllK.aoKZ0tF9iSekzjcNnMAvhcAKBqq6DY.qIBUs5yOoxqW4m-ga9drHp09VXIkn7st7J4IUlrMZFuVHnnzbeqD61AKKFiaRCqPee6Y88DqhsdNt7SzdA-xq9SKnJW67zsZTD0T9OoRl3.nLaSwoQ==&t=adi&prc=${censored}","tax":{"name":"test"},"hy":3020,"ny":2530,"arz":"<script type=\"text/javascript\">var _CDurl='';(function(){_CDurl=encodeURIComponent(document.URL);})();document.write('<scr' + 'ipt type=\"text/javascript\" src=\"https://censored/at?bad=&gad=&ic=EWRMh-.UTPdNvWB-JYa58c85N0fEPgXunKp3wwyPadp7jwqHgZbz4dG0A51OVO-2Gs0znYmLcPIH0ThEmpsYl8wKofo9ytJ2A3uWr9Kn-dNxeh.k8lIml9kavPk1.dk7f.46xKX7IVpf3.yU-Yx1KetQl3Q9f-iePn7B86yjVgMxkTNfhZAg0pP0kKZaJMd2orLXoV4xPXmwTdfJbWJU5bGAUROJT-Yd7yTHoVveuvOBClHzM4cgHFmGxzox6cCJ2gZB.7fqKkPzECXwdpobmO0RWxdu224-FADd.oM4DghIEpdZJe.FjEq0stQnJBT.puw0JamHgT15NdSQN7voBJ8UqGCDOu1qSLece6Iy.PN392xGWhxs0URbrWhSEgkhCr.R4ol9kjrMqK78shw2gHBJjEzKeBeo6lBzU8YfoKDM7oPlj5SwmL6sV2i2UaWJbEtreRt3oABDPab--AevfJW2rQ0-2iyt-rJSPjDHHoOQEFoh0G7cPm8SIZxk17ojWkFdM7CXlmuSN0paqMhp-4gWlfgvNq8a65I8GfY8cwVrW5KzRszHLhWYareVM3MNpejdcVH2kinEnYzBVyW0e8oN06LC2icG8FRlhOC2N8wni66liT73RvKyFFT1zW7SAoqtgn9KXY6m.EaZzSx3aapIMGpG9-S8q6mwAuwZId37ri4GTiLXp6OMABsLwT3sMUOm.Kktp.uYP1z2be2DFM6zKKPL7YJopAvdfS7TdhMfHD6Dfcv-EvK6Q0lNylaFIjegbNjPEPPXzLPdf8iwLK-dqfSe127TZcj5xJUFwo45IgFl0i0puKRIzsAtM2W3zM-TNc2HEc0nIllK.aoKZ0tF9iSekzjcNnMAvhcAKBqq6DY.qIBUs5yOoxqW4m-ga9drHp09VXIkn7st7J4IUlrMZFuVHnnzbeqD61AKKFiaRCqPee6Y88DqhsdNt7SzdA-xq9SKnJW67zsZTD0T9OoRl3.nLaSwoQ==&t=adj&prc=${censored}&tat='+_CDurl+'\"></scr' + 'ipt>');</script>"}],"tark":"1"}],"gno":"55f03d71-f021-49e0-a1a5-cae4315b3561"}
When I debug the error, I noticed that after half of the response is not visible.
Error statement:
"msg":"Request response decode error, Response: {\"tax\":{\"ver\":\"1.0\"},\"cur\":\"EUR\",\"rac\":[{\"btyrd\":[{\"zar\":2.599886212,\"domain\":[\"test.com\"],\"ney\":\"https://censored/nimp?fuid=&ic=EWRMh-.UTPdNvWB-JYa58c85N0fEPgXunKp3wwyPadp7jwqHgZbz4dG0A51OVO-2Gs0znYmLcPIH0ThEmpsYl8wKofo9ytJ2A3uWr9Kn-dNxeh.k8lIml9kavPk1.dk7f.46xKX7IVpf3.yU-Yx1KetQl3Q9f-iePn7B86yjVgMxkTNfhZAg0pP0kKZaJMd2orLXoV4xPXmwTdfJbWJU5bGAUROJT-Yd7yTHoVveuvOBClHzM4cgHFmGxzox6cCJ2gZB.7fqKkPzECXwdpobmO0RWxdu224-FADd.oM4DghIEpdZJe.FjEq0stQnJBT.puw0JamHgT15NdSQN7voBJ8UqGCDOu1q, Err: unexpected end of JSON input"
Response Model:
type Response struct {
Tax Tax `json:"tax"`
Cur string `json:"cur"`
Rack []Rack `json:"rack"`
Gno string `json:"gno"`
}
type Tax struct {
Ver string `json:"ver"`
}
type TaxOfTur struct {
Name string `json:"name"`
}
type Tur struct {
Zar float64 `json:"zar"`
Domain []string `json:"domain"`
Ney string `json:"ney"`
Tax TaxOfTur `json:"tax"`
Hy int `json:"hy"`
Ny int `json:"ny"`
Arz string `json:"arz"`
}
type Rack struct {
Tur []Tur `json:"tur"`
Tark string `json:"tark"`
}
Probably unrelated but requests are going concurrent and my test case is based on only one concurrent request. So how can I solve this problem?
I tried to implement your code at my end, and it's giving the expected output. The code I tried is given below.
Note: This is not a solution to the exact problem, since I was not able to reproduce the same.
// Response struct definition here...
// ..
// The trouble making JSON.
const msg = `{"tax":{"ver":"1.0"},"cur":"EUR","rack":[{"tur":[{"zar":2.599886212,"domain":["test.com"],"ney":"https://censored.com/nimp?fuid=&ic=EWRMh-.UTPdNvWB-JYa58c85N0fEPgXunKp3wwyPadp7jwqHgZbz4dG0A51OVO-2Gs0znYmLcPIH0ThEmpsYl8wKofo9ytJ2A3uWr9Kn-dNxeh.k8lIml9kavPk1.dk7f.46xKX7IVpf3.yU-Yx1KetQl3Q9f-iePn7B86yjVgMxkTNfhZAg0pP0kKZaJMd2orLXoV4xPXmwTdfJbWJU5bGAUROJT-Yd7yTHoVveuvOBClHzM4cgHFmGxzox6cCJ2gZB.7fqKkPzECXwdpobmO0RWxdu224-FADd.oM4DghIEpdZJe.FjEq0stQnJBT.puw0JamHgT15NdSQN7voBJ8UqGCDOu1qSLece6Iy.PN392xGWhxs0URbrWhSEgkhCr.R4ol9kjrMqK78shw2gHBJjEzKeBeo6lBzU8YfoKDM7oPlj5SwmL6sV2i2UaWJbEtreRt3oABDPab--AevfJW2rQ0-2iyt-rJSPjDHHoOQEFoh0G7cPm8SIZxk17ojWkFdM7CXlmuSN0paqMhp-4gWlfgvNq8a65I8GfY8cwVrW5KzRszHLhWYareVM3MNpejdcVH2kinEnYzBVyW0e8oN06LC2icG8FRlhOC2N8wni66liT73RvKyFFT1zW7SAoqtgn9KXY6m.EaZzSx3aapIMGpG9-S8q6mwAuwZId37ri4GTiLXp6OMABsLwT3sMUOm.Kktp.uYP1z2be2DFM6zKKPL7YJopAvdfS7TdhMfHD6Dfcv-EvK6Q0lNylaFIjegbNjPEPPXzLPdf8iwLK-dqfSe127TZcj5xJUFwo45IgFl0i0puKRIzsAtM2W3zM-TNc2HEc0nIllK.aoKZ0tF9iSekzjcNnMAvhcAKBqq6DY.qIBUs5yOoxqW4m-ga9drHp09VXIkn7st7J4IUlrMZFuVHnnzbeqD61AKKFiaRCqPee6Y88DqhsdNt7SzdA-xq9SKnJW67zsZTD0T9OoRl3.nLaSwoQ==&t=adi&prc=${censored}","tax":{"name":"test"},"hy":3020,"ny":2530,"arz":"<script type=\"text/javascript\">var _CDurl='';(function(){_CDurl=encodeURIComponent(document.URL);})();document.write('<scr' + 'ipt type=\"text/javascript\" src=\"https://censored/at?bad=&gad=&ic=EWRMh-.UTPdNvWB-JYa58c85N0fEPgXunKp3wwyPadp7jwqHgZbz4dG0A51OVO-2Gs0znYmLcPIH0ThEmpsYl8wKofo9ytJ2A3uWr9Kn-dNxeh.k8lIml9kavPk1.dk7f.46xKX7IVpf3.yU-Yx1KetQl3Q9f-iePn7B86yjVgMxkTNfhZAg0pP0kKZaJMd2orLXoV4xPXmwTdfJbWJU5bGAUROJT-Yd7yTHoVveuvOBClHzM4cgHFmGxzox6cCJ2gZB.7fqKkPzECXwdpobmO0RWxdu224-FADd.oM4DghIEpdZJe.FjEq0stQnJBT.puw0JamHgT15NdSQN7voBJ8UqGCDOu1qSLece6Iy.PN392xGWhxs0URbrWhSEgkhCr.R4ol9kjrMqK78shw2gHBJjEzKeBeo6lBzU8YfoKDM7oPlj5SwmL6sV2i2UaWJbEtreRt3oABDPab--AevfJW2rQ0-2iyt-rJSPjDHHoOQEFoh0G7cPm8SIZxk17ojWkFdM7CXlmuSN0paqMhp-4gWlfgvNq8a65I8GfY8cwVrW5KzRszHLhWYareVM3MNpejdcVH2kinEnYzBVyW0e8oN06LC2icG8FRlhOC2N8wni66liT73RvKyFFT1zW7SAoqtgn9KXY6m.EaZzSx3aapIMGpG9-S8q6mwAuwZId37ri4GTiLXp6OMABsLwT3sMUOm.Kktp.uYP1z2be2DFM6zKKPL7YJopAvdfS7TdhMfHD6Dfcv-EvK6Q0lNylaFIjegbNjPEPPXzLPdf8iwLK-dqfSe127TZcj5xJUFwo45IgFl0i0puKRIzsAtM2W3zM-TNc2HEc0nIllK.aoKZ0tF9iSekzjcNnMAvhcAKBqq6DY.qIBUs5yOoxqW4m-ga9drHp09VXIkn7st7J4IUlrMZFuVHnnzbeqD61AKKFiaRCqPee6Y88DqhsdNt7SzdA-xq9SKnJW67zsZTD0T9OoRl3.nLaSwoQ==&t=adj&prc=${censored}&tat='+_CDurl+'\"></scr' + 'ipt>');</script>"}],"tark":"1"}],"gno":"55f03d71-f021-49e0-a1a5-cae4315b3561"}`
func SendRequest() (*Response, error) {
url := "http://localhost:8080/foo" // dummy server
request, _ := http.NewRequest(http.MethodPost, url, nil)
request.Close = true
client := &http.Client{}
response, err := client.Do(request)
if err != nil {
return nil, err
}
defer response.Body.Close()
resBody, err := ioutil.ReadAll(response.Body)
if err != nil {
return nil, err
}
responseModel := &Response{}
err = json.Unmarshal(resBody, &responseModel)
if err != nil {
return nil, err
}
return responseModel, nil
}
func StartDummyServer() {
handler := func(w http.ResponseWriter, r *http.Request) {
fmt.Fprint(w, msg)
}
http.HandleFunc("/foo", handler)
log.Fatal(http.ListenAndServe(":8080", nil))
}
func main() {
// a dummy server is created to send the response
go StartDummyServer()
time.Sleep(time.Second)
resp, err := SendRequest()
if err != nil {
log.Fatal(err.Error())
}
fmt.Println(prettyPrint(resp))
time.Sleep(time.Second * 10)
}
func prettyPrint(i interface{}) string {
s, _ := json.MarshalIndent(i, "", " ")
return string(s)
}
Here is the console output:
{
"tax": {
"ver": "1.0"
},
"cur": "EUR",
"rack": [
{
"tur": [
{
"zar": 2.599886212,
"domain": [
"test.com"
],
"ney": "https://censored.com/nimp?fuid=\u0026ic=EWRMh-.UTPdNvWB-JYa58c85N0fEPgXunKp3wwyPadp7jwqHgZbz4dG0A51OVO-2Gs0znYmLcPIH0ThEmpsYl8wKofo9ytJ2A3uWr9Kn-dNxeh.k8lIml9kavPk1.dk7f.46xKX7IVpf3.yU-Yx1KetQl3Q9f-iePn7B86yjVgMxkTNfhZAg0pP0kKZaJMd2orLXoV4xPXmwTdfJbWJU5bGAUROJT-Yd7yTHoVveuvOBClHzM4cgHFmGxzox6cCJ2gZB.7fqKkPzECXwdpobmO0RWxdu224-FADd.oM4DghIEpdZJe.FjEq0stQnJBT.puw0JamHgT15NdSQN7voBJ8UqGCDOu1qSLece6Iy.PN392xGWhxs0URbrWhSEgkhCr.R4ol9kjrMqK78shw2gHBJjEzKeBeo6lBzU8YfoKDM7oPlj5SwmL6sV2i2UaWJbEtreRt3oABDPab--AevfJW2rQ0-2iyt-rJSPjDHHoOQEFoh0G7cPm8SIZxk17ojWkFdM7CXlmuSN0paqMhp-4gWlfgvNq8a65I8GfY8cwVrW5KzRszHLhWYareVM3MNpejdcVH2kinEnYzBVyW0e8oN06LC2icG8FRlhOC2N8wni66liT73RvKyFFT1zW7SAoqtgn9KXY6m.EaZzSx3aapIMGpG9-S8q6mwAuwZId37ri4GTiLXp6OMABsLwT3sMUOm.Kktp.uYP1z2be2DFM6zKKPL7YJopAvdfS7TdhMfHD6Dfcv-EvK6Q0lNylaFIjegbNjPEPPXzLPdf8iwLK-dqfSe127TZcj5xJUFwo45IgFl0i0puKRIzsAtM2W3zM-TNc2HEc0nIllK.aoKZ0tF9iSekzjcNnMAvhcAKBqq6DY.qIBUs5yOoxqW4m-ga9drHp09VXIkn7st7J4IUlrMZFuVHnnzbeqD61AKKFiaRCqPee6Y88DqhsdNt7SzdA-xq9SKnJW67zsZTD0T9OoRl3.nLaSwoQ==\u0026t=adi\u0026prc=${censored}",
"tax": {
"name": "test"
},
"hy": 3020,
"ny": 2530,
"arz": "\u003cscript type=\"text/javascript\"\u003evar _CDurl='';(function(){_CDurl=encodeURIComponent(document.URL);})();document.write('\u003cscr' + 'ipt type=\"text/javascript\" src=\"https://censored/at?bad=\u0026gad=\u0026ic=EWRMh-.UTPdNvWB-JYa58c85N0fEPgXunKp3wwyPadp7jwqHgZbz4dG0A51OVO-2Gs0znYmLcPIH0ThEmpsYl8wKofo9ytJ2A3uWr9Kn-dNxeh.k8lIml9kavPk1.dk7f.46xKX7IVpf3.yU-Yx1KetQl3Q9f-iePn7B86yjVgMxkTNfhZAg0pP0kKZaJMd2orLXoV4xPXmwTdfJbWJU5bGAUROJT-Yd7yTHoVveuvOBClHzM4cgHFmGxzox6cCJ2gZB.7fqKkPzECXwdpobmO0RWxdu224-FADd.oM4DghIEpdZJe.FjEq0stQnJBT.puw0JamHgT15NdSQN7voBJ8UqGCDOu1qSLece6Iy.PN392xGWhxs0URbrWhSEgkhCr.R4ol9kjrMqK78shw2gHBJjEzKeBeo6lBzU8YfoKDM7oPlj5SwmL6sV2i2UaWJbEtreRt3oABDPab--AevfJW2rQ0-2iyt-rJSPjDHHoOQEFoh0G7cPm8SIZxk17ojWkFdM7CXlmuSN0paqMhp-4gWlfgvNq8a65I8GfY8cwVrW5KzRszHLhWYareVM3MNpejdcVH2kinEnYzBVyW0e8oN06LC2icG8FRlhOC2N8wni66liT73RvKyFFT1zW7SAoqtgn9KXY6m.EaZzSx3aapIMGpG9-S8q6mwAuwZId37ri4GTiLXp6OMABsLwT3sMUOm.Kktp.uYP1z2be2DFM6zKKPL7YJopAvdfS7TdhMfHD6Dfcv-EvK6Q0lNylaFIjegbNjPEPPXzLPdf8iwLK-dqfSe127TZcj5xJUFwo45IgFl0i0puKRIzsAtM2W3zM-TNc2HEc0nIllK.aoKZ0tF9iSekzjcNnMAvhcAKBqq6DY.qIBUs5yOoxqW4m-ga9drHp09VXIkn7st7J4IUlrMZFuVHnnzbeqD61AKKFiaRCqPee6Y88DqhsdNt7SzdA-xq9SKnJW67zsZTD0T9OoRl3.nLaSwoQ==\u0026t=adj\u0026prc=${censored}\u0026tat='+_CDurl+'\"\u003e\u003c/scr' + 'ipt\u003e');\u003c/script\u003e"
}
],
"tark": "1"
}
],
"gno": "55f03d71-f021-49e0-a1a5-cae4315b3561"
}
Also, I used the same code you are using, after commenting off the request body part(since I don't have it) and changing the function signature(of SendRequest) a bit as given below. All others are as same as the code given above.
func SendRequest() (*Response, error) {
responseModel := &Response{}
// Commented off since the response body is not with me..
// byteData, err := json.Marshal(requestModel)
// if err != nil {
// zap.S().Errorf("Error marshalling request. Err: %v", err)
// return nil, err
// }
url := "http://localhost:8080/foo"
request, _ := http.NewRequest(http.MethodPost, url /*bytes.NewBuffer(byteData)*/, nil)
request.Close = true
client := &http.Client{}
response, err := client.Do(request)
if err, ok := err.(net.Error); ok && err.Timeout() {
log.Fatal("Response timeout exceed")
return nil, errors.New("Response timeout exceed")
}
if err != nil || response == nil {
errorMessage := "not respond"
log.Fatal(errorMessage)
return nil, errors.New(errorMessage)
}
defer response.Body.Close()
if response.StatusCode == http.StatusOK {
err = json.NewDecoder(response.Body).Decode(&responseModel)
if err != nil {
errorMessage := "Request response decode error"
log.Printf("%v, Err: %v", errorMessage, err)
return nil, errors.New(errorMessage)
}
return responseModel, nil
} else {
bodyBytes, _ := ioutil.ReadAll(response.Body)
errorMessage := "not respond"
log.Printf("%v, StatusCode %v, Response: %v\n", errorMessage, response.StatusCode, string(bodyBytes))
return nil, errors.New(errorMessage)
}
}
With this as well, I am able to Unmarshal the response successfully.
So, in my opinion, the incoming response is not correct(may be). Try printing the incoming data and verify.
resBody, err := ioutil.ReadAll(response.Body)
if err != nil {
return nil, err
}
fmt.Printf("%s", resBody) // 👈

How would I use an HTTP proxy when using a custom DialTLS Function in the transport

I'm trying to use a custom DialTLS Function in the http.Transport whilst using a HTTP proxy at the same time.
This is what I tried so far but whenever I configure the HTTP proxy it doesn't use the custom DialTLS Function :
tr, err := NewTransport("771,4865-4866-4867-49196-49195-49188-49187-49162-49161-52393-49200-49199-49192-49191-49172-49171-52392-157-156-61-60-53-47-49160-49170-10,65281-0-23-13-5-18-16-11-51-45-43-10-21,29-23-24-25,0")
if err != nil {
print(err)
}
tr.Proxy = http.ProxyURL(proxyURL)
tr.DisableKeepAlives = true
httpclient := &http.Client{Transport: tr, Timeout: 2 * time.Second}
proxyURL, err := url.Parse(fmt.Sprintf("http://%s", "127.0.0.1:80"))
if err != nil {
return errors.New("invalid proxy")
}
I have looked into the docs for the http.Transport and I found nothing that can help. Incase this is also required this is the custom DialTLS function I'm using:
dialtls := func(network, addr string) (net.Conn, error) {
dialConn, err := net.Dial(network, addr)
if err != nil {
return nil, err
}
config.ServerName = strings.Split(addr, ":")[0]
uTLSConn := tls.UClient(dialConn, config, tls.HelloCustom)
if err := uTLSConn.ApplyPreset(spec); err != nil {
return nil, err
}
if err := uTLSConn.Handshake(); err != nil {
return nil, err
}
return uTLSConn, nil
}
I'm trying to change the ja3 fingerprint a.k.a tls config. If anyone knows how I could achieve this please let me know

Go http request redirect

I am writing an API whichs has to redirect incoming requests to another service, the response must be forwarded to the original requester.
I figured a simple function like below should do the trick, but I was wrong.
I receive the data from my redirected response, however when I send it back to the initial request I receive this response without any data Could not get response. Error: socket hang up
If I try to execute the very same request using postman straight to the redirect URL it works perfectly fine.
func initialAssetsHandler(w http.ResponseWriter, r *http.Request) {
body, err := ioutil.ReadAll(r.Body)
if err != nil {
fmt.Println(err)
return
}
resp, err := http.Post(conf.redirectURL, "application/json", bytes.NewReader(body))
if err != nil {
log.Error(err)
}
defer resp.Body.Close()
buf := new(bytes.Buffer)
buf.ReadFrom(resp.Body)
log.Info(string(buf.Bytes()))
var data json.RawMessage
if err = json.NewDecoder(resp.Body).Decode(&data); err != nil {
fmt.Println(err)
return
}
helper.SendJsonRaw(w, 200, data)
}
Here is the SendJsonRaw function:
func SendJsonRaw(w http.ResponseWriter, status int, r json.RawMessage) error {
w.Header().Set(HeaderContentType, MimeApplicationJSON)
w.WriteHeader(status)
_, err := w.Write(r)
return err
}
The r.Body is read by the json decoder up to EOF, then when you pass it to the redirect request it looks empty to the http.Client and therefore it sends no body. You need to retain the content of the body.
For example you can do the following:
func initialAssetsHandler(w http.ResponseWriter, r *http.Request) {
body, err := ioutil.ReadAll(r.Body)
if err != nil {
fmt.Println(err)
return
}
var initialAssets TagAssets
if err := json.Unmarshal(&initialAssets, body); err != nil {
if !strings.Contains(err.Error(), "json: invalid use of ,string struct tag, trying to unmarshal") {
helper.SendJsonError(w, http.StatusBadRequest, err)
return
}
}
resp, err := http.Post(conf.redirectURL, "application/json", bytes.NewReader(body))
if err != nil {
log.Error(err)
}
defer resp.Body.Close()
log.Info(resp)
var data json.RawMessage
if err := json.NewDecoder(resp.Body).Decode(&data); err != nil {
fmt.Println(err)
return
}
helper.SendJsonOk(w, data)
}

How can I upgrade a client http connection to websockets in golang after sending the connection upgrade

I need a golang client that can upgrade from an http get response to a websocket connection.
I have a JS client that works and I've seen direct ws client connections but I have to upgrade from http. I have tried looking for other 3GL solutions (Java, C#, Python) but I need to be able to implement the upgrade in Go. I have seen Dart detaching the socket and creating a websocket from it.
WebSocket.fromUpgradedSocket
I noticed Client does not support Hijack but the discussion didn't get me anywhere.
I am using github.com/gorilla/websocket but can change that if it helps.
Server:
func main() {
srv := Srv{}
count = 0
http.HandleFunc("/", srv.handleRoot)
http.HandleFunc("/ws", srv.handleWs)
log.Fatal(http.ListenAndServe(":5002", nil))
}
func (tool *Srv) handleRoot(w http.ResponseWriter, r *http.Request) {
webSocketKey := hdr.Get("Sec-WebSocket-Key")
log.Printf("Socket key = '%v'", webSocketKey)
secWsAccept := computeAcceptKey(webSocketKey)
log.Printf("Accept = '%v'", secWsAccept)
w.Header().Add("sec-websocket-accept", secWsAccept)
w.Header().Add("upgrade", "websockt")
w.Header().Add("connection", "upgrade")
w.WriteHeader(101)
}
func (tool *Srv) handleWs(w http.ResponseWriter, r *http.Request) {
var upgrader = websocket.Upgrader{}
conn, err := upgrader.Upgrade(w, r, nil)
if err != nil {
log.Fatalf("Websocket fatal error. %v", err)
}
tool.conn = conn
go tool.serviceWsRequests()
}
func (tool *Srv) serviceWsRequests() {
for {
log.Printf("starting ws")
req := request{}
err := tool.conn.ReadJSON(&req)
if err != nil {
log.Printf("Failed to decode ws message. %v", err)
break
}
fmt.Printf("Got request. %v\n", req)
if req.Method == "ping" {
fmt.Printf("Param=%v\n", req.Parameters)
}
}
}
var keyGUID = []byte("258EAFA5-E914-47DA-95CA-C5AB0DC85B11")
func computeAcceptKey(challengeKey string) string {
h := sha1.New()
h.Write([]byte(challengeKey))
h.Write(keyGUID)
return base64.StdEncoding.EncodeToString(h.Sum(nil))
}
Client:
func main() {
tr := &http.Transport{
MaxIdleConns: 10,
IdleConnTimeout: 30 * time.Second,
DisableCompression: true,
}
client := &http.Client{
Transport: tr,
// Do NOT follow redirects
CheckRedirect: func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
},
}
wsKey, err := generateKey()
if err != nil {
log.Printf("Cannot generate challenge key %v", err)
}
// Get request for ws upgrade.
req, err := http.NewRequest("GET", "http://localhost:5002", nil)
req.Header.Add("Connection", "Upgrade")
req.Header.Add("Upgrade", "websocket")
req.Header.Add("Sec-WebSocket-Version", "13")
req.Header.Add("Sec-WebSocket-Key", wsKey)
log.Printf("ws key '%v'", wsKey)
resp, err := client.Do(req)
if err != nil {
log.Printf("Get error %v", err)
}
defer func() {
if resp != nil {
err = resp.Body.Close()
}
}()
log.Printf("Status='%v', proto='%v'", resp.Status, resp.Proto)
body, err := ioutil.ReadAll(resp.Body)
hdr := resp.Header
for k, v := range hdr{
log.Printf("%v : %v", k, v)
}
log.Printf("Body = %v", string(body))
resp, err = http.Get("ws://localhost:5002/ws")
if err != nil {
log.Printf("Error '%v'", err)
}
}
func generateKey() (string, error) {
p := make([]byte, 16)
if _, err := io.ReadFull(rand.Reader, p); err != nil {
return "", err
}
return base64.StdEncoding.EncodeToString(p), nil
}
var keyGUID = []byte("258EAFA5-E914-47DA-95CA-C5AB0DC85B11")
func computeAcceptKey(challengeKey string) string {
h := sha1.New()
h.Write([]byte(challengeKey))
h.Write(keyGUID)
return base64.StdEncoding.EncodeToString(h.Sum(nil))
}
I get an error
Error 'Get ws://localhost:5002/ws: unsupported protocol scheme "ws"'
Which doesn't surprise me because I haven't upgraded the connection.
So how do I go an upgrade in Go?
Use the Gorilla client to dial websocket connections:
func main() {
c, _ , err := websocket.DefaultDialer.Dial("ws://localhost:5002/ws", nil)
if err != nil {
// handle error
}
defer c.Close()
// do something with c, a *websocket.Conn
}
The Dial method issues a GET to the server requesting an upgrade to the WebSocket protocol. On successful completion of the upgrade, Dial returns a *websocket.Conn.

Reading content from http.Get in Golang

I'm having a tough time reading XML from a GET request in Go. I just started to learn Go and haven't found any resources on this topic. What I tried:
response, err := http.Get(url)
if err != nil {
log.Fatal(err)
} else {
defer response.Body.Close()
xml, _ := ioutil.ReadAll(response.Body)
if err != nil {
log.Fatal(err)
}
}
_, err := io.Copy(os.Stdout, response.Body) works but I'd like to store the XML for further processing.
Any help is greatly appreciated.
What you've tried is mostly good. Few things to improve it:
http.Get() returns an http.Response and an optional error. If there is no error, that only means that the HTTP GET operation succeeded, but the server might have responded with an error document. So you still have to check the response HTTP status code.
Also io.ReadAll() also returns an error (besides the read data), don't forget to check that too.
Let's wrap it in a function:
func getXML(url string) (string, error) {
resp, err := http.Get(url)
if err != nil {
return "", fmt.Errorf("GET error: %v", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return "", fmt.Errorf("Status error: %v", resp.StatusCode)
}
data, err := ioutil.ReadAll(resp.Body)
if err != nil {
return "", fmt.Errorf("Read body: %v", err)
}
return string(data), nil
}
Testing / using the above function:
if xmlStr, err := getXML("http://somehost.com/some.xml"); err != nil {
log.Printf("Failed to get XML: %v", err)
} else {
log.Println("Received XML:")
log.Println(xmlStr)
}
Also note that it would be the same to get the content of any other responses, so it's worth not "encoding" the string conversion and return type. This one is more general:
func getContent(url string) ([]byte, error) {
resp, err := http.Get(url)
if err != nil {
return nil, fmt.Errorf("GET error: %v", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("Status error: %v", resp.StatusCode)
}
data, err := ioutil.ReadAll(resp.Body)
if err != nil {
return nil, fmt.Errorf("Read body: %v", err)
}
return data, nil
}
Using this to get an XML doc:
if data, err := getContent("http://somehost.com/some.xml"); err != nil {
log.Printf("Failed to get XML: %v", err)
} else {
log.Println("Received XML:")
log.Println(string(data))
}

Resources