I have for loop in which I calling that function that takes response from osrm server, after some time ioutil.ReadAll(resp.Body) returns err that prints http2: server sent GOAWAY and closed the connection; LastStreamID=1999, ErrCode=NO_ERROR, debug=""
func RequestGET(req string) []byte {
reqst, err := http.NewRequest("GET", req, nil)
client := &http.Client{}
resp, err := client.Do(reqst)
if err != nil {
panic(err)
}
resp_data, err := ioutil.ReadAll(resp.Body)
resp.Body.Close()
if err != nil {
fmt.Println(err)
}
return resp_data
}
Isn't resp.Body.Close() closes a connection? I expected to get every time a new one.
The server is closing the connection for some reason; the backend closed, timed out, etc. There's not much you can do except retry, just as if you had any other connection error. There is some discussion in a arelated issue (https://golang.org/issue/18639) about retrying this automatically, but it generally looks like the client is working as intended.
Response.Body.Close does not close a connection, otherwise it would defeat the purpose of http1.1 and http2 using persistent connections. Reading and closing the response body is how you allow the connection to be reused by the http client.
I had this issue related to a GET request sending a header with more than 50k. By default nginx has a limit of 4k. So if your server is dropping the connection because the request header, you can get this message.
Related
I am implementing a retry using http.RoundTripper in Go. Here is an implementation example.
type retryableRoundTripper struct {
tr http.RoundTripper
maxRetryCount int
}
func (t *retryableRoundTripper) RoundTrip(req *http.Request) (resp *http.Response, err error) {
for count := 0; count < t.maxRetryCount; count++ {
log.Printf("retryableRoundTripper retry: %d\n", count+1)
resp, err = t.tr.RoundTrip(req)
if err != nil || resp.StatusCode != http.StatusTooManyRequests {
return resp, err
}
}
return resp, err
}
Questions
Is it necessary to read and close the response body to reuse the TCP connection on retry?
func (t *retryableRoundTripper) RoundTrip(req *http.Request) (resp *http.Response, err error) {
for count := 0; count < t.maxRetryCount; count++ {
log.Printf("retryableRoundTripper retry: %d\n", count+1)
resp, err = t.tr.RoundTrip(req)
if err != nil || resp.StatusCode != http.StatusTooManyRequests {
return resp, err
}
}
// add
io.Copy(ioutil.Discard, resp.Body)
resp.Body.Close()
return resp, err
}
As a side note, I've written a test and have confirmed that retries work as expected. (In Go Playground, it times out, but it works locally.)
https://play.golang.org/p/08YWV0kjaKr
Of course you need to read the connection to ensure that it can be reused, and Closing the connection is required as documented.
As stated in the docs:
The client must close the response body when finished with it
and
The default HTTP client's Transport may not
reuse HTTP/1.x "keep-alive" TCP connections if the Body is
not read to completion and closed.
If the server wants to send more data than fits in the initial read buffers, it is going to be blocked sending the response. This means that if the transport were to attempt to send a new request over that connection the server may not be able to handle it because it never completed the first request. This will usually result in a client error of connection reset by peer and a server error of write: broken pipe.
If you want to make an attempt to reuse the connection, but limit the amount read, use an io.LimitedReader and/or check the ContentLength value. This way you can discard the connection when it's faster to handle the errors and bring up a new connection than to read an unbounded amount of data. See Limiting amount of data read in the response to a HTTP GET request.
My program downloads a file from one server and then returns it to the user. Here is snippet of it:
// Make get request to target server
resp, httpErr := http.Get(url.String())
// Return error if http request is failed
if httpErr != nil {
fmt.Fprintln(w,"Http Request Failed :" ,httpErr.Error())
return
}
//Setting up headers
w.Header().Set("Content-Disposition", "attachment; filename="+vid.Title+"."+format.Extension)
w.Header().Set("Content-Type", r.Header.Get("Content-Type"))
w.Header().Set("Content-Length", strconv.Itoa(int(resp.ContentLength)))
// Copy instream of resp.Body to writer
io.Copy(w, resp.Body)
When the user stops the download or closes the connection I want to close the GET connection as well.
But it does not get closed as I found via usage graph. How can I close the connection of the user?
You should close the Body of the request in any case:
resp, httpErr := http.Get(url.String())
if httpErr != nil {
// handle error
return
}
// if it's no error then defer the call for closing body
defer resp.Body.Close()
More should not be necessary to do. When the client closes the connection io.Copy returns with an error. io.Copy returns the number of bytes written and an error. You could check that, if you want to know if the copy succeded.
written, err := io.Copy(w, resp.Body)
if err != nil {
// Copy did not succeed
}
I have an application on Bluemix written on Golang. I send to it a large chunked HTTP request and I see that my application gets a notice that it got this request only when the request fully reaches the destination instead of starting after the header appeared (in the beginning without waiting to the body).
The same code on regular server works fine without any delays, the problem seems to be only when it runs on Bluemix. I did several experiments, like listening to tcp instead of http (but still sending http requests), and again the first notice that there is some networking only when the request fully arrived. I also tried to stream the data very slowly, 1 byte in 100ms, and send small requests, and the result is the same.
I assume there is some buffering before redirecting the request to the dynamic port the App has, or maybe in some other place. Any help why does it happen and how to avoid it, will be very appreciated. Thank you.
UPD: Server code in Go that listens to TCP:
func main() {
var port string
if port = os.Getenv("PORT"); len(port) == 0 { port = DEFAULT_PORT }
l, err := net.Listen("tcp", ":"+port)
if err != nil {
fmt.Println("Error listening:", err.Error())
os.Exit(1)
}
// Close the listener when the application closes.
defer l.Close()
for {
// Listen for an incoming connection.
conn, err := l.Accept()
if err != nil {
fmt.Println("Error accepting: ", err.Error())
os.Exit(1)
}
// Handle connections in a new goroutine.
go handleTCPRequest(conn)
}
}
func handleTCPRequest(conn net.Conn) {
fmt.Println("Got something!!!", conn)
// Make a buffer to hold incoming data.
buf := make([]byte, 1024)
// Read the incoming connection into the buffer.
reqLen, err := conn.Read(buf)
if err != nil {
fmt.Println("Error reading:", err.Error(), conn)
} else {
fmt.Println("Read: ", reqLen, buf, conn)
}
// Close the connection when you're done with it.
conn.Close()
}
The client just sends a big http request or a throttled one.
I would expect to get print "Got Something" really fast, after first packages arrive, but this is not what I see. For example for a throttling client I will get all the package together and print "Got something" only after the full package arrives.
Code of throttling client in Go:
reader, writer := io.Pipe()
request, _ := http.NewRequest("PUT", URL, reader)
request.Header = http.Header{}
request.Header["Connection"]=[]string {"keep-alive"}
request.Header["X-Content-Type-Options"]=[]string {"nosniff"}
request.Header["Accept-Encoding"]=[]string{"gzip, deflate"}
request.Header["Accept"]=[]string{"*/*"}
request.Header["Transfer-Encoding"]=[]string {"chunked"}
request.TransferEncoding=[]string {"chunked"}
//puts one byte in 100ms asynchronously
go func(w *io.PipeWriter){
i:=0
for true {
i+=1
w.Write([]byte{0})
time.Sleep(100 * time.Millisecond)
fmt.Print("-")
if i > 1000 {
w.CloseWithError(io.EOF)
break
}
}
}(writer)
http.DefaultClient.Do(request)
I need to make a request to a server that return different responses at different times. I mean, the server generate different responses and these responses take different execution time, so server return the responses as soon as they are available.
And I want print in the screen (by the moment, I'd settle with that) these responses as soon as the server returns me.
All what I could do until now is print the responses but only when the server returns all the responses. So if the first response take 1sec, and the last response take 10sec, my code needs to wait 10sec to print all the messages.
EDIT: to add code I have:
//Config is gotten from yml file
RestConfig = Config["rest"].(map[string]interface{})
ServerConfig = Config["server"].(map[string]interface{})
RequestUrl := ServerConfig["url"]
RequestReader := bytes.NewReader(body)
Request, _ := http.NewRequest("POST", RequestUrl.(string), RequestReader)
//AppendHeaders append the needing headers to the request
client.AppendHeaders(Request, RestConfig["headers"])
//the type of client.HttpClient is *http.Client
Response, _ := client.HttpClient.Do(Request)
//And to print in the screen
defer Response.Body.Close()
fmt.Println( "-> Receiving response:\n---\n" )
fmt.Println( Response , "\n---\n-> Response body:\n---\n")
body_resp, _ := ioutil.ReadAll(Response.Body)
fmt.Println( string(body_resp) )
fmt.Println( "\n--\n")
Any way to do it??
Thank you very much.
Finally my code , is like this:
package main
import (
"fmt"
"log"
"bytes"
"strings"
"bufio"
"net/http"
)
func main() {
var body = "The body"
RequestReader := bytes.NewReader([]byte(body))
req, err := http.NewRequest("POST", "the_url", RequestReader)
if err != nil {
log.Fatal(err)
}
req.Header.Add("Accept", "application/xml")
req.Header.Add("Content-Type", "application/xml")
req.Header.Add("AG-Authorization", "key")
req.Header.Add("AG-Forwarded-Hosts", "*")
resp, err := (&http.Client{}).Do(req)
if err != nil {
log.Fatal(err)
}
reader := bufio.NewReader(resp.Body)
message := ""
for {
line, err := reader.ReadBytes('\n')
if err != nil {
log.Fatal(err)
}
message = message + string(line)
if strings.Contains(message, "<!-- End mark for each message -->"){
fmt.Println(message)
message = ""
}
}
}
Thank everyone.
The context package is what you are looking for.
The context package is responsible for signal cancelation and operation deadlines for processes and server requests. This has two public methods: WithCancel and WithTimeout. The Context associated with an incoming request is typically canceled when the request handler returns.
For your specific case you can use the WithTimeout method for setting a deadline on requests to backend servers.
// WithTimeout returns a copy of parent whose Done channel is closed as soon as
// parent.Done is closed, cancel is called, or timeout elapses. The new
// Context's Deadline is the sooner of now+timeout and the parent's deadline, if
// any. If the timer is still running, the cancel function releases its
// resources.
func WithTimeout(parent Context, timeout time.Duration) (Context, CancelFunc)
And here is a snippet taken from https://blog.golang.org/context/server/server.go
timeout, err := time.ParseDuration(req.FormValue("timeout")) // set a time limit in your post
if err == nil {
// The request has a timeout, so create a context that is
// canceled automatically when the timeout expires.
ctx, cancel = context.WithTimeout(context.Background(), timeout)
} else {
ctx, cancel = context.WithCancel(context.Background())
}
defer cancel() // Cancel ctx as soon as handleSearch returns.
For further reading take a look at this article:
https://blog.golang.org/context
I did some experiment on Golang net package.
When I use a client to connect a server (without close connections), I found that server had error like this:
"too many open files"
It makes sense because the file open limit set in my os.
However, I found that client never returned errors when connecting to server, which is weird since server had already rejected some of connection requests.
// Server code
package main
import (
"log"
"net"
)
func main() {
listener, err := net.Listen("tcp", "localhost:7070")
if err != nil {
log.Fatalln(err)
}
for {
conn, err := listener.Accept()
if err != nil {
log.Println(err, conn)
continue
}
}
}
// Client code
package main
import (
"log"
"net"
)
func main() {
cnt := 0
for {
conn, err := net.Dial("tcp", "localhost:7070")
if err != nil {
log.Println("!", conn.RemoteAddr())
continue
}
cnt++
n, err := conn.Write([]byte("hello"))
if err != nil {
log.Fatalln(err)
}
log.Println(cnt, n)
}
}
Can anyone tell me what happens here?
thx.
The TCP handshake is asynchronously performed on the server by the kernel TCP/IP stack before the server application accepts it, so even if your server code fails to accept the connection, it will be connected at the TCP level. That's why the net.Dial() succeeds.
When your server code accept of the connection fails, the TCP/IP stack will close the connection though.
net.Write() just passes data onto the local TCP/IP stack, which will asynchronously send the data. If this is done before the server code has had a chance to (fail to) accept the connection, there's no error as the tcp connection is still operational. If your client perform a 2. write, or perhaps waits a bit, net.Write() should fail too.