TCP connection pool - tcp

i'm new to socket and trying to create a connection pooling over tcp socket. my implementation send 32bit length then binary message for each call. But i'm having problem with sometimes the reader receiving previous response from server (could happened when client close and re-establish socket on send error). how do i flush socket (remaining bytes from previous call) before a new request. any suggestion?
Edit: i learned that tcp always stream 0s, what if i send byte(1) before message so i can have a flush function to check if socket not empty before a new call.

Your post actually asks several questions:
How to manage a connection pool?
How to handle communication over the sockets?
These are really two different things. A connection pool is just a way to manage a set of connections. A simple way to implement this is with a class such as:
package netpool
import (
"net"
)
const MaxConnections = 3
type Error string
func (e Error) Error() string {
return string(e)
}
var ErrMaxConn = Error("Maximum connections reached")
type Netpool struct {
name string
conns int
free []net.Conn
}
func NewNetpool(name string) *Netpool {
return &Netpool{
name: name,
}
}
func (n *Netpool) Open() (conn net.Conn, err error) {
if n.conns >= MaxConnections && len(n.free) == 0 {
return nil, ErrMaxConn
}
if len(n.free) > 0 {
// return the first free connection in the pool
conn = n.free[0]
n.free = n.free[1:]
} else {
addr, err := net.ResolveTCPAddr("tcp", n.name)
if err != nil {
return nil, err
}
conn, err = net.DialTCP("tcp", nil, addr)
if err != nil {
return nil, err
}
n.conns += 1
}
return conn, err
}
func (n *Netpool) Close(conn net.Conn) error {
n.free = append(n.free, conn)
return nil
}
I have created a stand-alone class here. It would typically be implemented as part of a higher-level class such as MyHTTPHost, or MyDatabase.
In this simple implementation, connections that are returned via netpool.Open() are not tracked. It's possible to leak connections by calling Open(), then closing the connections outside of netpool.Close(). It's possible to track them if you want to hold an active and inactive pool, for example, which would solve this problem.
A couple of other things you might want to add to a pooling implementation:
Threading protection (using sync.Mutex, for example)
Closing of connections in the freepool after some length of inactivity
Error checking to be sure that closed connections are still valid
Once you have a connection, you can call Read and Write on it normally. To flush all oustanding data on the socket, you can simply use the ioutil.ReadAll() helper function. By default, this will block indefinitely if there is no data available. To avoid that, add a read timeout using:
conn.SetReadDeadline(time.Now().Add(500 * time.Millisecond))
_, err = ioutil.ReadAll(conn)
neterr, ok := err.(net.Error)
if ok && neterr.Timeout() {
err = nil // timeout isn't an error in this case
}
if err != nil {
// handle the error case.
}
This will read all the data from the given connection if any is pending, or will return after 500ms with an I/O Timeout error if no data was pending.
The type assertion is required because ioutil.ReadAll() returns an Error interface, rather than a net.Error interface, and we need the latter to be able to easily find out if the call returned due to a timeout.

Related

check whether http connection is hijacked in go server

I am writing an HTTP server in Go, which uses the following pattern to handle API output:
func handler(w http.ResponsWriter, r *http.Request) {
defer reply(w, r, L)() //L is a Logger
//do things...
}
func reply(w http.ResponseWriter, r *http.Request, log Logger) func() {
cid := []byte{0, 0, 0, 0}
if log != nil {
rand.Read(cid)
log.Debug("[%x] %s %s", cid, r.Method, r.URL.String())
}
entry := time.Now()
return func() {
if log != nil {
defer log.Debug("[%x] elapsed %d millis", cid, time.Since(entry).Milliseconds())
}
_, err := w.Write(nil)
if err == http.ErrHijacked {
return //API is a WEBSOCKET entry point, do nothing
}
//handle common output logic for normal HTTP APIs...
}
}
The reason I do this, is that I found this comment in the standard library:
// ErrHijacked is returned by ResponseWriter.Write calls when
// the underlying connection has been hijacked using the
// Hijacker interface. A zero-byte write on a hijacked
// connection will return ErrHijacked without any other side
// effects.
ErrHijacked = errors.New("http: connection has been hijacked")
However following the Write() method, I got this comment:
// Write writes the data to the connection as part of an HTTP reply.
//
// If WriteHeader has not yet been called, Write calls
// WriteHeader(http.StatusOK) before writing the data. If the Header
// does not contain a Content-Type line, Write adds a Content-Type set
// to the result of passing the initial 512 bytes of written data to
// ...
Write([]byte) (int, error)
My questions are:
Is it OK to use my code to safely detect if a HTTP connection is hijacked? I only want to check the connection is hijacked or not, but do NOT want it to add headers for me!
Since the ResponseWriter is an interface, I cannot click through the source code to find out how the standard library implements that method. In general, how can I drill down to the standard library (or any open source code) to find out the implementation of an interface?
Thanks to Cerise, I found the source code of the standard response.Writer:
func (w *response) write(lenData int, dataB []byte, dataS string) (n int, err error) {
if w.conn.hijacked() {
if lenData > 0 {
caller := relevantCaller()
w.conn.server.logf("http: response.Write on hijacked connection from %s (%s:%d)", caller.Function, path.Base(caller.File), caller.Line)
}
return 0, ErrHijacked
}
... ....
So, as said in the document, there is NO side effect.

Go: start HTTP server asynchronously but return error if startup failed

I would like to start an HTTP server in a goroutine but have a its parent return an error if the startup failed. Is there a good way for doing this? I tried passing the error in a channel. But this makes the parent block even if the server started successfully (unless I introduce an arbitrary timeout for listening on that channel).
func Start() error {
go func() {
srv = &http.Server{...}
if err := srv.ListenAndServe(); err != http.ErrServerClosed {
// return err from Start()
}
}
// return nil if server started successfully
}
ListenAndServe is a helper function that opens a listening socket and then serves connections on that socket.
Startup errors in ListenAndServer are in the listen part of the function. Create the listener on the main goroutine and handle errors there. Run the server in a goroutine:
l, err := net.Listen("tcp", ":8080") // adjust arguments as needed.
if err != nil {
// Handle error.
}
srv = &http.Server{...}
go func() {
if err := srv.Serve(l); err != nil {
// Handle non-startup errors here.
}()
If you want to stick to that pattern, either send a nil error through the channel to signal a successful initialization or close the channel. Both actions can be picked up by the main thread and it will stop being blocked.

Is it necessary to read and close the response body to reuse the TCP connection on retry?

I am implementing a retry using http.RoundTripper in Go. Here is an implementation example.
type retryableRoundTripper struct {
tr http.RoundTripper
maxRetryCount int
}
func (t *retryableRoundTripper) RoundTrip(req *http.Request) (resp *http.Response, err error) {
for count := 0; count < t.maxRetryCount; count++ {
log.Printf("retryableRoundTripper retry: %d\n", count+1)
resp, err = t.tr.RoundTrip(req)
if err != nil || resp.StatusCode != http.StatusTooManyRequests {
return resp, err
}
}
return resp, err
}
Questions
Is it necessary to read and close the response body to reuse the TCP connection on retry?
func (t *retryableRoundTripper) RoundTrip(req *http.Request) (resp *http.Response, err error) {
for count := 0; count < t.maxRetryCount; count++ {
log.Printf("retryableRoundTripper retry: %d\n", count+1)
resp, err = t.tr.RoundTrip(req)
if err != nil || resp.StatusCode != http.StatusTooManyRequests {
return resp, err
}
}
// add
io.Copy(ioutil.Discard, resp.Body)
resp.Body.Close()
return resp, err
}
As a side note, I've written a test and have confirmed that retries work as expected. (In Go Playground, it times out, but it works locally.)
https://play.golang.org/p/08YWV0kjaKr
Of course you need to read the connection to ensure that it can be reused, and Closing the connection is required as documented.
As stated in the docs:
The client must close the response body when finished with it
and
The default HTTP client's Transport may not
reuse HTTP/1.x "keep-alive" TCP connections if the Body is
not read to completion and closed.
If the server wants to send more data than fits in the initial read buffers, it is going to be blocked sending the response. This means that if the transport were to attempt to send a new request over that connection the server may not be able to handle it because it never completed the first request. This will usually result in a client error of connection reset by peer and a server error of write: broken pipe.
If you want to make an attempt to reuse the connection, but limit the amount read, use an io.LimitedReader and/or check the ContentLength value. This way you can discard the connection when it's faster to handle the errors and bring up a new connection than to read an unbounded amount of data. See Limiting amount of data read in the response to a HTTP GET request.

Buffering of http requests to Bluemix App

I have an application on Bluemix written on Golang. I send to it a large chunked HTTP request and I see that my application gets a notice that it got this request only when the request fully reaches the destination instead of starting after the header appeared (in the beginning without waiting to the body).
The same code on regular server works fine without any delays, the problem seems to be only when it runs on Bluemix. I did several experiments, like listening to tcp instead of http (but still sending http requests), and again the first notice that there is some networking only when the request fully arrived. I also tried to stream the data very slowly, 1 byte in 100ms, and send small requests, and the result is the same.
I assume there is some buffering before redirecting the request to the dynamic port the App has, or maybe in some other place. Any help why does it happen and how to avoid it, will be very appreciated. Thank you.
UPD: Server code in Go that listens to TCP:
func main() {
var port string
if port = os.Getenv("PORT"); len(port) == 0 { port = DEFAULT_PORT }
l, err := net.Listen("tcp", ":"+port)
if err != nil {
fmt.Println("Error listening:", err.Error())
os.Exit(1)
}
// Close the listener when the application closes.
defer l.Close()
for {
// Listen for an incoming connection.
conn, err := l.Accept()
if err != nil {
fmt.Println("Error accepting: ", err.Error())
os.Exit(1)
}
// Handle connections in a new goroutine.
go handleTCPRequest(conn)
}
}
func handleTCPRequest(conn net.Conn) {
fmt.Println("Got something!!!", conn)
// Make a buffer to hold incoming data.
buf := make([]byte, 1024)
// Read the incoming connection into the buffer.
reqLen, err := conn.Read(buf)
if err != nil {
fmt.Println("Error reading:", err.Error(), conn)
} else {
fmt.Println("Read: ", reqLen, buf, conn)
}
// Close the connection when you're done with it.
conn.Close()
}
The client just sends a big http request or a throttled one.
I would expect to get print "Got Something" really fast, after first packages arrive, but this is not what I see. For example for a throttling client I will get all the package together and print "Got something" only after the full package arrives.
Code of throttling client in Go:
reader, writer := io.Pipe()
request, _ := http.NewRequest("PUT", URL, reader)
request.Header = http.Header{}
request.Header["Connection"]=[]string {"keep-alive"}
request.Header["X-Content-Type-Options"]=[]string {"nosniff"}
request.Header["Accept-Encoding"]=[]string{"gzip, deflate"}
request.Header["Accept"]=[]string{"*/*"}
request.Header["Transfer-Encoding"]=[]string {"chunked"}
request.TransferEncoding=[]string {"chunked"}
//puts one byte in 100ms asynchronously
go func(w *io.PipeWriter){
i:=0
for true {
i+=1
w.Write([]byte{0})
time.Sleep(100 * time.Millisecond)
fmt.Print("-")
if i > 1000 {
w.CloseWithError(io.EOF)
break
}
}
}(writer)
http.DefaultClient.Do(request)

rejected tcp connection does not return error

I did some experiment on Golang net package.
When I use a client to connect a server (without close connections), I found that server had error like this:
"too many open files"
It makes sense because the file open limit set in my os.
However, I found that client never returned errors when connecting to server, which is weird since server had already rejected some of connection requests.
// Server code
package main
import (
"log"
"net"
)
func main() {
listener, err := net.Listen("tcp", "localhost:7070")
if err != nil {
log.Fatalln(err)
}
for {
conn, err := listener.Accept()
if err != nil {
log.Println(err, conn)
continue
}
}
}
// Client code
package main
import (
"log"
"net"
)
func main() {
cnt := 0
for {
conn, err := net.Dial("tcp", "localhost:7070")
if err != nil {
log.Println("!", conn.RemoteAddr())
continue
}
cnt++
n, err := conn.Write([]byte("hello"))
if err != nil {
log.Fatalln(err)
}
log.Println(cnt, n)
}
}
Can anyone tell me what happens here?
thx.
The TCP handshake is asynchronously performed on the server by the kernel TCP/IP stack before the server application accepts it, so even if your server code fails to accept the connection, it will be connected at the TCP level. That's why the net.Dial() succeeds.
When your server code accept of the connection fails, the TCP/IP stack will close the connection though.
net.Write() just passes data onto the local TCP/IP stack, which will asynchronously send the data. If this is done before the server code has had a chance to (fail to) accept the connection, there's no error as the tcp connection is still operational. If your client perform a 2. write, or perhaps waits a bit, net.Write() should fail too.

Resources