I have a TCP server up and running listetning to a port and a go routine for handeling the connections. I wonder if it's possible to have a go routine running for every connection keeping them alive with net.SetKeepAlive(true). Also with error handling so that if the connection times out it will execute cleanup functions like removing the connection from a list?
Handle routine:
func handleConnection(conn net.Conn, rec chan string) {
var item QueItem
buf := make([]byte, bufSize)
l, err := conn.Read(buf)
if err != nil || l < 0 {
fmt.Println("Error reading from conn: ", conn)
fmt.Println("Error reading: ", err)
}
err = json.Unmarshal(buf[:l], &item)
if err != nil {
fmt.Println("Error converting to JSON", jErr)
}
fmt.Printf("Received : %+v\n", item)
fmt.Println("recived from:", conn.RemoteAddr())
rec <- item.IP
}
TCPserver:
for {
conn, err := ln.Accept()
if err != nil {
fmt.Println("No accept", err)
log.Println("Unable to accept connection", err)
}
go handleConnection(conn, recived)
}
To keep a check on the established TCP connections, one can do keep-alive mechanism in two ways.
At application level:
In idle conditions both client and server agree on a protocol to send pre-determined packets to each other. Lack of a message from peer with in certain time can signal a problem in the connection.
At TCP layer:
Enable the TCP stack to check the connection status. The TCP layer at which this mechanism is enabled will send keep-alive messages at regular determined interval. It would expect that the peer TCP stack send keep-alive-ack. Absence of ACK after needed re-transmission signals connection problem and the application would be duly notified.
I think net.SetKeepAlive(true) is a go way of informing the TCP to do keep-alive. You need not do any thing special constructs in go routine.
TCP keep-alive works well. Application need not be burdened to check for connection status
Related
Target
I want to send to data to my server from a read closer. (In the example a NopCloser, later it will be the Stdout of an exec.Command)
Problem
The Request never ends. Even if I manually close the cmdOut the program nevers ends. Concrete: It never reaches the "Request Done" line and there by never calls wg.Done()
Gotchas
All the data is sent correctly to the server (even with the exec.Command Stdout). But the http.DefaultClient.Do seems to be still listening on the ReadCloser after it is empty (and closed in the main routine)
Code
cmdOut := ioutil.NopCloser(bytes.NewBuffer([]byte("Hallo DU")))
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
// localhost:1234 is a netcat server: "nc -l -p 1234"
req, _ := http.NewRequest("PUT", "http://localhost:1234", cmdOut)
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Println(err)
return
}
// Never reaches this line
log.Println("Request Done")
}()
cmdOut.Close()
log.Println("Wait for go routine")
wg.Wait()
log.Println("DONE")
The problem isn't the sending of your request--it's the receiving of the response.
You're sending your request to netcat, which simply discards the request, and does nothing else. This leaves the HTTP client library waiting for an HTTP response, which never comes.
The solution is to talk to an actual HTTP server.
I am making an http post request to a server using golang. Suppose the server is currently turned off (Means the machine on which the server runs is turned off) then the request is stuck at the IP layer. So my program execution is unable to proceed further. It is unable to proceed to the Application layer. So is there any way in golang to stop this.
I am using the following code.
req, err := http.NewRequest("POST", url, bytes.NewReader(b))
if err != nil {
return errors.Wrap(err, "new request error")
}
resp, err := http.DefaultClient.Do(req)
if err != nil {
return errors.Wrap(err, "http request error")
}
defer resp.Body.Close()
Is there anything that can be added to this, to terminate the request if it doesn't find anything from the IP layer.
The default http Client has no timeout. You can create an explicit http.Client yourself and set the timeout:
var cl = &http.Client{
Timeout: time.Second * 10,
}
resp, err := cl.Do(req)
if err != nil {
// err will be set on timeout
return errors.Wrap(err, "http request error")
}
defer resp.Body.Close()
If the server does not answer any more in the middle of a request, you can handle the timeout.
Use a non-default http.Transport with its DialContext field set to a function which uses a custom context with the properly configured timeout/deadline. Another option is to use a custom net.Dialer.
Something like this:
cli := http.Client{
Transport: &http.Transport{
DialContext: func (ctx context.Context, network, address string) (net.Conn, error) {
dialer := net.Dialer{
Timeout: 3 * time.Second,
}
return dialer.DialContext(ctx, network, address)
},
},
}
req, err := http.NewRequest(...)
resp, err := cli.Do(req)
Note that as per the net.Dialer's docs the context passed to its DialContext might trump the timeout set on the dialer itself—this is
exactly what we need: the dialer's Timeout field controls exactly the
"dialing" (TCP connection establishment) while you might also arm your
HTTP request with a context (using http.Request.WithContext) controlling
the timeout of the whole request, and also be able to cancel it at any time (including the dialing step).
Playground example.
The Transport #kostix refers to is definitely what you're looking for in this case. Transports as well as Clients are safe for concurrent use as well. But please read about the Transport (and I also advise reading about the Client as well) as there are a number of different ways to affect how you handle idle connections, not just the pre-mentioned DialContext.
As you may want to set your ResponseHeaderTimeout:
ResponseHeaderTimeout, if non-zero, specifies the amount of
time to wait for a server's response headers after fully
writing the request (including its body, if any). This
time does not include the time to read the response body.
Or, if you are using a secure connection, you may want to set your TLSHandshakeTimeout:
TLSHandshakeTimeout specifies the maximum amount of time waiting to
wait for a TLS handshake. Zero means no timeout.
For readability and maintainability, I suggest also maybe creating a function to build your Client, something along the lines of:
func buildClient(timeout time.Duration) *http.Client {
tr := &http.Transport{
IdleConnTimeout: timeout,
ResponseHeaderTimeout: timeout,
TLSHandshakeTimeout: timeout,
}
client := &http.Client{
Transport: tr,
Timeout: timeout,
}
return client
}
My client receives every message from the server just fine after connecting, but the server's output is always this:
Client #1 connected.
first packet error: timeout
Looping client #1
message error: timeout
Looping client #1
message error: timeout
Looping client #1
message error: timeout
...etc...
I've messed around with some values on each side. It usually does nothing, but sometimes when messing with the server's timeout values it makes it so the server just hangs forever on client:send() or client:receive(). I can never get it so that both of them can exchange messages with eachother. Why?
My goal is to have a client and server that spend minimal time (<2-3ms) every loop so that I can spend a lot of time doing other things (this was part of a bigger program but I isolated it for easier testing of the issue) on both the client or server. I can't afford to have long, blocking calls so I set the timeout really low.
Server source:
socket = require("socket");
server = assert(socket.bind("*", 534));
ip, port = server:getsockname();
server:settimeout(0.03);
nClients = 0;
clients = {};
print("Server started.");
while true do
local client, err = server:accept();
if (not err) then
nClients = nClients + 1;
print("Client #"..nClients.." connected.");
clients[nClients] = client;
clients[nClients]:settimeout(0.03);
clients[nClients]:send("test");
local msg, err = clients[nClients]:receive();
if (not err) then
print("we got it");
else
print("first packet error: "..err);
end
end
for i=1,nClients do
if (clients[i] ~= nil) then
print("Looping client #"..i);
local msg, err = clients[i]:receive();
if (not err) then
print("message received: "..msg);
else
print("message error: "..err);
end
clients[nClients]:send("another message");
end
end
socket.sleep(0.01);
end
Client source:
socket = require("socket");
tcp = assert(socket.tcp());
ip, port = "127.0.0.1", 534;
print("Client started.");
tcp:settimeout(0.03);
tcp:connect(ip, port);
while true do
local a, b, msg = tcp:receive();
if (msg and #msg > 1) then
print("message received: "..msg);
end
tcp:send("message");
socket.sleep(0.01);
end
I have a example web server below where if you call curl localhost:3000 -v then ^C (cancel) it immediately (before 1 second), it will report write tcp 127.0.0.1:3000->127.0.0.1:XXXXX: write: broken pipe.
package main
import (
"fmt"
"net/http"
"time"
)
func main() {
log.Fatal(http.ListenAndServe(":3000", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
time.Sleep(1 * time.Second)
// Why 8061 bytes? Because the response header on my computer
// is 132 bytes, adding up the entire response to 8193 (1 byte
// over 8kb)
if _, err := w.Write(make([]byte, 8061)); err != nil {
fmt.Println(err)
return
}
})))
}
Based on my debugging, I have been able to conclude that this will only happen if the entire response is writing more than 8192 bytes (or 8kb). If my entire response write less than 8192, the broken pipe error is not returned.
My question is where is this 8192 bytes (or 8kb) buffer limit set? Is this a limit in Golang's HTTP write buffer? Is this related to the response being chunked? Is this only related to the curl client or the browser client? How can I change this limit so I can have a bigger buffer written before the connection is closed (for debugging purposes)?
Thanks!
In net/http/server.go the output buffer is set to 4<<10, i.e. 4KB.
The reason you see the error at 8KB, is that it takes at least 2 writes to a socket to detect a closed remote connection. The first write succeeds, but the remote host sends an RST packet. The second write will be to a closed socket, which is what returns the broken pipe error.
Depending on the socket write buffer, and the connection latency, it's possible that even more writes could succeed before the first RST packet is registered.
It is broken pipe, but u should use ioutil.ReadAll for small data size of response or io.copy for large data size of response.
For ioutil.ReadAll
defer response.Body.Close()
body, err := ioutil.ReadAll(response.Body)
if err != nil {
logger.Errorf(ctx, "err is %+v", err)
return nil, err
}
For io.copy
// 10MB
var wb = make([]byte, 0, 10485760)
buf := bytes.NewBuffer(wb)
written, err := io.Copy(buf, response.Body)
body := wb[:written]
I'm trying to write unit tests for some server-side code, but I'm having trouble being deterministic with my shutdown test cases. It seems a loopback TCP connection isn't correctly handling a clean shutdown. I've reprod this in a sample app which does the following in lockstep:
Create a client & server connection.
Verify connectivity by sending a message successfully from client to server.
Use channels to tell the server to call conn.Close() and wait until that call has completed.
(Try to) verify the connection is cleanly broken by calling Write on the client connection again.
Step 4 succeeds without error. I've tried using a json.Encoder and a bare call to TCPConn.Write. I checked the traffic with WireShark. The server sent a FIN packet, but the client never does (even with a 1s sleep) The server even sent a RST packet in response to (4) and the client conn.Write still returned nil for its error.
This seems totally bonkers. Am I missing something here? Currently running Go v1.2.1/Darwin
Edit: Obligatory repro
package main
import (
"bufio"
"fmt"
"net"
)
var (
loopback = make(chan string)
shouldClose = make(chan struct{})
didClose = make(chan struct{})
)
func serve(listener *net.TCPListener) {
conn, err := listener.Accept()
if err != nil {
panic(err)
}
s := bufio.NewScanner(conn)
if !s.Scan() {
panic(fmt.Sprint("Failed to scan for line: ", s.Err()))
}
loopback <- s.Text() + "\n"
<-shouldClose
conn.Close()
close(didClose)
if s.Scan() {
panic("Expected error reading from a socket closed on this side")
}
}
func main() {
listener, err := net.ListenTCP("tcp", &net.TCPAddr{})
if err != nil {
panic(err)
}
go serve(listener)
conn, err := net.Dial("tcp", listener.Addr().String())
if err != nil {
panic(fmt.Sprint("Dialer got error ", err))
}
oracle := "Mic check\n"
if _, err = conn.Write([]byte(oracle)); err != nil {
panic(fmt.Sprint("Dialer failed to write oracle: ", err))
}
test := <-loopback
if test != oracle {
panic("Server did not receive the value sent by the client")
}
close(shouldClose)
<-didClose
// For giggles, I can also add a <-time.After(500 * time.Millisecond)
if _, err = conn.Write([]byte("This should fail after active disconnect")); err == nil {
panic("Sender 'successfully' wrote to a closed socket")
}
}
This is how an active close of a TCP connection works. When the client detects that the server has closed, it is then expected to close its half of the connection.
In your case, instead of closing the client you're sending more data. This causes the server to send an RST packet to force the connection closed since the message received isn't valid.
If you're still unsure, here's and equivalent python client+server which displays the same behavior. (I find using python helpful, since it closely follows the underlying BSD socket API, without using C)
Server:
import socket, time
server = socket.socket()
server.bind(("127.0.0.1", 9999))
server.listen(1)
sock, addr = server.accept()
msg = sock.recv(1024)
print msg
print "closing"
sock.close()
time.sleep(3)
print "done"
Client:
import socket, time
sock = socket.socket()
sock.connect(("127.0.0.1", 9999))
sock.send("test\n")
time.sleep(1)
print "sending again!"
sock.send("no error here")
time.sleep(1)
print "sending one last time"
sock.send("broken pipe this time")
To properly detect a remote close on the connection, you should do Read(), and look for an io.EOF error in return.
// we technically need to try and read at least one byte,
// or we will get an EOF even if the connection isn't closed.
buff := make([]byte, 1)
if _, err := conn.Read(buff); err != io.EOF {
panic("connection not closed")
}