EOF error when reading from port with bufio - networking

I am trying to learn the net package. I am listening for local connections on a port, and sending data to that port using echo -n "Server test.\n" | nc localhost 5000.
I always get an EOF error when reading the data, however. I checked the docs and this is only supposed to happen when there is no more input available, however I don't understand why this is happening here.
This is my code:
package main
import (
"bufio"
"fmt"
"net"
"os"
)
// Connection details
type connection struct {
host string
port string
network string
}
// Initialise a Listener on a given port
// Pass handling into seperate goroutine
func main() {
localConn := connection{
host: "", // Localhost
port: "5000",
network: "tcp",
}
listener, err := net.Listen(localConn.network, localConn.host+":"+localConn.port)
checkError("Error listening: ", err)
conn, err := listener.Accept()
for {
checkError("Error accepting: ", err)
go handleRequest(conn)
}
}
// Delegate handling of requests
func handleRequest(conn net.Conn) {
// Read message up until newline delimiter
message, err := bufio.NewReader(conn).ReadString('\n')
checkError("Error reading: ", err)
fmt.Println("Message recieved: ", string(message))
conn.Write([]byte("Recieved message: " + string(message) + "\n"))
conn.Close()
}
// Check if an error exists
// If so, print and exit program. (Not super robust!)
func checkError(message string, err error) {
if err != nil {
fmt.Println(message, err.Error())
os.Exit(1)
}
}

You appear to be using echo incorrectly. The flag -e interprets the two characters \n as a newline (check here).
Use the following command to send data to server:
echo -e "Server test.\n" | nc localhost 5000
Other than that you should also fix for loop:
for {
conn, err := listener.Accept()
checkError("Error accepting: ", err)
go handleRequest(conn)
}
In your original code you are only one connection is ever accepted. After that the for loop simply starts more goroutines that try to read on a closed connection (error or not, first handleRequest call closes the connection).

One problem is on these lines:
conn, err := listener.Accept()
for {
checkError("Error accepting: ", err)
go handleRequest(conn)
}
The application starts goroutines in a loop to read the single connection. The first goroutine to read the connection is successful. The subsequent goroutines report an error.
Change the code to:
for {
conn, err := listener.Accept()
checkError("Error accepting: ", err)
go handleRequest(conn)
}
The client is not sending a newline as expected by the server. Use this command to send the message:
echo "Server test." | nc localhost 5000

Related

Cannot use socks5 proxy in golang - read: connection reset by peer

I have a dockerfile which runs tor -
FROM alpine:edge
RUN apk update && apk add tor
EXPOSE 9050
USER tor
CMD ["/usr/bin/tor"]
and ran it using command - docker run --name tor -p 11000:9050 tor
and checked connection using - telnet 127.0.0.1 11000 and it showed connected
Now I want to use tor as proxy while any request from go program. I tried -
package main
import (
"fmt"
"net/http"
"net/url"
"time"
)
func main() {
proxyUrl, err := url.Parse("socks5://127.0.0.1:11000")
if err != nil {
// TODO handle me
panic(err)
}
cl := http.Client{
Transport: &http.Transport{
Proxy: http.ProxyURL(proxyUrl),
},
Timeout: 18000 * time.Millisecond,
}
resp, err := cl.Get("http://google.com")
if err != nil {
// TODO handle me
panic(err)
}
// TODO work with the response
fmt.Println(resp)
}
But running this program threw error -
panic: Get http://google.com: socks connect tcp 127.0.0.1:11000->google.com:80: read tcp 127.0.0.1:59630->127.0.0.1:11000: read: connection reset by peer
goroutine 1 [running]: <stacktrace>
exit status 2
I tried other approaches also, notably mentioned here and here but kept getting same error - read: connection reset by peer
Please help which part is incorrect here.
Thanks.
--------------------another approach that I tried ----------------
As mentioned in one of the links, I tried this code also -
const (
PROXY_ADDR = "127.0.0.1:11000"
URL = "http://facebookcorewwwi.onion"
)
func main() {
// create a socks5 dialer
dialer, err := proxy.SOCKS5("tcp", PROXY_ADDR, nil, proxy.Direct)
if err != nil {
fmt.Fprintln(os.Stderr, "can't connect to the proxy:", err)
os.Exit(1)
}
dialContext := func(ctx context.Context, network, address string) (net.Conn, error) {
// do anything with ctx
return dialer.Dial(network, address)
}
// setup a http client
httpTransport := &http.Transport{
DialContext: dialContext,
}
httpClient := &http.Client{Transport: httpTransport}
// create a request
req, err := http.NewRequest("GET", URL, nil)
if err != nil {
fmt.Fprintln(os.Stderr, "can't create request:", err)
os.Exit(2)
}
resp, err := httpClient.Do(req)
if err != nil {
fmt.Fprintln(os.Stderr, "cannot make get request: ", err)
os.Exit(2)
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
fmt.Fprintln(os.Stderr, "cannot read response body: ", err)
os.Exit(2)
}
fmt.Println("received response -> ", body)
}
but received error -
cannot make get request: Get http://facebookcorewwwi.onion: socks connect tcp 127.0.0.1:11000->facebookcorewwwi.onion:80: read tcp 127.0.0.1:59826->127.0.0.1:11000: read: connection reset by peer
exit status 2
Any help is appreciable.
After making sure tor is working properly on port 9050.
Try the following curl command to ensure tor is working properly.
curl --socks5 localhost:9050 --socks5-hostname localhost:9050 -s https://wtfismyip.com/json
Can you try this
package main
import (
"context"
"fmt"
"io/ioutil"
"net"
"net/http"
"golang.org/x/net/proxy"
)
func main() {
proxyUrl := "127.0.0.1:9050"
dialer, err := proxy.SOCKS5("tcp", proxyUrl, nil, proxy.Direct)
dialContext := func(ctx context.Context, network, address string) (net.Conn, error) {
return dialer.Dial(network, address)
}
transport := &http.Transport{DialContext: dialContext,
DisableKeepAlives: true}
cl := &http.Client{Transport: transport}
resp, err := cl.Get("https://wtfismyip.com/json")
if err != nil {
// TODO handle me
panic(err)
}
body, err := ioutil.ReadAll(resp.Body)
// TODO work with the response
if err != nil {
fmt.Println("body read failed")
}
fmt.Println(string(body))
}
As suggested in the answer/comment above, main problem was socks5 connection to dockerized tor container.
Found the solution here, i just had to modify dockerfile like -
FROM alpine:edge
RUN apk update && apk add tor
RUN cp /etc/tor/torrc.sample /etc/tor/torrc && echo "SocksPort 0.0.0.0:9050" > /etc/tor/torrc
EXPOSE 9050
CMD ["/usr/bin/tor"]
Other code pieces are working fine.

Accepting a Message Before Writing One

I have a server that listens to 2 ports and transfers messages between the 2 ports. The only problem is that each client (or each port) has to submit a message before they can receive the previous message that the other port sent. For example, if client 1 says "hello", client 2 will not receive the message unless he sends a message first.
I want to find a way to make it so that each client will have to receive the previous message before being able to send a new one. (Since Client 1 will not have anything to receive until Client 2 says something, I am planning to send it a string that has placeholder text.)
Can anyone help me with how to go about this? I try putting my listening code and my code for writing a message for each client into their own go routines, but that did not do the job. Any help will be appreciated.
Here is my code:
Server
package main
import (
"fmt"
"log"
"net"
)
var message string = "Client 2 is receiving your message"
func main() {
fmt.Println("The server is listening on Port 3000 and 8080")
//Set up listeners for the ports each client is using
listener, err := net.Listen("tcp", "localhost:3000")
if err != nil {
log.Fatal(err)
}
listener2, err := net.Listen("tcp", "localhost:8080")
if err != nil {
log.Fatal(err)
}
go acceptLoop(listener)
acceptLoop(listener2) // run in the main goroutine
}
func acceptLoop(l net.Listener) {
defer l.Close()
for {
c, err := l.Accept()
if err != nil {
log.Fatal(err)
}
fmt.Println("New connection found!")
go listenConnection(c)
}
}
func listenConnection(conn net.Conn) {
fmt.Println("Yay")
for {
buffer := make([]byte, 1400)
dataSize, err := conn.Read(buffer)
if err != nil {
fmt.Println("Connection has closed")
return
}
//This is the message you received
data := buffer[:dataSize]
fmt.Print("Received message: ", string(data))
// Send the message back
_, err = conn.Write([]byte(message))
if err != nil {
log.Fatalln(err)
}
fmt.Print("Message sent: ", string(data))
message = string(data)
}
}
Client 1
package main
import (
"fmt"
"log"
"net"
"bufio"
"os"
)
func main() {
conn, err := net.Dial("tcp", "localhost:3000")
if err != nil {
log.Fatalln(err)
}
for {
reader := bufio.NewReader(os.Stdin)
fmt.Print("Enter text: ")
text, _ := reader.ReadString('\n')
_, err = conn.Write([]byte(text))
if err != nil {
log.Fatalln(err)
}
for {
buffer := make([]byte, 1400)
dataSize, err := conn.Read(buffer)
if err != nil {
fmt.Println("The connection has closed!")
return
}
data := buffer[:dataSize]
fmt.Println("Received message: ", string(data))
break
}
}
}
My second client is the same as my first except "localhost:3000" is replaced with "localhost:8080".
I will appreciate any help! I'm fairly new to networking and Go, so any tips will be great.
Right now your Server is just an echo. It responds with the message received from the client.
If you want to relay the message between two clients, the first thing you need to figure out is what should happen if 3 clients connect (2 to the first port, 1 to the other)?
If you expect to have just one client connected to each port you could do something like this (pseudo-go-code):
func main() {
// ...
# Channels must be buffered or else clients will deadlock.
relay := make(chan string, 1)
go acceptLoop(listener1, relay)
acceptLoop(listener2, relay)
}
func acceptLoop(l, relay) {
for {
conn := l.Accept()
for {
relay <- conn.Read()
conn.Write(<-relay)
}
}

How can I start the browser AFTER the server started listening?

In Go, how can I start the browser AFTER the server started listening?
Preferably the simplest way possible.
My code so far, super dumbed down to the point:
package main
import (
// Standard library packages
"fmt"
"net/http"
"github.com/skratchdot/open-golang/open"
// Third party packages
"github.com/julienschmidt/httprouter"
)
// go get github.com/toqueteos/webbrowser
func main() {
// Instantiate a new router
r := httprouter.New()
// Add a handler on /test
r.GET("/test", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
// Simply write some test data for now
fmt.Fprint(w, "Welcome!\n")
})
//open.Run("https://google.com/")
// open.Start("https://google.com")
// http://127.0.0.1:3000/test
// Fire up the server
http.ListenAndServe("localhost:3000", r)
fmt.Println("ListenAndServe is blocking")
open.RunWith("http://localhost:3000/test", "firefox")
fmt.Println("Done")
}
Open the listener, start the browser and then enter the server loop:
l, err := net.Listen("tcp", "localhost:3000")
if err != nil {
log.Fatal(err)
}
// The browser can connect now because the listening socket is open.
err := open.Start("http://localhost:3000/test")
if err != nil {
log.Println(err)
}
// Start the blocking server loop.
log.Fatal(http.Serve(l, r))
There's no need to poll as shown in another answer. The browser will connect if the listening socket is open before the browser is started.
ListenAndServe is a convenience function that opens a socket and calls Serve. The code in this answer splits out these steps so the browser can be opened after listening starts but before the blocking call to Serve.
If there is no error, http.ListenAndServe() will never return. So you shouldn't add code after that except code that handles failure.
You have to start a new goroutine, so ListenAndServe() is called in one goroutine, and code checking if it is up should run on the other goroutine.
And you can check if your server is up by making a simple HTTP GET call to it, for example using http.Get().
The following example delays startup for 7 seconds on purpose. The new goroutine starts an endless for loop that checks if server is up, sleeping 1 second between attempts.
Example:
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("Hi!"))
})
go func() {
for {
time.Sleep(time.Second)
log.Println("Checking if started...")
resp, err := http.Get("http://localhost:8081")
if err != nil {
log.Println("Failed:", err)
continue
}
resp.Body.Close()
if resp.StatusCode != http.StatusOK {
log.Println("Not OK:", resp.StatusCode)
continue
}
// Reached this point: server is up and running!
break
}
log.Println("SERVER UP AND RUNNING!")
}()
log.Println("Starting server...")
time.Sleep(time.Second * 7)
log.Fatal(http.ListenAndServe(":8081", nil))
Example output:
2015/09/23 13:53:03 Starting server...
2015/09/23 13:53:04 Checking if started...
2015/09/23 13:53:06 Failed: Get http://localhost:8081: dial tcp [::1]:8081: connectex: No connection could be made because the target machine actively refused it.
2015/09/23 13:53:07 Checking if started...
2015/09/23 13:53:09 Failed: Get http://localhost:8081: dial tcp [::1]:8081: connectex: No connection could be made because the target machine actively refused it.
2015/09/23 13:53:10 Checking if started...
2015/09/23 13:53:10 SERVER UP AND RUNNING!
The API is not absolutely terrible, but let's just say "It takes some getting used to". Here is how you use custom attributes on the Server struct:
s := &http.Server{
Addr: cnf.API_SERVER_ADDRESS,
Handler: h,
ReadTimeout: 0, // 1 * time.Minute,
WriteTimeout: 30 * time.Minute,
MaxHeaderBytes: 1 << 20,
}
go func() {
l, err := net.Listen("tcp", cnf.API_SERVER_ADDRESS)
if err != nil {
log.Fatal(err)
}
fmt.Println(`{"server_state":"listening"}`)
log.Fatal(s.Serve(l));
}()
because if you instead use:
http.Serve(l, handler)
then you can't define custom properties on the server

Accept() take too long time to return in golang

I work on a tcp server in golang ,recently I found a problem, when the client connected to the server, use netstat -nat|grep -i "55555" to check , it tells me the connection has established and Recv-Q has message , but Accept() need take about several seconds to return . I'm confused about it, could anyone tell me why or give any advice how to find the problem?
Here is my main code
tcpAddr, err := net.ResolveTCPAddr("tcp4", server.Host)
if err != nil {
log.Error(err.Error())
os.Exit(-1)
}
listener, err := net.ListenTCP("tcp", tcpAddr)
if err != nil {
log.Error(err.Error())
os.Exit(-1)
}
for {
conn, err := listener.Accept()
if err != nil {
continue
}
log.Debug("Accept a new connection")
go handleClient(conn)//deal the connection
}
P.S.
In handleClient, it's too long, I can't paste it here, only one more thing, I set tcp keep alive to the connection , use this package github.com/felixge/tcpkeepalive
kaConn, err := tcpkeepalive.EnableKeepAlive(connection)
if err != nil {
log.Debug("EnableKeepAlive err ", err)
}else{
kaConn.SetKeepAliveIdle(30*time.Second)
kaConn.SetKeepAliveCount(4)
kaConn.SetKeepAliveInterval(5*time.Second)
}
Update :
When I stop tcp keep alive, it works and Accept return instantly .
But even if not stop tcp keep alive, Accept return instantly when I use a script to test.Is it to do with the client?

golang TCPConn.SetWriteDeadline doesn't seem to work as expected

I'm trying to detect sending failures by inspecting the error returned by golang TCPConn.Write, but it's nil. I also tried using TCPConn.SetWriteDeadline without success.
That's how things happen:
the server starts
a client connects
the server sends a message and the client receives it
the client shuts down
the server sends one more message: no error
the server sends the third message: only now the error appears
Question: why only the second message to a non-existing client results in an error? How should the case be handled properly?
The code follows:
package main
import (
"net"
"os"
"bufio"
"fmt"
"time"
)
func AcceptConnections(listener net.Listener, console <- chan string) {
msg := ""
for {
conn, err := listener.Accept()
if err != nil {
panic(err)
}
fmt.Printf("client connected\n")
for {
if msg == "" {
msg = <- console
fmt.Printf("read from console: %s", msg)
}
err = conn.SetWriteDeadline(time.Now().Add(time.Second))
if err != nil {
fmt.Printf("SetWriteDeadline failed: %v\n", err)
}
_, err = conn.Write([]byte(msg))
if err != nil {
// expecting an error after sending a message
// to a non-existing client endpoint
fmt.Printf("failed sending a message to network: %v\n", err)
break
} else {
fmt.Printf("msg sent: %s", msg)
msg = ""
}
}
}
}
func ReadConsole(network chan <- string) {
console := bufio.NewReader(os.Stdin)
for {
line, err := console.ReadString('\n')
if err != nil {
panic(err)
} else {
network <- line
}
}
}
func main() {
listener, err := net.Listen("tcp", "localhost:6666")
if err != nil {
panic(err)
}
println("listening on " + listener.Addr().String())
consoleToNetwork := make(chan string)
go AcceptConnections(listener, consoleToNetwork)
ReadConsole(consoleToNetwork)
}
The server console looks like this:
listening on 127.0.0.1:6666
client connected
hi there!
read from console: hi there!
msg sent: hi there!
this one should fail
read from console: this one should fail
msg sent: this one should fail
this one actually fails
read from console: this one actually fails
failed sending a message to network: write tcp 127.0.0.1:51194: broken pipe
The client looks like this:
package main
import (
"net"
"os"
"io"
//"bufio"
//"fmt"
)
func cp(dst io.Writer, src io.Reader, errc chan<- error) {
// -reads from src and writes to dst
// -blocks until EOF
// -EOF is not an error
_, err := io.Copy(dst, src)
// push err to the channel when io.Copy returns
errc <- err
}
func StartCommunication(conn net.Conn) {
//create a channel for errors
errc := make(chan error)
//read connection and print to console
go cp(os.Stdout, conn, errc)
//read user input and write to connection
go cp(conn, os.Stdin, errc)
//wait until nil or an error arrives
err := <- errc
if err != nil {
println("cp error: ", err.Error())
}
}
func main() {
servAddr := "localhost:6666"
tcpAddr, err := net.ResolveTCPAddr("tcp", servAddr)
if err != nil {
println("ResolveTCPAddr failed:", err.Error())
os.Exit(1)
}
conn, err := net.DialTCP("tcp", nil, tcpAddr)
if err != nil {
println("net.DialTCP failed:", err.Error())
os.Exit(1)
}
defer conn.Close()
StartCommunication(conn)
}
EDIT: Following JimB's suggestion I came up with a working example. Messages don't get lost any more and are re-sent in a new connection. I'm not quite sure though how safe is it to use a shared variable (connWrap.IsFaulted) between different go routines.
package main
import (
"net"
"os"
"bufio"
"fmt"
)
type Connection struct {
IsFaulted bool
Conn net.Conn
}
func StartWritingToNetwork(connWrap * Connection, errChannel chan <- error, msgStack chan string) {
for {
msg := <- msgStack
if connWrap.IsFaulted {
//put it back for another connection
msgStack <- msg
return
}
_, err := connWrap.Conn.Write([]byte(msg))
if err != nil {
fmt.Printf("failed sending a message to network: %v\n", err)
connWrap.IsFaulted = true
msgStack <- msg
errChannel <- err
return
} else {
fmt.Printf("msg sent: %s", msg)
}
}
}
func StartReadingFromNetwork(connWrap * Connection, errChannel chan <- error){
network := bufio.NewReader(connWrap.Conn)
for (!connWrap.IsFaulted) {
line, err := network.ReadString('\n')
if err != nil {
fmt.Printf("failed reading from network: %v\n", err)
connWrap.IsFaulted = true
errChannel <- err
} else {
fmt.Printf("%s", line)
}
}
}
func AcceptConnections(listener net.Listener, console chan string) {
errChannel := make(chan error)
for {
conn, err := listener.Accept()
if err != nil {
panic(err)
}
fmt.Printf("client connected\n")
connWrap := Connection{false, conn}
go StartReadingFromNetwork(&connWrap, errChannel)
go StartWritingToNetwork(&connWrap, errChannel, console)
//block until an error occurs
<- errChannel
}
}
func ReadConsole(network chan <- string) {
console := bufio.NewReader(os.Stdin)
for {
line, err := console.ReadString('\n')
if err != nil {
panic(err)
} else {
network <- line
}
}
}
func main() {
listener, err := net.Listen("tcp", "localhost:6666")
if err != nil {
panic(err)
}
println("listening on " + listener.Addr().String())
consoleToNetwork := make(chan string)
go AcceptConnections(listener, consoleToNetwork)
ReadConsole(consoleToNetwork)
}
This isn't Go specific, and is a artifact of the underlying TCP socket showing through.
A decent diagram of the TCP termination steps is at the bottom of this page:
http://www.tcpipguide.com/free/t_TCPConnectionTermination-2.htm
The simple version is that when the client closes its socket, it sends a FIN, and receives an ACK from the server. It then waits for the server to do the same. Instead of sending a FIN though, you're sending more data, which is discarded, and the client socket now assumes that any more data coming from you is invalid, so the next time you send you get an RST, which is what bubbles up into the error you see.
Going back to your program, you need to handle this somehow. Generally you can think of whomever is in charge of initiating a send, is also in charge of initiating termination, hence your server should assume that it can continue to send until it closes the connection, or encounters an error. If you need to more reliably detect the client closing, you need to have some sort of client response in the protocol. That way recv can be called on the socket and return 0, which alerts you to the closed connection.
In go, this will return an EOF error from the connection's Read method (or from within the Copy in your case). SetWriteDeadline doesn't work because a small write will go though and get dropped silently, or the client will eventually respond with an RST, giving you an error.

Resources