I'm trying to send a SYN packet that has been received in one network interface into another one on the same host (while changing the IP addresses), using a raw socket.
I'm on Windows Server 2008 R2, and while sending and receiving UDP packets is going very well, when I'm trying to send a TCP packet, the other side just not receiving it (although it shown correctly in Wireshark). The other side normally listens to SOCK_STREAM data.
I've just read in this site that:
Even though you can send TCP packets with a raw socket, the operating system's TCP/IP stack will process incoming TCP and UDP packets. When you send a TCP SYN packet with a raw socket, the destination's response will be rejected by the operating system because it is unaware of the SYN you sent. Therefore, you cannot complete a three-way handshake.
However, Microsoft noted that:
These above restrictions do not apply to Windows Server 2008 R2, Windows Server 2008 , Windows Server 2003, or to versions of the operating system earlier than Windows XP with SP2.
Is it that TCP data cannot be sent over raw sockets? However, if yes, is there a way to establish the TCP connection anyway? Maybe some WinAPI to add the stream identifier?
I'd love to hear any solution for establishing TCP connections with a raw socket. Thanks!
Related
I think the question explains it fully.
Another way to say it: Are the packets read via a raw socket removed from the communication channel completely, so that they are not even delivered to the TCP socket that they were intended for in the first place. For e.g., if there is a web server running on my host, listening on port 80, and I open a raw socket listening for all TCP protocol packets, will the web server never receive those packets being sent to port 80?
Thanks.
From my experience packets read on the raw socket are not removed from the communication. You could even have multiple raw sockets at the same time and all get the same packages. But to cite from Linux: man raw:
Raw sockets may tap all IP protocols in Linux, even protocols like ICMP or TCP which have a protocol module in the kernel. In this case, the packets are passed to both the kernel module and the raw socket(s). This should not be relied upon in portable programs, many other BSD socket implementation have limitations here.
I have around 20 clients communicating together with a central server in the same LAN. The clients can make transaction simultaneously with the server. The server forward each transaction to external appliance in the network. Sometimes it works, sometimes my application shows a "time out" message in a client screen (randomly)
I mirrored all traffic and found TCP Retransmission after TCP Reset packets for the first TCP Sequence. I immediately thought about packet loss but all my cables/NIC are fine, and I do not see DUP ACK in the capture.
It seems that RST packets may have different significations.
What causes those TCP Reset?
Where should I focus my investigation: network or application design ?
I would appreciate any help. Thanks in advance.
Judging by the capture, I assume your central server is 137.56.64.31. What's happening is the clients are initiating a connection to the server with a SYN packet and the server responds with a RST. This is typical if the server has no application listening on that particular port e.g. the webserver application isn't running and a client tries to connect to port 80.
The clients are all connecting to different ports on the server, which is unusual for an central server, but not unheard of. The destination ports the clients are connecting to on the server are: 11007, 11012, 11014, 11108, and 11115. Is that normal for the application? If not, the clients should be connecting to whatever port the application server is listening on.
The reason for the retransmits is that instead of giving up on the connection upon receiving a RST from the server, the client tries to initiate the connection again so Wireshark considers it a retransmission.
I use raw socket to build a tcp client program and run it on machine A
and I run a regular tcp server program on machine B
the raw socket-based client program first send a SYN packet
and then it receives a SYN/ACK packet from the remote tcp server
then the kernel of machine A sends a RST to the remote tcp server
the sequence number and ack-sequence number is fine
what are potential reasons?
and how to deal with it? thanks!
BTW: I used tcpdump to capture packets on the remote machine B
and it shows "TCP port numbers reused" for the SYN packet from client,
actually before the client send the SYN, I used
netstat -tnp
to check on-going tcp sessions, and it shows nothing
This is perfectly normal. If a machine receives a SYN/ACK packet it doesn't expect, it should respond with a RST to let the other side know that it has no knowledge of or interest in that connection. The kernel sent a RST because that's what it's supposed to do -- it has no idea what your program is doing.
If you're trying to run your own TCP stack on a machine that already has a TCP stack, you'll have to prevent the regular TCP stack from responding to machines your stack is trying to talk to -- otherwise, they'll be talking to two TCP stacks which can't possibly work.
The socket API is the de-facto standard for TCP/IP and UDP/IP communications (that is, networking code as we know it). However, one of its core functions, accept() is a bit magical.
To borrow a semi-formal definition:
accept() is used on the server side.
It accepts a received incoming attempt
to create a new TCP connection from
the remote client, and creates a new
socket associated with the socket
address pair of this connection.
In other words, accept returns a new socket through which the server can communicate with the newly connected client. The old socket (on which accept was called) stays open, on the same port, listening for new connections.
How does accept work? How is it implemented? There's a lot of confusion on this topic. Many people claim accept opens a new port and you communicate with the client through it. But this obviously isn't true, as no new port is opened. You actually can communicate through the same port with different clients, but how? When several threads call recv on the same port, how does the data know where to go?
I guess it's something along the lines of the client's address being associated with a socket descriptor, and whenever data comes through recv it's routed to the correct socket, but I'm not sure.
It'd be great to get a thorough explanation of the inner-workings of this mechanism.
Your confusion lies in thinking that a socket is identified by Server IP : Server Port. When in actuality, sockets are uniquely identified by a quartet of information:
Client IP : Client Port and Server IP : Server Port
So while the Server IP and Server Port are constant in all accepted connections, the client side information is what allows it to keep track of where everything is going.
Example to clarify things:
Say we have a server at 192.168.1.1:80 and two clients, 10.0.0.1 and 10.0.0.2.
10.0.0.1 opens a connection on local port 1234 and connects to the server. Now the server has one socket identified as follows:
10.0.0.1:1234 - 192.168.1.1:80
Now 10.0.0.2 opens a connection on local port 5678 and connects to the server. Now the server has two sockets identified as follows:
10.0.0.1:1234 - 192.168.1.1:80
10.0.0.2:5678 - 192.168.1.1:80
Just to add to the answer given by user "17 of 26"
The socket actually consists of 5 tuple - (source ip, source port, destination ip, destination port, protocol). Here the protocol could TCP or UDP or any transport layer protocol. This protocol is identified in the packet from the 'protocol' field in the IP datagram.
Thus it is possible to have to different applications on the server communicating to to the same client on exactly the same 4-tuples but different in protocol field. For example
Apache at server side talking on (server1.com:880-client1:1234 on TCP)
and
World of Warcraft talking on (server1.com:880-client1:1234 on UDP)
Both the client and server will handle this as protocol field in the IP packet in both cases is different even if all the other 4 fields are same.
What confused me when I was learning this, was that the terms socket and port suggest that they are something physical, when in fact they're just data structures the kernel uses to abstract the details of networking.
As such, the data structures are implemented to be able to distinguish connections from different clients. As to how they're implemented, the answer is either a.) it doesn't matter, the purpose of the sockets API is precisely that the implementation shouldn't matter or b.) just have a look. Apart from the highly recommended Stevens books providing a detailed description of one implementation, check out the source in Linux or Solaris or one of the BSD's.
As the other guy said, a socket is uniquely identified by a 4-tuple (Client IP, Client Port, Server IP, Server Port).
The server process running on the Server IP maintains a database (meaning I don't care what kind of table/list/tree/array/magic data structure it uses) of active sockets and listens on the Server Port. When it receives a message (via the server's TCP/IP stack), it checks the Client IP and Port against the database. If the Client IP and Client Port are found in a database entry, the message is handed off to an existing handler, else a new database entry is created and a new handler spawned to handle that socket.
In the early days of the ARPAnet, certain protocols (FTP for one) would listen to a specified port for connection requests, and reply with a handoff port. Further communications for that connection would go over the handoff port. This was done to improve per-packet performance: computers were several orders of magnitude slower in those days.
Some things look strange to me:
What is the distinction between 0.0.0.0, 127.0.0.1, and [::]?
How should each part of the foreign address be read (part1:part2)?
What does a state Time_Wait, Close_Wait mean?
etc.
Could someone give a quick overview of how to interpret these results?
0.0.0.0 usually refers to stuff listening on all interfaces.
127.0.0.1 = localhost (only your local interface)
I'm not sure about [::]
TIME_WAIT means both sides have agreed to close and TCP
must now wait a prescribed time before taking the connection
down.
CLOSE_WAIT means the remote system has finished sending
and your system has yet to say it's finished.
I understand the answer has been accepted but here is some additional information:
If it says 0.0.0.0 on the Local Address column, it means that port is listening on all 'network interfaces' (i.e. your computer, your modem(s) and your network card(s)).
If it says 127.0.0.1 on the Local Address column, it means that port is ONLY listening for connections from your PC itself, not from the Internet or network. No danger there.
If it displays your online IP on the Local Address column, it means that port is ONLY listening for connections from the Internet.
If it displays your local network IP on the Local Address column, it means that port is ONLY listening for connections from the local network.
Foreign Address - The IP address and port number of the remote computer to which the socket is connected. The names that corresponds to the IP address and the port are shown unless the -n parameter is specified. If the port is not yet established, the port number is shown as an asterisk (*). (from wikipedia)
What is the distinction between 0.0.0.0, 127.0.0.1, and [::]?
0.0.0.0 indicates something that is listening on all interfaces on the machine.
127.0.0.1 indicates your own machine.
[::] is the IPv6 version of 0.0.0.0
My machine also shows *:\* for UDP which shows that UDP connections don't really have a foreign address - they receive packets from any where. That is the nature of UDP.
How should each part of the foreign address be read (part1:part2)?
part1 is the hostname or IP addresspart2 is the port
127.0.0.1 is your loopback address also known as 'localhost' if set in your HOSTS file. See here for more info: http://en.wikipedia.org/wiki/Localhost
0.0.0.0 means that an app has bound to all ip addresses using a specific port. MS info here: http://support.microsoft.com/default.aspx?scid=kb;en-us;175952
'::' is ipv6 shorthand for ipv4 0.0.0.0.
Send-Q is the amount of data sent by the application, but not yet acknowledged by the other side of the socket.
Recv-Q is the amount of data received from the NIC, but not yet consumed by the application.
Both of these queues reside in kernel memory.
There are guides to help you tweak these kernel buffers, if you are so inclined. Although, you may find the default params do quite well.
This link has helped me a lot to interpret netstat -a
A copy from there -
TCP Connection States
Following is a brief explanation of this handshake. In this context the "client" is the peer requesting a connection and the "server" is the peer accepting a connection. Note that this notation does not reflect Client/Server relationships as an architectural principal.
Connection Establishment
The client sends a SYN message which contains the server's port and the client's Initial Sequence Number (ISN) to the server (active open).
The server sends back its own SYN and ACK (which consists of the client's ISN + 1).
The Client sends an ACK (which consists of the server's ISN + 1).
Connection Tear-down (modified three way handshake).
The client sends a FIN (active close). This is a now a half-closed connection. The client no longer sends data, but is still able to receive data from the server. Upon receiving this FIN, the server enters a passive close state.
The server sends an ACK (which is the clients FIN sequence + 1)
The server sends its own FIN.
The client sends an ACK (which is server's FIN sequence + 1). Upon receiving this ACK, the server closes the connection.
A half-closed connection can be used to terminate sending data while sill receiving data. Socket applications can call shutdown with the second argument set to 1 to enter this state.
State explanations as shown in Netstat:
State Explanation
SYN_SEND Indicates active open.
SYN_RECEIVED Server just received SYN from the client.
ESTABLISHED Client received server's SYN and session is established.
LISTEN Server is ready to accept connection.
NOTE: See documentation for listen() socket call. TCP sockets in listening state are not shown - this is a limitation of NETSTAT. For additional information, please see the following article in the Microsoft Knowledge Base:
134404 NETSTAT.EXE Does Not Show TCP Listen Sockets
FIN_WAIT_1 Indicates active close.
TIMED_WAIT Client enters this state after active close.
CLOSE_WAIT Indicates passive close. Server just received first FIN from a client.
FIN_WAIT_2 Client just received acknowledgment of its first FIN from the server.
LAST_ACK Server is in this state when it sends its own FIN.
CLOSED Server received ACK from client and connection is closed.
For those seeing [::] in their netstat output, I'm betting your machine is running IPv6; that would be equivalent to 0.0.0.0, i.e. listen on any IPv6 address.