Two servers communicate via UDP. When a large packet is send, it is not received at the other end.
I added the following registry entries
But this has not solved the problem. Below is the wireshark info regarding the lost packet. The size of the lost packet is 2706
How can I make sure that the packets are fragmented. Should I enable or disable Fragmetnation. I was not able to find a registry entry for MTU for UDP.
I'm troubleshooting a MTU/MSS issue that is causing fragmentation over a PPPoE service. Below is a packet dump of a TCP 3-Way Handshake from a different service (that is working as expected) that relates to my question.
I understand the way PMTUD works as this: by setting the Don't Fragment (DF) bit to 1 in the IP header, a router along the path to the destination that requires fragmentation of the packet sends an ICMP back to the host to adjust the MSS size accordingly. However, my understanding is that this will only happen when fragmentation occurs (packets greater than the path MTU). This suggests that PMTUD works during the data exchange phase, NOT when TCP 3-Way Handshake is negotiated (since these are small packets, 78 bytes in this case).
In the above packet capture the SYN packet sends a MSS=1460 (which is too large, due to the 8 byte overhead of PPPoE) and the SYN/ACK response from the server sends back the correct MSS=1452. What mechanism does TCP use to determine the MSS during this exchange?
Maybe, the server hasn't computed the MSS during this three-way handshake. For instance, if the system administrator has observed a lot of fragmentation, he can have set the MSS of the whole system to 1452 (with the command ip tcp adjust-mss 1452), so when you are doing the three-way handshake, the server only advertises its default MSS. Is it applicable to your case?
What you're probably seeing here is the result of what's known as MSS clamping where the network on which the server is attached to modifies the MSS in the outgoing SYN/ACK packets to signal to the sender to use a lower MSS. This is commonly done on networks that perform some form of tunnelling such as PPPoE on ADSL.
From what I have read about UDP, it has no error handling, no checking for things like sequence of data sent/recieved, no checking for duplicate packets, no checking for corrupt packets and obviously no guarantee that the packets sent are even received...
So with that in mind, why an earth is there actually an option to use checksums in UDP?? Because surely if you want to make sure the data being sent is received in the correct order (and not corrupt and so on) then you would use TCP...
UDP packets include a field for a 16 bit CRC checksum which the receiving operating system will use to check for packet corruption. If the checksum is present and fails, then the packet will be silently discarded. It is up to the application to notice that the packet disappeared and take corrective action.
UDP checksums are enabled by default on all modern operating systems. It is possible to disable UDP checksums in IPv4, either at the socket or OS level. Doing so would reduce the CPU overhead of processing each packet at both the sender and receiver. This might be desirable if, for example, the application were calculating its own checksum separately. Without any checksum, there would be no guarantee that the bytes received are the same as the bytes sent.
The task of UDP is to transport datagrams, which are "network data packets". For UDP, every data packet is a transmission of its own. If you send 3 packets, those are three independent transmissions for UDP. Whether the content of these 3 packets somehow belongs together or if these are three individual requests (think of DNS requests, where every request is sent as an own UDP packet), UDP doesn't know and doesn't care. All that UDP guarantees is that a packet is either transmitted as a whole or not at all; either the entire packet arrives or the entire packet is lost, you will never see "half of a packet" arriving. So if you just want to send a bunch of data packets, you use UDP.
The task of TCP, on the other hand, is to transport a stream of data. It's not about packets. It's about a stream of bytes somehow making it from one host to another. How this happens, e.g. how TCP is breaking the data stream into chunks and sending these chunks over the network and ensuring that no data is lost and all data is in order, is up to TCP. All that TCP guarantees is that the bytes will arrive correctly and in order at the other side, unless the TCP connection is lost, in which case the stream ends abruptly somewhere in the middle but all data, that arrived up to that point, did arrive correctly and in correct order. So despite TCP also working with packets, the transmission behaves like a stream that has no internal "data units". When sending 80 bytes over TCP, there may be one packet with 80 bytes or 10 packets with each 8 bytes or anything in between, you cannot know and you don't have to.
But just because you use UDP doesn't mean you don't care for data corruption in UDP packets. Keep in mind that corruption may not just affect your data, it may also affect the UDP header itself. If only a single bit swaps, the UDP packets may have an incorrect destination port. So they added a checksum which ensures that neither the UDP header nor the data payload has been corrupted but made it optional, so it's up to you whether you want to use it or not. If used, corrupt packets are dropped and thus behave like lost packets. If your code takes care of lost packets, it will automatically take care of corrupt packets, too.
With IPv6 though, the checksum was dropped from the IP header, which means that IP header corruptions are no longer detected. But this was seen as a small problem, as most layer 2 protocols have their own mechanism to detect corrupt data (e.g. Ethernet and WiFi already guarantee that data is not corrupted on its way through the network) and the checksums of UDP/TCP also cover some of the IP header fields, so even without layer 2 error checking, the recipient would notice if the IP addresses in the header have been corrupted along the way and drop the packet. As a consequence, the UDP checksum is no longer optional with IPv6.
Actually, I have two related questions.
I'm capturing filtered network traffic by libpcap on Debian. Then I need to replay this traffic on Win2k3 server. Sometimes I capture packets, both TCP and UDP, much larger than 1500 bytes (default MTU size for Ethernet). E.g., 2000+ bytes. I did no specific changes to MTU size on that Linux. So question #1:
What's the reason for these packets much larger than default MTU? Jumbo frames? This Wikipedia article states that "network interface cards capable of jumbo frames require explicit configuration to use jumbo frames", but I'm not aware about any such configuration. Also ifconfig shows me "MTU:1500". Can it be somehow related with "interrupt-combining" technique (or "interrupt coalescing" as in this article)? Can I supress such packets?
Then, question #2:
How can I send such packets by pcap_sendpacket on Windows? I receive error message "send error: PacketSendPacket failed" only for packets larger than 1500 bytes. Seems I cannot use jumbo frames because I'm sending data to directly connected custom "net tap" like pci card and I'm not sure I can configure its NIC. What else? Should I fragment these packets according to the protocol rules?
EDIT:
Checked fragmentation by NIC as Guy Harris suggested:
~# ethtool -k eth0
Offload parameters for eth0:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp-segmentation-offload: off
udp-fragmentation-offload: off
generic-segmentation-offload: off
generic-receive-offload: off
large-receive-offload: off
ntuple-filters: off
receive-hashing: off
The same for eth1 and br0 - network bridge between eth0 and eth1 which I'm sniffing.
And I still receive large UDP packets.
Your network adapter is probably doing TCP segmentation/desegmentation offloading and IP fragmentation/reassembly offloading, so:
UDP packets being sent by your machine that are larger than will fit in a single Ethernet frame are being handed to the network adapter without being fragmented, with the network adapter doing the fragmentation, and those are also handed to libpcap before being fragmented;
UDP fragments being received by your network adapter that are larger than will fit in a single Ethernet frame are being reassembled by the network adapter before being handed to the host, and are being handed to libpcap after being reassembled;
chunks of TCP stream data being sent by your machine that are too big to fit in a single Ethernet frame are being handed to the network adapter, with the network adapter breaking the chunks up into smaller TCP segments, and the full chunk is being handed to libpcap;
TCP segments received by your network adapter are being reassembled into larger chunks of TCP data and the chunks are being handed to the host and then to libpcap;
so what libpcap is seeing are not Ethernet packets and are not limited to the Ethernet frame size.
(I.e., Nikolai Fetissov was probably correct; what you're receiving might look like Ethernet frames, but that's because the network adapter and driver make them look that way. They are, in fact, not Ethernet frames transmitted on or received from the Ethernet.)
You can only suppress them by turning off whatever form of segmentation/desegmentation/fragmentation/reassembly is being done on your network adapter using the ethtool command; turn off options such as TCP Segementation Offload, UDP Fragmentation Offload, General Segmentation Offload, Large Receive Offload, and Generic Receive Offload.
Once you've disabled those options, you should no longer have those large packets, and thus you should be able to replay them with no problem. There is no easy way to replay the reassembled/un-fragmented-or-segmented packets you've captured so far - you'd have to write your own code to fragment them, and there's no guarantee that they'd be re-fragmented/re-segmented in the same way that they were originally fragmented/segmented on the wire.
¿Are you using the wireshark to capture?
It's important beacause by default wireshark reassemble fragmented ip datagrams (and stores them in a pcap file as reassembled MTU-higger single packages without fragmentation).
To disable:
Edit->preferences->Protocols->ipV4-> and uncheck "Reassemble fragmented IPv4 datagrams".
Why is the IP called a connectionless protocol? If so, what is the connection-oriented protocol then?
Thanks.
Update - 1 - 20:21 2010/12/26
I think, to better answer my question, it would be better to explain what "connection" actually means, both physically and logically.
Update - 2 - 9:59 AM 2/1/2013
Based on all the answers below, I come to the feeling that the 'connection' mentioned here should be considered as a set of actions/arrangements/disciplines. Thus it's more an abstract concept rather than a concrete object.
Update - 3 - 11:35 AM 6/18/2015
Here's a more physical explanation:
IP protocol is connectionless in that all packets in IP network are routed independently, they may not necessarily go through the same route, while in a virtual circuit network which is connection oriented, all packets go through the same route. This single route is what 'virtual circuit' means.
With connection, because there's only 1 route, all data packets will arrive in the same order as they are sent out.
Without connection, it is not guaranteed all data packets will arrive
in the same order as they are sent out.
Update - 4 - 9:55 AM 2016/1/20/Wed
One of the characteristics of connection-oriented is that the packet order is preserved. TCP use a sequence number to achieve that but IP has no such facility. Thus TCP is connection-oriented while IP is connection-less.
The basic idea is pretty simple: with IP (on its own -- no TCP, UDP, etc.) you're just sending a packet of data. You simply send some data onto the net with a destination address, but that's it. By itself, IP gives:
no assurance that it'll be delivered
no way to find out if it was
nothing to let the destination know to expect a packet
much of anything else
All it does is specify a minimal packet format so you can get some data from one point to another (e.g., routers know the packet format, so they can look at the destination and send the packet on its next hop).
TCP is connection oriented. Establishing a connection means that at the beginning of a TCP conversation, it does a "three way handshake" so (in particular) the destination knows that a connection with the source has been established. It keeps track of that address internally, so it can/will/does expect more packets from it, and be able to send replies to (for example) acknowledge each packet it receives. The source and destination also cooperate to serial number all the packets for the acknowledgment scheme, so each end knows whether packets it sent were received at the other end. This doesn't involve much physically, but logically it involves allocating some memory on both ends. That includes memory for metadata like the next packet serial number to use, as well as payload data for possible re-transmission until the other side acknowledges receipt of that packet.
TCP/IP means "TCP over IP".
TCP
--
IP
TCP provides the "connection-oriented" logic, ordering and control
IP provides getting packets from A to B however it can: "connectionless"
Notes:
UDP is connection less but at the same level as TCP
Other protocols such as ICMP (used by ping) can run over IP but have nothing to do with TCP
Edit:
"connection-oriented" mean established end to end connection. For example, you pick up the telephone, call someone = you have a connection.
"connection-less" means "send it, see what happens". For example, sending a letter via snail mail.a
So IP gets your packets from A to B, maybe, in any order, not always eventually. TCP sorts them out, acknowledges them, requests a resends and provides the "connection"
Connectionless means that no effort is made to set up a dedicated end-to-end connection, While Connection-Oriented means that when devices communicate, they perform handshaking to set up an end-to-end connection.
IP is an example of the Connectionless protocols , in this kind of protocols you usually send informations in one direction, from source to destination without checking to see if the destination is still there, or if it is prepared to receive the information . Connectionless protocols (Like IP and UDP) are used for example with the Video Conferencing when you don't care if some packets are lost , while you have to use a Connection-Oriented protocol (Like TCP) when you send a File because you want to insure that all the packets are sent successfully (actually we use FTP to transfer Files). Edit :
In telecommunication and computing in
general, a connection is the
successful completion of necessary
arrangements so that two or more
parties (for example, people or
programs) can communicate at a long
distance. In this usage, the term has
a strong physical (hardware)
connotation although logical
(software) elements are usually
involved as well.
The physical connection is layer 1 of
the OSI model, and is the medium
through which the data is transfered.
i.e., cables
The logical connection is layer 3 of
the OSI model, and is the network
portion. Using the Internetwork
Protocol (IP), each host is assigned a
32 bit IP address. e.g. 192.168.1.1
TCP is the connection part of TCP/IP. IP's the addressing.
Or, as an analogy, IP is the address written on the envelope, TCP is the postal system which uses the address as part of the work of getting the envelope from point A to point B.
When two hosts want to communicate using connection oriented protocol, one of them must first initiate a connection and the other must accept it. Logically a connection is made between a port in one host and other port in the other host. Software in one host must perform a connect socket operation, and the other must perform an accept socket operation. Physically the initiator host sends a SYN packet, which contains all four connection identifying numbers (source IP, source port, destination IP, destination port). The other receives it and sends SYN-ACK, the initiator sends an ACK, then the connection are established. After the connection established, then the data could be transferred, in both directions.
In the other hand, connectionless protocol means that we don't need to establish connection to send data. It means the first packet being sent from one host to another could contain data payloads. Of course for upper layer protocols such as UDP, the recipient must be ready first, (e.g.) it must perform a listen udp socket operation.
The connectionless IP became foundation for TCP in the layer above
In TCP, at minimal 2x round trip times are required to send just one packet of data. That is : a->b for SYN, b->a for SYN-ACK, a->b for ACK with DATA, b->a for ACK. For flow rate control, Nagle's algorithm is applied here.
In UDP, only 0.5 round trip times are required : a->b with DATA. But be prepared that some packets could be silently lost and there is no flow control being done. Packets could be sent in the rate that are larger than the capability of the receiving system.
In my knowledge, every layer makes a fool of the one above it. The TCP gets an HTTP message from the Application layer and breaks it into packets. Lets call them data packets. The IP gets these packets one by one from TCP and throws it towards the destination; also, it collects an incoming packet and delivers it to TCP. Now, TCP after sending a packet, waits for an acknowledgement packet from the other side. If it comes, it says the above layer, hey, I have established a connection and now we can communicate! The whole communication process goes on between the TCP layers on both the sides sending and receiving different types of packets with each other (such as data packet, acknowledgement packet, synchronization packet , blah blah packet). It uses other tricks (all packet sending) to ensure the actual data packets to be delivered in ordered as they were broken and assembled. After assembling, it transfers them to the above application layer. That fool thinks that it has got an HTTP message in an established connection but in reality, just packets are being transferred.
I just came across this question today. It was bouncing around in my head all day and didn't make any sense. IP doesn't handle transport. Why would anyone even think of IP as connectionless or connection oriented? It is technically connectionless because it offers no reliability, no guaranteed delivery. But so is my toaster. My toaster offers no guaranteed delivery, so why not call aa toaster connectionless too?
In the end, I found out it's just some stupid title that someone somewhere attached to IP and it stuck, and now everyone calls IP connectionless and has no good reason for it.
Calling IP connectionless implies there is another layer 3 protocol that is connection oriented, but as far as I know, there isn't and it is just plain stupid to specify that IP is connectionless. MAC is connectionless. LLC is connectionless. But that is useless, technically correct info.