From what I have read about UDP, it has no error handling, no checking for things like sequence of data sent/recieved, no checking for duplicate packets, no checking for corrupt packets and obviously no guarantee that the packets sent are even received...
So with that in mind, why an earth is there actually an option to use checksums in UDP?? Because surely if you want to make sure the data being sent is received in the correct order (and not corrupt and so on) then you would use TCP...
UDP packets include a field for a 16 bit CRC checksum which the receiving operating system will use to check for packet corruption. If the checksum is present and fails, then the packet will be silently discarded. It is up to the application to notice that the packet disappeared and take corrective action.
UDP checksums are enabled by default on all modern operating systems. It is possible to disable UDP checksums in IPv4, either at the socket or OS level. Doing so would reduce the CPU overhead of processing each packet at both the sender and receiver. This might be desirable if, for example, the application were calculating its own checksum separately. Without any checksum, there would be no guarantee that the bytes received are the same as the bytes sent.
The task of UDP is to transport datagrams, which are "network data packets". For UDP, every data packet is a transmission of its own. If you send 3 packets, those are three independent transmissions for UDP. Whether the content of these 3 packets somehow belongs together or if these are three individual requests (think of DNS requests, where every request is sent as an own UDP packet), UDP doesn't know and doesn't care. All that UDP guarantees is that a packet is either transmitted as a whole or not at all; either the entire packet arrives or the entire packet is lost, you will never see "half of a packet" arriving. So if you just want to send a bunch of data packets, you use UDP.
The task of TCP, on the other hand, is to transport a stream of data. It's not about packets. It's about a stream of bytes somehow making it from one host to another. How this happens, e.g. how TCP is breaking the data stream into chunks and sending these chunks over the network and ensuring that no data is lost and all data is in order, is up to TCP. All that TCP guarantees is that the bytes will arrive correctly and in order at the other side, unless the TCP connection is lost, in which case the stream ends abruptly somewhere in the middle but all data, that arrived up to that point, did arrive correctly and in correct order. So despite TCP also working with packets, the transmission behaves like a stream that has no internal "data units". When sending 80 bytes over TCP, there may be one packet with 80 bytes or 10 packets with each 8 bytes or anything in between, you cannot know and you don't have to.
But just because you use UDP doesn't mean you don't care for data corruption in UDP packets. Keep in mind that corruption may not just affect your data, it may also affect the UDP header itself. If only a single bit swaps, the UDP packets may have an incorrect destination port. So they added a checksum which ensures that neither the UDP header nor the data payload has been corrupted but made it optional, so it's up to you whether you want to use it or not. If used, corrupt packets are dropped and thus behave like lost packets. If your code takes care of lost packets, it will automatically take care of corrupt packets, too.
With IPv6 though, the checksum was dropped from the IP header, which means that IP header corruptions are no longer detected. But this was seen as a small problem, as most layer 2 protocols have their own mechanism to detect corrupt data (e.g. Ethernet and WiFi already guarantee that data is not corrupted on its way through the network) and the checksums of UDP/TCP also cover some of the IP header fields, so even without layer 2 error checking, the recipient would notice if the IP addresses in the header have been corrupted along the way and drop the packet. As a consequence, the UDP checksum is no longer optional with IPv6.
Related
I send mixtures of large UDP packets back-to-back with small UDP packets. The large packets get fragmented to my MTU.
On RHEL6 (CentOS6), the small UDP packets always arrive at the receivers in the correct order with respect to the final fragment of any previous large packet.
On RHEL7, this no longer is the case. The small packet can get transmitted in-between the fragments of the larger packet, thus causing the receiver to see the small packet BEFORE the reassembled large packet.
As near as I can tell with ethtool, the configuration of the NIC is the same on both machines (It's actually the same machine and I swap hard drives).
So, my question is... What controls this behavior in RHEL7+? It's not udp-fragmentation-offload (That's set the same in both configurations). I'd like to find out how to force the fragments to be transmitted as a complete group, with no interfering packets, in RHEL7+.
Thanks,
XL600
I've been doing some work with C# Networking using UDP. I'm getting on fine but need the answer to a couple of fundamental questions I'm having problems testing:
Currently I'm sending data in a ~16000 byte datagrams, which according to wireshark is getting split into several 1500 byte packets (because of max packet size limits) and then reassembled at the other end.
Am I right in understanding the datagram will be received complete at the other end OR not at all. IE it's an all or nothing thing. There is no chance of ending up with a fragmented datagram due to packet loss?
Therefore, I only need to ACK per datagram, rather than ensuring my datagrams are < 1500 bytes and ACK each one?
I've looked in a lot of places but there seems to be a lot of confusion between the differences between datagrams and the underlying packets...
Thanks for you help!
There is no chance of ending up with a fragmented datagram due to packet loss?
I believe that's true: that fragmentation and fragment reassembly is handled by the protocol layer below UDP, i.e. that it's handled by the "IP" layer, which will error if it fails to reassemble the packet-fragments into a datagram (for example, search for "fragment" in RFC 792).
http://www.pcvr.nl/tcpip/udp_user.htm#11_5 says,
"The IP layer at the destination performs the reassembly. The goal is to make fragmentation and reassembly transparent to the transport layer (TCP and UDP), which it is, except for possible performance degradation."
As you may now 16 bit UDP length field indicates that you can send a total of 65535 bytes. However, the data can be theoretically (sizeof(IP Header) + sizeof(UDP Header)) = 65535-(20+8) = 65507 bytes.
But this does not mean that all applications that are using UDP will send this amount of data as an example DNS packets limits to 512 bytes. This is because you don't get any ACK packets from server. This is one reason that packets may get lost in the network (packet transmission problems and loss). Secondly intermediate nodes may encapsulate datagrams inside of another protocol, as an example IPSEC or other protocols do that.
For UDP there is no ACK packets, so in your case if underlying application uses UDP you should not see any ACK packets. Secondly, some of the server limit their sizes to the max UDP packets depending on the application, so if you have data transfer from client to server you should see same bytes e.g 512 bytes. going and coming back in wireshark. Mostly, source makes the request and destination sends X bytes UDP datagrams back.
These links may be good for your questions:
Wireshark UDP analysis
RFC 1122 (states that 576 is the minimum maximum reassembly buffer size)
Am I right in understanding the datagram will be received complete at the other end OR not at all. IE it's an all or nothing thing. There is no chance of ending up with a fragmented datagram due to packet loss?
That is correct.
Therefore, I only need to ACK per datagram, rather than ensuring my datagrams are < 1500 bytes and ACK each one?
I don't understand this question. You need to ACK each datagram regardless of its size, and you should make them < 1500 bytes so they won't get fragmented. Otherwise you may never be able to transmit any specific datagrams at all, if it repeatedly gets fragmented and a fragment repeatedly gets lost.
I've been writing a program that uses a stop and wait protocol on top of UDP to send packets over LAN and also over WAN. I've recently been testing my program and have noticed that the packet loss rate is higher for larger packets (approaching 64k bytes). Intuitively this makes sense but what are the actual reasons for this?
UDP packets greater than the MTU size of the network that carries them will be automatically split up into multiple packets, and then reassembled by the recipient. If any of those multiple sub-packets gets dropped, then the receiver will drop the rest of them as well.
So for example if you send a 63k UDP packet, and it goes over Ethernet, it will get broken up into 47+ smaller "fragment" packets (because Ethernet's MTU is 1500 bytes, but some of those are used for UDP headers, etc, so the amount of user-data-space available in a UDP packet is smaller than that). The receiver will only "see" that UDP packet if all 47+ of those fragment-packets make it through okay. If just one of those fragment-packets gets dropped, the whole operation fails.
Well, data networks are far from reliable; packets get dropped all the time. Overloaded routers, full buffers and corrupt packets are some of the reasons. Since UDP has no flow control capabilities, it can't slow down if for example the receiving end is overloaded.
As Jeremy explained, the bigger the payload, the more packets it is going to be split into, and therefore a bigger chance of losing some of them.
UDP is used in cases where a dropped packet here in there won't affect anything or cases that you need something to get there in time or not at all. (VOIP, streaming video etc)
Its all about IP fragmentation and defragmentation. Packet more than MTU would be fragmented and has to be defragmented at the final host, there are also chances the fragments gets fragmented again on the path and which again can add the delay. sometimes if some N/W element is configured for layer 4 filtering then it defragments(not the final host) apply rules and then again frgaments and forward. Thats the reason the applicaiton which need performance always try to send data with size <= (MTU-ETHHDR-IPHDR)
If a TCP connection is established between a client and server, is sending data faster on this connection-oriented route compared to a connectionless given there is less header info in the packets? So a TCP connection is opened and bytes sent down the open connection as and when required. Or would UDP still be a better choice via a connectionless route where each packet contains the destination address?
Is sending packets via an established TCP connection (after all hand shaking has been done) a method to be faster than UDP?
I suggest you read a little bit more about this topic.
just as a quick answer. TCP makes sure that all the packages are delivered. So if one is dropped for whatever reason. The sender will continue sending it until the receiver gets it. However, UDP sends a packet and just forgets it, so you might loose some of the packets. As a result of this, UDP sends less number of packets over network.
This is why they use UDP for videos because first loosing small amount of data is not a big deal plus even if the sender sends it again it is too late for receiver to use it, so UDP is better. In contrast, you don't want your online banking be over UDP!
Edit: Remember, the speed of sending packets for both UDP and TCP is almost same, and depends on the network! However, after handshake is done in TCP, still receiver needs to send the acks, and sender has to wait for ack before sending new batch of data, so still would be a little slower.
In general, TCP is marginally slower, despite less header info, because the packets must arrive in order, and, in fact, must arrive. In a UDP situation, there is no checking of either.
Using the TCP/IP protocol, given a connection between a client and a server, are the packets sent by the client to the server always received in the same order they were sent?
For example, if the client sends 3 packets of data, A, B and C, will the server always receive A first followed by B and C or is it possible for the server to receive C first, followed by A and B?
At IP level, packets may arrive in any order (if they arrive). At TCP level, the data stream is guaranteed to be ordered in the same manner on both ends.
That means yes, the server will always receive A then B then C. As long as you are using TCP.
When using TCP, data is received by the destination application in the same order as it is sent by the source application.
See the following for more details:
http://en.wikipedia.org/wiki/Transmission_Control_Protocol#Data_transfer
TCP is a transmission protocol, and it transmits data by sending the data out in IP packets over the underlying IP network. TCP is responsible for ensuring the correct transmission of the data, which includes ordering the arriving packets, re-requesting missing ones and discarding duplicates.
TCP as such does not expose any notion of "packet" to the user; the fact that the data is chunked into IP packets is a detail of the "over IP" implementation. A different implementation, e.g. TCP-over-bicycle-courier, might employ an entirely different scheme.
It cannot happen that you receive data in a different order on the application side over a TCP socket.
It may happen that packets are received in a different order by the networking layer of the OS, but TCP makes it a requirement that the upper levels get data in order. It is the OS' role to ask again for unreceived fragments etc and assemble these fragments. So, you need not worry.
UDP, on the other hand, offers no such guarantee.
The server (as the physical NIC of the machine) might receive them in any order. Your OS might receive them in any order again - that will mostly (but not allways) be the order of physical reception. Your client application is guaranteed to receive them in correct order, thats a property of TCP
In general, packets will be received in the same order they are transmitted. But the network may drop or reorder packets. For example, packets may take different routes and arrive out of order. Packets may be lost or even duplicated on the network. The TCP implementation is responsible for retransmitting packets that are lost, acknowledging packets that are received, ignoring duplicated packets, all with the objective of accurately reconstructing the transmitted byte stream at the receiver.
At the application level, you send a stream of bytes and receive a stream of bytes. TCP does whatever is needed to ensure the received stream of bytes is identical to the sent stream of bytes, regardless of what happens to the packets on the network.