We have recently upgraded to DPDK 18.08 version.
After upgrading to the latest version observing issue with UDP packets transmission error for few packets.
No issue observed while transferring UDP packets with size 28 bytes and 48 bytes.
I tried to print the packet length calculation in my program just before sending it out to the Kernel using rte_kni_tx_burst.
The packet length calculation seems correct to me.
1.)
size_udp:48
sizeof(struct udp_hdr):8
size_ApplMsg:40
udphdr->dgram_len:12288
m->data_len:82
size_ip:68
l2_data_shift:14
2.)
size_udp:28
sizeof(struct udp_hdr):8
size_ApplMsg:20
udphdr->dgram_len:7168
m->data_len:62
ip->total_length:12288
size_ip:48
l2_data_shift:14
Packets with UDP size 736 are not getting transmitted to the receiving end and getting dropped.
3.)
size_udp:736
sizeof(struct udp_hdr):8
size_ApplMsg:728
udphdr->dgram_len:57346
m->data_len:770
size_ip:756
l2_data_shift:14
Also MTU is set to 1500 in my program. So it shouldn't be an issue to transfer 736 bytes UDP data which is less than 1500 bytes MTU.
I tried to increase the kernel buffer size but that didn't help.
netstat -su -> output shows 0 send/receive buffer errors.
What has changed in DPDK 18.08 with respect to UDP packets?
Please suggest if I need to consider tuning udp, offloading udp traffic to resolve this issue.
Thanks,
Related
I've noticed in wireshark that I'm able to send 4096 bytes of data to a HTTP webserver (from uploading a file) however the server only seems to be acknowledging data 1460 bytes at a time. Why is this the case?
The size of TCP segments is restricted to the MSS (Maximum Segment Size), which is basically the MTU (Maximum Transmission Unit) less the bytes comprising the IP and TCP overhead. On a typical Ethernet link, the MTU is 1500 bytes and basic IP and TCP headers comprise 20 bytes each, so the MSS is 1460 (1500 - 20 - 20).
If you're seeing packets indicated with a length field of 4096 bytes, then it almost certainly means that you're capturing on the transmitting host and Wireshark is being handed the large packet before it's segmented into 1460 byte chunks. If you were to capture at the receiving side, you would see the individual 1460 byte segments arriving and not a single, large 4096 byte packet.
For further reading, I would encourage you to read Jasper Bongertz's blog titled, "The drawbacks of local packet captures".
TCP by default uses path MTU discovery:
When system send packet to the network it set don't fragment flag (DF) in IP header
When IP router or you local machine see DF packet that should be fragmented to match MTU of the next hop link it sends feedback (RTCP fragmentation need) that contains new MTU
When system receives fragmentation needed ICMP it adjusts MSS and send data again.
This procedure is performed to reduce overall load on the network and increase probability of each packet delivery.
This is why you see 1460 packets.
Regarding to you question: the server only seems to be acknowledging data 1460 bytes at a time. Why is this the case?
TCP keep track window that defines "how many bytes of data you can send without acknowledge". Its purpose is to provide flow control mechanisms (sender can't send too much data that can't be processed) and congestion control mechanisms (sender can't send too much data to overload network). Window is defined by receiver side and may be increased during connection when TCP will estimate real channel bandwidth. So you may see one ACK that acknowledges several packets.
When send a UDP datagram larger than the MTU size, only the last fragment of the UDP datagram is putted out to the destination. The rest of fragments are silently discarded. Sendto() return the sendlength (no error).
Envirenment:
OS: vxWorks6.8
CPU: ARM9
MTU: 1500
The vxWorks board is direct connected via ethernet cable to the Win7 PC (with wireshark).
The DF flag isn't set.
Example 1:
sendto with 1400 Bytes
wireshark shows: udp with 1400 Databytes4
Example 2:
sendto with 1800 Bytes
wireshark shows: Fragmented IP protocol (proto=udp 0x11, off=1496, id0a00) -> Data (312 bytes)
Example 3:
sendto with 4000 Bytes
wireshark shows: Fragmented IP protocol (proto=udp 0x11, off=1496, id0a00) -> Data (1016 bytes)
Example 4:
sendto with 7800 Bytes
wireshark shows: Fragmented IP protocol (proto=udp 0x11, off=1496, id0a00) -> Data (328 bytes)
I use the same test on a second vxworks board with CPU PPC and this work fine and the ip-fragmentation work properly.
Why the first fregments of the udp packet are always discarded?
Many Thanks
I've been doing some work with C# Networking using UDP. I'm getting on fine but need the answer to a couple of fundamental questions I'm having problems testing:
Currently I'm sending data in a ~16000 byte datagrams, which according to wireshark is getting split into several 1500 byte packets (because of max packet size limits) and then reassembled at the other end.
Am I right in understanding the datagram will be received complete at the other end OR not at all. IE it's an all or nothing thing. There is no chance of ending up with a fragmented datagram due to packet loss?
Therefore, I only need to ACK per datagram, rather than ensuring my datagrams are < 1500 bytes and ACK each one?
I've looked in a lot of places but there seems to be a lot of confusion between the differences between datagrams and the underlying packets...
Thanks for you help!
There is no chance of ending up with a fragmented datagram due to packet loss?
I believe that's true: that fragmentation and fragment reassembly is handled by the protocol layer below UDP, i.e. that it's handled by the "IP" layer, which will error if it fails to reassemble the packet-fragments into a datagram (for example, search for "fragment" in RFC 792).
http://www.pcvr.nl/tcpip/udp_user.htm#11_5 says,
"The IP layer at the destination performs the reassembly. The goal is to make fragmentation and reassembly transparent to the transport layer (TCP and UDP), which it is, except for possible performance degradation."
As you may now 16 bit UDP length field indicates that you can send a total of 65535 bytes. However, the data can be theoretically (sizeof(IP Header) + sizeof(UDP Header)) = 65535-(20+8) = 65507 bytes.
But this does not mean that all applications that are using UDP will send this amount of data as an example DNS packets limits to 512 bytes. This is because you don't get any ACK packets from server. This is one reason that packets may get lost in the network (packet transmission problems and loss). Secondly intermediate nodes may encapsulate datagrams inside of another protocol, as an example IPSEC or other protocols do that.
For UDP there is no ACK packets, so in your case if underlying application uses UDP you should not see any ACK packets. Secondly, some of the server limit their sizes to the max UDP packets depending on the application, so if you have data transfer from client to server you should see same bytes e.g 512 bytes. going and coming back in wireshark. Mostly, source makes the request and destination sends X bytes UDP datagrams back.
These links may be good for your questions:
Wireshark UDP analysis
RFC 1122 (states that 576 is the minimum maximum reassembly buffer size)
Am I right in understanding the datagram will be received complete at the other end OR not at all. IE it's an all or nothing thing. There is no chance of ending up with a fragmented datagram due to packet loss?
That is correct.
Therefore, I only need to ACK per datagram, rather than ensuring my datagrams are < 1500 bytes and ACK each one?
I don't understand this question. You need to ACK each datagram regardless of its size, and you should make them < 1500 bytes so they won't get fragmented. Otherwise you may never be able to transmit any specific datagrams at all, if it repeatedly gets fragmented and a fragment repeatedly gets lost.
I am analyzing wireshark log files, when I make a request to a web page using firefox through a proxy server.
Following are details of connection establishment:
I have noted "maximum segment size" when I open options branch in the TCP segment details of the [SYN] message from my PC to the proxy server - it says 1460 bytes
Similarly, maximum segment size eof the [SYN,ACK] message from the proxy server to my PC - it says 1460 bytes
After establishing the TCP connection, should not each of the TCP frames sent from proxy server to my PC be of 1460 bytes? I am puzzled that why are they 590 bytes. Please advice how the 590 size is being set
A plausible explanation is that 590 turns out to be the Path MTU for the particular connection.
In other words whereby the client (one of the end nodes of the connection)accepts packets of a maximum of of 1460 bytes payload, some node(s) on the way accepts smaller packets. For efficiency purposes, the Path MTU Discovery allows the originator of a packet to size it so that it would fit the smaller MTU encountered on the path, and hence avoid fragmentation.
BTW:
1460 is a very common MTU (well MSS), because it it corresponds to 1500, Ethernet v2's maximum, minus 20+20= 40 bytes for the IP header overhead)
See the following Wikipedia entry for an overview of MTU (Maximum Transmission Unit) and a basic description of the Path MTU Discovery method (Basically setting the the DF i.e. do-not-fragment flag and relying on the ICMP ""Destination Unreachable (Datagram Too Big)" messages to detect that some node on the way couldn't handle the packet, and hence try with smaller size until it goes through).
Also, I suggest inspecting the packets when the connection is to a different host, maybe a peer on the very same network segment, without going through the proxy mentioned. Chances are you will then start seeing 1460 bytes frames.
How do you get the maximum number of bytes that can be passed to a sendto(..) call for a socket opened as a UDP port?
Use getsockopt(). This site has a good breakdown of the usage and options you can retrieve.
In Windows, you can do:
int optlen = sizeof(int);
int optval;
getsockopt(socket, SOL_SOCKET, SO_MAX_MSG_SIZE, (int *)&optval, &optlen);
For Linux, according to the UDP man page, the kernel will use MTU discovery (it will check what the maximum UDP packet size is between here and the destination, and pick that), or if MTU discovery is off, it'll set the maximum size to the interface MTU and fragment anything larger. If you're sending over Ethernet, the typical MTU is 1500 bytes.
On Mac OS X there are different values for sending (SO_SNDBUF) and receiving (SO_RCVBUF).
This is the size of the send buffer (man getsockopt):
getsockopt(sock, SOL_SOCKET, SO_SNDBUF, (int *)&optval, &optlen);
Trying to send a bigger message (on Leopard 9216 octets on UDP sent via the local loopback) will result in "Message too long / EMSGSIZE".
As UDP is not connection oriented there's no way to indicate that two packets belong together. As a result you're limited by the maximum size of a single IP packet (65535). The data you can send is somewhat less that that, because the IP packet size also includes the IP header (usually 20 bytes) and the UDP header (8 bytes).
Note that this IP packet can be fragmented to fit in smaller packets (eg. ~1500 bytes for ethernet).
I'm not aware of any OS restricting this further.
Bonus
SO_MAX_MSG_SIZE of UDP packet
IPv4: 65,507 bytes
IPv6: 65,527 bytes