What is the overhead for PPP and Ethernet sending 5000 bytes?
Frame size for Point-to-Point Protocol: 8 bytes
MTU: 500 bytes
Frame size for Ethernet: 18 bytes
MTU: 1500 bytes
Both sending 5000 bytes..
I know this is just a calculation, but I am not sure how to do it. I can't find it anywhere. I would think that since a PPP frame takes 8 bytes and maximum transmission unit is 500 then it can send (500 - 8)bytes of information in one go. It sends 10 frames, resulting in 4920 bytes sent. Then sends the final (80+8)bytes with the last frame.
Similar for Ethernet. (1500 - 18)bytes with each frame. 3 frames sent means 4446 bytes sent. Sending (554+18)bytes in the last frame.
This obviously doesn't answer the "overhead" question. Anyone have any ideas?
It really depends on how you define overhead. This answer will assume overhead is the number of bytes that you need to transmit in addition to the data itself.
For Ethernet, assuming the 5000 byte payload is not encapsulated in an IP + TCP/UDP frame, you will have 18 bytes of overhead for every packet sent. That means each transmission with an MTU of 1500 will be able to hold 1482 bytes. To transmit 5000 bytes, this means 4 packets must be transmitted, which means an overhead of 72 bytes (18 * 4). Note that the overhead becomes bigger when you include things like the IP frame which contains a TCP frame.
For PPP, as you've already shown, you can send 492 bytes per frame. Eleven frames means 88 bytes of overhead (11 * 8) - again, not including any additional protocol frames within the payload.
In both these examples any protocols that build on top of these link layer protocols will contribute to overhead. For example, an Ethernet packet sent with an IPv4 frame which contains a UDP datagram will have an additional 28 bytes consumed by headers and not data (20 bytes in an IPv4 header and 8 in a UDP header, assuming no IP options). Considering the original Ethernet example, this means the amount of data per packet becomes 1454 bytes, which luckily still comes to 4 packets (the extra spills over into the smaller 4th packet), with 144 bytes of overhead.
You can read more here (I find that page a little hard to read though).
Related
MTU (Maximum transmission unit) is the maximum frame size that can be transported.
When we talk about MTU, it's generally a cap at the hardware level and is for the lower level layers - DataLink and Physical layer.
Now, considering the OSI layer, it does not matter how efficient are the upper layers or what kind of magic-sauce they are applying, data-link layer will always construct frames of size < 1500 bytes (or whatever is the MTU) and anything in the "internet" will always be transmitted at that frame size.
Does the internet's transmission rate really capped at 1500 bytes. Now-a-days, we see speeds in 10-100 Mbps and even Gbps. I wonder for such speeds, does the frames still get transmitted at 1500 bytes, which would mean lots and lots and lots of fragmentation and re-assembly at the receiver. At this scale, how does the upper layer achieve efficiency ?!
[EDIT]
Based on below comments, I re-frame my question:
If data-layer transmits at 1500 byte frames, I want to know how is upper layer at the receiver able to handle such huge incoming data-frames.
For ex: If internet speed in 100 Mbps, upper layers will have to process 104857600 bytes/second or 104857600/1500 = 69905 frames/second. Network layer also need to re-assemble these frames. How network layer is able to handle at such scale.
If data-layer transmits at 1500 byte frames, I want to know how is
upper layer at the receiver able to handle such huge incoming
data-frames.
1500 octets is a reasonable MTU (Maximum Transmission Unit), which is the size of the data-link protocol payload. Remember that not all frames are that size, it is just the maximum size of the frame payload. There are many, many things with much smaller payloads. For example, VoIP has very small payloads, often smaller than the overhead of the various protocols.
Frames and packets get lost or dropped all the time, often on purpose (see RED, Random Early Detection). The larger the data unit, the more data that is lost when a frame or packet is lost, and with reliable protocols, such as TCP, the more data must be resent.
Also, having a reasonable limit on frame or packet size keeps one host from monopolizing the network. Hosts must take turns.
For ex: If internet speed in 100 Mbps, upper layers will have to
process 104857600 bytes/second or 104857600/1500 = 69905
frames/second. Network layer also need to re-assemble these frames.
How network layer is able to handle at such scale.
Your statement has several problems.
First, 100 Mbps is 12,500,000 bytes per second. To calculate the number of frames per second, you must take into account the data-link overhead. For ethernet, you have 7 octet Preabmle, a 1 octet SoF, a 14 octet frame header, the payload (46 to 1500 octets), a four octet CRC, then a 12 octet Inter-Packet Gap. The ethernet overhead is 38 octets, not counting the payload. To now how many frames per second, you would need to know the payload size of each frame, but you seem to wrongly assume every frame payload is the maximum 1500 octets, and that is not true. You get just over 8,000 frames per second for the maximum frame size.
Next, the network layer does not reassemble frame payloads. The payload of the frame is one network-layer packet. The payload of the network packet is the transport-layer data unit (TCP segment, UDP datagram, etc.). The payload of the transport protocol is application data (remember that the OSI model is just a model, and OSes do not implement separate session and presentation layers; only the application layer). The payload of the transport protocol is presented to the application process, and it may be application data or an application-layer protocol, e.g. HTTP.
The bandwidth, 100 Mbps in your example, is how fast a host can serialize the bits onto the wire. That is a function of the NIC hardware and the physical/data-link protocol it uses.
which would mean lots and lots and lots of fragmentation and
re-assembly at the receiver.
Packet fragmentation is basically obsolete. It is still part of IPv4, but fragmentation in the path has been eliminated in IPv6, and smart businesses, do not allow IPv4 packet fragments due to fragmentation attacks. IPv4 packets may be fragmented if the DF bit is not set in the packet header, and the MTU in the path shrinks smaller than the original MTU. For example, a tunnel will have a smaller MTU because of the tunnel overhead. If the DF bit is set, then a packet too large for the MTU on the next link, the packet is dropped. Packet fragmentation is very resource intensive on a router, and there is a set of steps that must be performed to fragment a packet.
You may be confusing IPv4 packet fragmentation and reassembly with TCP segmentation, which is something completely different.
I have been learning wireshark lately. While inspecting TCP segments, I saw a strange situation, at least for me. There was a mismatching SEQ,ACK numbers. Then I realized that difference between two ACK's same as 1 and half package size. However, as far as I know, ACKs are only growing by the full packet size. So what happened here?
SEQ ACK
1 1
2897 8689
5793 13033 <--
8689 14481
11585
14481
Well, there's not much to go by here, but I'm going to guess that you are capturing packets on the machine sending the large packets, i.e., the packets with 2896 bytes of TCP payload. As such, you are seeing the packets before they are IP-fragmented and actually transmitted out on the wire.
But you can't just send 2896 bytes of data onto the wire; Ethernet links typically impose a 1500 byte MTU (Maximum Transmission Unit), and when you account for IP and TCP header overhead, you usually end up with 1460 bytes of available payload or MSS (Maximum Segment Size). In your case it looks like you're only getting 1448 bytes of available MSS, most likely due to the addition of one or more IP and/or TCP header options.
In any case, the 2896 bytes of payload are going to be fragmented over 2 IP fragments, each containing 1448 bytes of TCP payload. I'm pretty sure that what you're seeing is an ACK from the receiver after having received 1 full segment plus 1 IP fragment from the next segment.
The previous ACK number was 8689, and 8689 + 2896 = 11585. Now add 1/2 of the data segment (2896 / 2 = 1448) and you get 11585 + 1448 = 13033. That's the ACK number you're seeing. Now add the other 1/2 and you get 13033 + 1448 = 14481, which is the ACK number of the next packet.
I hope that makes sense?
For an in depth look at the drawbacks of local packet captures, I direct you to a well-written blog by Jasper Bongertz titled, "The drawbacks of local packet captures".
Since I do not have reputation to comment the answer of Christopher Maynard, I comment it here.
As Christopher stated correctly the MTU (Maximum Transmission Unit) is standard Ethernet networks is 1500Bytes. However, crafting 1460Bytes (1500Bytes - 20Bytes IP Header - 20Bytes TCP Header, ignoring possible TCP options) TCP segments imposes a high load on the network stack since every segment must be handled separately.
As an example, if I want to transmit 1Mb (equals to 1024*1024Bytes = 1048576Bytes) of data, it would result in 1Mb/1460Bytes = 719 segments. In contrast to 1Mb/65525Bytes (theoretical max TCP segment size) = 16 segments. Since the overhead in the Kernel is mostly independent of the segment/packet size, small segments require much more processing.
To counteract this fact, TCP Segmentation Offloading (TSO) was developed. TSO allows the Kernel to craft TCP segments with maximum size and the NIC (Network Interface Card) uses hardware acceleration to split these segments into the separate TCP segments of 1460Bytes. Therefore, no IP fragmentation is required.
By IEEE 802.3, an Ethernet frame has to carry a payload of at least 46 bytes. This is for collision detection-- collisions of smaller frames (may) go undetected.
The Q is: what if the payload to be carried is shorter? what kind of padding is used to scale the frame up to the slot size-- 64 bytes?
TIA.
To quote from Data and Computer Network Communication (emphasis mine);
If the network layer wishes to send less than 46 bytes of data the MAC protocol adds sufficient number of zero bytes (0x00, is also known as null padding characters) to satisfy the requirement.
Some buggy drivers fail to do this though as noted by Adaptec.
I am currently going through my networking slides and was wondering if someone could help me with the concept of fragmentation and reassembly.
I understand how it works, namely how datagrams are split into smaller chunks because network links have a MTU. However the example in the picture is confusing me.
So the first two sections show a length of 1500, because this is the MSU, but shouldn't this mean that the last one should have 1000 (for a total of 4000 bytes) and not 1040? Where did these extra 40 bytes come from? My guess is that because the previous two fragments both had a header of 20 bytes, this extra 40 bytes of data needed to go somewhere, so it will arrive in the last fragment?
Fragflag essentially means that there is another fragment, so all of them will have a Fragflag of 1 except the last fragment which will be at zero. However I don't understand what offset is or how it is calculated. Why is the first offset at zero? Why did we divide the bytes in the datafield (1480) by 8 to get the second offset? Where did this 8 come from? Aside from that, I am assuming that each fragments offset will just increase by this value?
For example, the first fragment will have a offset of 0, the second 185, the third 370 and the fourth 555? (370+185)
Thanks for any help!
There is a 20 byte header in each packet. So the original packet contains 3,980 bytes of data. The fragments contain 1480, 1480, and 1020 bytes of data. 1480 + 1480 + 1020 = 3980
Every bit in the header is precious. Dividing the offset by 8 allows it to fit in 13 bits instead of 16. This means every packet but the last must contain a number of data bytes that is a multiple of 8, which isn't a problem.
The fragmentation and Reassembly has been exclusively explained in the RFC 791. Do go through the Internet Protocol Specification RFC. The RFC has various sections explaining the sample fragmentation and reassembly. All your doubts and questions are well catered in it.
Ans 1: Regarding the lengths of the packet: The original Packet contains 4000 Bytes. This packet is a fully IP packet and hence contains the IP header as well . Thus the payload length is actually 4000 - ( IP Header Length i. e. 20 ).
Actual Payload Length = 4000 - 20 = 3980
Now the packet is fragmented owing to the fact that the length is greater than the MTU ( 1500 Bytes).
Thus the 1st packet contains 1500 Bytes which includes IP header + Payload Fraction.
1500 = 20 ( IP header ) + 1480 ( Data Payload )
Similarly for the other packet.
The third packet shall contain remaining left over data ( 3980 - 1480 -1480 ) = 1020
Thus length of the packet is 20 ( IP Header ) + 1020 ( payload ) = 1040
Ans 2: The offset is the address or the locator from where the data starts with reference to the original data payload. For IP the data payload comprises all the data thats after the IP header and Options header. Thus the system/router takes the payload and divides it into smaller parts and keeps the track of the offset with reference to the original packet so that reassembly can be done.
As given in the RFC Page 12.
"The fragment offset field tells the receiver the position of a fragment in the original datagram. The fragment offset and length determine the portion of the original datagram
covered by this fragment. The more-fragments flag indicates (by being reset) the last fragment. These fields provide sufficient information to reassemble datagrams. "
The fragment offset is measured in Units of 8 bytes each. It has 13 bit field in the IP header. As said in the RFC page 17
"This field indicates where in the datagram this fragment belongs.The fragment offset is measured in units of 8 octets (64 bits). The first fragment has offset zero."
Thus as you asked in the question where did this 8 come from, its the standard thats been defined for IP protocol specification, where 8 octets are taken as one value. This also helps us to transmit large packets via this.
Page 28 of the RFC writes:
*Fragments are counted in units of 8 octets. The fragmentation strategy is designed so than an unfragmented datagram has all zero fragmentation information (MF = 0, fragment offset =
0). If an internet datagram is fragmented, its data portion must be
broken on 8 octet boundaries. This format allows 2**13 = 8192 fragments of 8 octets each for a
total of 65,536 octets. Note that this is consistent with the the
datagram total length field (of course, the header is counted in the
total length and not in the fragments).*
the offset size is 13 bits in the IP header but we need 16 bits as in worst case. So we use a scaling factor of 8 i.e. (2^16/2^13).
those are not extra bits but the total length of last fragment.
as 1500 is MTU this means there can be 1500 byte of data in one fragment including header. Header is appended with every fragment. this means in fragment we are capable of sending 1500-20 =1480 byte of data.
it is given there is 4000B datagram .datagram is nothing but a packet encapsulation of data at network layer.so the total data we have to send is 4000-20=3980 . then it is fragmented into 3parts (ceil(3980/1480)) each of length 1480,1480,1020 respectively . hence when 20B header is appended to last fragment its length becomes 1020+20=1040 .
What is the size of an empty UDP datagram? And that of an empty TCP packet?
I can only find info about the MTU, but I want to know what is the "base" size of these, in order to estimate bandwidth consumption for protocols on top of them.
TCP:
Size of Ethernet frame - 24 Bytes
Size of IPv4 Header (without any options) - 20 bytes
Size of TCP Header (without any options) - 20 Bytes
Total size of an Ethernet Frame carrying an IP Packet with an empty TCP Segment - 24 + 20 + 20 = 64 bytes
UDP:
Size of Ethernet frame - 24 Bytes
Size of IPv4 Header (without any options) - 20 bytes
Size of UDP header - 8 bytes
Total size of an Ethernet Frame carrying an IP Packet with an empty UDP Datagram - 24 + 20 + 8 = 52 bytes
Himanshus answer is perfectly correct.
What might be misleading when looking at the structure of an Ethernet frame [see further reading], is that without payload the minimum size of an Ethernet frame would be 18 bytes: Dst Mac(6) + Src Mac(6) + Length (2) + Fcs(4), adding minimum size of IPv4 (20) and TCP (20) gives us a total of 58 bytes.
What has not been mentioned yet is that the minimum payload of an ethernet frame is 46 byte, so the 20+20 byte from the IPv4 an TCP are not enough payload! This means that 6 bytes have to be padded, thats where the total of 64 bytes is coming from.
18(min. Ethernet "header" fields) + 6(padding) + 20(IPv4) + 20(TCP) = 64 bytes
Hope this clears things up a little.
Further Reading:
Ethernet_frame
Ethernet
See User Datagram Protocol. The UDP Header is 8 Bytes (64 bits) long.
The mimimum size of the bare TCP header is 5 words (32bit word), while the maximum size of a TCP header is 15 words.
Best wishes,
Fabian
If you intend to calculate the bandwidth consumption and relate them to the maximum rate of your network (like 1Gb/s or 10Gb/s), it is necessary, as pointed out by Useless, to add the Ethernet framing overhead at layer 1 to the numbers calculated by Felix and others, namely
7 bytes preamble
1 byte start-of-frame delimiter
12 bytes interpacket gap
i.e. a total of 20 more bytes consumed per packet.
If you're looking for a software perspective (after all, Stack Overflow is for software questions) then the frame does not include the FCS, padding, and framing symbol overhead, and the answer is 54:
14 bytes L2 header
20 bytes L3 header
20 bytes L4 header
This occurs in the case of a TCP ack packet, as ack packets have no L4 options.
As for FCS, padding, framing symbol, tunneling, etc. that hardware and intermediate routers generally hide from host software... Software really only cares about the additional overheads because of their effect on throughput. As the other answers note, FCS adds 4 bytes to the frame, making it frame 58 bytes. Therefore 6 bytes of padding are required to reach the minimum frame size of 64 bytes. The Ethernet controller adds an additional 20 bytes of framing symbols, meaning that a packet takes a minimum of 84 byte times (or 672 bit times) on the wire. So, for example, a 1Gbps link can send one packet every 672ns, corresponding to a max packet rate of roughly 1.5MHz. Also, intermediate routers can add various tags and tunnel headers that further increase the minimum TCP packet size at points inside the network (especially in public network backbones).
However, given that software is probably sharing bandwidth with other software, it is reasonable to assume that this question is not asking about total wire bandwidth, but rather how many bytes does the host software need to generate. The host relies on the Ethernet controller (for example this 10Mbps one or this 100Gbps one) to add FCS, padding, and framing symbols, and relies on routers to add tags and tunnels (A virtualization-offload controller, such as in the second link, has an integrated tunnel engine. Older controllers rely on a separate router box). Therefore the minimum TCP packet generated by the host software is 54 bytes.