Sorry for what is a stupid question.
Function of transport layer is reliable delivery of messages. UDP is inherently unreliable, why do we use it at Transport layer then?
Thanks
EDIT: Just to clarify, I have read the Wiki and other sources. My question is
UDP is Unreliable (I know why and the advantages and where it is used etc.) , why not use it(UDP) at some other layer, rather than Transport layer which implies reliability.
Sometimes it is more important that the data be sent quickly and without pauses than that the stream be reliable. DNS uses UDP because the transaction between a DNS server and client consists of only one packet each way. If the packet is lost, it will be re transmitted at the request of the client.
Similarly, streaming video often uses UDP as a transport protocol because the occasional loss of a packet is acceptable. It is preferable that the image quality suffer as a result of lost packets, rather than the video stream suffer jitter or pauses (lag) as a result of TCP synchronization.
Games also often use UDP, sacrificing engine accuracy for improved speed/user experience.
These and more examples can be found in the relevant portions of the wikipedia article.
EDIT
UDP is used at the transport layer because it is a transport layer protocol. It provides "provides end-to-end communication services for applications" (RFC1122).
Reliability services are optional for transport layer protocols.
... rather than Transport layer which implies reliability
There's more than one dimension within "reliability." It's interesting to note that UDP is reliable in that it provides a checksum to prevent against corruption.
Stream protocols like TCP create problems for latency-sensitive applications. For latency-sensitive apps, UDP's natural limitation (to shed traffic during congestion) is a huge boon.
why not use it(UDP) at some other layer
IP datagrams are designed to be small enough to make the next hop transit. UDP datagrams can span IP datagrams, so there's some value added there. But if TCP were a layer above UDP, it would be limited by UDP's semantics (TCP ports are bound to a connection, UDP datagrams are not).
The reason why UDP is used at the transport layer is because the way these layers are set up. UDP is inherently a protocol for transferring data from point A to point B, not as an application or at the hardware layer.
At the transport layer there is no assumption of reliability, but rather that UDP is a protocol for transferring data. Under the 7 layer style of networking, it falls in the interface between the network and session layers. The name Transport layer simply says what it does. Reference wikipedia for more information on the OSI model.
TLDR The reason UDP is in the transport layer is because it is a protocol for data transport, and is therefore in the transport layer. All protocols that deal with data transport fall under this category
Transport layer classes
Class 0 - Simple class
Class 1 - Basic error recovery class
Class 2 - Multiplexing class
Class 3 - Error Recovery and multiplexing class
Class 4 - Error detection and recovery class
Related
I wonder
which protocol is responsible for managing retransmission? the ethernet protocol or tcp or both?
I was attending a TCPIP course and it is not clear for me which protocol is responsible for managing retransmission
To answer your question, TCP handles retransmission of corrupt frames. Ethernet only detects it.
Systems communicating over Ethernet divide a stream of data into shorter pieces called frames. Each frame contains source and destination addresses, and error-checking data so that damaged frames can be detected and discarded; most often, higher-layer protocols trigger retransmission of lost frames. Per the OSI model, Ethernet provides services up to and including the data link layer.
So the “responsibility” is on TCP. Note that speaking of responsibilities is misleading. You can have UDP (OSI layer 4) over Ethernet (OSI layer 2) in which case the “responsibility” for handling missing data is on the application which may choose to NOT resend (I.e. video streaming in which it doesn’t make sense).
On other data link protocols the resending of frames can be built into the protocol itself.
In that sense, handling retransmissions would be more of a property or feature of the TCP protocol rather than a “responsibility”
I have heard, Many companies like facebook are using UDP to fetch data from memcached. I have a doubt, How they make sure there is NO packet loss and order of received packet is per requirement.As we know tcp provide such facility but udp does not.
OSI Model has 7 layers which are:
Application Layer
Presentation Layer
Session Layer
Transport Layer
Network Layer
Data Link Layer
Physical Layer
Splitting things to layers is very good approach to solve problems but it doesn't mean you have to do all the network operations in network layer.
As you have mentioned, TCP provides feedback to end systems when UDP doesn't but UDP has it's own advantages. First of UDP's datagram is simpler than TCP's one. And also most of huge systems like Facebook uses UDP because using TCP for these kind systems would not be very clever since all the data senders would have to keep track of sending rate, retransmissions rate for many many receivers. So if they've used TCP, their network layer would be under very very big presure.
So they make flow control in the application layer to reduce network traffic.
i have a question about the trasport layer of the iso/osi network stack.
It's common to think that TCP is reliable but slow and UDP is fast but not reliable.
In several network applications, it is mandatory to implement an application-level ACK system also with TCP transport protocol in order to manage link interruptions and so on.
Assuming that the application scenario is this: local (small) network, typical small packet size (max 512 bytes), application-level ack system implementation. Why not use the protocol UDP as transport protocol in order to exploit its features such as speed, low latency and broadcast communication?
Thanks to all.
If you don't care about the Delivery (failure or successful delivery) of your message than UDP will be always a best choice.. But if you want to be sure whether the message is delivered or failed than think about other protocols rather than UDP.
In a small LAN it shouldnt be a big problem to use UDP, when the transferd Data isnt too "important" (which means if a packet is lost its no problem). More informations about the scenario are nessesary, if you use UDP you have to implement functions that check if you received all the information of the sender manually.
if you need the speed and if some lost packets are no problem (Voice Chat for instance) then use UDP.
I had a strange idea. I heard of software which from my understanding uses UDP to transfer files decreasing the overhead found in TCP packets.
If my app requires TCP and my LAN has software set up to communicate with another datacenter on the other side of the coast with software setup on their end. Would it be possible to send the actual data via UDP but than simulating TCP on both ends?
Does anyone have any ideas or information about such projects?
If you're asking if you can use UDP as a Layer 2, then the answer is yes, sort of. There are various protocols that allow you to create a tunnel to another network using a UDP transport, such as L2TP and even IPsec (with NAT traversal). You could also do it at the application layer.
If you're asking if TCP can be implemented in UDP, the answer is no. First, TCP packets and UDP packets have an incompatible format. Second, TCP and UDP have different protocol numbers (seen in the IP header) which means that TCP traffic destined for a UDP port would not be passed to the correct upper-layer protocol.
Both TCP and UDP are built on top of the IP, but the TCP uses different packet structure and at the layer-2 it is not possible to mimic the TCP using UDP packets.
Of course, if you have the control on both the source and destination, then it is possible to create a reliable UDP tunnel for the TCP packets. This would require some internal information (packet number, ack/nack flags) in the body of the UDP packet.
There is an interesting project http://udt.sourceforge.net/
It is a broadcast-capable reliable file transfer mechanism built on top the UDP.
PseudoTCP is a protocol which implements TCP algorithms on top of the UDP. It was introduced since the NAT traversal for TCP is much more complicated than UDP. But some P2P applications do need a reliable data transfer among nodes.
So far as I know, there are two PseudoTCP variations: Libjingle and Libnice.Libjingle is an open source library from google which was initially for gtalk. You could take a look at file sharing example from libjingle: https://developers.google.com/talk/libjingle/file_share. Recently, Chrome desktop also use PseudoTCP implementation from libjingle for reliable connections.
Yes, you can develop a protocol on UDP that simulates TCP. However, if you simulated TCP fully, it would technically have more overhead. Because TCP is implement as the packet and your simulated TCP is implemented in the body of the packet.
If you only need one or two features of TCP (such as basic ordering), then implementing it in UDP is useful.
Halo uses 2-3 (IIRC) UDP protocols that simulate different features of TCP, then full fledged TCP for initializing game-states. I Shot You First Networking, GDC publication
For example, in one case, they send 3 duplicate UDP packets to overcome packet loss.
If you control the software on both ends, and it is cost-effective to build your own protocol, then UDP can be versatile.
One way to do it now on Linux-3.18+ is to use Foo over UDP (FOU) which implements Generic UDP Encapsulation (GUE). Here's a good introduction to FOU, and the man page for ip-fou.
Or if you want an [open source] UDP based file transfer system there are things like UDT, UFTP, Tsunami-UDP, and even Google's QUIC (Now deprecated in favour of IETF QUIC).
Update: The QUIC protocol now has been standardised by the IETF which provides for secure reliable and unreliable transport over UDP as an alternative to TCP. There's a wide range of QUIC implementations available. There is also a growing set of protocol mappings on to QUIC such as HTTP/3, DNS over QUIC, etc
If my app requires TCP and my LAN has software setup to communicate
with another datacenter on the other side of the coast with software
setup on their end. Would it be possible to send the actual data via
UDP but than simulating TCP on both ends?
No. A UDP socket is in a different namespace from a TCP socket. You will be unable to write UDP at one end and send or receive TCP at the other end. TCP and UDP are peer protocols; both exist at the layer above IP. You can't use one to spoof the other.
Hmm, I believe so. You'd need to use a proxy at both ends, but it should be possible.
The biggest problem you are going to run into is that UDP is designed with the idea that you don't care if some of the packets don't ever make it to the other end.
Here's a link with some more info:
http://www.cyberciti.biz/faq/key-differences-between-tcp-and-udp-protocols/
IMHO, it's not a good idea to transmit files via UDP.
TCP's problems are in its algorithms, not its headers.
You certainly could implement the TCP algorithms on top of UDP. That would effectively be the same as tunneling TCP datagrams inside of UDP datagrams. But all this accomplishes is to add a few more bytes of overhead to each packet, and require another endpoint to unwrap the packets.
UDP itself is just thin shim on top of IP: its a convenient way to access IP packet switched networking without having to dive into kernels or receive special handling from routers. The main reason to implement reliable transport on top of UDP is to get away from TCP algorithms in favor of something more efficient. FileCatalyst was mentioned above as one company which does this, and my own company Data Expedition, Inc. does so as well.
So you could implement TCP algorithms on top of UDP, but you wouldn't want to.
You can simulate something like a connection over UDP, and you as well can add reliability checks and ordering and retransmission and so on. - but then, it still isn't TCP, it just acts the way.
Of course, one of the ends can be a kind of "hub" or "proxy" which does an adaption. Then you don't have a 2-end solution, but in fact a 4 end solution - one pair with "real" TCP and the other with the "self-knitted" "TCP" - which you put together with an appropriately crafted program.
This might be a silly question:
Does HTTP ever use the User Datagram Protocol?
For example:
If one is streaming MP3 or video using HTTP, does it internally use UDP for transport?
From RFC 2616:
HTTP communication usually takes place
over TCP/IP connections. The
default port is TCP 80, but other
ports can be used. This does not
preclude HTTP from being implemented
on top of any other protocol on the
Internet, or on other networks. HTTP
only presumes a reliable transport;
any protocol that provides such
guarantees can be used; the mapping
of the HTTP/1.1 request and response
structures onto the transport data
units of the protocol in question is
outside the scope of this
specification.
So although it doesn't explicitly say so, UDP is not used because it is not a "reliable transport".
EDIT - more recently, the QUIC protocol (which is more strictly a pseudo-transport or a session layer protocol) does use UDP for carrying HTTP/2.0 traffic and much of Google's traffic already uses this protocol. It's currently progressing towards standardisation as HTTP/3.
Typically, no.
Streaming is seldom used over HTTP itself, and HTTP is seldom run over UDP. See, however, RTP.
For something as your example (in the comment), you're not showing a protocol for the resource. If that protocol were to be HTTP, then I wouldn't call the access "streaming"; even if it in some sense of the word is since it's sending a (possibly large) resource serially over a network. Typically, the resource will be saved to local disk before being played back, so the network transfer is not what's usually meant by "streaming".
As commenters have pointed out, though, it's certainly possible to really stream over HTTP, and that's done by some.
Maybe just a bit of trivia, but UPnP will use HTTP formatted messages over UDP for device discovery.
Yes, HTTP, as an application protocol, can be transferred over UDP transport protocol.
Here are some of the services that use UDP and an underlying protocol for transferring HTTP data and streaming it to the end-user:
XMPP's Jingle Raw UDP Transport Method
A number for services that use UDT --- UDP-based Data Transfer Protocol, which is the a superset of UDP protocol.
The Transport Layer Security (TLS) protocol encapsulating HTTP as well as the above mentioned XMPP and other application protocols does have an implementation that uses UDP in its transport layer; this implementation is called Datagram Transport Layer Security (DTLS).
Push notifications in GNUTella are HTTP requests sent over UDP transport.
This article contains further details on streaming over UDP and its reliable superset, the RUDP: Reliable UDP (RUDP): The Next Big Streaming Protocol?
Of course, it doesn't necessarily have to be transmitted over TCP. I implemented HTTP on top of UDP, for use in the Satellite TV Broadcasting industry.
If you are streaming an mp3 or video that may not necessarily be over HTTP, in fact I'd be suprised if it was. It would probably be another protocol over TCP but I see no reason why you cannot stream over UDP.
If you do you have to take into account that there is no certainty that your data will arrive at the other end, but I can take it that you know about UDP.
To answer you question, No, HTTP does NOT use UDP.
For what you talk about though, mp3/video streaming COULD happen over UDP and in my opinion should never happen over HTTP.
Maybe some change on this topic with QUIC
QUIC (Quick UDP Internet Connections, pronounced quick) is an experimental transport layer network protocol developed by Google and implemented in 2013. QUIC supports a set of multiplexed connections between two endpoints over User Datagram Protocol (UDP), and was designed to provide security protection equivalent to TLS/SSL, along with reduced connection and transport latency, and bandwidth estimation in each direction to avoid congestion. QUIC's main goal is to optimize connection-oriented web applications currently using TCP.
I think some of the answers are missing an important point. The choice between UDP and TCP should not be based on the type of data (e.g., audio or video) or whether the application starts to play it before the transfer is completed ("streaming"), but whether it is real time. Real time data is (by definition) delay-sensitive, so it is often best sent over RTP/UDP (Real Time Protocol over UDP).
Delay is not an issue with stored data from a file, even if it's audio and/or video, so it is probably best sent over TCP so any packet losses can be corrected. The sender can read ahead and keep the network pipe full and the receiver can also use lots of playout buffering so it won't be interrupted by the occasional TCP retransmission or momentary network slowdown. The limiting case is where the entire recording is transferred before playback begins. This eliminates any risk of a playback stall, but is often impractical.
The problem with TCP for real-time data isn't retransmissions so much as excessive buffering as TCP tries to use the pipe as efficiently as possible without regard to latency. UDP preserves application packet boundaries and has no internal storage, so it does not introduce any latency.
(This is an old question, but it deserves an updated answer.)
In all likelihood, HTTP/3 will be using the QUIC protocol, which is described as
multiplexed transport over UDP
So, from a certain point of view, you could say that HTTP/3 will be using UDP.
The answer: Yes
Reason: See the OSI model.
Explaination:
HTTP is an application layer protocol, which could be encapsulated with a protocol that uses UDP, providing arguably faster reliable communication than TCP. The server daemon and client would obviously need to support this new protocol. Quake 2 protocol proves that UDP can be used over TCP to provide a basis for a structured communication system insuring flow control (e.g. chunk ids).
http over udp is used by some torrent tracker implementations (and supporteb by all main clients)
In theory yes it is possible to use UDP for http but that might be problematic. Say for instance in your example a mp3 or a video is being streamed there will be problem of ordering and some bits might go missing as UDP is not connection oriented there is no retransmit mechanism.
HTTP/3 (aka QUIC) uses UDP instead of TCP.
https://http3-explained.haxx.se/en/the-protocol/feature-udp
UDP is the best protocol for streaming, because it doesn't make demands for missing packages like TCP. And if it doesn't make demands, the flow is far more faster and without any buffering.
Even the stream delay is lesser than TCP. That is because TCP (as a far more secure protocol) makes demands for missing packages, overwriting the existing ones.
So TCP is a protocol too advanced to be used for streaming.