So we know that UDP is message-oriented and TCP is stream-oriented.
Why is the SMTP using TCP instead of UDP?
Why is the SMTP using TCP instead of UDP?
Mainly because TCP is reliable and UDP is not, i.e. with UDP the application would need to care about lost, duplicated or reordered data.
Also, a mail is not the same as a "message" in UDP. This UDP message just means a datagram with a specific length and no predefined inner structure. A mail has some inner structure though. The length of a mail can also easily exceed the maximum size of a UDP message (64k).
The usual rules for deciding whether to use TCP or UDP is as follows:
If you need all the features TCP provides that UDP doesn't, use TCP. You're not going to make a better TCP.
If there are features TCP provides that have high cost in your application that you can avoid by using UDP, use UDP.
So let's apply that logic to SMTP for the features TCP provides that UDP doesn't:
Connection oriented: Well, you need that for SMTP. You want to start a connection to deliver mail. You can't make an email a single chunk of data because they can several megabytes.
Slow start: Well, you need that for SMTP. You can't just send a bunch of emails at full speed, you'll get packet loss. You need to start sending data slowly and asses available network bandwidth.
Reordering, duplicate detection, and lost datagram retransmission: Well, you need that for SMTP. You need to retransmit dropped datagrams and you need to fix any parts of an email received out of order.
Send and receive windows: Well, you need that for SMTP. It's hard to transfer large amounts of data efficiently without that.
So if SMTP was build over UDP, SMTP itself would need to implement the majority of the feature set of TCP because SMTP needs it. TCP was designed by top experts, is heavily optimized on all popular platforms, and network engineers optimize networks for its performance. You're not going to beat that with something designed specifically for SMTP and it wouldn't make sense for every top layer protocol to implement its own version of all those features.
Related
I'm currently developping a highly responsive game in python using client-server model. The speed of the data exchanges between the client and the server needs to be very fast.
Is it possible to make both, udp and tcp connections ? Udp would be used only during the game. And Tcp will be used for more reliability-needed messages like connection, name changes, chat,...
Is it also a good way of thinking ? Or should i use only UDP ?
Yes, this can be a good idea. With UDP, a single lost packet does not stall the entire stream. On the other hand you need retry and congestion control.
I'd try to send messages using UDP and if no confirmation arrives within a short amount of time re-send them on a TCP connection that has been kept open. That should move 99% of the load to UDP and use TCP for congestion control and reliability.
The H2O database does it that way.
You can bind your UDP and TCP connections on different ports or even on the same port. As for which to use, it is up to you. Try both out and if TCP is too slow or UDP is too unreliable then you always have the option to switch.
You can also use a 3rd-party library that builds reliability layer over UDP and specify required reliability per packet. As an example you can check Raknet.
TCP has a greater computation overhead to ensure reliable delivery of packets. But, since modern networks are fast, is there any scenario in which performance of UDP outweighs the reliability of TCP?
Is there any other particular advantage of UDP over TCP?
I can see two cases, where UDP would have an upper hand over TCP.
First, one of the attractive features of UDP is that since it does not need to retransmit lost packets nor does it do any connection setup, sending data incurs less delay. This lower delay makes UDP an appealing choice for delay-sensitive applications like audio and video.
Second, multicast applications are built on top of UDP since they have to do point to multipoint. Using TCP for multicast applications would be hard since now the sender would have to keep track of retransmissions/sending rate for multiple receivers.
It depends on your usage. If your application is time sensitive, like Voice over IP, then you don't care about missing packets. What you care about is the delay in that case.
You should have a look at this answer: What are examples of TCP and UDP in real life?
You could also look at the Wikipedia related section: http://en.wikipedia.org/wiki/User_Datagram_Protocol#Comparison_of_UDP_and_TCP
Applications that require constant data flow, bulk data and which require fastness than reliability uses UDP over TCP.
udp provides better application level control over what data is sent....since the data is packaged in a udp segment and immediately passed over to the network layer......hence no-frills segment delivery service is observed.
There is no need for connection establishment hence no delay(unlike tcp...which requires handshaking before the actual data transfer)
There is no need to maintain connection state in the end systems(ie no need for send and receive buffers,congestion control parameters and sequence and acknowledgement number parameters)..hence more active clients could be supported
Small packet header overhead for udp(only 8 bytes) where as tcp has 20 bytes of header
Facebook uses UDP connections instead of TCP/IP to connect to theirs Memcached Servers
There are couple of differences of UDP over TCP.
First, TCP is connection-based whereas UDP is connectionless.
Connection-based: Make sure that all messages will arrive and arrive in the correct order.
Connectionless: It does not guarantee order or completeness.
Second, Here is why UDP is faster over TCP:
UDP does not require ACK message back
UDP has no flow control
No duplication verification at the receiving end
Shorter header
For a service which just returns a small number when queried such as 30, or 10, but would have to handle up to 5 or so requests at any instance, would TCP or UDP be a better protocol? I am leaning towards UDP, but I wanted some expert opinions. I am looking for relatively quick reply times as well. Could you tell me what the advantages of each would be for a service like this? Thanks.
TCP is a reliable connection-based protocol. So, you are guaranteed that data is sent/received - the packets are automatically re-sent if they are not verified to be received on the other end. However, there is the overhead of the three-way handshake for establishing the connection.
TCP is used for protocols like HTTP where there is a one-time exchange of information (the HTTP Request and Reply).
UDP is an unreliable connection-less protocol. So you can simply send / receive a packet but you have no (automatic, OS stack-provided) way of verifying that the other end got your message. If you care, you have to implement some kind of ACK yourself.
UDP is used often for more continuous, "streaming" type protocols. For example, many online multiplayer games use UDP to exchange game information to/from the host. They do this on a continual, periodic basis. So if a packet is lost, it's not really a big deal, because another update is just around the corner. It would be far worse for the gameplay if you had to wait for that (now stale) update to be re-transmitted.
DNS is also implemented over UDP.
Ultimately the choice is yours. I would probably default to TCP for most cases, and only use UDP in a scenario like I described.
I had a strange idea. I heard of software which from my understanding uses UDP to transfer files decreasing the overhead found in TCP packets.
If my app requires TCP and my LAN has software set up to communicate with another datacenter on the other side of the coast with software setup on their end. Would it be possible to send the actual data via UDP but than simulating TCP on both ends?
Does anyone have any ideas or information about such projects?
If you're asking if you can use UDP as a Layer 2, then the answer is yes, sort of. There are various protocols that allow you to create a tunnel to another network using a UDP transport, such as L2TP and even IPsec (with NAT traversal). You could also do it at the application layer.
If you're asking if TCP can be implemented in UDP, the answer is no. First, TCP packets and UDP packets have an incompatible format. Second, TCP and UDP have different protocol numbers (seen in the IP header) which means that TCP traffic destined for a UDP port would not be passed to the correct upper-layer protocol.
Both TCP and UDP are built on top of the IP, but the TCP uses different packet structure and at the layer-2 it is not possible to mimic the TCP using UDP packets.
Of course, if you have the control on both the source and destination, then it is possible to create a reliable UDP tunnel for the TCP packets. This would require some internal information (packet number, ack/nack flags) in the body of the UDP packet.
There is an interesting project http://udt.sourceforge.net/
It is a broadcast-capable reliable file transfer mechanism built on top the UDP.
PseudoTCP is a protocol which implements TCP algorithms on top of the UDP. It was introduced since the NAT traversal for TCP is much more complicated than UDP. But some P2P applications do need a reliable data transfer among nodes.
So far as I know, there are two PseudoTCP variations: Libjingle and Libnice.Libjingle is an open source library from google which was initially for gtalk. You could take a look at file sharing example from libjingle: https://developers.google.com/talk/libjingle/file_share. Recently, Chrome desktop also use PseudoTCP implementation from libjingle for reliable connections.
Yes, you can develop a protocol on UDP that simulates TCP. However, if you simulated TCP fully, it would technically have more overhead. Because TCP is implement as the packet and your simulated TCP is implemented in the body of the packet.
If you only need one or two features of TCP (such as basic ordering), then implementing it in UDP is useful.
Halo uses 2-3 (IIRC) UDP protocols that simulate different features of TCP, then full fledged TCP for initializing game-states. I Shot You First Networking, GDC publication
For example, in one case, they send 3 duplicate UDP packets to overcome packet loss.
If you control the software on both ends, and it is cost-effective to build your own protocol, then UDP can be versatile.
One way to do it now on Linux-3.18+ is to use Foo over UDP (FOU) which implements Generic UDP Encapsulation (GUE). Here's a good introduction to FOU, and the man page for ip-fou.
Or if you want an [open source] UDP based file transfer system there are things like UDT, UFTP, Tsunami-UDP, and even Google's QUIC (Now deprecated in favour of IETF QUIC).
Update: The QUIC protocol now has been standardised by the IETF which provides for secure reliable and unreliable transport over UDP as an alternative to TCP. There's a wide range of QUIC implementations available. There is also a growing set of protocol mappings on to QUIC such as HTTP/3, DNS over QUIC, etc
If my app requires TCP and my LAN has software setup to communicate
with another datacenter on the other side of the coast with software
setup on their end. Would it be possible to send the actual data via
UDP but than simulating TCP on both ends?
No. A UDP socket is in a different namespace from a TCP socket. You will be unable to write UDP at one end and send or receive TCP at the other end. TCP and UDP are peer protocols; both exist at the layer above IP. You can't use one to spoof the other.
Hmm, I believe so. You'd need to use a proxy at both ends, but it should be possible.
The biggest problem you are going to run into is that UDP is designed with the idea that you don't care if some of the packets don't ever make it to the other end.
Here's a link with some more info:
http://www.cyberciti.biz/faq/key-differences-between-tcp-and-udp-protocols/
IMHO, it's not a good idea to transmit files via UDP.
TCP's problems are in its algorithms, not its headers.
You certainly could implement the TCP algorithms on top of UDP. That would effectively be the same as tunneling TCP datagrams inside of UDP datagrams. But all this accomplishes is to add a few more bytes of overhead to each packet, and require another endpoint to unwrap the packets.
UDP itself is just thin shim on top of IP: its a convenient way to access IP packet switched networking without having to dive into kernels or receive special handling from routers. The main reason to implement reliable transport on top of UDP is to get away from TCP algorithms in favor of something more efficient. FileCatalyst was mentioned above as one company which does this, and my own company Data Expedition, Inc. does so as well.
So you could implement TCP algorithms on top of UDP, but you wouldn't want to.
You can simulate something like a connection over UDP, and you as well can add reliability checks and ordering and retransmission and so on. - but then, it still isn't TCP, it just acts the way.
Of course, one of the ends can be a kind of "hub" or "proxy" which does an adaption. Then you don't have a 2-end solution, but in fact a 4 end solution - one pair with "real" TCP and the other with the "self-knitted" "TCP" - which you put together with an appropriately crafted program.
I was told that for a High Frequency Trading (HFT) system that requires low-latency, TCP is used over UDP. I was told that with TCP you can make point to point connections, whereas you cannot with UDP, however from my understanding you can send UDP packets to specific IP/port.
There are several arguments used in this article as to why UDP > TCP for gaming but I can see relevance for HFT.
Why would TCP be a better protocol to use for HFT?
(Admins: My previous post of this question was silently removed with no explanation. If I am violating terms of use please alert me of this instead of silently removing the question)
UDP is superior to TCP if you don't need some of the features TCP provides. Every feature has a cost, and so if you don't need features, you are paying that cost for no reason.
In an HFT application, you need pretty much every feature TCP requires. So if you picked UDP, you'd have to implement those features yourself. That means you'd have to implement connection establishment, connection teardown, retransmissions, transmit pacing, windows, and so on.
If there was a way to do all those things that was better than the way TCP was doing it, TCP would be doing it that way. You'd have one hand tied behind your back because TCP is heavily optimized by some of the best minds on the planet and implemented in/with the kernel.
There's no reasons to expect a stream of data over an already-established TCP connection would be slower than the same data over UDP, plus you get checksumming, retries, and all the other TCP goodness. UDP mainly wins in cases where you can afford to discard the reliability or where the overhead of many TCP handshakes would be too expensive, such as with common DNS queries.
TCP is faster for when using a few connections, the important difference is that modern NICs perform significant amounts of acceleration on TCP and not really that much for UDP. This means there is more overhead to process each UDP packet and as such they cannot compete unless you need to send to multiple recipients simultaneously.
However the UDP multicast route still suffers the same problems as unicast UDP per datagram overheads. Therefore many HFT systems use hardware accelerated systems that can multiplex the streams across many NICs via TCP, example Solace.
These days though you want to completely bypass the kernel with say a userspace IP stack such as by Solarflare or Mellanox, or even skip both the kernel and IP stack with RDMA.
Quite simply, if you need connection reliability (ensuring that every byte of data transmitted is received), you should be using TCP regardless.
As you mentioned, UDP is more suitable for games, where 100% accurate real-time tracking of every object would use quite a large amount of bandwidth and is unnecessary (this is where slow connections encounter lag).
There is no special difference between a TCP port and a UDP port, beyond the type of connection being used (send the packet and forget it, UDP style, or negotiate a connection and sustain it, TCP style) and the service listening on the server side. e.g. TCP/25 would usually reveal a SMTP server, whereas UDP/25 would not.
Basically, modern TCP implementations are going to be just as fast as UDP, if you're keeping the connection alive. If TCP is having to resend a packet, you'd need to resend it in UDP too. Plus for UDP you're going to end up implementing the same reliability code (retransmission of dropped packets) that TCP has already implemented.