Related
I am using TCP for communicating with an arduino (just open a socket and wait for a connection)using an ethernet shield, While watching/reading about various other project that use some sort of network interface for communication they all seem to use UDP instead of TCP for communication. What I was wondering is what would be my gain if I use UDP instead?
A UDP stack is considerably simpler than a TCP stack. You can easily write a UDP stack from scratch on your own, TCP is a bit harder, doable but harder. TCP has built in retries and other things so you dont lose reliability with UDP directly, it is what you do with it that could compare. UDP is significantly faster than TCP and is why it is or was used for video and various things back in the day. Also things like video could stand to lose a packet here and there and didnt care. For embedded UDP is quite nice for being small, fast, etc. If you are using someone elses library then UDP is likely not going to save you much on memory/flash resources, it will still be a bit faster. It is when you implement your own UDP that you save quite a bit on memory, because you can cut corners. You can do things like only implement arp and udp and nothing else (although ping is useful but somehow painful), and you can cut corners on arp/rarp depending on what you need to do with this thing. You can implement support only for the packet size you are interested in. Numbering your packets and having the requesting side send two or three of everything and responding to every request can greatly reduce the lost packet problem. Keeping the packet size very small helps both the embedded resource problem and avoids any mtu or other problems along the way. For simplicity you can even force a specific packet length.
I always ask the question the other way, what would I stand to gain by using TCP. There are times when it is useful, embedded, desktop or server though I still ask that question every time and have to justify the use of TCP over UDP otherwise I wont use it.
You gain multicasting, but lose reliability.
You gain code space, data memory, and determinism.
A fair amount of memory is required to reassemble a TCP stream, unless you want to NAK every packet that's not in-order. They are never guaranteed to come in order....
An asynchronous command-response protocol with timeouts, where all commands and responses fit into a single UDP packet, and commands are idempotent (can be applied many times and maintain the correct result) is a pretty robust protocol.
I have just started writing socket programs. Came to know that single UDP packet has source port destination port and some MAC address representing router..etc. I wonder why anybody cannot create custom packets with a fake information in and send it over internet. I would like to know how safe are our PCs. What should be done to secure it ?
There are a couple of different aspects to the answer.
One is that the web relies on TCP, not UDP. Which means that it is connection-oriented. Your package will be rejected, unless it appears to be part of an existing connection (which means, among other things, that it has to have the right source IP and port as well. And it has to have the right sequence number to fit into the receive window). This can still be faked without too much trouble, of course. But it does require you to know a bit about the packets being sent on the original connection.
Another part is that whenever we need to be sure that the sender of a packet is who they claim to be, we use encryption. :)
Most packets don't really need this. It's not a huge deal if someone sends a request to Google which appears to come from my IP. But when making credit card transactions, it becomes a bit more important.
Most of the TCP/IP stack "leaks trust", as I once put it -- and there isn't much that you, as a software developer (assuming you're looking for a programming solution, otherwise, stackoverflow's the wrong forum, go to serverfault or superuser;-) can do about it -- beyond choosing and carefully implemented protocols that are reasonable in terms of security expectation.
HTTPS (with strong checks of certificates, etc) is one reasonably strong approach; for stronger security, look into SSH and VPN-based approaches. Of course, nobody should assume privacy or strong authentication is in place unless they've taken specific steps towards it (if they HAVE taken such steps, they may be still subject to successful attacks, which is why using existing, more or less "proven" solutions such as HTTPS, SSH, VPNs, is advisable;-).
Yes, anyone can create packets with whatever data they want and send them out over the internet. Especially with UDP, you can pretend to be anyone you want (unless your ISP does egress filtering). Source addresses for UDP cannot be trusted. Source addresses for TCP can to an extent (you know the data has to be coming from the IP address in question, or someone along the route).
Welcome to the internet :)
Edit: just to clarify egress filtering is something the sending ISP would have to do. As a reciever, there's not really anything you can do to verify the address on a UDP packet without communicating back to the sender. The only reason you can at least partially trust an incoming TCP connection is that TCP requires certain control data flow back to the sender (and hence needs a valid IP address/port to set the connection up and maintain it).
Well, many many people create invalid packets and send them over Internet; for instance, read Ping of death.
A [completly] secure computer is a computer turned off. To make your running PC more secure from this thread kind, you should rely on firewall softwares/hardwares, which can detect that malformed packets.
Custom packets with fake information can easily be created. Therefore you have to make sure you're not vulnerable to them.
I am considering using SCTP instead of TCP for a p2p app written in C. Should I do it? Also how does the speed of SCTP compare to the speed of TCP?
EDIT:
I found that SCTP can be tunneled over UDP with the only problem being tunneled SCTP is not interoperable with untunneled SCTP.
Have you considered whether your target systems will all have SCTP pre-installed on them or whether your application will need to include SCTP itself? In my experience I would not expect all systems to have SCTP installed on them, and I would expect them not to if it were Windows.
If you include SCTP in the application itself then that will more than double the number of messages being passed into an out of the Kernel which will impact performance when compared with using the pre installed TCP.
Have you considered what benefits you want from SCTP? You mentioned fault tolerance but for this to work with SCTP it requires the application to have multiple ethernet ports and and IP addresses. Is this likely on your app?
As much as I love SCTP (!) I would seriously consider sticking with TCP unless you are sure SCTP is needed or unless you control the hosts your app is deployed on.
Regards
If it's for a local area network, sure go for it.
Note however that if you plan to use it on the open internet many consumer grade firewalls aren't flexible enough to permit unrecognised IP protocols through them.
How does it help you?
You're P2P, so every peer must have at least one socket open to every other peer.
If you've got a socket open, then you can do everything you need to do over that. If you've taken the approach of one socket per file and you have multiple files being tranferred concurrently between two given peers, then SCTP will save you one socket per file. However, on a normal P2P network of any size, you will almost never have multiple files being transferred concurrently between two peers.
Just have one socket and have your own little protocol; send a packet with a header, the header indicates content type, e.g. a command, or part a file - and if so, which file, and which byte range.
Of course, you get a little overhead for that, whereas if you have one socket for commands and one per file, you're more efficient. Is saving one socket per peer (assuming one download at a time) worth the time/hassle/complexity of using SCTP?
How can I detect if another host is using the same MAC address as the current host, e.g. because the other host is spoofing?
I'm working in an embedded environment, so looking for answers on a protocol level, rather than “use such and such a tool”.
Edit: RARP does not solve this problem. For RARP to get any reply at all, there has to be at least one host on the segment which supports RARP. Since RARP is obsolete, modern operating systems don't support it. Furthermore, all RARP can do is tell you your own IP address - the response won't be any different if there’s another host on the segment with the same MAC, unless that host has itself used a different IP address.
This question is too interesting to put down! After several false starts I started thinking about the essential components of the problem and scoured the RFCs for advice. I haven't found a definitive answer, but here's my thought process, in the hope that it helps:
The original question asks how to detect another device with your MAC address. Assuming you're on an IP network, what's required to accomplish this?
The passive method would be simply to listen to traffic and look for any packets that you didn't transmit but have your MAC address. This may or may not occur, so although it can tell you definitively if a duplicate exists, it cannot tell you definitively that it doesn't.
Any active method requires you to transmit a packet that forces an impostor to respond. This immediately eliminates any methods that depend on optional protocols.
If another device is spoofing you, it must (by definition) respond to packets with your MAC address as the destination. Otherwise it's snooping but not spoofing.
The solution should be independent of IP address and involve only the MAC address.
So the answer, it seems, would be to transmit either a broadcast (ethernet) packet or a packet with your MAC address as its destination, that requires a response. The monkeywrench is that an IP address is usually involved, and you don't know it.
What sort of protocol fits this description?
Easy Answer:
If your network supports BOOTP or DHCP, you're done, because this authoritatively binds a MAC address to an IP address. Send a BOOTP request, get an IP address, and try to talk to it. You may need to be creative to force the packet onto the wire and prevent yourself from responding (I'm thinking judicious use of iptables and NAT).
Not-so-easy Answers:
A protocol that's independent of IP: either one that doesn't use the IP layer, or one that allows broadcasts. None comes to mind.
Send any packet that would normally generate a response from you, prevent yourself from responding, and look for a response from another device. It would seem sensible to use your IP address as the destination, but I'm not convinced of that. Unfortunately, the details (and, therefore the answer) are left as an exercise for the OP ... but I hope the discussion was helpful.
I suspect the final solution will involve a combination of techniques, as no single approach seems to guarantee a dependable determination.
Some information is available at http://en.wikipedia.org/wiki/ARP_spoofing#Defenses
If all else fails, you may enjoy this: http://www.rfc-editor.org/rfc/rfc2321.txt
Please post a follow-up with your solution, as I'm sure it will be helpful to others. Good luck!
You could send an ARP request for each possible ip in the subnet.
Of course the source address of the ARP request must be ff:ff:ff:ff:ff:ff, otherwise you might not see the response.
I forged a packet like this with bittwiste and replayed it with PReplay and all the hosts on the network got the response. (I don't know if these forged ARP packets are legal or not... some OSes might ignore them)
Here is what the forged package looked like:
Here is what the reply looked like:
If you watch the responses and see your MAC address in one of the packets (in the red rectangle) , than someone has the same MAC address as you do...
Unfortuantely I couldn't test the theory fully because none of my (Windows) machines care about me trying to set the nic's MAC address...
Two hosts using same MAC address on a single network segment would probably make switches go nuts and you could probably detect it by having an extremely unreliable network connection (as the switches would send some portion of packets that belong to your host to the second one, depending on which one of you sent the last packet in their direction).
This is very late, and a non-answer, but I wanted to follow up with exactly what I did in case anyone else is interested.
I was working with some very weird embedded hardware that doesn’t have a MAC address assigned at manufacture. That means we needed to assign one in software.
The obvious solution is to have the user pick a MAC address that they know is available on their network, preferably from the locally-administered range, and that’s what I did. However, I wanted to pick a reasonably safe default, and also attempt to warn the user if a conflict occurred.
In the end I resorted to picking a random-ish default in the locally-administered range, chosen by making some hardware readings that have moderate entropy. I deliberately excluded the beginning and end of the range on the assumption that those are moderately more likely to be chosen manually. The chances are that there will only be one of these devices on any given network, and certainly less than 20, so the chances of a conflict are very low, albeit not as low as they could be due to the somewhat predictable random numbers.
Given the low chances of there being a problem, and despite the excellent answers above, I decided to dispense with the conflict detection and make do with a warning to the user to look out for MAC conflict problems.
If I did decide to implement conflict detection, then given that I control the whole network stack, I would probably look out for excessive unknown or missing packets, and then trigger a change of MAC address or warn the user when that happens.
Hopefully that will help someone else somewhere – but probably not!
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
The community reviewed whether to reopen this question 1 year ago and left it closed:
Original close reason(s) were not resolved
Improve this question
For general protocol message exchange, which can tolerate some packet loss. How much more efficient is UDP over TCP?
People say that the major thing TCP gives you is reliability. But that's not really true. The most important thing TCP gives you is congestion control: you can run 100 TCP connections across a DSL link all going at max speed, and all 100 connections will be productive, because they all "sense" the available bandwidth. Try that with 100 different UDP applications, all pushing packets as fast as they can go, and see how well things work out for you.
On a larger scale, this TCP behavior is what keeps the Internet from locking up into "congestion collapse".
Things that tend to push applications towards UDP:
Group delivery semantics: it's possible to do reliable delivery to a group of people much more efficiently than TCP's point-to-point acknowledgement.
Out-of-order delivery: in lots of applications, as long as you get all the data, you don't care what order it arrives in; you can reduce app-level latency by accepting an out-of-order block.
Unfriendliness: on a LAN party, you may not care if your web browser functions nicely as long as you're blitting updates to the network as fast as you possibly can.
But even if you care about performance, you probably don't want to go with UDP:
You're on the hook for reliability now, and a lot of the things you might do to implement reliability can end up being slower than what TCP already does.
Now you're network-unfriendly, which can cause problems in shared environments.
Most importantly, firewalls will block you.
You can potentially overcome some TCP performance and latency issues by "trunking" multiple TCP connections together; iSCSI does this to get around congestion control on local area networks, but you can also do it to create a low-latency "urgent" message channel (TCP's "URGENT" behavior is totally broken).
In some applications TCP is faster (better throughput) than UDP.
This is the case when doing lots of small writes relative to the MTU size. For example, I read an experiment in which a stream of 300 byte packets was being sent over Ethernet (1500 byte MTU) and TCP was 50% faster than UDP.
The reason is because TCP will try and buffer the data and fill a full network segment thus making more efficient use of the available bandwidth.
UDP on the other hand puts the packet on the wire immediately thus congesting the network with lots of small packets.
You probably shouldn't use UDP unless you have a very specific reason for doing so. Especially since you can give TCP the same sort of latency as UDP by disabling the Nagle algorithm (for example if you're transmitting real-time sensor data and you're not worried about congesting the network with lot's of small packets).
UDP is faster than TCP, and the simple reason is because its non-existent acknowledge packet (ACK) that permits a continuous packet stream, instead of TCP that acknowledges a set of packets, calculated by using the TCP window size and round-trip time (RTT).
For more information, I recommend the simple, but very comprehensible Skullbox explanation (TCP vs. UDP)
with loss tolerant
Do you mean "with loss tolerance" ?
Basically, UDP is not "loss tolerant". You can send 100 packets to someone, and they might only get 95 of those packets, and some might be in the wrong order.
For things like video streaming, and multiplayer gaming, where it is better to miss a packet than to delay all the other packets behind it, this is the obvious choice
For most other things though, a missing or 'rearranged' packet is critical. You'd have to write some extra code to run on top of UDP to retry if things got missed, and enforce correct order. This would add a small bit of overhead in certain places.
Thankfully, some very very smart people have done this, and they called it TCP.
Think of it this way: If a packet goes missing, would you rather just get the next packet as quickly as possible and continue (use UDP), or do you actually need that missing data (use TCP). The overhead won't matter unless you're in a really edge-case scenario.
When speaking of "what is faster" - there are at least two very different aspects: throughput and latency.
If speaking about throughput - TCP's flow control (as mentioned in other answers), is extremely important and doing anything comparable over UDP, while certainly possible, would be a Big Headache(tm). As a result - using UDP when you need throughput, rarely qualifies as a good idea (unless you want to get an unfair advantage over TCP).
However, if speaking about latencies - the whole thing is completely different. While in the absence of packet loss TCP and UDP behave extremely similar (any differences, if any, being marginal) - after the packet is lost, the whole pattern changes drastically.
After any packet loss, TCP will wait for retransmit for at least 200ms (1sec per paragraph 2.4 of RFC6298, but practical modern implementations tend to reduce it to 200ms). Moreover, with TCP, even those packets which did reach destination host - will not be delivered to your app until the missing packet is received (i.e., the whole communication is delayed by ~200ms) - BTW, this effect, known as Head-of-Line Blocking, is inherent to all reliable ordered streams, whether TCP or reliable+ordered UDP. To make things even worse - if the retransmitted packet is also lost, then we'll be speaking about delay of ~600ms (due to so-called exponential backoff, 1st retransmit is 200ms, and second one is 200*2=400ms). If our channel has 1% packet loss (which is not bad by today's standards), and we have a game with 20 updates per second - such 600ms delays will occur on average every 8 minutes. And as 600ms is more than enough to get you killed in a fast-paced game - well, it is pretty bad for gameplay. These effects are exactly why gamedevs often prefer UDP over TCP.
However, when using UDP to reduce latencies - it is important to realize that merely "using UDP" is not sufficient to get substantial latency improvement, it is all about HOW you're using UDP. In particular, while RUDP libraries usually avoid that "exponential backoff" and use shorter retransmit times - if they are used as a "reliable ordered" stream, they still have to suffer from Head-of-Line Blocking (so in case of a double packet loss, instead of that 600ms we'll get about 1.5*2*RTT - or for a pretty good 80ms RTT, it is a ~250ms delay, which is an improvement, but it is still possible to do better). On the other hand, if using techniques discussed in http://gafferongames.com/networked-physics/snapshot-compression/ and/or http://ithare.com/udp-from-mog-perspective/#low-latency-compression , it IS possible to eliminate Head-of-Line blocking entirely (so for a double-packet loss for a game with 20 updates/second, the delay will be 100ms regardless of RTT).
And as a side note - if you happen to have access only to TCP but no UDP (such as in browser, or if your client is behind one of 6-9% of ugly firewalls blocking UDP) - there seems to be a way to implement UDP-over-TCP without incurring too much latencies, see here: http://ithare.com/almost-zero-additional-latency-udp-over-tcp/ (make sure to read comments too(!)).
Which protocol performs better (in terms of throughput) - UDP or TCP - really depends on the network characteristics and the network traffic. Robert S. Barnes, for example, points out a scenario where TCP performs better (small-sized writes). Now, consider a scenario in which the network is congested and has both TCP and UDP traffic. Senders in the network that are using TCP, will sense the 'congestion' and cut down on their sending rates. However, UDP doesn't have any congestion avoidance or congestion control mechanisms, and senders using UDP would continue to pump in data at the same rate. Gradually, TCP senders would reduce their sending rates to bare minimum and if UDP senders have enough data to be sent over the network, they would hog up the majority of bandwidth available. So, in such a case, UDP senders will have greater throughput, as they get the bigger pie of the network bandwidth. In fact, this is an active research topic - How to improve TCP throughput in presence of UDP traffic. One way, that I know of, using which TCP applications can improve throughput is by opening multiple TCP connections. That way, even though, each TCP connection's throughput might be limited, the sum total of the throughput of all TCP connections may be greater than the throughput for an application using UDP.
Each TCP connection requires an initial handshake before data is transmitted. Also, the TCP header contains a lot of overhead intended for different signals and message delivery detection. For a message exchange, UDP will probably suffice if a small chance of failure is acceptable. If receipt must be verified, TCP is your best option.
I will just make things clear. TCP/UDP are two cars are that being driven on the road. suppose that traffic signs & obstacles are Errors TCP cares for traffic signs, respects everything around. Slow driving because something may happen to the car. While UDP just drives off, full speed no respect to street signs. Nothing, a mad driver. UDP doesn't have error recovery, If there's an obstacle, it will just collide with it then continue. While TCP makes sure that all packets are sent & received perfectly, No errors , so , the car just passes obstacles without colliding. I hope this is a good example for you to understand, Why UDP is preferred in gaming. Gaming needs speed. TCP is preffered in downloads, or downloaded files may be corrupted.
UDP is slightly quicker in my experience, but not by much. The choice shouldn't be made on performance but on the message content and compression techniques.
If it's a protocol with message exchange, I'd suggest that the very slight performance hit you take with TCP is more than worth it. You're given a connection between two end points that will give you everything you need. Don't try and manufacture your own reliable two-way protocol on top of UDP unless you're really, really confident in what you're undertaking.
There has been some work done to allow the programmer to have the benefits of both worlds.
SCTP
It is an independent transport layer protolol, but it can be used as a library providing additional layer over UDP. The basic unit of communication is a message (mapped to one or more UDP packets). There is congestion control built in. The protocol has knobs and twiddles to switch on
in order delivery of messages
automatic retransmission of lost messages, with user defined parameters
if any of this is needed for your particular application.
One issue with this is that the connection establishment is a complicated (and therefore slow process)
Other similar stuff
https://en.wikipedia.org/wiki/Reliable_User_Datagram_Protocol
One more similar proprietary experimental thing
https://en.wikipedia.org/wiki/QUIC
This also tries to improve on the triple way handshake of TCP and change the congestion control to better deal with fast lines.
Update 2022: Quic and HTTP/3
QUIC (mentioned above) has been standardized through RFCs and even became the basis of HTTP/3 since the original answer was written. There are various libraries such as lucas-clemente/quic-go or microsoft/msquic or google/quiche or mozilla/neqo (web-browsers need to be implementing this).
These libraries expose to the programmer reliable TCP-like streams on top the UDP transport. RFC 9221 (An Unreliable Datagram Extension to QUIC) adds working with individual unreliable data packets.
Keep in mind that TCP usually keeps multiple messages on wire. If you want to implement this in UDP you'll have quite a lot of work if you want to do it reliably. Your solution is either going to be less reliable, less fast or an incredible amount of work. There are valid applications of UDP, but if you're asking this question yours probably is not.
If you need to quickly blast a message across the net between two IP's that haven't even talked yet, then a UDP is going to arrive at least 3 times faster, usually 5 times faster.
It is meaningless to talk about TCP or UDP without taking the network condition into account.
If the network between the two point have a very high quality, UDP is absolutely faster than TCP, but in some other case such as the GPRS network, TCP may been faster and more reliability than UDP.
The network setup is crucial for any measurements. It makes a huge difference, if you are communicating via sockets on your local machine or with the other end of the world.
Three things I want to add to the discussion:
You can find here a very good article about TCP vs. UDP in the
context of game development.
Additionally, iperf (jperf enhance iperf with a GUI) is a
very nice tool for answering your question yourself by measuring.
I implemented a benchmark in Python (see this SO question). In average of 10^6 iterations the difference for sending 8 bytes is about 1-2 microseconds for UDP.