how to use "udptunnel" so that i will be able to send and receive udp packets over tcp
probably with an example
You start udptunnel on two PCs in different networks and connect them as decribed here: http://www.cs.columbia.edu/~lennox/udptunnel/ with an example: http://wiki.leipzig.freifunk.net/Udptunnel
Related
topology
This is my experimental setup in Mininet. VM1 and VM2 are separate Virtualbox VM instances running on my computer connected by Bridged adapter, and S1 and S2 are connected with vxlan forwarding.
Then I used D-ITG on H1 and H2 to generate traffic. I send TCP traffic from H1 to H2 and use wireshark to capture. During a 10sec TCP flow, I used a python script that changes the tunnel id of the first rule on S1 from 100 to 200.
If the packet/sec rate and payload size is small enough, the TCP session does not seem to be affected, but when I start sending around 100 packet/sec each with payload of 64 bytes, TCP stop sending after receiving a dup ACK. Here is the wireshark capture:
wireshark1
wireshark2
On the link between H1 and S1 I received ICMP destination unreachable (fragmentation needed).
After the two errors, TCP stopped sending. I understand that the "previous segment not captured" is caused by the fact that when I alter the S1 routing table, there is some down time and packets are dropped by the switch. However, I don't understand why TCP does not initiate retransmission.
This does not happen if I reduce the packet rate or the payload to a smaller amount, or if I use UDP. Is this an issue with the TCP stack, or maybe D-ITG? Or maybe it is an issue with the sequence numbers? Is there a range where if very previous packets are not ACKed, they will not be retransmitted?
This problem has been bothering me for a while, so I hope someone here can maybe provide some clarification. Thanks a lot for reading XD.
I suspected it may be a problem with mininet NICs, so I tried to disable TCP fragmentation offload, and it worked much better. I suppose that the virtual NICs in mininet in a VM could not handle the large amount of traffic generated by D-ITG, so using TCP fragmentation offload can overload? the NIC and cause segmentation errors.
This is just my speculation, but disabling TSO did help my case. Additional input is welcomed!
I want the PING package to go through the proxy program, but the program only supports forwarding packets from the TCP layer and does not support forwarding ICMP packets.
Is there any solution?
If you need a tool, you can use a software like psping, paping and tcping. They test time response similar as ICMP Ping but over TCP.
If you need to implement by hand, you may use SYN package and its response (SYN ACK) to measure. For C#, this thread is discussing the implementation.
I've seen while searching that it is possible to use two different programs on the same computer communicating over the network using the same port and same network interface provided one use UDP and the other TCP. However I didn't get a good explanation, how does it actually work and why this is possible?
Is it also possible for multiple programs to use the same UDP port since UDP does not establish a real connection between the peers, but just sends the packets to an address? I understand it's not possible with TCP as it creates a synchronized connection between the server and the client, but what about UDP?
Please explain in details if possible, or link a good article on the topic.
The other answers are correct but somewhat incomplete.
An IP (aka "INET") socket "connection" (i.e. communication between two processes, possibly on different machines) is defined by a 5-tuple: protocol, source address, source port, destination address, destination port. You can see that this is not limited to a stateful connection such as TCP.
This means that you can bind different processes to any unique instance of that 5-tuple. Because the "protocol" (e.g. TCP and UDP) is part of the differentiating factor, each can have a different process.
Theoretically, you could bind different services to the same TCP port if they bind to different interfaces (network cards, loopback, etc.) though I've never tried it.
It is standard practice, however, to always use the same service on the same port number. If both UDP and TCP are supported, they're just different ways of communicating with that same service. DNS, for example, uses UDP on port 53 for lookup because they are small requests and it's faster than creating a TCP connection but DNS also uses TCP on port 53 for "transfers" which are infrequent and can have large amounts of data.
Lastly, in complete accuracy, it isn't necessarily a 5-tuple. IP uses the "protocol" to pass to the next layer such as TCP and UDP though there are others. TCP and UDP each seperately differentiate connections based on the remaining 4 items. It's possible to create other protocols on top of IP that use completely different (perhaps port-less) differentiation mechanisms.
And then there are different socket "domains", such as the "unix" socket domain, which is completely distinct from "inet" and uses the filesystem for addressing.
The destination isn't identified by IP Addr:Port alone. There is another thing - IP header has a field called Protocol which differentiates the TCP and UDP endpoint. As such it becomes possible for two process to bind to same IP:Port as long as communication protocol is different.
The endpoint of a connection is for UDP and TCP defined by IP, protocol (TCP or UDP) and port. This means as long as you use a different protocol the endpoint of the communication is different too.
Because they are not the only component of the means of address. It's the same as why you can have two houses with the same number on different streets, or why you know John Whorfin is not the same Red Lectroid as John Bigbooté.
Each IP packet contains a field that says which transport-layer protocol is to be used, and within the domain of that protocol is a set of ports that can be the same as in any other protocol because they are actually a completely separate set.
As for the second question, there are answers elsewhere.
I had a strange idea. I heard of software which from my understanding uses UDP to transfer files decreasing the overhead found in TCP packets.
If my app requires TCP and my LAN has software set up to communicate with another datacenter on the other side of the coast with software setup on their end. Would it be possible to send the actual data via UDP but than simulating TCP on both ends?
Does anyone have any ideas or information about such projects?
If you're asking if you can use UDP as a Layer 2, then the answer is yes, sort of. There are various protocols that allow you to create a tunnel to another network using a UDP transport, such as L2TP and even IPsec (with NAT traversal). You could also do it at the application layer.
If you're asking if TCP can be implemented in UDP, the answer is no. First, TCP packets and UDP packets have an incompatible format. Second, TCP and UDP have different protocol numbers (seen in the IP header) which means that TCP traffic destined for a UDP port would not be passed to the correct upper-layer protocol.
Both TCP and UDP are built on top of the IP, but the TCP uses different packet structure and at the layer-2 it is not possible to mimic the TCP using UDP packets.
Of course, if you have the control on both the source and destination, then it is possible to create a reliable UDP tunnel for the TCP packets. This would require some internal information (packet number, ack/nack flags) in the body of the UDP packet.
There is an interesting project http://udt.sourceforge.net/
It is a broadcast-capable reliable file transfer mechanism built on top the UDP.
PseudoTCP is a protocol which implements TCP algorithms on top of the UDP. It was introduced since the NAT traversal for TCP is much more complicated than UDP. But some P2P applications do need a reliable data transfer among nodes.
So far as I know, there are two PseudoTCP variations: Libjingle and Libnice.Libjingle is an open source library from google which was initially for gtalk. You could take a look at file sharing example from libjingle: https://developers.google.com/talk/libjingle/file_share. Recently, Chrome desktop also use PseudoTCP implementation from libjingle for reliable connections.
Yes, you can develop a protocol on UDP that simulates TCP. However, if you simulated TCP fully, it would technically have more overhead. Because TCP is implement as the packet and your simulated TCP is implemented in the body of the packet.
If you only need one or two features of TCP (such as basic ordering), then implementing it in UDP is useful.
Halo uses 2-3 (IIRC) UDP protocols that simulate different features of TCP, then full fledged TCP for initializing game-states. I Shot You First Networking, GDC publication
For example, in one case, they send 3 duplicate UDP packets to overcome packet loss.
If you control the software on both ends, and it is cost-effective to build your own protocol, then UDP can be versatile.
One way to do it now on Linux-3.18+ is to use Foo over UDP (FOU) which implements Generic UDP Encapsulation (GUE). Here's a good introduction to FOU, and the man page for ip-fou.
Or if you want an [open source] UDP based file transfer system there are things like UDT, UFTP, Tsunami-UDP, and even Google's QUIC (Now deprecated in favour of IETF QUIC).
Update: The QUIC protocol now has been standardised by the IETF which provides for secure reliable and unreliable transport over UDP as an alternative to TCP. There's a wide range of QUIC implementations available. There is also a growing set of protocol mappings on to QUIC such as HTTP/3, DNS over QUIC, etc
If my app requires TCP and my LAN has software setup to communicate
with another datacenter on the other side of the coast with software
setup on their end. Would it be possible to send the actual data via
UDP but than simulating TCP on both ends?
No. A UDP socket is in a different namespace from a TCP socket. You will be unable to write UDP at one end and send or receive TCP at the other end. TCP and UDP are peer protocols; both exist at the layer above IP. You can't use one to spoof the other.
Hmm, I believe so. You'd need to use a proxy at both ends, but it should be possible.
The biggest problem you are going to run into is that UDP is designed with the idea that you don't care if some of the packets don't ever make it to the other end.
Here's a link with some more info:
http://www.cyberciti.biz/faq/key-differences-between-tcp-and-udp-protocols/
IMHO, it's not a good idea to transmit files via UDP.
TCP's problems are in its algorithms, not its headers.
You certainly could implement the TCP algorithms on top of UDP. That would effectively be the same as tunneling TCP datagrams inside of UDP datagrams. But all this accomplishes is to add a few more bytes of overhead to each packet, and require another endpoint to unwrap the packets.
UDP itself is just thin shim on top of IP: its a convenient way to access IP packet switched networking without having to dive into kernels or receive special handling from routers. The main reason to implement reliable transport on top of UDP is to get away from TCP algorithms in favor of something more efficient. FileCatalyst was mentioned above as one company which does this, and my own company Data Expedition, Inc. does so as well.
So you could implement TCP algorithms on top of UDP, but you wouldn't want to.
You can simulate something like a connection over UDP, and you as well can add reliability checks and ordering and retransmission and so on. - but then, it still isn't TCP, it just acts the way.
Of course, one of the ends can be a kind of "hub" or "proxy" which does an adaption. Then you don't have a 2-end solution, but in fact a 4 end solution - one pair with "real" TCP and the other with the "self-knitted" "TCP" - which you put together with an appropriately crafted program.
is it possible to send multiple tcp or udp packets on a single ip packet? are there any specifications in the protocol that do not allow this.
if it is allowed by the protocol but is generally not done by tcp/udp implementations could you point me to the relevant portion in the linux source code that proves this.
are there any implementations of tcp/udp on some os that do send multiple packets on a single ip packet. (if it is allowed).
It is not possible.
The TCP seqment header does not describe its length. The length of the TCP payload is derived from the length of the IP packet(s) minus the length of the IP and TCP headers. So only one TCP segment per IP packet.
Conversely, however, a single TCP segment can be fragmented over several IP packets by IP fragmentation.
Tcp doesn't send packets: it is a continuous stream. You send messages.
Udp, being packet based, will only send one packet at a time.
The protocol itself does not allow it. It won't break, it just won't happen.
The suggestion to use tunneling is valid, but so is the warning.
You might want to try tunneling tcp over tcp, although it's generally considered a bad idea. Depending on your needs, your mileage may vary.
You may want to take a look at the Stream Control Transmission Protocol which allows multiple data streams across a single TCP connection.
EDIT - I wasn't aware that TCP doesn't have it's own header field so there would be no way of doing this without writing a custom TCP equivalent that contains this info. SCTP may still be of use though so I'll leave that link.
TCP is a public specification, why not just read it?
RFC4164 is the roadmap document, RFC793 is TCP itself, and RFC1122 contains some errata and shows how it fits together with the rest of the (IPv4) universe.
But in short, because the TCP header (RFC793 section 3.1) does not have a length field, TCP data extends from the end of the header padding to the end of the IP packet. There is nowhere to put another data segment in the packet.
You cannot pack several TCP packets into one IP packet - that is a restriction of specification as mentioned above. TCP is the closest API which is application-oriented. Or you want to program sending of raw IP messages? Just tell us, what problem do you want to solve. Think about how you organize the delivery of the messages from one application to another, or mention that you want to hook into TCP/IP stack. What I can suggest you:
Consider packing whatever you like into UDP packet. I am not sure, how easy is to initiate routing of "unpacked" TCP packages on remote side.
Consider using PPTP or similar tunnelling protocol.