I try to make a P2P Program and need help with getting through the NAT of the clients.
I have read many questions here on stackoverflow, but i never got what the drawbacks and benefits of all the Methods to get through a NAT are.
How many routers support which methods?
Which methods are commonly used by the big companies? (BitTorrent, TeamViewer,...)
What are the drawbacks/benefits of the three methods i listed and which else exists?
UPnP requires software support in your router. Even when routers have it, it's sometimes turned off due to security concerns.
STUN uses a simple public server to detect the presence of NAT and the real IP addresses. This is useful for other, higher-level protocols that do the useful work.
Hole punching is a simple technique to connect two computers. It requires a 3rd public computer to communicate between the 2 computers behind NAT. You could run your own server with a custom protocol, our you could use STUN and existing STUN servers. (Only a few packets are needed for setup, the rest are peer-to-peer).
Of course, some NAT firewalls are incompatible with the above, so you may need a custom protocol. Bittorrent does this: they use different techniques depending on which clients are behind firewalls and whatnot.
Related
As far as I understand, ICE protocol is used for discovering the nodes/devices from the end-user device to "the outside".
I don't understand why it's needed. Isn't packet-routing is the responsibility of network devices like routers and switches? They should find the shortest path from the gateway to the end-user device (Actually, routers remembers those routes they previously discovered).
Moreover, NAT protocol is used to convert from an "internal ip" to "external ip" and vice-versa.
So again,
Why does the other user needs to be familiar with my internal network setup?
NAT is a kludge, put in place to try to conserve IPv4 addresses until IPv6 becomes ubiquitous, and it breaks the end-to-end connectivity which is the promise of IP. Because of that, some things don't work correctly through NAT. There are various kludges to work around the NAT kludge, and ICE is part of that. This is explained in RFC 5245, Interactive Connectivity Establishment (ICE): A Protocol for Network Address Translator (NAT) Traversal for Offer/Answer Protocols:
Introduction
RFC 3264 [RFC3264] defines a two-phase exchange of Session Description
Protocol (SDP) messages [RFC4566] for the purposes of establishment of
multimedia sessions. This offer/answer mechanism is used by protocols
such as the Session Initiation Protocol (SIP) [RFC3261].
Protocols using offer/answer are difficult to operate through Network
Address Translators (NATs). Because their purpose is to establish a
flow of media packets, they tend to carry the IP addresses and ports
of media sources and sinks within their messages, which is known to be
problematic through NAT [RFC3235]. The protocols also seek to create
a media flow directly between participants, so that there is no
application layer intermediary between them. This is done to reduce
media latency, decrease packet loss, and reduce the operational costs
of deploying the application. However, this is difficult to
accomplish through NAT. A full treatment of the reasons for this is
beyond the scope of this specification.
Numerous solutions have been defined for allowing these protocols to
operate through NAT. These include Application Layer Gateways (ALGs),
the Middlebox Control Protocol [RFC3303], the original Simple
Traversal of UDP Through NAT (STUN) [RFC3489] specification, and Realm
Specific IP [RFC3102] [RFC3103] along with session description
extensions needed to make them work, such as the Session Description
Protocol (SDP) [RFC4566] attribute for the Real Time Control Protocol
(RTCP) [RFC3605]. Unfortunately, these techniques all have pros and
cons which, make each one optimal in some network topologies, but a
poor choice in others. The result is that administrators and
implementors are making assumptions about the topologies of the
networks in which their solutions will be deployed. This introduces
complexity and brittleness into the system. What is needed is a
single solution that is flexible enough to work well in all
situations.
This specification defines Interactive Connectivity Establishment
(ICE) as a technique for NAT traversal for UDP-based media streams
(though ICE can be extended to handle other transport protocols, such
as TCP [ICE-TCP]) established by the offer/answer model. ICE is an
extension to the offer/answer model, and works by including a
multiplicity of IP addresses and ports in SDP offers and answers,
which are then tested for connectivity by peer-to-peer connectivity
checks. The IP addresses and ports included in the SDP and the
connectivity checks are performed using the revised STUN specification
[RFC5389], now renamed to Session Traversal Utilities for NAT. The
new name and new specification reflect its new role as a tool that is
used with other NAT traversal techniques (namely ICE) rather than a
standalone NAT traversal solution, as the original STUN specification
was. ICE also makes use of Traversal Using Relays around NAT (TURN)
[RFC5766], an extension to STUN. Because ICE exchanges a multiplicity
of IP addresses and ports for each media stream, it also allows for
address selection for multihomed and dual- stack hosts, and for this
reason it deprecates RFC 4091 [RFC4091] and [RFC4092].
Firewalls. They're typically configured to bounce any unsolicited traffic from the world wide web to you. They only approve of you initiating contact with a server, which only then is allowed to back-traffic to you, and that's pretty much it. Unless your friends all own static IPs (which few people can justify) this is a hostile environment for peer to peer communication.
ICE tries to solve this, by enumerating addresses and ports at which the other side may be reached, and trying to connect to these addresses, by initiating outbound requests on both ends, or if all else fails, falling back to communicating through a TURN server, if specified.
See this WebRTCHacks article for more on the problem.
Why does the other user needs to be familiar with my internal network setup?
Because the other user is sometimes on your internal network. e.g. LAN games.
I wonder if there exist any other technologies used to establish internet connection between applications. Are there any other? I am searching and so far I haven't found anything else described.
There are many abstractions on top of sockets, if you don't want to deal directly with a socket API. UDP, TCP/IP, various RPC protocols, HTTP (which is on top of TCP/IP), etc. Many programming languages have easy methods of doing, say, an HTTP request and getting the resulting document. You can use that to allow applications to talk to each other over the internet without using a socket API.
What are you trying to accomplish?
If you want to skip sockets you basically have to implement your own means of talking to the network card hardware and telling it to communicate with other devices. A socket is just the abstraction chosen for *nix and Windows machines.
I am working in a project to migrate a firewall application from IPv4 to IPv6. I have several questions:
What changes and modifications might be needed?
Will the popular protocols such as FTP, HTTP, POP3 also need to be adapted/modified?
Which IPv6 components should or must be implemented?
Which tunneling/transition mechanism to prefer?
As I am new to this network security field, I hope you guys could give me some valuable input. Thanks in advance.
There are a lot of things to consider. Off the top of my head:
Learn the difference between link-local (fe80::/10), global unicast, and multicast address ranges. Make sure you support interface scoping with link-local addresses (you will see addresses like fe80::1%eth1, which will indicate the link-local address on the eth1 interface).
ARP equivalent (IPv6 neighbor discovery) is now part of ICMP. This is important because if the user wants to block ICMP packets and isn't careful, they could lose all their connectivity!
Most (sane) protocols will not need major changes. FTP is one protocol that will potentially need changes, since it sometimes passes network addresses within the protocol itself (rather than letting the lower-level protocols take care of it)
The most basic tunneling/transition mechanism you will need is called 6in4; it simply encapsulates IPv6 packets within IPv4 packets and allows the user to manually configure the endpoints of the tunnel. Automatic tunneling mechanisms like 6to4 and Teredo can also be useful in some situations.
If you are selling a commercial product, I recommend you take a look at the USGv6 test selection tables. Also, read through the USGv6 profile which has pointers to many of the RFCs you will need to understand in order to develop an IPv6-compliant product. Not supporting the USGv6 profile for a network protection device (NPD) could severely limit your market. Finally, get some training! IPv6 is vastly different from IPv4 in many ways. If your employer wants this project to succeed, training will be critical given that it appears that many project members are new to both IPv6 and network security. (do you have a mentor on the team to ask questions?)
I've seen and read a lot of similar questions, and the corresponding Wikipedia articles (NAT traversal, STUN, TURN, TCP hole punching), but the overwhelming amount of information doesn't really help me with my very simple problem:
I'm writing a P2P application, and I want two users of my application behind NAT to be able to connect to each other. The connection must be reliable (comparable to TCP's reliability) so I can't just switch to UDP. The solution should work on today's common systems without reconfiguration. If it helps, the solution may involve a connectible 3rd-party, as long as it doesn't have to proxy the entire data (for example, to get the peers' external (WAN) IP addresses).
As far as I know, my only option is to use a "reliable UDP" library + UDP hole punching. Is there a (C/C++) library for this? I found enet in a related question, but it only takes care of the first half of the solution.
Anything else? Things I've looked at:
Teredo tunnelling - requires support from the operating system and/or user configuration
UPnP port forwarding - UPnP isn't present/enabled everywhere
TCP hole punching seems to be experimental and only work in certain circumstances
SCTP is even less supported than IPv6. SCTP over UDP is just fancy reliable UDP (see above)
RUDP - nearly no mainstream support
From what I could understand of STUN, STUNT, TURN and ICE, none of them would help me here.
ICE collects a list of candidate IP/port targets to which to connect. Each peer collects these, and then each runs a connectivity check on each of the candidates in order, until either a check passes or a check fails.
When Alice tries to connect to Bob, she somehow gets a list of possible ways - determined by Bob - she may connect to Bob. ICE calls these candidates. Bob might say, for example: "my local socket's 192.168.1.1:1024/udp, my external NAT binding (found through STUN) is 196.25.1.1:4454/udp, and you can invoke a media relay (a middlebox) at 1.2.3.4:6675/udp". Bob puts that in an SDP packet (a description of these various candidates), and sends that to Alice in some way. (In SIP, the original use case for ICE, the SDP's carried in a SIP INVITE/200/ACK exchange, setting up a SIP session.)
ICE is pluggable, and you can configure the precise nature/number of candidates. You could try a direct link, followed by asking a STUN server for a binding (this punches a hole in your NAT, and tells you the external IP/port of that hole, which you put into your session description), and falling back on asking a TURN server to relay your data.
One downside to ICE is that your peers exchange SDP descriptions, which you may or may not like. Another is that TCP support's still in draft form, which may or may not be a problem for you. [UPDATE: ICE is now officially RFC 6544.]
Games often use UDP, because old data is useless. (This is why RTP usually runs over UDP.) Some P2P applications often use middleboxes or networks of middleboxes.
IRC uses a network of middleboxes: IRC servers form networks, and clients connect to a near server. Messages from one client to another may travel through the network of servers.
Failing all that, you could take a look at BitTorrent's architecture and see how they handle the NAT problem. As CodeShadow points out in the comments below, BitTorrent relies on reachable peers in the network: in a sense some peers form a network of middleboxes. If those middleboxes could act as relays, you'd have an IRC-like architecture, but one that's set up dynamically.
I recommend libjingle as it is used by some major video game companies which heavily relies on P2P network communication. (Have you heard about Steam? Vavle also uses libjingle , see the "Peer-to-peer networking" session in the page: https://partner.steamgames.com/documentation/api)
However, the always-work-solution would be using a relay server. Since there is no "standard" way to go through NAT, you should have this relay server option as a fall-back strategy if a connection has to be always established between any peers.
I am trying to implement a peer to peer communication ..
I use sockets for communication between them. I want to know if there are any ways i can use the same sort of communication when the two peers are behind two private Networks (They dont know each others public ip address.) I can think of a shared buffer in a well known location as a means of communication. But is there some other way to get this done?
You may also have to consider the case where two peers don't even have public IP addresses, in which case you could use STUN techniques to avoid the need for a central server. STUN has been used effectively in products such as Skype (I think), but it remains quite a black art at this stage.
For a central approach, consider XMPP, of which ejabberd is a very popular implementation. Various other messaging protocols exist as well, such as the emerging AMPQ standard.
These technologies are likely to be more robust and scalable than a home-grown socket-based protocol.
STUN and TURN are the common ways to achieve this as already posted and are standards based approaches. However, I have seen a really interesting approach to doing this using icmp messages recently. It's called pwnat, and available here
Since both are in different private networks, both peers cannot communicate with each other.
You need to have a public server in place, so that you can establish connection via that server.
Refer resiprocate , Resiprocate for details