I am trying to connect to peer via add_peer() function in LibTorrent. But what if the peer from which I want to download the file is behind NAT? Is there something for NAT Traversal in Libtorrent?
The NAT traversal in libtorrent is limited to:
Explicit port forwards using UPnP, NAT-PMP and PCP.
Implicit (opportunistic) attempts to reach peers via their external port
The peer receiving the connection attempt is not behind a NAT, but the initiating peer is. This is the case NATs are meant to support.
It sounds like you're mostly interested in (2), where we assume both peers are behind a NAT. This is commonly referred to UDP hole-punching.
Generally, if you don't control or have any influence over the peer you're trying to connect to, you're limited in what measures you can take.
Also, if the neither NAT is a full-cone (or let's say, p2p-friendly) it may not be possible for the peers to connect. A p2p-friendly NAT generally accepts incoming connections from IPs they have not had any interaction with previously.
The main two approaches used by libtorrent (and bittorrent clients generally) are:
commonly connected peers may introduce two NATed peers to each other via the peer exchange extension. In this mode both peers try to connect to each other simultaneously, hoping that both NATs will open up pin-holes for the ports that are being attempted. This only works if the swarm has at least one peer that's not behind a NAT. You can find more information about this in BEP 55
Sharing the UDP port for uTP, DHT and UDP trackers and having the listen port be implied by the source port of the tracker and DHT announce. With some luck, that source port can also be used by other hosts to reach the NATes client. This works because uTP connections also run over UDP.
Related
I started to explore the option of connecting with other using a p2p connection, so I coded a simple socket program in JAVA for android devices in which the users can share simple messages p2p (I didn't have any idea about NAT then). I got to know about NAT, so I now need to establish a TCP connection with another user which uses a server for discovery but payload is transferred p2p. I have also looked at XMPP(a very good and detailed explanation of how protocol works is here) and UPnP but I dont know how to implement them.
Another interesting question that arises is of BitTorrent because they can work on any device and even behind a NAT. I am not able to get any explanation of how BitTorrent works.
I have researched a lot but I am stuck.
My questions are:
A detailed explanation of BitTorrent(like here, not how torrents work) and how is it able to work around NAT ?
Is there a way to make a NAT entry programmatically ?
Is socket programming sufficient for p2p ?
How difficult is it to create your own protocol and how can I build one ?
If two devices D1 and D2 want to communicate p2p and they know each other's IP. D1 sends a request to D2 and that can't get through the D2's NAT, but there should be an entry created in D1's NAT. So when D2 tries to send something D1's NAT should discover an entry with D2's IP. Then why is the packet not allowed by it ?
Another interesting question that arises is of BitTorrent because they can work on any device and even behind a NAT. I am not able to get any explanation of how BitTorrent works.
This statement looks like you assume that bittorrent needs full connectivity to operate.
That is incorrect.
Behind a NAT device you will still be able to establish outgoing TCP connections. Which generally is sufficient for bittorrent as long as there are other, non-NATed (or NATed but properly port-forwarded) clients in the network that can accept incoming connnections.
NAT has no impact on the flow direction of the data because connections are bi-directional once they are established. It only is problematic for the initial connection setup.
This works perfectly fine for bittorrent because bittorent does not care from which specific node you get your data.
Although better connectivity generally does improve performance.
If the identity of the node matters or one-on-one transfers are an important use-case then other p2p protocols usually attempt NAT traversal first and if that fails rely on 3rd party nodes relaying traffic between those nodes who cannot connect to each other directly.
Additionally, IPv6 support will become essential in the future to maintain end-to-end connectivity because more and more ISPs are starting to roll out carrier-grade NAT for IPv4 while IPv6 will remain non-NATed
One thing need to be clear is that 100% P2P between all type of NAT is impossible right now. There is no practical way to establish P2P connectivity between **Symmetric and Symmetric/PRC NAT. In this scenario connection is established through a relay server called TURN.
I am answering from your 2nd question because I don't know much about the first one.
2) Yes. You can send a packet through your NAT and there will be a mapping between your internal IP:Port to your NAT's external IP:Port. You can know these external IP:Port by sending a stun request. Note that this technique doesn't work for Symmetric NAT.
3)Yes socket programming sufficient for p2p.
4)Why do you need a protocol when there already exists several. ICE protocol is the best today for NAT traversal and I don't think it was easy to create. UPnP and NAT-PMP is really vulnerable in terms of security.
5)I think what happens is usually NAT blocks unknown packets coming to it. So when D1 sends a packet to D2, its NAT blocks all packets incoming from D1s IP:Port. That is why connection establishment fails. You have to employ hole punching technique for D1 and D2 to successfully establish P2P connectivity.
**By symmetric NAT I mean symmetric NAT with random port allocation.
There is a paper on "Peer-to-Peer Communication Across Network Address Translators" which describes the UDP hole punching method and extends it to be used over TCP as well.
Of course, you will always need a relay server for the cases where hole punching is not supported.
Recent versions of BitTorrent use µTP, which is layered above UDP, not TCP. µTorrent uses a private extension (ut_holepunch) that performs UDP hole punching, most other implementations don't bother (with the notable exception of Tixati).
Some NAT routers accept port forwarding requests using either the uPNP or the PMP protocol. Whether this is supported depends on the particular brand of router and its configuration.
Yes, socket programming is enough for P2P.
Difficult to answer. I suggest that you read the wikified and annotated BitTorrent specification for a start.
Yes, this is the principle behind UDP hole punching.
Closed. This question is not about programming or software development. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed last month.
Improve this question
I am trying to experiment with some p2p networking. Upon doing some research, one of the biggest obstacle I learnt is "What if a client is behind a NAT/Firewall", later on I discovered about Hole Punching but that it is not always guaranteed to work.
As far a I understand, I don't understand why it might fail, This is what I know so far:
Based on the diagram above, this is how I understand how a successful connection can be established.
Alice joins the network (1) by creating connection to a directory-server. When this happens, Alice's NAT creates a mapping from her public ip to her local ip.
The directory server receives the connection and store Alice's public ip:port in the directory
Bob does the same (2), Joins the network and publishes his ip:port in the directory
Alice wants to communicate with bob. So she looks up Bob's ip:port from the directory. (3)
Alice sends data on Bob's ip:port which she got from the server. (5)
Since Bob also has a mapping from is ip:port to his local ip:port, the NAT simply forwards any data received on Bob's public ip:port to his computer.
Same works for Alice
I hope I was clear in my explanation of what I understand. My question is, what is so hard or unreliable about this? i must be clearly missing something. Can you explain me what it is?
One problem is that the NAT mappings in Alice's NAT server may time out, either after a fixed time, or after a period of inactivity.
A second potential problem is that the NAT server could make the restriction that Alice's NAT mapping is only "good" for TCP connections established by Alice, or connections between Alice and the initial IP "she" connected to. (In other words, direct communication between Alice & Bob may be blocked.)
And so on.
The problem is that the behaviour of a NAT server is highly dependent on how the managing organization's configuration / policy decisions. Many of these decisions could mean that your particular P2P usage pattern won't work reliably ... or at all.
So then is my whole idea about hole punching wrong?
No. It just means that it won't always work.
Possibly the biggest problem in NAT holepunching is lack of port consistency. For your implementation to work, at least one of the two NATs must support it.
Port consistency is where the same (local ip, local port) is mapped to the same (external ip, external port) regardless of the target (destination ip, destination port). Without this, the port seen by the directory server is not helpful to the client since it will not be the same port the clients will need to talk to each other.
(Note that this is a weaker requirement than port preservation, where external port == local port.)
Unfortunately for P2P communication, most NATs are some flavor of Symmetric NAT and do not have consistent port mappings.
Firewalls are typically stateful. Bob (2) establishing communications with the outside directory server sets up a rule in his NAT server that allows Bob and the directory server to communicate. When the NAT server sees packets from Alice, it rejects/drops them because it hasn't seen Bob establish communications with Alice.
First of all there are 2 types of hole punching
1.UDP hole punching
2.TCP hole punching
UDP hole punching success rate is 82%
TCP hole punching success rate is 64%
I have done many UDP hole punching experiments and they were mostly all successful but not same in the case of TCP hole punching.
The reason behind the failure of TCP hole punching is only the router NAT table. I will try to explain my best:
Client 1 --> connect(client2) --Internet-- connect(client1)<-- Client 2
Now if Client1 **SYN Packet**** reaches to the client2 and **client2 **SYN packet wasn't released** , the ROUTER of client2 can do 2 things:
1. send RST packet back as connection refused to client1.
2. drop packet immediately and no reply send to client1.
If this happens no connection will be established.
I can only suggest a solution that time difference between connect call from both the client should be very less. The connect call difference should be in milli-seconds
TIP: if you are in local network , put disable your firewall
for ubuntu user : sudo ufw disable
I think understanding how the Hole Punching really works would assist to get into why it may fail.
It was first explored by Dan Kagel, read here. In this technique, both peers are generally
assumed to be behind two different NATs. Both peers must be connected to an
intermediate server called a Rendezvous/Signaling server; there are many well-known Rendezvous protocols and SIP (RFC 3261) is the most famous one. As they are
connected to the server, they get to know about each other’s public transport
addresses through it. The public transport addresses are allocated by the NATs
in front of them. The Hole Punching process in short:
Peer 1 and Peer 2 first discover their Public Transport Addresses using
STUN Bind Request as described in RFC 8589.
Using a signaling/messaging mechanism, they exchange their Public Transport Addresses. SDP(Session Description Protocol)[16] may be used to
complete this. The Public Transport Address of Peer 1 and Peer 2 will be
assigned by NAT 1 and NAT 2 respectively.
Then both peers attempt to connect to each other using the received Public
Transport addresses. With most NATs, the first message will be dropped
by the NAT except for Full Cone NAT. But the subsequent packets will
penetrate the NAT successfully as by this time the NAT will have a mapping.
NAT can be of any type. If the NAT is, let's say, Symmetric NAT RFC 8489, Hole Punching won’t be possible. Because only an external host that receives a packet from an internal host can send a packet back. If this is the case, then the only possible way is Relaying.
Learn more about the current state of P2P communication: read RFC 5128.
I tried to implement UDP hole punching algorithm for my application.
When both peers contacted the server revealing their public IP
the 3G cellular NAT assigned constant external port for the same internal UDP port,however,
the 3G cellular NAT changed the internal->external port mapping depending on destination.
Hence, for example, if C is the static IP server A->C was mapped to port 1234
whereas A->B was mapped to port 5678. This way UDP hole punching failed.
As those cellular NATs use CGN, there is no UPnP/NAT-PMP support.
I have read about PCP for CGN with functionality similar to these, however,
I didn't find any protocol information on PCP.
Does anybody know if there is a way to overcome this destination-variable port mapping issue?
Either by port forwarding(like PCP) or traversal(PREFFERED).
One last thing. There are proofs of concept like skype, viber and MOST IMPORTANTLY torrent downloaders like those that depend on vuze-core(frostwire) that work on android over 3G and other cellular networks. So they obviously must have found a solution for that...
Thanks in advance!
I was trying to understand how torrents work?
And after reading a lot on web I now know the basics about it but I have a very
important question related to working of torrents!
In torrents how do peer-to-peer connections take place?
Almost all the peers have private-IP(for e.g 192.x.x.x) addresses then how does connections take place without a server(As I have read: There is no server involved in torrents) ?
Thanks a lot!
There are a few alternatives:
Peers behind NAT simply don't connect to other peers behind NATs. This creates two classes of peers, where the ones that are connectable will have an advantage when trading pieces, and typically achieve faster download rates.
Peers behind NAT use UPnP or NAT-PMP to set up port forwarding in order to be connectable by other peers
peers using uTP and Peer exchange can support a simple hole-punching mechanism (uTorrent and libtorrent supports this for instance). A peer can help in introducing two of its connections to each other, they try to connect to at the same time and of one of them have a full-cone NAT, they are very likely to succeed in establishing the connection.
Peers supporting DHT and uTP may use a relatively new feature where the port announced to the DHT is derived from their UDP packets. Using the same socket for DHT and uTP increases the chances that a peer behind a full-cone NAT can accept incoming connections without UPnP or NAT-PMP set up. Simply because the DHT traffic will keep a pinhole open on the NAT.
If you have a swarm of only peers behind symmetric NATs, nobody is going to be able to connect to anyone else, and bittorrent is not going to work. In practice (at least in moderately large swarms) there are always some peers that are connectable.
Would anyone know the answer to this?
I was reading Practical JXTA II (also at http://www.scribd.com/doc/47538921/Practical-JXTA-II). I'm confused by the statement on page 92 second paragraph concerning establishing communication with a peer behind a NAT : "such peers remain inaccessible ...until either ... or b) inaccessible peer establishes a connection to the remote peer spontaneously."
This seems to imply that the NAT translation of IPv4 local addresses to public addresses is always the same. If the router is mapping a large set of addresses to a smaller set of public addresses wouldn't the results vary? Once the HTTP response is received then the session would be terminated and someone else could use that public IP, right? Once the HTTP session is over the router would no longer record the mapping used.
I'm trying to implement an idea for Web Services where an aspect of the application is P2P (I need both nodes to act as both client and server at times). The central server can have a DNS registered address but the various potentially mobile nodes might be behind NATs etc. After reading this I thought I would be ok if I had the nodes behind NATs establish a connection when they start up, telling the central DNS register node their public address, but now I'm thinking that address would likely change.
Her is my understanding of what Jérôme meant.
Say peer A is WAN-visible and peer B is behing a firewall. Peer A can send data to Peer B, when
Peer A and Peer B both establish outbound connections to a relay server. Peer A sends data to the relay on the outbound request that the relay forwards to Peer B on the synchronous response (of peer B connection to the relay).
Peer B establishes a connection to Peer A, and peer A sends data back to Peer B on the synchronous response. A "reverse-invoke" mechanism.
wrt to JXTA, a peer publishes to the network a local address and optionally a WAN address (address is the couple host+port). There can only be one WAN address per peer if you want to establish direct connection to that peer using NAT.
Having a central server is a bad idea in a P2P network: you create a single point of failure, which is exactly what P2P networks excel at avoiding.
Yet, as you hint, there is still a need to maintain a registry of "adresses/peer locations". This registry has to be distributed however. This would need a book, but here are two approaches:
a Distributed Hash Table (DHT) on the nodes: every node holds and share a copy of part of the registry. JXTA has such a mechanism but check Kademlia on Wikipedia for a very successful algorithm.
a Global Index Nodes approach (I believe Skype-like): a limited number of dedicated peers/nodes that hold the registry using DHTs or other replication algorithms. The peers connect to the GINs for addresses using firewall friendly protocols (HTTP) and the GINs talk to each other using fast socket-to-socket connections (check Hazelcast for a quick way of implementing GINs).