I'm looking to build a bittorrent client in Ruby (although language is not important over here).
I read a BEP specification which says querying at /announce (without DHT support) to tracker would give a list of peers currently connected for a given info-hash.
To examine this. I created a torrent file and I found that tracker return the IP of my machine along with the port (which i confirmed is the running port of Bittorrent client on my machine i.e Utorrent)
But here is the problem the Peers info returned the IP of my ISP i.e 111.125.209.41 (the public facing IP since I'm behind the NAT).
Now I cant connect (via TCP) to the Process running on my machine with the public facing IP of my ISP.
Can anyone suggest how does bitorrent work on this and what should I do to solve this.
It could that over UDP Bittorrent would be using UDP hold punching not sure what happen on TCP.
There are two separate concerns.
How do peers on the internet (aka 99.99999% of the world) connect to your NATed node
How do peers inside your network connect to a node on the same network.
The first is achieved by various nat traversal methods, including negotiating with the nat device or manually configuring it.
The second issue either requires a router capable of hairpin routing in combination with a forwarded port or local peers discovering the internal address through other means, such as LSD.
Related
I'm trying to develop an applicaton for p2p communication between two android devices. In order to punch a hole through my NAT(s), I'd need to know my external ip address and port.
To that end, I've developed a java server on GAE to report my "remote" ip address and port. The problem is that on GAE I can get my ip address, but not my port. Without it, I'm unable to successfully punch the hole.
So, my question is what's the best, free method to find out my external IP address and port?
That's a question that has no answer with TCP.
Here's the problem: your "port" is not a fixed value. You don't have "an" external port. You typically get one dynamically assigned for each outbound connection.
As answers you should see from the test sites posted in another answer clearly indicate, it's a moving target (though it may stay stationary for a short time due to the browser using HTTP/1.1 keepalives and actually reusing the same connection, not just the same port)... but if you hit the site repeatedly, you'll see it either drift around randomly, or increment. Trying it from two different web browsers on the same machine, you'd never see the same port number -- the port corresponds to the specific source connection, not the machine sourcing the connection.
Sometimes, you may find that it's the same port number as the port your machine's stack opened for the outbound connection, but even when it is, it doesn't matter, because no traffic should be able to return to your machine on that port unless it is from the IP address and port of the machine to which you made the outbound connection. Any decent network address translating device would never accept traffic from another source IP address and/or port, other than the one you addressed in the outbound connection.
There is no standard, simple, predictable, reliable, or consistent way to punch a hole in TCP NAT and then exploit that hole for a peer-to-per connection. To the extent that such things are possible in a given NAT implementation, that is an implementation that is shoddy, broken, defective, and insecure.
See also: https://www.rfc-editor.org/rfc/rfc5128
Sounds like your app could use a STUN server to get its external address.
I have an application that relies on IP addresses for communication (Domain names simply does not work. :(... )
Its function is to connect to its peer on the other machine and send data over after establishing trust. During the "trust establishing" phase they both exchange their IPs for future communication. They both are behind the two different firewalls and are NATted. One is in our NATted office network and other is in the cloud NATted behind their firewall. The applications knows their respective private IPs and exchange that (the 10.x.xxx.xxx range), when they try to connect back to each other (using the private IPs with range 10.x.xxx.xxx) for transferring data they fail. The connection is TCP and the port range is pretty varied.
I am curious if there is anyway I can hard code (for this one time) a rule (at may be firewall level or some place outside my application) that says if there is a connection being initiated for IP address 10.x.xxx.xxx then redirect it to 205.x.xxx.xxx?
Private IP address ranges like 10.x.y.z are, by their very nature, private.
You can't do any meaningful resolution unless each node in between the endpoints has rules in place to translate these.
Translation is tricky, all the main tools you would use cater for static translation (port forwarding, e.g. where a particular port is forwarded to a particular IP). This is one avenue, but it is a hacky one (it requires you to open lots of ports, procedurally update your router and probably have some sort of broker server to maintain mappings).
Alternatively, you could run the isolated networks over a VPN, which would give your endpoints mutual private IPs which you can use to connect to eachother. It would simply be a case of binding to this new address and communicating across the VPN. This would also potentially encrypt your communication over the internet.
Other possibilities are to use NAT/TCP punchthrough techniques which can allow traversal, but these are really a patch to a broken network topology (Read up on IPv6 to see how this can be alleviated).
Alternatively, you could route all the connections over a proxy, but this will complicate matters compared to a VPN.
To answer the question about hardcoding a rule, port forwarding is the solution here. It will obviously depend on your router configuration for the peer accepting the connection, but this client should have the port target port forwarded to the machine. This will obviously not scale very well and is really shifting to a server/client architecture for one connection!
Depending on your hardware, you may be able to forward a range of ports (if a single port cannot be established) and limit the port forwarding to certain incoming connections (the external IPs).
Information on port forwarding can be found at http://portforward.com/
This sounds a lot like what you'd want out of a VPN. Is there anyway that you could set one up? Basically the Site-To-Site VPN between you and the cloud would say 'oh hey, here is an ip located on the remote network, go ahead and connect through the link'. Would this kind of solution work in your case?
Something along these lines: http://i.msdn.microsoft.com/dynimg/IC589512.jpg
I'm writing a P2P application. Peers regularly ping a main server to update their current IP/port, so when a peer wants to reach another one it can ask the server for that information. For now peers use UPnP to configure the NAT (for classic home setups) to be accessible from outside.
So everything works, except when a peer's client tries to reach another (or the same) peer's server and both are behind the same NAT. Since in that case the client is trying to reach its own "external" (public) IP address from behind the NAT, the NAT doesn't do the port forwarding and is unable to route the IP packet.
For now I'm thinking of two solutions:
query the NAT with UPnP to see to which local IP the port is forwarded
store on the main server the internal IPs of the peers
Can you think of other solutions? What strategies do mainstream P2P applications implement to solve this problem?
Since in that case the client is trying to reach its own "external"
(public) IP address from behind the NAT, the NAT doesn't do the port
forwarding and is unable to route the IP packet.
This is known as the hairpin condition. Not all router/NAT solve this properly. The solutions are:
a) Check whether your router/NAT can be configured to enable 'hairpining'. This solution works iff you control all router/NATs in your deployment.
b) Buy another router/node allowing this. Just like a), it works iff your control all router/NATs in your deployment.
c) If you can get obtain the port information for from UPnP, this is a solution too, but not all Router/NAT know or support UPnP. It does not cover for all cases in large deployment.
d) Using multicasting to 'discover' other nodes on the LAN and even communicate with them is a common solution to this problem. You need to agree on an IP address and have peers listen to it.
e) Storing the private IP address on the server is a solution too, but it requires more storage capacity on the server than solution d. There is a timeout (i.e., expiration of data validity) to handle too.
f) Use a TURN like communication between peers (i.e., communication between nodes pass through central server). This solution is rock solid, but not the most efficient in terms of bandwidth consumption.
Hope this helps.
Interactive Connectivity Establishment (ICE) was specifically developed for solving that type of NAT problems. It uses STUN and TURN to achieve the result and is used in modern p2p applications. (e.g. Voicechat)
The PJNATH library has a document explaining
unlike standalone STUN solution, it (ICE) solves the hairpinning issue, since it also offers host candidates.
we have little network devices which are shipped with IP address 0.0.0.1 to ensure that they never collide with any other device in their new environment (thus none of the 10.x.x.x, 172.16.x.x or 192.168.x.x ranges) until configuration. DHCP is no solution since there might be no DHCP server in the field.
The devices would listen to UDP broadcasts and answer with broadcasts until they are given their new IP address this way.
This worked fine with Windows XP - but sucks with Windows 7: the config program does not receive the answer packets from the devices which still have 0.0.0.1. Wireshark sees the packets, then they are dumped by the system.
Question: Is there any reason (RFC?) that actually prohibits using this address in a local environment? Or is it just MS that was overcautious? Where can I read why they treat this address "invalid"? Which ranges are really "invalid" now, too?
Any idea of a workaround on the PC side (Win 7)?
I know that it is not recommended to use 0.xxx addresses for work places, but for this very reason - having a not-used address - it works perfectly.
Edit: there is a device out there called "Netburner" which might have faced the similar issue, according to their forum. See: http://forum.embeddedethernet.com/viewtopic.php?f=5&t=612&p=2198 Does - by coincidence - anybody know some background information?
It sounds as if your configuration application is listening for broadcast packets on all network interfaces and expecting to receive packets from foreign subnets.
That should not work - the OS should only pass-on broadcast packets from the subnets each network interface is on, not from all subnets on the same physical (e.g. Ethernet) segment. I am reasonably certain that doing otherwise is broken behaviour WRT the IP protocol.
The are two ways to deal with this:
Make sure that your network interface has an IP address in the target subnet. You can have more than one IP addresses for each network card, so that should not interfere with normal network operations.
Configure or modify you application to use raw sockets, like Wireshark. Keep in mind, however, that this overrides all normal checks and balances and should be avoided, since it can cause behaviour that is almost impossible to diagnose - which is why it is frowned upon by meny network administrators.
Can you you add new routing table entries to Windows machines easily? Windows has to know which interface to use when routing a broadcast packet to the 0.0.0.x network.
The Unix machines I'm familiar with have a routing table that maps network/netmask entries to either gateways or interfaces (if the network is a local network). The local network (192.168.0.0/16 for my home network) gets sent to interface eth0. Everything else 0.0.0.0/0 gets sent to a specific gateway machine 192.168.0.1.
If my machine sent a UDP broadcast message to network 0.0.0.0/24 (in other words, UDP broadcast sent to 0.0.0.255, then my machine would forward the packet to the gateway machine (which it can look up via arp). The switches in the middle wouldn't propagate the packet to other network devices, because the MAC address is set.
If my machine had another routing entry for 0.0.0.0/24 to the local interface, then my machine would send the packet on the wire using an ethernet broadcast group, and the switches would forward the packet to all connections. (Yay! Just like hubs in the 90s! :)
So I figure you need to add a routing entry for 0.0.0.0/24 to your client machines, so that they can properly address the broadcast packet.
I was wondering that how application like skype ( a popular chat client ) works in local network with one router, How it can listen on particular port?
for example:=
In one network A and B are two machines running skype , gateway of both is G1,
now how A and B will have same IP on internet that is of G1, but how can they ensure that they are listening on different ports? How can they ask to router G1 for unique port.
I want to make a simple text chat server on linux. How can I have connections between two different computers in two different networks?
Solution to your problem is to have a forwarding server somewhere in the net.
Different programs use different means to connect to each other. But every chat server, including Skype, has a server, which forwards data or information about subnet IP/port availability.
There are two types of clients: "listening" clients and "passive" ones. Listening clients have direct access to Internet via router port forwarding, and "passive" ones have to use additional tricks to get their hands on external data, line external servers or additional ports to listen.
The point is, not clients connect to each other, but they connect to a server, which then connects back to them to verify they are available, and, if at least one of them is not firewalled, direct another on to connect to the first one, excludint itself from further communication. And if both are firewalled, then is has to forward their messages through itself.
Host Discovery
Manual discovery, client A knowns who client B is
Discovery through broadcast UDP which is used by lot of games for LAN play. A client sends out a packet to the broadcast address for their subnet. The peers can choose to pick up this broadcast and respond. The downside is that this is limited to the current subnet. The more general INADDR_BROADCAST (255.255.255.255) works for all subnets on the local-link, but it cannot be routed, so won't work over internet (this is what DHCP auto-configuration uses).
Discovery through a central (Rendezvous) server. Each individual client knows the address of the server, and the latter informs them about each other. This technique is used by IRC, Voip, IMs and by most 'peer-to-peer' networks.
Communication
After the initial discovery is done you want to be able to talk to eachother. On the internet this can get tricky. Most people nowadays have their own router and sit behind a NAT, so direct connections are impossible.
Using a Rendezvous server, you can possibly talk to each other using the server itself. client A tells the server what to say, and it in turn tells client B, since both clients have an outbound connection to the server.
It is possible for the clients to talk to each other without the server proxying. This requires either DMZ, port forwarding or UPnP. DMZ will basically forward all incoming connections on all the ports to a given local IP. Port forwarding only forwards certain ports to local IPs. UPnP is a bit more advanced, the client requests that the router temporarily forwards a port to it, and you tell the other client via the rendezvous server where to connect.
Chatting app implementation
The easiest solution to your problem is most likely to use a central server, which is known by all the clients, that proxies host discovery and possibly the communication between the clients. If you want the clients to communicate directly, you can just proxy host discovery, and then let either DMz, manual port forwarding or UPnP do the rest.
Another solution would be to just have direct communication through NAT traversal techniques discussed above, and do manual host discovery.
Yet another solution would be to use a public webserver and 'abuse' its ability to insert content to chat with each other.
You need a central UDP Rendezvous Server.
After the initial connection from the client to the server the UDP clients can be redirected to talk to eachother directly even if firewalled.
The trick is to open an UDP connection from the inside.
Check out Real-Time Media Flow Protocol and how they use it.
Check out UDP Hole Punching
alt text http://labs.adobe.com/technologies/stratus/images/p2pvideo_250x215.jpg
Traditional NAT servers replace the source address and port with the address and a random port number of the external interface of the NAT server. This works well for simple protocols such as HTTP and SMTP, but it can create problems for more complex protocols that require multiple response ports on the external interface of the NAT server. NAT servers also aren’t aware of information stored in the data portion of the application layer header without the help of NAT editors and similar software fixes.
Windows XP’s answer to these problems is NAT Traversal, which can automatically allow the UPnP-enabled NAT client application to communicate with a UPnP NAT device. NAT Traversal provides methods to allow the UPnP client to learn the public IP address of the NAT server and to negotiate dynamically assigned port mappings for UPnP NAT client applications.
NAT Traversal features can be built into any hardware device or software application. Applications that commonly cause troubles for NAT devices but work well when UPnP-enabled include the following:
Multiplayer Internet games
Audio and video communications
Terminal Services clients and servers
Peer-to-peer file sharing applications
When these applications are UPnP-enabled, access through the Windows XP ICS allows them to work seamlessly.
Unless A and B are actually "listening" to the responses to outgoing requests, your router will need to be cofigured to forward the relevant port numbers to the relevant hosts. This isn't something that you can request in the code, it's something you need to configure on the router itself.