As far as I understand, ICE protocol is used for discovering the nodes/devices from the end-user device to "the outside".
I don't understand why it's needed. Isn't packet-routing is the responsibility of network devices like routers and switches? They should find the shortest path from the gateway to the end-user device (Actually, routers remembers those routes they previously discovered).
Moreover, NAT protocol is used to convert from an "internal ip" to "external ip" and vice-versa.
So again,
Why does the other user needs to be familiar with my internal network setup?
NAT is a kludge, put in place to try to conserve IPv4 addresses until IPv6 becomes ubiquitous, and it breaks the end-to-end connectivity which is the promise of IP. Because of that, some things don't work correctly through NAT. There are various kludges to work around the NAT kludge, and ICE is part of that. This is explained in RFC 5245, Interactive Connectivity Establishment (ICE): A Protocol for Network Address Translator (NAT) Traversal for Offer/Answer Protocols:
Introduction
RFC 3264 [RFC3264] defines a two-phase exchange of Session Description
Protocol (SDP) messages [RFC4566] for the purposes of establishment of
multimedia sessions. This offer/answer mechanism is used by protocols
such as the Session Initiation Protocol (SIP) [RFC3261].
Protocols using offer/answer are difficult to operate through Network
Address Translators (NATs). Because their purpose is to establish a
flow of media packets, they tend to carry the IP addresses and ports
of media sources and sinks within their messages, which is known to be
problematic through NAT [RFC3235]. The protocols also seek to create
a media flow directly between participants, so that there is no
application layer intermediary between them. This is done to reduce
media latency, decrease packet loss, and reduce the operational costs
of deploying the application. However, this is difficult to
accomplish through NAT. A full treatment of the reasons for this is
beyond the scope of this specification.
Numerous solutions have been defined for allowing these protocols to
operate through NAT. These include Application Layer Gateways (ALGs),
the Middlebox Control Protocol [RFC3303], the original Simple
Traversal of UDP Through NAT (STUN) [RFC3489] specification, and Realm
Specific IP [RFC3102] [RFC3103] along with session description
extensions needed to make them work, such as the Session Description
Protocol (SDP) [RFC4566] attribute for the Real Time Control Protocol
(RTCP) [RFC3605]. Unfortunately, these techniques all have pros and
cons which, make each one optimal in some network topologies, but a
poor choice in others. The result is that administrators and
implementors are making assumptions about the topologies of the
networks in which their solutions will be deployed. This introduces
complexity and brittleness into the system. What is needed is a
single solution that is flexible enough to work well in all
situations.
This specification defines Interactive Connectivity Establishment
(ICE) as a technique for NAT traversal for UDP-based media streams
(though ICE can be extended to handle other transport protocols, such
as TCP [ICE-TCP]) established by the offer/answer model. ICE is an
extension to the offer/answer model, and works by including a
multiplicity of IP addresses and ports in SDP offers and answers,
which are then tested for connectivity by peer-to-peer connectivity
checks. The IP addresses and ports included in the SDP and the
connectivity checks are performed using the revised STUN specification
[RFC5389], now renamed to Session Traversal Utilities for NAT. The
new name and new specification reflect its new role as a tool that is
used with other NAT traversal techniques (namely ICE) rather than a
standalone NAT traversal solution, as the original STUN specification
was. ICE also makes use of Traversal Using Relays around NAT (TURN)
[RFC5766], an extension to STUN. Because ICE exchanges a multiplicity
of IP addresses and ports for each media stream, it also allows for
address selection for multihomed and dual- stack hosts, and for this
reason it deprecates RFC 4091 [RFC4091] and [RFC4092].
Firewalls. They're typically configured to bounce any unsolicited traffic from the world wide web to you. They only approve of you initiating contact with a server, which only then is allowed to back-traffic to you, and that's pretty much it. Unless your friends all own static IPs (which few people can justify) this is a hostile environment for peer to peer communication.
ICE tries to solve this, by enumerating addresses and ports at which the other side may be reached, and trying to connect to these addresses, by initiating outbound requests on both ends, or if all else fails, falling back to communicating through a TURN server, if specified.
See this WebRTCHacks article for more on the problem.
Why does the other user needs to be familiar with my internal network setup?
Because the other user is sometimes on your internal network. e.g. LAN games.
Related
I try to make a P2P Program and need help with getting through the NAT of the clients.
I have read many questions here on stackoverflow, but i never got what the drawbacks and benefits of all the Methods to get through a NAT are.
How many routers support which methods?
Which methods are commonly used by the big companies? (BitTorrent, TeamViewer,...)
What are the drawbacks/benefits of the three methods i listed and which else exists?
UPnP requires software support in your router. Even when routers have it, it's sometimes turned off due to security concerns.
STUN uses a simple public server to detect the presence of NAT and the real IP addresses. This is useful for other, higher-level protocols that do the useful work.
Hole punching is a simple technique to connect two computers. It requires a 3rd public computer to communicate between the 2 computers behind NAT. You could run your own server with a custom protocol, our you could use STUN and existing STUN servers. (Only a few packets are needed for setup, the rest are peer-to-peer).
Of course, some NAT firewalls are incompatible with the above, so you may need a custom protocol. Bittorrent does this: they use different techniques depending on which clients are behind firewalls and whatnot.
We're prototyping a jgroups-based cluster node messaging system that will replace one that was JDBC-based. There are a lot of folks in my organization who are concerned about adding more multicast traffic to an already busy network, so I'm getting some pushback on a UDP/multicast solution.
I know JGroups can be configured to be TCP only, but I do not want to have to force a configuration step into the application where each node has to be identified ahead of time in a config file.
What I'd like then is to see if we can get a hybrid working here where multicast is used ONLY for group membership operations (discovery, heartbeats, failure detection), but messaging is all TCP-based.
I'm not finding examples of that in my searches, however, and am therefore questioning whether JGroups can be configured this way.
Can it, and any example configs showing how?
Thanks!
As for discovery, using MPING you can do that - it uses IP multicast to discover new nodes, although these later respond via the main transport (TCP in your case).
As for FD/FD_ALL, I don't think it's possible, the protocols are designed to use the main transport. You'd have to write your own FD protocol, shouldn't be that complicated.
However, if you can use UDP, you probably should. It's up to you whether you send a message to one node, multiple ones or all of them - for one destination there will be unicast UDP, for few destinations (if you set the anycast option) will be multiple unicasts used and only for all nodes the UDP will reduce the network load by multicasting. It's really up to the application, UDP just allows multicasts.
I am trying to write a program that scan an ip range and detect if an ip is address of a router or not.
Currently i used traceroute from my computer to all host in the network. However, i believe there must be some way to directly "ask" a host at an ip if it is a router or not?
by the way, do you know any program/ opensource already does this?
Routers are supposed to talk couple of protocols (actually a neat bunch) that regular IP nodes do not, and then there are some which are more common (i.e. even non-router nodes do).
Router-only protocols:
VRRP
IGRP / EIGRP
OSPF
BGP
RIP
You could do active-probing on those, i.e. send a packet (behaving as if you are another router, or an end-node) and checking to see what kind of response the router (if at all) sends.
Alternatively you could do passive-probing, like 'sniffing', i.e. watching out for the kind of IP packets being sent out by various nodes. There are some which are usually sent out by Routers only (again, mostly from the above list).
Common protocol, but that can actually tell you a lot:
SNMP (esply the unsecure one's like v1/v2, are easy to deal with, without having to establish a secure session)
Other ways:
Portscanning (actually can tell you a real lot), for example all routers have some management ports (although, often they are locked down due to security concerns)
What you want to do is often what many 'Network Management' software do, to "discover" capabilities / functionality of other nodes in the network. And, there isn't a single size-fits all solution. They use bunch of different methods, heuristics to finally figure out what the other node is.
Any node which is hopped to and not just an endpoint is a router. However, this doesn't allow you to detect routers with no reachable devices hooked up. (Any input as to whether my answer has merit would be great!)
I am working in a project to migrate a firewall application from IPv4 to IPv6. I have several questions:
What changes and modifications might be needed?
Will the popular protocols such as FTP, HTTP, POP3 also need to be adapted/modified?
Which IPv6 components should or must be implemented?
Which tunneling/transition mechanism to prefer?
As I am new to this network security field, I hope you guys could give me some valuable input. Thanks in advance.
There are a lot of things to consider. Off the top of my head:
Learn the difference between link-local (fe80::/10), global unicast, and multicast address ranges. Make sure you support interface scoping with link-local addresses (you will see addresses like fe80::1%eth1, which will indicate the link-local address on the eth1 interface).
ARP equivalent (IPv6 neighbor discovery) is now part of ICMP. This is important because if the user wants to block ICMP packets and isn't careful, they could lose all their connectivity!
Most (sane) protocols will not need major changes. FTP is one protocol that will potentially need changes, since it sometimes passes network addresses within the protocol itself (rather than letting the lower-level protocols take care of it)
The most basic tunneling/transition mechanism you will need is called 6in4; it simply encapsulates IPv6 packets within IPv4 packets and allows the user to manually configure the endpoints of the tunnel. Automatic tunneling mechanisms like 6to4 and Teredo can also be useful in some situations.
If you are selling a commercial product, I recommend you take a look at the USGv6 test selection tables. Also, read through the USGv6 profile which has pointers to many of the RFCs you will need to understand in order to develop an IPv6-compliant product. Not supporting the USGv6 profile for a network protection device (NPD) could severely limit your market. Finally, get some training! IPv6 is vastly different from IPv4 in many ways. If your employer wants this project to succeed, training will be critical given that it appears that many project members are new to both IPv6 and network security. (do you have a mentor on the team to ask questions?)
I've seen and read a lot of similar questions, and the corresponding Wikipedia articles (NAT traversal, STUN, TURN, TCP hole punching), but the overwhelming amount of information doesn't really help me with my very simple problem:
I'm writing a P2P application, and I want two users of my application behind NAT to be able to connect to each other. The connection must be reliable (comparable to TCP's reliability) so I can't just switch to UDP. The solution should work on today's common systems without reconfiguration. If it helps, the solution may involve a connectible 3rd-party, as long as it doesn't have to proxy the entire data (for example, to get the peers' external (WAN) IP addresses).
As far as I know, my only option is to use a "reliable UDP" library + UDP hole punching. Is there a (C/C++) library for this? I found enet in a related question, but it only takes care of the first half of the solution.
Anything else? Things I've looked at:
Teredo tunnelling - requires support from the operating system and/or user configuration
UPnP port forwarding - UPnP isn't present/enabled everywhere
TCP hole punching seems to be experimental and only work in certain circumstances
SCTP is even less supported than IPv6. SCTP over UDP is just fancy reliable UDP (see above)
RUDP - nearly no mainstream support
From what I could understand of STUN, STUNT, TURN and ICE, none of them would help me here.
ICE collects a list of candidate IP/port targets to which to connect. Each peer collects these, and then each runs a connectivity check on each of the candidates in order, until either a check passes or a check fails.
When Alice tries to connect to Bob, she somehow gets a list of possible ways - determined by Bob - she may connect to Bob. ICE calls these candidates. Bob might say, for example: "my local socket's 192.168.1.1:1024/udp, my external NAT binding (found through STUN) is 196.25.1.1:4454/udp, and you can invoke a media relay (a middlebox) at 1.2.3.4:6675/udp". Bob puts that in an SDP packet (a description of these various candidates), and sends that to Alice in some way. (In SIP, the original use case for ICE, the SDP's carried in a SIP INVITE/200/ACK exchange, setting up a SIP session.)
ICE is pluggable, and you can configure the precise nature/number of candidates. You could try a direct link, followed by asking a STUN server for a binding (this punches a hole in your NAT, and tells you the external IP/port of that hole, which you put into your session description), and falling back on asking a TURN server to relay your data.
One downside to ICE is that your peers exchange SDP descriptions, which you may or may not like. Another is that TCP support's still in draft form, which may or may not be a problem for you. [UPDATE: ICE is now officially RFC 6544.]
Games often use UDP, because old data is useless. (This is why RTP usually runs over UDP.) Some P2P applications often use middleboxes or networks of middleboxes.
IRC uses a network of middleboxes: IRC servers form networks, and clients connect to a near server. Messages from one client to another may travel through the network of servers.
Failing all that, you could take a look at BitTorrent's architecture and see how they handle the NAT problem. As CodeShadow points out in the comments below, BitTorrent relies on reachable peers in the network: in a sense some peers form a network of middleboxes. If those middleboxes could act as relays, you'd have an IRC-like architecture, but one that's set up dynamically.
I recommend libjingle as it is used by some major video game companies which heavily relies on P2P network communication. (Have you heard about Steam? Vavle also uses libjingle , see the "Peer-to-peer networking" session in the page: https://partner.steamgames.com/documentation/api)
However, the always-work-solution would be using a relay server. Since there is no "standard" way to go through NAT, you should have this relay server option as a fall-back strategy if a connection has to be always established between any peers.