Reading about Webrtc i get the feeling that "it will drop server bandwidth usage dramatically" except for "a few corner enterprise-firewall cases" where one needs a TURN server which relays the whole traffic between the peers.
For example, altough not webrtc related but the idea is similar, the wikipedia article of Chatroulette states: The website uses Adobe Flash to display video and access the user's webcam. Flash's peer-to-peer network capabilities (via RTMFP) allow almost all video and audio streams to travel directly between user computers, without using server bandwidth. However, certain combinations of routers will not allow UDP traffic to flow between them, and then it is necessary to fall back to RTMP.
Also similar articles on Webrtc focus on "yeah there might be problems with firewalls so you need a TURN server but ignore this and look at my awesome PeerConnection javascript code".
What i don't understand:
A Connection between two peers requires a server socket to be open so the peers can connect to it. Even UDP requires the concept of a udp server socket. Since nearly all not-server internet connected peers are behind some kind of router. E.g. every smartphone uses a wifi router, desktop PC's use the router of the service provider, ...
It shouldnt be possible to connect to a server socket hosted on a smartphone (browser webrtc server socket) or desktop cause of the router/firewall.
Thus my understanding is practically no two peers which need to send their traffic through the internet will be able to use a direct P2P connection, right?
So the only useful case to use Webrtc is in a LAN like environment, right?
Furtherly in case of a video chat service like chatroulette based on webrtc would need to use a bunch of TURN servers to relay nearly ALL traffic. Which makes Webrtc equally costly regarding server bandwidth like hosting my own solution.
So my question is: Am i right? If not what is the technical detail that allows a PeerConnection to be used without a TURN server but for two nodes separated by the Internet? How is the connection established on Layer 4 the TCP/UDP Transport Layer? Is it using UDP and all wifi routers allow hosting UDP server sockets or such? Which wouldnt make much sense cause of NAT and security.
UPDATE 1:
Digging a bit further i found what "symmetric nat" means and what it has to do with enterprises: In most enterprises it seems that the device connected to the internet has symmetric nat implemented. This means that the routing table which maps internal "internal-ip:internal-port" tuples to "internet-ip:internet-port" also stores "destination-ip:destination-port". So such routes/nats store a table for every (tcp?) connection having 6 columns "internal-ip:internal-port:internet-ip:internet-port:destination-ip:destination-port". This means no one else but the destination is allowed to communicate with internal-ip:internal-port.
Whereas non-enterprise-routers seem to only store the "internal-ip:internal-port:internet-ip:internet-port" combination. Thats also what is meant as "poke a hole in the firewall".
You're not right. All peers have IP addresses in order to communicate, and can be reached on those same addresses, provided a firewall allows it.
NATs tend to be optimized for client-initiated client-server traffic only. That typically means they initially allow outbound traffic only, and only allow inbound traffic on the same line after outbound traffic has happened. Perfect for servers. See this WebRTCHacks article for an intro to the problem.
This is where ICE comes in to attempt to poke holes in the firewall from the inside (client-side), in order to establish a line of communication directly between two peers, without needing any "server" socket, whatever that means.
How ICE works is quite complicated, and is explained in detail in the RFC.
But in broad terms it works in a number of steps:
Each peer (e.g. browser) has an "ICE agent" that collects candidates. Candidates are addresses (IP:port numbers) at which this peer can be reached, including:
Host candidates: e.g. immediate LAN/wifi/VPN IPs of the machine.
Server-reflexive candidates: public (outside-NAT) addresses of the machine, obtained by bouncing requests off mirroring (STUN) servers on the internet.
Relay candidates: addresses to a shared TURN server to forward data if all else fails.
Once discovered, candidates are inserted into the local SDP, and trickled over the signaling channel to the other peer, where they are inserted in it's remote description, where the other agent sees them.
Once an ICE agent has both local and remote candidates, it starts pairing local and remote candidates, and checks them for connectivity by sending STUN requests on them (effectively attempts at reaching the peer).
Successful pairs are ones both ICE agents have gotten a response back on (a 4-way handshake if you will).
If there's more than one successful pair, they're sorted by some metric, and the best pair becomes selected.
The selected pair is then used to send media over. One pair is needed for each track of (video or audio) media.
If a better pair is found later, the selected pair may change, affecting what address media is sent on.
TURN should only be needed in cases either where both clients are behind symmetrical NATs, or UDP traffic is blocked entirely.
I work in web development, but I don't have a great understanding of network protocols. I recall hearing an analogy that TCP, HTTP, and SSL can be thought of as a series of nested envelopes around the actual request content.
I also have a fuzzy idea TCP consists of packets, which are verified on the other end. But I'm sort of picturing the HTTP request being chopped into packets, too...
So basically, I don't understand this stuff well at all. Can anybody give a good overview of this? Also, is there a beginner-friendly book or other resource that you'd recommend?
Since I asked this question, I've learned more about this topic, so I'll take a crack at answering it myself.
The easiest way to picture the protocol stack is as a letter, wrapped in a series of envelopes. Each envelope has a different role in getting the letter to its recipient, and envelopes are added and removed as needed along the journey.
The Application Layer
The letter itself is an application-layer request. For example, you've typed "StackOverflow.com" in your browser and pressed enter. Your browser needs to ask the StackOverflow server for its home page. So it writes a letter saying, "Dear StackOverflow, would you please send me your home page?"
If the writer of the letter is your browser, the recipient of the letter is the web server program running on StackOverflow. The browser wants the web server to "write back" with a response in the form of a web page. Both the browser and server are applications - programs running on specific computers.
Because browsers speak HTTP, that's what it uses to make the request: the letter says something like "GET http://stackoverflow.com". The browser also writes down any cookie information it got from StackOverflow last time ("remember me? You told me my login ID was X") and adds some miscellaneous labeled information called "headers" (things like "I'm Firefox" and "I can accept HTML or text" and "it's OK with me if you compress the content with gzip"). All that information will help the server know how to personalize or customize its response.
At that point, the browser is basically done. It hands this letter to the operating system and says, "would you please send this for me?" The OS says, "Sure." It then does some work to connect to StackOverflow (more on that in a minute), then tells the browser, "I'm working on it. By the way, here's a little mail bin I made for you, called a socket. When I hear back from StackOverflow, I'll put its letter in there and you can read it just like a file." The browser then happily awaits the response.
The IP layer
To send the request from the browser to StackOverflow, the operating system has to do several things.
First, it has to look up the address for StackOverflow.com - specifically, the IP address. It does this using DNS (which I won't go into here). Once it knows the IP address, it will know how to wrap the request in one of the "envelopes" called the IP layer.
Why do we need the IP layer? Well, once upon a time, we didn't.
Why we need IP
Have you ever seen an old movie where someone makes a phone call by asking the operator to connect them? The operator would physically connect the wire from Person #1's house to the wire for Person #2's house. Before the protocol stack was invented, connecting computers was a lot like that phone call: you needed a dedicated wire from point to point.
So, for example, if the computer scientists at Stanford wanted to exchange data with the ones at Harvard, they'd pay a bunch of money to rent a dedicated wire between the two places (a "leased line"). Any data that went into one end came out reliably at the other end. However, this was very expensive: imagine paying for a separate line for every place you want to connect to!
People realized that this wouldn't scale up. We needed a way to have a network that was shared by all users, like a giant spiderweb of wires spread out all over the map. That way, each user would only need one connection to the network and could reach any other user through it.
But that presented a problem. If everyone's communications went on the same lines, how would the data get to the right place? Imagine a bunch of letters dumped on a conveyor belt. Obviously, every letter needs to be addressed to someone, or else they can't be delivered.
That's the basic idea of IP: every machine needs to have an IP address that uniquely identifies it. Messages are placed in IP packets, which are like envelopes with addresses and return addresses.
So, once the OS has looked up the IP address for Stackoverflow.com, it puts the HTTP request in an IP envelope. If it's a "long letter", too big for one envelope, the OS cuts it into pieces and puts it in several IP envelopes. Each envelope says something like "FROM: (your IP address); TO: (The Server's IP address." Like the HTTP request, the IP packet has some other miscellaneous header information, which we won't go into here, but the basic idea is just "to" and "from."
So, at this point, the letter is ready to go, right?
The messiness of IP
Not quite. This letter could easily get lost! See, with IP, we no longer have a dedicated line from place to place. If we did, we'd be sure that our letters were getting delivered: as long as the line wasn't broken, everything would go through.
But with IP, everyone's packets get dumped onto conveyor belts and carried along. The belts lead to little sorting stations, called "routers". If you imagine the routers like physical mail centers, you could picture one in, say, New York City.
"Here's a letter headed for Mexico City. I don't know exactly how to get there, but the station in Houston should be able to get it closer, so I'll send it there. Ah, here's a letter that's going to Atlanta. I'll send it to Charlotte; they should be able to forward it a step closer."
Generally this system works OK, but it's not as reliable as having your own dedicated line. Nearly anything could happen en route: a conveyor belt could break or catch fire, and everything on it could be lost. Or one could get bogged down for a while, so that its packets are delivered very late.
Besides that, because these conveyor belts and stations are used by everyone, nobody's letters get treated specially. So what happens if a router gets more letters than it can possibly handle? For a while, it can stack them in a corner (maybe in RAM), but eventually, it runs out of space.
What it does then may seem shocking: it starts throwing them away.
Yep. That's it. You might think that it would at least be kind enough to send back a note to you, saying, "sorry, we couldn't deliver your letter." But it doesn't. If you think about it, if the router is overwhelmed, it's probably because there's too much traffic on the lines already. Adding apology notes would only make the problem worse. So it throws away your packet and doesn't bother telling anyone.
Obviously, this is a problem for our HTTP request. We need it to get there, and we need the response to get back reliably, too.
To make sure it gets there, we want some kind of "delivery confirmation" service. For that, we'll wrap another envelope around our HTTP request before putting into IP packets. That layer is called TCP.
TCP
TCP stands for "transfer control protocol." It exists to control what would otherwise be a messy, error-prone delivery process.
As implied before, TCP lets us add some "delivery confirmation" to this messy delivery system. Before we wrap our HTTP request in IP packets, we first put it into TCP packets. Each one gets a number: packet 1 of 5, 2 of 5, etc. (The numbering scheme is actually more complicated and counts bytes rather than packets, but let's ignore that for now.)
The basic idea of TCP is this:
First, the client and server - in this case, your operating system and the StackOverflow server's operating system - do a "handshake" to establish a "connection". Both words needs quotes because the "handshake" is actually a few messages back and forth, proving that packets can get successfully there and back, and the "connection" is really nothing more than each side deciding that they'll keep track of the packets flowing between them.
Next, they send packets back and forth; the client maybe requesting a web page, and the server maybe sending it back (in as many packets as that takes).
As one side receives packets, it sends back confirmation messages, saying "so far I've received your packets up to packet 100" and so forth. If one party sends packets and doesn't hear a confirmation for a while, it will assume they were lost and re-send them.
(Getting confirmations when things arrive at the other end is better than getting error reports when a router drops things along the way for a couple of reasons. One is that confirmations go back over a working connection, whereas errors would further clog a non-working connection. Another is that we don't have to trust the intermediary routers to do the right thing; the client and server are the ones who care most about this particular conversation, so they're the ones who take charge of being sure that it works.)
Besides making sure that all the data gets to the other end, TCP also makes sure that the received data gets put back into the right order before handing it up the stack, in case earlier packets got resent and arrived later, or packets in the middle took a longer route, or whatever.
That's basically it - having this kind of delivery confirmation makes the unreliable IP network reliable.
Why wasn't it built straight into IP?
UDP
Well, confirmation has a drawback: it makes things slower. If something is missed, it must be repeated. In some cases, that would be a waste of time, because what you really want is a real-time connection. For example, if you're having a phone conversation over IP, or you're playing a real-time game over the internet, you want to know what's happening right now, even if it means you miss a bit of what happened a second ago. If you stop to repeat things, you'll fall out of sync with everyone else. In cases like that, you can use a cousin of TCP called UDP, which doesn't re-send lost packets. UDP stands for "user datagram protocol", but many people think of it as "unreliable data protocol". That's not an insult; sometimes reliability is less important than staying current.
Since both of these are valid use cases, it makes sense that the IP protocol stayed neutral on the issue of reliability; those who use it can choose whether to add reliability or not.
Both TCP and UDP add one other important piece of information to the request: a port number.
Port numbers
Remember, our original request is comes from a browser and is going to a web server program. But the IP protocol only has addresses that specify computers, not the applications running on them. The machine with StackOverflow's web server may also have other server programs that are listening for requests: a database server, an FTP server, etc. When that machine gets the request, how will it know which program should handle it?
It will know because the TCP request has a port number on it. This is just a number, nothing fancy, but by convention, certain numbers are interpreted to mean certain things. For example, using a port number of 80 is a conventional way of saying "this is a request for a web server." Then the server machine's operating system will know to hand that request to the web server program and not, say, the FTP server program.
When the TCP packets start streaming back to your computer, they will also have a port number, to let your machine know which program to give the response to. That number will vary based on the socket that your machine created initially.
Wait, what's a socket?
Sockets
Remember earlier when the browser asked the OS to send the request? The OS said it would set up a "mail bin" for any response it got back. That bin is called a socket.
You can think of a socket sort of like a file. A file is an interface that the OS provides. It says, "you can read and write data here, and I will take care of figuring out how to actually store it on the hard drive or USB key or whatever." The thing that uniquely identifies a file is the combination of path and filename. In other words, you can only have one file located in the same folder with the same name.
Similarly, a socket is an interface the OS provides. It says, "you can write requests here and read responses." The thing that uniquely identifies a socket is the combination of four things:
Destination IP
Destination Port
Source IP
Source Port
So, you can only have one socket on a system with the same combination of all of those. Notice that you could easily have several sockets open to the same destination IP and port - say, StackOverflow's web server - as long as they all have different source ports. The OS will guarantee that they do by choosing an arbitrary source port for each request, which is why you can have several tabs or several browsers all requesting the same web site simultaneously without anything getting confused; the packets coming back all say which port on your computer they're headed for, which lets the OS know "ah, this packet is for tab 3 in Firefox" or whatever.
Summing up so far
We've been thinking of the protocols as a series of envelops wrapped around the letter. In our example, the letter was an HTTP request, which got wrapped in TCP, then in IP. The IP packets get sent to the right destination computer. That computer removes the IP "envelope" and finds a TCP packet inside. The TCP packet has a port number, which lets the operating system know which port to collect its information in. It replies saying that it got that packet, and it puts its contents (the HTTP request) into the correct socket for the appropriate program to read from. When that program writes a reponse to the socket, the OS sends it back to the requester.
So our "stack" is:
An HTTP request (a "letter"). This is the application layer.
Wrapped in TCP packets ("envelopes"). This is the transport layer.
Wrapped in IP packets ("envelopes"). This is the IP layer.
It's important to understand that this stack is totally customizable. All of these "protocols" are just standard ways of doing things. You can put anything you want inside of an IP packet if you think the receiving computer will know what to do with it, and you can put anything you want inside a TCP or UDP packet if you think the receiving application will know what to do with it.
You could even put something else inside your HTTP request. You could say that some JSON data in there is the "phone number exchange protocol," and as long as both ends know what to do with it, that's fine, and you've just added a higher-level protocol.
Of course, there's a limit to how "high" you can go in the stack - that is, you can put a smaller envelope inside HTTP, and a smaller one inside that, etc, but eventually you won't have any room to go smaller; you won't have any bits for actual content.
But you can easily go "lower" in the stack; you can wrap more "envelopes" around the existing ones.
Other protocol layers
Once common "envelope" to wrap around IP is Ethernet. For example, when your computer decides to send IP packets to Google, it wraps them up as we've described so far, but to send them, it gives them to your network card. The network card may then wrap the IP packets in Ethernet packets (or token ring packets, if you've got an antique setup), addressing them to your router and sending them there. Your router removes those Ethernet "envelopes", checks the IP address, decides who the next closest router is, wraps another Ethernet envelope addressed to that router, and sends the packet along.
Other protocols could be wrapped as well. Maybe two devices are only connected wirelessly, so they wrap their Ethernet packets in a Wi-Fi or Bluetooth or 4G protocol. Maybe your packets need to cross a village with no electricity, so someone physically prints the packets on paper with numbered pages, rides them across town on a bicycle, and scans them into another computer in the order of the page numbers. Voila! A print-to-OCR protocol. Or maybe, I don't know, TCP over carrier pigeon would be better.
Conclusion
The protocol stack is a beautiful invention, and it works so well that we generally take it for granted.
It is a great example of abstracting functionality: each layer has its own work to do and can rely on others to deal with the rest.
The application layer is only concerned with applications talking to each other: "Firefox wants to talk to the web server at StackOverflow.com."
The transport layer is only concerned with getting a stream of packets delivered correctly from one app to another: "all the packets from port 123 on machine 1 need to get to port 80 on machine 2".
The IP layer is only concerned with routing individual packets: "this packet needs to get to the following IP address."
The link layer is only concerned with getting packets from one waypoint to the next: "this ethernet packet needs to get from the network card to the router."
The physical layer is only concerned with signal transmission: "these pulses need to be sent over this wire."
(Although these layer terms are borrowed from OSI, OSI was actually a competing standard to TCP/IP, and included things like the "session layer" and "presentation layer" that TCP/IP doesn't use. OSI was intended to be a more sane and standardized alternative to the scrappy hacked-together TCP/IP stack, but while it was still being discussed, TCP/IP was already working and was widely adopted.)
Because the layers can be mixed and matched as needed, the stack is flexible enough to accommodate nearly any use we can think of, so it's probably going to be around for a long time. And hopefully now you can appreciate it a bit more.
For the throughout description of TCP/IP networking (without physical layer, e.g., Ethernet), pick TCP/IP Illustrated by Stevens. If you going to do some low-level network programming, Unix network programming by the same author is the best.
There's a reason you'll often hear of TCP/IP implementations called a "stack". Part of the concept is that you have a low-level protocol (Ethernet, PPP, what-have-you), slightly higher-level protocols built on top of it (IP), and so on. It's quite similar to the OSI model, and can be described in terms of that model, though TCP/IP breaks up the layers just a bit differently. Anyway, programs generally send data using one of the upper-level protocols, and let the TCP/IP stack handle the details of getting the data from point A to point B.
TCP sits on top of IP and lets you think of the data flowing in and out as a pair of streams (one in, one out) rather than getting raw IP packets and having to figure out what to do with them. (Big BIG benefit: it simplifies multiplexing. Without TCP or UDP or the like, IP would be near useless -- only one program could normally communicate with the network at a given time.)
SSL sits on top of TCP, and lets you send data over the stream that TCP provides without having to get involved in the ugly details of encrypting and decrypting data, verifying certificates, etc.
HTTP sits on top of TCP (or SSL, in the case of HTTPS), and provides a way for a client and server to pass entire requests and responses, along with metadata describing them.
Network protocols are formal standards and policies comprised of rules, procedures and formats that define communication between two or more devices over a network. Network protocols govern the end-to-end processes of timely, secure and managed data or network communication.
There are several broad types of networking protocols, including:
• Network communication protocols: Basic data communication protocols, such as TCP/IP and HTTP.
• Network security protocols: Implement security over network communications and include HTTPS, SSL and SFTP.
• Network management protocols: Provide network governance and maintenance and include SNMP and ICMP.
The different layers of the Open Systems Interconnection (OSI) reference model are:
Application layer: This is the upper most layer in the OSI reference model. The application layer provides the means by which application processes can access network services, and is therefore associated with services that include direct support for applications.
Presentation layer: This layer in the OSI reference model deals with specifying the format which should be utilized to enable network data to be communicated between computers in the network. The presentation layer adds formatting, encryption, and data compression to the packet.
Session layer: This layer enables applications that reside on different computers to create and close network sessions. It also manages open network connections, or sessions that are open.
Transport layer: The transport layer is responsible for ensuring that data is delivered in sequence, error-free, and efficiently over the network. The transport layer also identifies duplicated packets, and drops them. Transport layer protocols include Transmission Control Protocol (TCP) and Sequenced Packet Exchange (SPX). These protocols open packets at the receiving computer, and reassemble the original messages as well.
Network layer: This layer of the OSI reference model provides addressing for messages for all networks. It translates your logical addresses and names to physical addresses, and then identifies the preferred route from the source computer to the destination computer.
Data Link layer: The Data Link layer prepares data for the physical connection by defining the means by which software drivers can access the physical medium. The Data Link layer transmits frames from the Network layer to the Physical layer.
Physical layer: This layer places the data on the physical medium which is carrying the data. It is responsible for the actual physical connection between two computers on the network that are exchanging data.
The function of protocols at the sending computer is summarized below:
• Segment data into smaller more manageable chunks or packets.
• Append addressing to the packets.
• Ensure that data is ready for sending via the network interface card (NIC) to the network cable
The function of protocols at the receiving computer is summarized below:
• Remove packets from the network cable, and move the packets through the NIC to the computer.
• Remove all information that relate to the sending of the packet. This is information added to the packet by the sending computer.
• Move the packets to the buffer for the reassembly process.
• Convey the data to the particular application.
Internet Protocol :
Internet protocol suite is the set of communication protocols that implement the protocol stack on which the internet runs. The Internet protocol suite is sometimes called the TCP/IP protocol suite, after TCP\IP, which refers to the important protocols in it, the Transmission Control Protocol(TCP) and the Internet Protocol(IP). The Internet protocol suite can be described by the analogy with the OSI model, but there are some differences. Also not all of the layers correspond well.
Protocol Stack:
A protocol stack is the complete set of protocol layers that work together to provide networking capabilities.
Transmission Control Protocol (TCP):
The Transmission Control Protocol is the core protocol of the internet protocol suite. It originated in the network implementation in which it complemented the Internet Protocol. Therefore the entire suite is commonly referred to as TCP/IP. TCP provides reliable delivery of a stream of octets over an IP network. Ordering and error-checking are main characteristics of the TCP. All major Internet applications such as World Wide Web, email and file transfer rely on TCP.
Internet Protocol(IP):
The Internet Protocol is the principal protocol in the Internet protocol suite for relaying data across networks. Its routing function essentially establishes the internet. Historically it was the connectionless datagram service in the original Transmission Control Program; the other being the connection oriented protocol(TCP). Therefore, the Internet protocol suite is referred as TCP/IP.
Is there a beginner-friendly book or other resource that you'd
recommend?
Data Communications and Networking by Behrouz Forouzan:
This contains introductory material and the explanation is beginner friendly. At the same time, it is not dumbed down and the material gets a bit more challenging as you go on. There are very good diagrams explaining concepts too. The typesetting is awesome and you'll have lots of interesting tips surrounding the content. The chapters are ordered according to the OSI stack as mentioned in other answers here. But a lot of the math and derivations for formulas for protocol efficiencies aren't explained.
Computer Networks by Andrew S. Tanenbaum
Everything found in Behrouz Forouzan + lots of equations.
My recommendation is to read the first book first and if you are particularly curious about the math, go to the second one.
We had computer networking on school and we had to buy this book it really helps. It explains every layer of the OSI model. (From the internetcabel and routers up to the tcp udp protecol layers up to the application layer). If you want to have more basic knowledge of how it all works this is a must read.
Is there any field/option/anything that I can put in a TCP packet (be it a syn or an ack or just plain data) that I can be sure will be returned by the other end intact?
For eg. I want to "tag" a particular connection (src, srcport, dst, dstport) with a number that I can always read from a packet belonging to that connection. That means I can identify the connection without using the 4-tuple (as given above).
Yes: it is called a Client protocol encapsulated in the TCP server protocol.
In other words: define the Client protocol to meet your needs. Don't try to "shove" extra bits in the TCP overhead.
There are of course the 'options' overhead in TCP but I doubt you'll find an easy way to access these... and in any case, you shouldn't.
You could possibly abuse the TCP Timestamp option for this. It does not seem like a great idea, though.
You can have a lookup table in your application where you associate your tag with the socket.
No, there isn't any facility for what you describe.
Typically what you would do if you're writing a socket application with multiple connections to other systems, is keep track of the socket handle that belongs to each remote system. When receiving data, you are using the socket handle (in some form, don't know which OS or language you're using) so you can take appropriate action based on whichever socket handle that is.
I've never seen a server application that keeps track of connections based on the 4-tuple of address/ports. That seems like way too much work.
On rereading your question, it seems like you may be asking this from the point of view of the TCP driver level. What sort of software are you writing here?
In UDP, destination IP and destination port number are used to demultiplex the packets, but in TCP destination IP, source IP, destination port number and source port numbers (4-tuple) all needed to distinguish between the connections why reasoning for this usage.
I'm working on a network client program that connects to public servers, specified by the user. If the user gives me a hostname to connect to that has both IPv4 and IPv6 addresses (commonly, a DNS name with both A and AAAA records), I'm not sure how I should decide which address I should connect to.
The problem is that it's quite common for machines to support both IPv4 and IPv6, but only to have global connectivity over IPv4. The most common case of this is when only IPv6 link-local addresses are configured. At the moment the best alternatives I can come up with are:
Try the IPv6 address(es) first - if the connection fails, try the IPv4 address(es); or
Just let the user specify it as a config setting ("prefer_ipv6" versus "prefer_ipv4").
The problem I can see with option 1 is that the connection might not fail straight away - it might take quite a while to time out.
Please do try IPv6. In the significant majority of installations, trying to create an IPv6 connection will fail right away if it can't succeed for some reason:
if the system doesn't support IPv6 sockets, creating the socket will fail
if the system does support IPv6, and has link-local addresses configured, there won't be any routing table entry for the global IPv6 addresses. Again, the local kernel will report failure without sending any packets.
if the system does have a global IP address, but some link necessary for routing is missing, the source should be getting an ICMPv6 error message, indicating that the destination cannot be reached; likewise if the destination has an IPv6 address, but the service isn't listening on it.
There are of course cases where things can break, e.g. if a global (or tunnel) address is configured, and something falsely filters out ICMPv6 error messages. You shouldn't worry about this case - it may be just as well that IPv4 connectivity is somehow broken.
Of course, it's debatable whether you really need to try the IPv6 addresses first - you might just as well try them second. In general, you should try addresses in the order in which they are returned from getaddrinfo. Today, systems support configuration options that let administators decide in what order addresses should be returned from getaddrinfo.
Subsequent to the question being asked the IETF has proposed an answer to this question with RFC6555, a.k.a. Happy Eyeballs.
The pertinent point being the client and server may both have IPv4 and IPv6 but a hop in between may not so it is impossible to reliably predict which path will work.
You should let the system-wide configuration decide thanks to getaddrinfo(). Just like Java does. Asking every single application to try to cater for every single possible IPv6 (mis)configuration is really not scalable! In case of a misconfiguration it is much more intuitive to the user if all or none applications break.
On the other hand you want to try to log annoying delays and time-outs profusely, so users can quickly identify what to blame. Just like every other delays ideally, including (very common) DNS time-outs.
This talk has the solution. To summarize;
Sometimes there are problems with either DNS lookups or the subsequent connection to the resolved address
You don't want to wait for connecting to an IPv6 address to timeout before connecting to the IPv4 address, or vice versa
You don't want to wait for a lookup for an AAAA record to timeout before looking for an A record or vice versa
You don't want to stall while waiting for both AAAA and A records before attempting to connect with whichever record you get back first.
The solution is to lookup AAAA and A records simultaneously and independently, and to connect independently to the resolved addresses. Use whatever connection succeeds first.
The easiest way to do this is to allow the networking API do it for you using connect-by-name networking APIs. For example, in Java:
InetSocketAddress socketAddress = new InetSocketAddress("www.example.com", 80);
SocketChannel channel = SocketChannel.open(socketAddress);
channel.write(buffer);
The slide notes say at this point:
Here we make an opaque object called an InetSocketAddress from a host
and port, and then when we open that SocketChannel, that can complete
under the covers, doing whatever is necessary, without the
application ever seeing an IP address.
Windows also has connect-by-name APIs. I don’t have code fragments for
those here.
Now, I’m not saying that all implementations of these APIs necessarily
do the right thing today, but if applications are using these APIs,
then the implementations can be improved over time.
The di!erence with getaddrinfo() and similar APIs is that they
fundamentally can’t be improved over time. The API definition is that
they return you a full list of addresses, so they have to wait until
they have that full list to give you. There’s no way getaddrinfo can
return you a partial list and then later give you some more.
Some ideas:
Allow the user to specify the preference on a per-site basis.
Try IPv4 first.
Attempt IPv6 in parallel upon the first connection.
On subsequent connections, use IPv6 if the connection was successful previously.
I say to try IPv4 first because that is the protocol which is better established and tested.