I have a simple client application which sends udp datagram to the server with known ip address and port and waits for respond. Client can be started on any computer or mobile device and it can be located behind several routers and firewalls... Server application listens certain port for client's datagram and replies to the client's endpoint with an answer. Server application works on windows computer with properly configured firewall, etc. So, as I understand, this simple scheme should work regardless to client's location and his firewall settings. But it looks like it doesn't work in about 75% of configurations. Server receives request from the client in 100% of cases, but in 75% of cases client can't receive response from the server, i.e. it looks like it's always blocked by something (server attempts 10 times to send an answer to the client, but without luck, i.e. client doesn't receive anything). I tried many different configurations of client computers to figure out the reasons of these issues and what I've found:
In some cases simple windows firewall can block respond packets (But how it could be possible? As I understand, all respond packets must be forwarded back to the client regardless to firewall settings.)
Some Hardware firewalls or NATs also can block respond UDP packets. And again, I can't understand why it could be possible?
The question is - is there any reliable method to deliver an answer to client? As I know, many programs, such as Skype works fine with UDP even with all these network "obstacles".
Thank you!
Related
Let's assume that I start a server at one of the computers in my private network (192.168.10.10:9900).
Now when making a request from some other computer in the same network, how does the client computer (OS?) knows which protocol to use / which protocol the server follows ? [TCP or UDP]
EDIT: As mentioned in the answers, I was basically looking for a default protocol which will be used by the client in the absence of any transport protocol information.
TCP / UDP protocols work at the transport layer level (TCP / IP MODEL) and its main difference is that TCP has a method to ensure the arrival of messages while UDP is lighter because of its virtue is to be faster in Information delivery. The use of one protocol or another is always defined by the application that will use it.
So the reference you put on the private server with ip: port 192.168.10.10:9900 is very vague to be more precise we could say that we have an Apache web server running on the ip: port 192.168.10.10:9900 (the port for default is 80 when installing the server, but it can be changed in the configuration).
Now the web servers (apache, IIS, etc.) work using the TCP protocol because when a client (computer, cell phone, etc.) consults a page through a browser (Chrome, Firefox, etc.), the ideal thing is that all the website and not just some pieces. This is why this type of servers chose and use this protocol in the first instance since they seek that in the end the result is that the user obtains the complete page regardless of whether a few more milliseconds are sacrificed in the validations involved in using TPC.
Now going to the client side. The user when visiting a web page from any browser (Chrome, Firefox, etc.) will use TCP since this protocol is already configured in the browser to send the query messages and subsequently receive the messages with the same form Website information.
Now this behavior is going to be repeated for any client / server application. For example, to change the type of application on the UDP side, we can observe the operation of DHCP services which are used to receive an IP when connecting any device to a Wi-Fi network. In this case, this service seeks to be as fast as possible (instead of the most reliable) since you want the device to connect as quickly as possible to the network, so use the UDP protocol and in this case any equipment when connecting To a WIFI network you will send your messages using this protocol.
Finally, if you want to know promptly about the type of TCP / UDP protocol used by a specific application, you can search on the Wireshark application which allows you to scan the messages that leave the device or show the protocol used in the different layers of the application.
There is no reason any client would make a request to your server, so why would it care what protocol it follows? Clients don't just randomly connect to things to see if there's a server there. So it doesn't make any difference to any client.
Normally, the client computer will use the TCP protocol as default. If you start the server using UDP protocol mode, then when you use curl -XGET 192.168.10.10:9900/test-page, it will give you back an curl: (7) Failed to connect to 192.168.10.10 port 9900: Connection refused error. You can try it, use the nc -lvp 9900 -u, it will give you that result.
The answers here are pointing to some default protocol. Its' not that, Whenever you start an application let say HTTP server, the server's internal has code to open a socket(which can be TCP or UDP), since HTTP:80 is a TCP protocol the code creates a TCP socket. Similarly for other network application it depends on their requirement what kind of transport layer protocol to use (TCP Or UDP). Like a DNS client will create a UDP socket to connect to DNS server, since DNS:53 is mostly over UDP. Both TCP and UDP have different use cases, advantages and disadvantages. Depending on there uses cases / advantages / disadvantages of UDP/TCP decision is taken to implement server using either of them.
I need to establish a connection between a server and clients which can both be behind any type of NAT. For this purpose I have a dedicated host on the internet with clean IP for hosting STUN/TURN server. I'm not going to use WebRTC, I just want to use STUN/TURN server for messaging between clients and a server. After reading RFC's, SO, etc I have some questions left unclear:
In which case STUN is used? In my understanding STUN is used only for Full-cone NAT. In all other cases TURN server must be used, because of host and/or port restriction. Is that correct?
It seems I need a signalling server to notify clients about server address and vice versa. But as soon as client/server sends a message to the signalling server, I know their outer host:port, so I let each side know other's side host:port, each side can send messages to this signalling server containing peer's host:port data, which the signalling server can use to detect which peer this message is for and forward it to corresponding peer. At first sight this logic seems to me pretty straight-forward and my signalling server becomes a TURN server - is that how TURN server is implemented? But if so, I don't understand, why would I need a TURN server like "coturn", "reTurn", etc? I know they implement ICE, but how this ICE will work, if my signalling server received message from concrete host:port of a peer, so that is the only candidate that can be used for connection with the peer?
In case of restricted NAT (port, address or symmetric), how long a client outer (public) port is open on router for receiving UDP datagrams? I read that TURN client sends refresh messages to server to keep channel open, is this how client also prevents ports from closing?
STUN can bridge P2P connections for most NATs except the symmetric variety, which have unpredictable port mapping. TURN is needed for the latter.
Signaling is typically done with TCP and a different socket. P2P media is typically UDP. So there's that distinction. You might discover the IP address with the signaling servers help, but you won't reliably discover the port. Even if both are TCP, you probably want a separate socket connection for the signaling service than the media.
From my experience: anywhere from 1-2 minutes. Sometimes longer. In the absence of data flowing in both directions, have keep alive messages that flow every 45 seconds to keep the session from getting dropped.
I'm trying to learn the basics of ARP/TCP/HTTP (in sort of a scatter-shot way).
As an example, what happens when I go to google.com and do a search?
My understanding so far:
For my machine to communicate with others (the gateway in this case),
it may need to do an ARP Broadcast (if it doesn't already have the
MAC address in the ARP cache)
It then needs to resolve google.com's IP address. It does this by
contacting the DNS server. (I'm not completely sure how it knows
where the DNS server is? Or is it the gateway that knows?)
This involves communication through the TCP protocol since HTTP is
built on it (TCP handshake: SYN, SYN/ACK, ACK, then requests for
content, then RST, RST/ACK, ACK)
To actually load a webpage, the browser gets the index.html, parses
it, then sends more requests based on what it needs? (images,etc)
And finally, to do the actual google search, I don't understand how
the browser knows to communicate "I typed something in the search box
and hit Enter".
Does this seem about right? / Did I get anything wrong or leave out anything crucial?
Firstly try to understand that your home router is two devices: a switch and a router.
Focus on these facts:
The switch connects all the devices in your LAN together(including the router).
The router merely connects your switch(LAN) with the ISP(WAN).
Your LAN is essentially an Ethernet network which works with MAC addresses.
For my machine to communicate with others (the gateway in this case),
it may need to do an ARP Broadcast (if it doesn't already have the MAC
address in the ARP cache)
Correct.
When you want to send a file from your dekstop to your laptop, you do not want to go through the router. You want to go through the switch, as that is faster(lower layer). However you only know the IP of the laptop in your network. For that reason you need to get its MAC address. That's where ARP kicks in.
In this case you would broadcast the ARP request in the LAN until someone responds to you. This could be the router or any other device connected to the switch.
It then needs to resolve google.com's IP address. It does this by
contacting the DNS server. (I'm not completely sure how it knows where
the DNS server is? Or is it the gateway that knows?)
If you use DHCP, then that has already provided you with the IP of the DNS server. If not, then it means that you manually provided the IP of the DNS. So the IP of the DNS server is stored locally on your computer.
Making a DNS request is just about putting its IP in the packet with the request and forwarding the packet to the network.
Sidenote: DHCP also provides the IP address of the router.
This involves communication through the TCP protocol since HTTP is
built on it (TCP handshake: SYN, SYN/ACK, ACK, then requests for
content, then RST, RST/ACK, ACK)
Yes. To clarify things: When your computer sends the request
FRAME[IP[TCP[GET www.google.com]]]
The frame is being sent to your LAN's switch which forwards it to the MAC of the router. Your router will open the frame to check the destination IP and route it accordingly(in this case to the WAN). Finally when the frame arrives at the server, the server will open the TCP segment and read the payload, which is the HTTP message. The ACK/SYN etc. messages are being processed just by your computer and the server and not any router or switch.
To actually load a webpage, the browser gets the index.html, parses
it, then sends more requests based on what it needs? (images,etc)
Yes. An HTML file is essentially a tree structure which can have embedded resources like images, javafiles, CSS etc. For each such resource a new request has to be sent.
Once your browser gets all these recourses, it will render the webpage.
And finally, to do the actual google search, I don't understand how
the browser knows to communicate "I typed something in the search box
and hit Enter".
When you type a single character, it is being sent to the server. The server then responds with its suggestions. Easy as that.
References(good reads):
http://www.tcpipguide.com/free/t_TheNeedForAddressResolution.htm
http://www.howtogeek.com/99001/htg-explains-routers-and-switches/
http://www.eventhelix.com/realtimemantra/networking/ip_routing.htm#.UsrYAvim3yO
http://en.wikipedia.org/wiki/Dynamic_Host_Configuration_Protocol
I have two applications that talk via TCP, both of which run on Windows XP machines. The client is a third-party application for which I have only the executable, no source. The IP address of the server it connects to is set in a text configuration file. The server is an application I am writing.
All netmasks are 255.255.255.0.
In all cases, the client runs on 192.168.142.202.
I am seeing a case where if I run my server on 192.168.142.207, everything works, but if I move my server over to another machine on the same subnet (192.168.142.105), everything does not work fine. Specifically, the connection does not seem to get properly established. I have looked at what's going on in Wireshark and would like to request assistance interpeting what I see.
On the server side, I see the 3-way handshake: SYN, SYN/ACK, ACK. I get no error codes on the return of accept(), and netstat shows the connection as established.
On the client side, the connection does not seem to be established properly. This causes the client to reconnect periodically, and it will also occasionally close all of the not-correctly-connected sockets that get created as a result. When I look at the client side in Wireshark, I most often see a SYN, SYN, SYN pattern, rather that the expected 3-way handshake. Occassionally, the 3-way handshake does appear, but even then, the client doesn't seem to be happy with the connection because it closes it.
I will note that there are actually two TCP connections between the client and server. The other connection (i.e. not the problematic connection I described above) works just fine. The problematic connection has listening port 5004; the good connection has listening port 1234.
I have placed both .txt and .pcap versions of the client and server Wireshark captures at this link: https://skydrive.live.com/redir.aspx?cid=c5beaf58ac752bb0&resid=C5BEAF58AC752BB0!105&parid=root
As far as the physical network setup goes, there is one switch in between the client and server in the case that works, and there are two switches in between the client and server in the case that doesn't work. All ping tests are successful. There are no wireless connections involved; everything is wired.
All firewalls are off.
Does anybody have any thoughts on either what the problem is or what further data I could gather to solve the problem?
Well, it appears this is not a network or network programming problem at all. I've figured out by trial and error that the third-party software that connects to me wants the machine it runs on to have a smaller IP address than the machine my software runs on. This seems completely arbitrary to me, but empirically, this very strongly appears to be the case. Arghhhh............
Thanks to any and all who may have spent time poring over the Wiresharks dumps I provided...
I've noticed recently that I don't have to forward ports for mmorpg's that I play.
I'm thinking about working on a game that people can play online and had a question.
Why is this the case given its a two way socket connection that is constantly sending data back and forth? Doesn't their server need to get through my firewall in order to connect to me?
TCP crash lesson: TCP is a two-way protocol. The challenge is that at least one host needs to initiate. Since within an MMORPG, your own computer is never acting as a server, nobody has to connect to it. All the information about game state is passed through the company's public facing servers that have public facing IP addresses (and hey, maybe they actually use port forwarding there, just to confuse my explanation... but you never have to see their pretty network internals, proxies, and other wizardry.).
Anyway, when you connect to Stackoverflow, you're making one outbound connection that requests data from the server, and then over that same connection you're receiving it back. Same exact scenario, only with a webserver instead of a game.
UPnP allows you to tackle many routers. There's also NAT Punch-Through if you have access to a third party that isn't behind a router.
Either way, port forwarding is only necessary if you wish to act as a server (or the sender in a P2P relationship). A client does not need to forward ports.
You don't need to forward ports to access the web either, despite data coming in as well.
When you make an outgoing TCP connection, your NAT router puts the connection in a table, so that when data comes in, it knows what machine in your LAN to send the packet to.
Everyone mentions TCP, but NAT works for UDP as well: The first outgoing UDP packet associates that source port with the internal IP address, and your NAT device will forward incoming traffic to that port to the correct host on the internal network.
In other words, if your computer requests the connection (outbound) first, the router opens up the port automatically, on the assumption that you're going to want data back. But if you want remote users to connect to your computer without your computer requesting it, the router would normally drop the packets since it wouldn't know where to send them (they were unsolicited). So instead, you need to tell the router to deliver any unsolicited packets at port N to your computer.
Sorry to add another answer so late, and I know one was already accepted, but I personally found the other answers to be more confusing than this simple explanation.