My question is the same as this one but hopefully adds clarity to get an answer. After reading this fantastic article on the specifics behind NAT Traversal along with a general summary of methods found here, I'm wondering if the scenario has been accomplished or is possible. I'm writing software that serves web pages on any specified port, and am wondering if it is possible to have a web client from the WAN side connect to this server that is behind a NAT router. The reason this I'm finding this difficult is because:
I don't want to tell the user (who owns the web server) to configure their router to port forward (and many cases the user may not have privileges to do so).
UPnP I believe is often default-disabled, and is another configuration privilege not afforded to the user.
UDP Hole Punching looked promising until I realized the client is using a browser with http, and thus can communicate only through TCP, and limits my capability further by restricting options to browser-scripts.
I haven not found a successful implementation of TCP Hole Punching, considering the difficulties of maintaining state information (currently I'm looking at chownat, but am wondering how to implement TCP over a UDP tunnel from a web browser (or if that's even possible?).
Using a proxy to forward all traffic doesn't scale well (though using an external server, that is not behind a NAT, would be perfectly fine for setting up the initial connection or NAT traversal). By Scaling, I mean if many many users have their own web servers, not for the one user's web server to have high traffic (which is not a concern given the user's upload-bandwidth is often severely limited).
Right now I'm starting to think there will have to be some client-side browser script to help implement this, so the task won't be completely handled by the server. If anybody has any ideas or experience with trying to have a user connect to a web server behind a NAT router, I'd greatly help some direction! Thanks!
Related
I want my client program to communicate with a server without making the user add an exception to Windows Firewall in elevated mode. Is there a way to do this? HTTP? For instance, uTorrent and Google Chrome can both be installed by a regular (non-admin) user, and both programs network quite extensively - how do they do this? Am I missing something about how the firewall and/or ports works?
Yes there is a way. Assuming that your client program is the one running on the users machine and that your client program is the one initiating communication with the server then your client program generally would not need to require end user to open any exceptions in the windows firewall as long as you stick to using http over port 80. Http on port 80 is generally open for outbound traffic (initiated by the client) and therefor you could build your communication (and if needed your own protocol) on top of the http protocol. This is the typical scenario for webserver and webbrowsers (clients).
If you need the server to initiate the communication it becomes more complex and a lot of different approaches could be used. Choice of communication channels and structure should depend on factors like whether you would want to communicate to one client at a time or many (broadcast/multicast), do you need encryption, what are your needs for speed (throughput and latency), what kind of system are you trying to build and so on.
Many webapplications achieve an effect of a server initiated communication by using special techniques called polling, long polls, comet, websockets and so on. these work through http on top of tcp/ip on port 80. Other systems employs subscription mechanisms to be able to get notified through a third part if something new has happened. If you need server initiated communications please let me now and i will try to give a better explanation on the options.
My college has different proxies for accessing Internet like 192.168.0.2/3/4 and also a specific port number.What is the advantage of using this ? I also would like to know what exactly happens there.I also heard that my institution has different ISP connections shared over the same network. What is the role of proxy there?
It will be very easy to know if you understand what proxies do and why they are used generally. Which could be found on a magical website called www.google.com. By using a proxy, you get more control over the network because all request go through there.Your school may want to do stuffs like traffic shaping, content filtering etc. Using the proxy server will make sure all request to the internet are routed there first.
Proxies are good for a few things:
Filtering. By using a proxy, your college can filter out viruses, porn, Facebook or torrent downloads.
Logging. By requiring a username and password, the college can track what you do with your internet time, and can tell you off if you go somewhere you shouldn't or help you be allowing them to do traffic shaping, or other network maintenance.
Line Bonding. For example, if you have two ADSL lines of 5Mb, you can bond those to get a 10Mb line (normally this is done at the gateway stage, and not the proxy, but it is possible to do it at this stage of the network)
Failover. Again, this would normally be done at the gateway/router stage. This detects which lines are active and routes your traffic to those lines.
Network Connectivity. If your college is in-turn part of a bigger academic network, this could allow crossing those network boundaries to get internet access.
Although those are valid possibilities, it's probably just for Filtering...
In the wider internet, proxies are in use for allowing access to blocked content - like giving China access to Google...
Hey I am writing an app in Twisted, and as it stands I have 4 servers bound two different ports all communicating with the client via JSON. Is there anyway to bind these 4 servers to the same port and have the interactions remain the same?
For instance say the client subscribes to two different feeds, transmitted via a direct socket.
Right now I just do like
server1.read_string()
server2.read_string()
and it will read the correct JSON string from the respective feeds. Is there anyway to maintain this type of functionality but contact my server on the same port?
I do not want to throw all of the server functionality into one massive server and partition the data by header prefixes.
I don't want to do something like
s = server.read_string()
header = s.split(//some delimiter)[0]
if (header == "SERVER1")
{
// Blahh
}
It sounds like you have many clients interacting with your servers via HTTP. The standard solution is to throw a reverse proxy between the client and your servers - that proxy then forwards connections to the appropriate server depending on the URL. The reverse proxy can run on any one of your existing servers or on its own server to lighten the load.
If your data is cachable, the reverse proxy can do caching on your results too.
There are many reverse proxies available and you will want to choose one based on what sort of workload you have. Do you need it to be highly configurable? Is the data public or based on logins? How long does each connection last / how many connections to you want to hold open at once?
Squid, Varnish, HAProxy are good reverse proxies and even Apache could do this for you.
I plan to use HAProxy for Gridspy, my project as I have many ongoing connections with my clients and want to place an orbited server in the same URL path as my django server. See This tutorial for more information on how to forward many connections on port 80 from one server to many. This tutorial is focused on Comet, but your problem is even simpler than that.
If you are considering an ongoing tcp/ip connection from the browser back to your servers, seriously consider Orbited. See this tutorial about graphs via orbited and morbidQ. Orbited will also punch through firewalls and proxies better than most custom solutions will, as it looks like normal HTTP traffic.
In order to have multiple servers running on the same machine all bound to the same port, they need to be bound to different IP addresses. The only way to bind to the same port on the same IP is to enable the socket's SO_REUSESOCKET option, but then multiple servers would be able to receive each other's inbound data, really messing up your communications.
Otherwise, having a single server that uses headers to identifies the particular feeds is best. Why do you not want to do that?
Users connect to our webserver via https, and stay on a secured connection throughout their use of our service. A typical user session will establish a small handful of connections to the server (one or two).
There are a very small number of exceptions we are trying to track down. Particular users will intermittently have handfuls of hundreds of connections established. When we happen to catch the problem in the act, we can see the exchange of the SSL handshake, and from the perspective of the server, all appears to be in order. Yet we never observe a payload - the client instead connects on a new port and initiates a new handshake.
We do not have access to the client, and cannot observe the behavior from that side of the connection. Nor do we have a local scenario that can reproduce the problem.
It is our belief (though not confirmed) that the user agent is connecting to our server directly, and not through a proxy.
Does anybody recognize these symptoms? Can anyone suggest steps to further identify the problem?
Are there any patterns you can see to this traffic, aside from making many repeated requests?
For example, do the requests come from the same IP ranges? Possibly search engines or other spiders, or maybe from countries that you normally don't get users from, possibly indicating some sort of weird botnet or at least something you could block?
Do these rogue requests always negotiate to use a particular cipher suite, potentially indicating the client software?
Does it make any difference if you change the available cipher suites available for negotiation?
What server software are you using, and are there any firewalls within your network that could potentially be dropping some responses to the user?
i've seen a botnet flooding https sites being mentoned.
this is probably not your situation, but i thought i mention it.
i'm seeing chrome (12.0.742.60 beta) flooding my server with https connections, some half a dozen or more connections for a single static picture being served... as if it had an optimization to build up connections with ready https handshakes waiting for requests to send, and then after the page (file) has been served it closes them all.
on plain http i see only two connections (one extra for favicon.ico).
I have 50 machines in a LAN and each of these have internet access. Can a program be developed using vc++ which will tell what are all the websites which is being opened by users in each machine?
You can easily accomplish this by writing an application which captures packets outbound on port 80 (and the associated DNS information). The problem is that this application must run on every client computer which you want to trace. The easier method, as stated by others, is to take advantage of your network architecture and tunnel all traffic through a central proxy which can record the same information.
There are many-many enterprise tools suited for just this task in the latter instance.
Route your internet traffic through a centralized proxy and monitor the traffic from proxy say using Fiddler, or something else. In case proxying is not possible, use Fiddler to generate data at known location and then collate it at required intervals.
Install a firewall, if you don't already have one, and use it to log connections.