I work for a company and we have a device that we are installing in small shops for their payment transactions. This device uses internet connection as the primary connection and in case internet goes down, it fails to 3G connection. During this time there is a downtime for few minutes
But we are having issues, where customers are calling us and says that their site goes down repeatedly throughout the day. When we look into our logs we see that our device has indeed failed over and back a number of times from primary to 3G and back to primary. We advise them that they need to check with ISP and make sure there is no internet drops.
Often customer say that they have consulted with ISP and they seem to say there are no issues from their end.
The only other possible reason that I can think of as to why the device keeps falling is due to faulty cabling. Are there are other way that we can test out that the problem is to do with Internet and not our device?
Perhaps you ought to expand the test routines included in the device, assuming the device has the memory capacity and/or libraries and computing power available.
For example, does your device determine the Internet is down only if it cannot reach a certain IP destination? If so, you may want to expand this by 1) testing to ensure timeouts aren't too short due to upstream congestion, 2) testing another known location such as Google's DNS server 8.8.8.8 when the intended destination IP fails, and 3) testing the internal gateway to determine if the ISP modem/router has rebooted for some reason.
Related
I'm facing a challenging problem here that don't know how to resolve:
Context: I have a game launcher that connects to my server and if doesn't detect any cheating software on the player computer, launches the game and tells the server to allow that IP to connect to the game server.
This has many potential issues like if there are multiple players under the same IP but I make a queue in that case so all is fine until here.
Now the main problem is that I don't have control over what information sends the game, I can only modify the launcher. For this reason all is IP based as that's the only way I have to identify that a certain player is logging in and has been authorised by launcher. It goes:
Launcher connects to Server and tells him to Allow IP A.
Server replies: ok (save IP A)
Launcher starts game.
Player tries to login.
A connection is established to the server, server checks if origin IP (IP A) is allowed to log in, if yes, go ahead.
So, the system even though far from ideal, does the job, and considering game is compiled and we cannot modify it, I couldn't think of better way.
Anyway now we come to the problem:
Certain players, when they open the launcher, all goes fine, game launches but then when player tries to login server denies connection as it comes from a different origin IP!
That broke up my mind, how can two tcp connections made within a few seconds of difference from client A to server B have a different client source IP? obviously this ruins all my system. I even tried to periodically fetch IP from sites like whatismyip to see if it was changing overtime but it wasn't the case, it seems like maybe because it goes to another port, or I don't know the reason, sometimes changes it and sometimes it doesn't.
It seems to be related to players being using tethering internet connections,as I e never seen this before on a common internet connection.
So basically, I'm not sure what could I do to identify/relate those two connections and this is a big problem as many players are unable to join my game and I cannot let them join without the launcher for obvious reasons.
My random ideas to resolve it range from bad to terrible:
open multiple connections to server on different ports and see if that gives different source iOS
let player connect and then do some kind of validation based on netstat check on client: when player is connected to game server I should see it there and could send that info to server, server would kick any client connected if there's no validation from launch, however, I think I would still have the problem to link both connections.
maybe there's another way that I'm not aware of to identify this connections. Assume I have full control in server side and in launcher, but I cannot change the game server packet that does the "login" attempt.
Based on your assumptions (IP-based only, game/server unmodifiable), it looks like we are hitting a wall indeed..
For the moment the only thing that comes to mind is performing multiple requests to the server instead of one, and until the user finally logs in.
I mean:
Periodically: launcher connects to server and tells him to Allow current IP. Server saves this IP and hopefully at some point you will have discovered all IPs.
Do this in the backgound until the player is finalizing its login (or a fixed period of time)
With some luck, if you open multiple connections during the whole period of time needed to start the game and login, you will have discovered and allowed all IPs of the user. This will mitigate the issue but not eliminate it.
I'll edit this post if I think about something else.
I wrote a Java application that communicates with a 3rd-party server that is hosted on AWS. According to what I've read, there are no connection limits on their REST API, yet regardless of how many threads I try to hit them with I am seeing a limit of 64 outgoing connections.
My client is written in Java, running under Windows 10. This number smells like an intentional limit, but I have been unable to find any documented limits specific to the aforementioned environment.
I tried pointing my application to https://test.com/ and got a limit of 128 outgoing connections which leads me to believe the limit is server-side.
How can one determine (from the client's side) whether outgoing connections are being restricted by the server-side? (I tried netstat -an but I only saw ESTABLISHED connections)
Seeing as my application is not receiving any connection failures from the server, could they be intentionally withholding SYN-ACK responses until older connections are closed?
I'm pretty sure there's no reliable way to tell from a client's side what the outgoing connections are being restricted to (and that would seem like a security issue to me if you could).
As for what the limits are, I think that also could be a difficult thing to determine from the server itself (unless it was configured at some lower bound) as there would probably be a lot of variables such as pretty much everything, from the amount of memory, ports, CPU on the machine you are connected to, from and every single one in between.
Also 64 and 128 don't seem like human set limits to me, but instead limits on something internal as it's a power of 2.
I'm searching for a solution to work on a Windows machine with multiple NICs/networks. While working within the network of a customer I tend to get a lot of problems. I need an internet connection for several things like git/sourcesafe/development system etc. but clients often can't offer internet access. So the solution is to use a mobilephone to connect to the internet and ethernet to access the client network/database.
Windows seems to be pretty random here. Sometimes everything works, some mintues later it trys to access the internet over the clients gataway or access the database via the internet. So I end up deactivting/activating NIC's all day long.
Is there some easy way to use the ethernet connection only to access the clients network and the mobile for everything else?
From what you described I think this might help: turn off DHCP for the interface that connects to your customers network and set a static address for that (without a gateway). Then configure your other interface to use DHCP. This should set ONE default gateway (your phone) and all traffic will run through it, except endpoints in your customer's net.
BTW: You can only have one default gateway at a time, of course, but in your case the gateway that was set the latest won, so to speak.
I have a questions regarding WebSocket communications in mobile connections.
I was wondering how the long-lived TCP connections can be handled for a long time in mobility networks when the user migrate among different networks. What happens to already established TCP connections when handover (hand-off) occurs?
Do different technologies (3G, 4G or etc) behave differently in this case?
I will appreciate if you could leave some online sources or articles as well that I can read more in this regard?
Thank you in advance :)
The hand-off is always transparent to the user — all TCP and voice connections are always kept active when transitioning between the towers on a commercial mobile network like LTE, UMTS etc. You might experience some periods of time where the data stops flowing, but that's about it.
I've had several opportunities to verify this myself through an interesting experiment on a T-Mobile USA's HSPA+ nationwide network. Take a 12-hour-plus drive from one major city to another one, without turning your phone off. Take a look at the area where the external IPv4-address terminates (by using traceroute). You might as well notice that it's still at the same area where you've started your trip. Now reboot the phone, and see where the external IPv4 address is routed to now. You'll notice that now it's likely terminated in a major metro area closer to where you are. I.e., your connection within the core network of the operator follows you along not just within a given city, metro or state, but also between the states and the timezones.
The reason for this is that the carrier has a Core Network, and all external connections are handled by the Packet Gateway of the Core Network, which keeps track of all the connections. More on this is documented in Chapter 7 of the book called High Performance Browser Networking (HPBN.co).
This is not really a SO but more a programmers question and I don't see what you have researched for yourself, but you certainly can't rely on a connection to stay alive, mobile or not.
In fact mobile operators kill long-living connections by resetting them after a certain amount of time or data. So you should be ready to reconnect upon a socket exception anyway.
I am to design a server that needs to serve millions of clients that are simultaneously connected with the server via TCP.
The data traffic between the server and the clients will be sparse, so bandwidth issues can be ignored.
One important requirement is that whenever the server needs to send data to any client it should use the existing TCP connection instead of opening a new connection toward the client (because the client may be behind a firewall).
Does anybody know how to do this, and what hardware/software is needed (at the least cost)?
What operating systems are you considering for this?
If using a Windows OS and using something later than Vista then you shouldn't have a problem with many thousands of connections on a single machine. I've run tests (here: http://www.lenholgate.com/blog/2005/11/windows-tcpip-server-performance.html) with a low spec Windows Server 2003 machine and easily achieved more than 70,000 active TCP connections. Some of the resource limits that affect the number of connections possible have been lifted considerably on Vista (see here: http://www.lenholgate.com/blog/2005/11/windows-tcpip-server-performance.html) and so you could probably achieve your goal with a small cluster of machines. I don't know what you'd need in front of those to route the connections.
Windows provides a facility called I/O Completion Ports (see: http://msdn.microsoft.com/en-us/magazine/cc302334.aspx) which allow you to service many thousands of concurrent connections with very few threads (I was running tests yesterday with 5000 connections saturating a link to a server with 2 threads to process the I/O...). Thus the basic architecture is very scalable.
If you want to run some tests then I have some freely available tools on my blog that allow you to thrash a simple echo server using many thousands of connections (1) and (2) and some free code which you could use to get you started (3)
The second part of your question, from your comments, is more tricky. If the client's IP address keeps changing and there's nothing between you and them that is providing NAT to give you a consistent IP address then their connections will, no doubt, be terminated and need to be re-established. If the clients detect this connection tear down when their IP address changes then they can reconnect to the server, if they can't then I would suggest that the clients need to poll the server every so often so that they can detect the connection loss and reconnect. There's nothing the server can do here as it can't predict the new IP address and it will discover that the old connection has failed when it tries to send data.
And remember, your problems are only just beginning once you get your system to scale to this level...
This problem is related to the so-called C10K problem. The C10K page lists a large number of good resources for addressing the problems you will encounter when you try to allow thousands of clients to connect to the same server.
I've come across the APE Project
a while back. It seems like a dream come true. They can support up to 100k concurrent clients on a single node. Spread them across 10 or 20 nodes, and you can serve millions. Perfect for RESTful applications. Might want to look deeper for any shared namespace. One drawback is that this is a standalone server, as in supplementary to a web server. This server is of course Open Source, so any cost is hardware/ISP related.
You cannot use UDP. If the client sends a request and you don't reply immediately, a router is going to forget the reverse route in 30 seconds or less, so your server will never be able to reply to the client.
TCP is the only option, and it, too, will give you headaches. Most routers are going to forget the route and/or drop the connection after a few minutes, so your client/server code is going to have to send "keep alives" fairly often.
I recommend setting up a "sniffer", to see how the phone companies are staying in touch with your smartphone for their "push" technology. Copy whatever they're doing, because that stuff works!
As Greg mentioned, the problem you are describing is C10K (or rather "C1M" in your case )
I recently made a simple TCP echo server on linux that scales very well with the number of sessions (only tested up to 200.000 though), by using the epoll queue. On BSD, you have something similar called kqueue.
You can check out the code if you want to. Hope this helps and good luck!
EDIT: As noted in the comments below, my original assertion that there is a 64K limit based on the number of ports is incorrect, however there is a 32K limit on the number of socket handles, so my suggested design is valid.
With a typical TCP/IP server design, you're limited in the number of simultaneous open connections you can have. The server has one listening port, and when a client connects to it the server makes an accept call, and that creates a new socket on a random port for the rest of the connection.
To handle more than 64K simultaneous connections I think you need to use UDP instead. You only need one port for the server to listen on, and you need to manage the connections using a 32-bit client ID in the packet data instead of having a separate port for each client. The 32-bit client ID could be the client's IP address, and the client can listen on a known UDP port for messages coming back from the server. That port would be the only one that needs to be open on the firewall.
With this approach, your only limitation is how quickly you can handle and respond to UDP messages. With millions of clients, even sparse traffic could give you large spikes, and if you don't read the packets fast enough your input queue will fill up and you'll start dropping packets. The C10K page Greg points to will give you strategies for that.