I'm facing a challenging problem here that don't know how to resolve:
Context: I have a game launcher that connects to my server and if doesn't detect any cheating software on the player computer, launches the game and tells the server to allow that IP to connect to the game server.
This has many potential issues like if there are multiple players under the same IP but I make a queue in that case so all is fine until here.
Now the main problem is that I don't have control over what information sends the game, I can only modify the launcher. For this reason all is IP based as that's the only way I have to identify that a certain player is logging in and has been authorised by launcher. It goes:
Launcher connects to Server and tells him to Allow IP A.
Server replies: ok (save IP A)
Launcher starts game.
Player tries to login.
A connection is established to the server, server checks if origin IP (IP A) is allowed to log in, if yes, go ahead.
So, the system even though far from ideal, does the job, and considering game is compiled and we cannot modify it, I couldn't think of better way.
Anyway now we come to the problem:
Certain players, when they open the launcher, all goes fine, game launches but then when player tries to login server denies connection as it comes from a different origin IP!
That broke up my mind, how can two tcp connections made within a few seconds of difference from client A to server B have a different client source IP? obviously this ruins all my system. I even tried to periodically fetch IP from sites like whatismyip to see if it was changing overtime but it wasn't the case, it seems like maybe because it goes to another port, or I don't know the reason, sometimes changes it and sometimes it doesn't.
It seems to be related to players being using tethering internet connections,as I e never seen this before on a common internet connection.
So basically, I'm not sure what could I do to identify/relate those two connections and this is a big problem as many players are unable to join my game and I cannot let them join without the launcher for obvious reasons.
My random ideas to resolve it range from bad to terrible:
open multiple connections to server on different ports and see if that gives different source iOS
let player connect and then do some kind of validation based on netstat check on client: when player is connected to game server I should see it there and could send that info to server, server would kick any client connected if there's no validation from launch, however, I think I would still have the problem to link both connections.
maybe there's another way that I'm not aware of to identify this connections. Assume I have full control in server side and in launcher, but I cannot change the game server packet that does the "login" attempt.
Based on your assumptions (IP-based only, game/server unmodifiable), it looks like we are hitting a wall indeed..
For the moment the only thing that comes to mind is performing multiple requests to the server instead of one, and until the user finally logs in.
I mean:
Periodically: launcher connects to server and tells him to Allow current IP. Server saves this IP and hopefully at some point you will have discovered all IPs.
Do this in the backgound until the player is finalizing its login (or a fixed period of time)
With some luck, if you open multiple connections during the whole period of time needed to start the game and login, you will have discovered and allowed all IPs of the user. This will mitigate the issue but not eliminate it.
I'll edit this post if I think about something else.
Related
I'm setting up my first server on a Raspberry Pi 4 but after reading some articles online I was wondering whether my server is ready to be open to the internet or not. I premise I'm just an individual who would like to publish some programming projects on a site that is accessible on a browser.
After some concerns I designed a PHP page which checks the client IP and returns a 403 header until i give that user the permission to access. Is it enough? Is it necessary?
And also, are there ports that are more safe to open than others?
You "can" open ports 80 and/or 443 for displaying webpages - depending on SSL certificates
I do it myself (not for web hosting) and restrict the open ports to certain IPs - my friends (not smart enough to levy an attack 😂). Though IPs are likely to change every so often and your firewall will need updating.
It's a key thing to remember that anything is open to exploitation if it's not properly maintained/set up. Also displaying a 403 isn't a silver bullet.
Port 25 would give a user access to the files on your device if proper authorisation isn't set up. Opening ports 80 and 443 will give users access to webpages but makes your device/network exposed to DoS attacks or platform level attacks. If there's a known exploit for your version of PHP or your firewall/router or possibly the device itself then an attacker will exploit it.
Hosting providers have layers upon layers of security and are constantly updating devices throughout their network. Keeping your device and platform up to date will help - but it may be worth instead investing a little in a host (from about £4 a month).
There are loads more things I can touch on but will leave it at that for now
Edit after comment:
my website is just a little project i mean who could casually target it?
Strictly speaking, anyone. "Who would want to?" Again, anyone. Sure you're a small target that wouldn't provide any useful data. But your device, once hacked, can be used as a DoS zombie or as a crypto-miner and you probably wouldn't even realise.
And also can't I use whatever port like 6969 or 45688?
Yes, strictly speaking, you can. You could tell your device to listen on that port and reply with the website data. To do this you would also need to provide the port number on the end of the URL in the format www.example.com:6969. Though, again, this isn't a silver bullet. Most security issues aren't with port-forwarding but with poor management/security and bugs in the components themselves. All a port forwarder is doing is saying "oh, device X wants data on this port... here you go".
Another point is, data sent on "Well-known ports" (1-1023) tend to have their headers checked for irregularities by the firewall - which can dispose of any irregular packets. By using a custom port the firewall doesn't really know what to expect, so it sends it anyway. Also, steer away from "Private ports" (49152-65535) these are used as source ports, not destination ports.
I want to create a program in which a user enters data and saves it on a text file. For example they enter a name and their age and it saves it and loads it. The thing is that i want this program to be available to the user at all times on any computer which means having the program online.
Do i need to make the program have networking capabilities? Or can i just have a server or host computer to hold the program and have the client access that directly?(like using the spreadsheets from google).
I don't know what you mean by "do I need to make the program have networking capabilities". If you choose to avoid the web site route then you need to have a server and client side app. If you want the program to be able to save data to a server then you need to do some socket programming.
To create a server, you need to:
create a socket
bind the socket to an address and port
listen for incoming connections
wait for clients
accept a client
send and receive data
To create a client, you need to:
create a socket
connect to a server
send and receive data
Hope this helps!
There are may ways to accomplish your goal, but the route that is probably easiest and most useful is to create a web page that implements this functionality.
Some of the many benefits include
No special software to install. Just requires a web browser.
Runs on any platform (including mobile) that has a web browser.
No software updates to push out to users. Update your website, and everyone gets the latest code.
I'm not sure if I'm asking the question properly. I'm referring to locally installed software, often called an "Agent" that keeps in regular communication with some host via HTTP. e.g. When you install LogMeIn, the Agent keeps in communication with the logmein.com server so that when you visit logmein.com with your web browser and connect to the agent, the server is able to initiate communication. The Agent, however, isn't a webserver, nor are any ports forwarded to the Agent. So, is the Agent constantly polling the server asking like a broken record, "Can I help you? Can I help you? Can I help you?" Or is the http connection from Agent to server somehow kept open? I know you can keep an http connection open, but A) how, and B) for how long? Does the Agent need to act like a less annoying broken record asking, "Can I help you? Yet? Yet? Yet?" with much more time in between each question? Or can the Agent ask once and wait indefinitely, asking again only once it learns that the connection has been dropped?
Bottom line is, I'd like to create a small little sample program for trying my hand at writing a client/server application that communicates via the Internet using HTTP. Either side needs to be able to initiate commands / requests. The Agent would likely communicate with the Server using some sort of API, perhaps RESTful. When I start the experiment, I'll be using Perl. It'd be fun to create a Hello World project that would have samples in many languages for many platforms how to write the agent and how to communicate with the server. The agent code would do client side things (e.g. determine public IP address) and send the data to the server. The server would act on the data (e.g. store IP address in a database). The server might also initiate a command to the Agent (e.g. Hey, Agent! What's your CPU type?) Proper authentication / authorization between Agent and Server is of course a necessity.
Are there any existing projects to model off of? Any existing documents? Perhaps I'm just missing terminology and if I just knew that everything I was asking can be summarized by the term foo, then the doors would be opened wide for what I could find in searches!
I looked into the code of Ubuntu's Landscape. It uses Python's Twister -- a web server for HTML5 Websockets. So I'd say what I was looking for in an answer is Websockets (bi-directional communication). That now has opened up a wealth of options, node.js, twister, mojolicious, and many many more as web servers. Turns out using Ajax to poll every few seconds is a very bad idea -- an overwhelming slam on web servers. Keep the connection open.
I have a somewhat basic understanding of network programming (and networking concepts in general) from taking a networking course in university a few years ago.
I remember being able to create a simple chat application, where the chat server is used as a central directory aware of which clients are currently online, but once a client knows another client it wants to chat with, the actual messages between them don't need to go through the server. I remember we could only test this over a bunch of LAN machines.
This C# chat program also has several comments mentioning that the program does not work over the internet: http://www.geekpedia.com/tutorial239_Csharp-Chat-Part-1---Building-the-Chat-Client.html
My question is why do these applications not work over the internet when "commercial" chat applications can. Surely, there is some way to make my computer accessible to the outer network even if its IP address is not valid outside the network of the ISP.
I see no problem with the linked-to code. The server doesn't even bind to a local address, which means it will listen for connections on all ip-addresses on the computer. There is however a comment for in the server article where the user changed the TcpListener object creation to bind to a specific address, which means clients only can connect to that specific address.
In the original server design, with using TcpListenet with only a port number, there should be nothing preventing its use on an Internet connected computer, unless there is a firewall blocking access.
Were you aware of networkComms.net and in particular the short chat example demonstrating the functionality here (It's less than 15 lines of code)? This was written specifically for people writing server-client apps in c# and given most of the problems you might come across will already have been solved and it might save you some time. This library is completely plug & play and has no issues working over the internet (as long as you can setup the necessary port forwarding where necessary).
Generally if both of your targets are behind NAT (so no true external ip addresses) and you are unable to configure port forwarding you need to look at 'TCP / UDP hole punching', quite an advanced technique.
I am to design a server that needs to serve millions of clients that are simultaneously connected with the server via TCP.
The data traffic between the server and the clients will be sparse, so bandwidth issues can be ignored.
One important requirement is that whenever the server needs to send data to any client it should use the existing TCP connection instead of opening a new connection toward the client (because the client may be behind a firewall).
Does anybody know how to do this, and what hardware/software is needed (at the least cost)?
What operating systems are you considering for this?
If using a Windows OS and using something later than Vista then you shouldn't have a problem with many thousands of connections on a single machine. I've run tests (here: http://www.lenholgate.com/blog/2005/11/windows-tcpip-server-performance.html) with a low spec Windows Server 2003 machine and easily achieved more than 70,000 active TCP connections. Some of the resource limits that affect the number of connections possible have been lifted considerably on Vista (see here: http://www.lenholgate.com/blog/2005/11/windows-tcpip-server-performance.html) and so you could probably achieve your goal with a small cluster of machines. I don't know what you'd need in front of those to route the connections.
Windows provides a facility called I/O Completion Ports (see: http://msdn.microsoft.com/en-us/magazine/cc302334.aspx) which allow you to service many thousands of concurrent connections with very few threads (I was running tests yesterday with 5000 connections saturating a link to a server with 2 threads to process the I/O...). Thus the basic architecture is very scalable.
If you want to run some tests then I have some freely available tools on my blog that allow you to thrash a simple echo server using many thousands of connections (1) and (2) and some free code which you could use to get you started (3)
The second part of your question, from your comments, is more tricky. If the client's IP address keeps changing and there's nothing between you and them that is providing NAT to give you a consistent IP address then their connections will, no doubt, be terminated and need to be re-established. If the clients detect this connection tear down when their IP address changes then they can reconnect to the server, if they can't then I would suggest that the clients need to poll the server every so often so that they can detect the connection loss and reconnect. There's nothing the server can do here as it can't predict the new IP address and it will discover that the old connection has failed when it tries to send data.
And remember, your problems are only just beginning once you get your system to scale to this level...
This problem is related to the so-called C10K problem. The C10K page lists a large number of good resources for addressing the problems you will encounter when you try to allow thousands of clients to connect to the same server.
I've come across the APE Project
a while back. It seems like a dream come true. They can support up to 100k concurrent clients on a single node. Spread them across 10 or 20 nodes, and you can serve millions. Perfect for RESTful applications. Might want to look deeper for any shared namespace. One drawback is that this is a standalone server, as in supplementary to a web server. This server is of course Open Source, so any cost is hardware/ISP related.
You cannot use UDP. If the client sends a request and you don't reply immediately, a router is going to forget the reverse route in 30 seconds or less, so your server will never be able to reply to the client.
TCP is the only option, and it, too, will give you headaches. Most routers are going to forget the route and/or drop the connection after a few minutes, so your client/server code is going to have to send "keep alives" fairly often.
I recommend setting up a "sniffer", to see how the phone companies are staying in touch with your smartphone for their "push" technology. Copy whatever they're doing, because that stuff works!
As Greg mentioned, the problem you are describing is C10K (or rather "C1M" in your case )
I recently made a simple TCP echo server on linux that scales very well with the number of sessions (only tested up to 200.000 though), by using the epoll queue. On BSD, you have something similar called kqueue.
You can check out the code if you want to. Hope this helps and good luck!
EDIT: As noted in the comments below, my original assertion that there is a 64K limit based on the number of ports is incorrect, however there is a 32K limit on the number of socket handles, so my suggested design is valid.
With a typical TCP/IP server design, you're limited in the number of simultaneous open connections you can have. The server has one listening port, and when a client connects to it the server makes an accept call, and that creates a new socket on a random port for the rest of the connection.
To handle more than 64K simultaneous connections I think you need to use UDP instead. You only need one port for the server to listen on, and you need to manage the connections using a 32-bit client ID in the packet data instead of having a separate port for each client. The 32-bit client ID could be the client's IP address, and the client can listen on a known UDP port for messages coming back from the server. That port would be the only one that needs to be open on the firewall.
With this approach, your only limitation is how quickly you can handle and respond to UDP messages. With millions of clients, even sparse traffic could give you large spikes, and if you don't read the packets fast enough your input queue will fill up and you'll start dropping packets. The C10K page Greg points to will give you strategies for that.