I'm thinking about making a networked game. I'm a little new to this, and have already run into a lot of issues trying to put together a good plan for dead reckoning and network latency, so I'd love to see some good literature on the topic. I'll describe the methods I've considered.
Originally, I just sent the player's input to the server, simulated there, and broadcast changes in the game state to all players. This made cheating difficult, but under high latency things were a little difficult to control, since you dont see the results of your own actions immediately.
This GamaSutra article has a solution that saves bandwidth and makes local input appear smooth by simulating on the client as well, but it seems to throw cheat-proofing out the window. Also, I'm not sure what to do when players start manipulating the environment, pushing rocks and the like. These previously neutral objects would temporarily become objects the client needs to send PDUs about, or perhaps multiple players do at once. Whose PDUs would win? When would the objects stop being doubly tracked by each player (to compare with the dead reckoned version)? Heaven forbid two players engage in a sumo match (e.g. start pushing each other).
This gamedev.net bit shows the gamasutra solution as inadequate, but describes a different method that doesn't really fix my collaborative boulder-pushing example. Most other things I've found are specific to shooters. I'd love to see something more geared toward games that play like SNES Zelda, but with a little more physics / momentum involved.
Note: I'm not asking about physics simulation here -- other libraries have that covered. Just strategies for making games smooth and reactive despite network latency.
Check out how Valve does it in the Source Engine: http://developer.valvesoftware.com/wiki/Source_Multiplayer_Networking
If it's for a first person shooter you'll probably have to delve into some of the topics they mention such as: prediction, compensation, and interpolation.
I find this network physics blog post by Glenn Fiedler, and even more so the response/discussion below it, awesome. It is quite lengthy, but worth-while.
In summary
Server cannot keep up with reiterating simulation whenever client input is received in a modern game physics simulation (i.e. vehicles or rigid body dynamics). Therefore the server orders all clients latency+jitter (time) ahead of server so that all incomming packets come in JIT before the server needs 'em.
He also gives an outline of how to handle the type of ownership you are asking for. The slides he showed on GDC are awesome!
On cheating
Mr Fiedler himself (and others) state that this algorithm suffers from not being very cheat-proof. This is not true. This algorithm is no less easy or hard to exploit than traditional client/server prediction (see article regarding traditional client/server prediction in #CD Sanchez' answer).
To be absolutely clear: the server is not easier to cheat simply because it receives network physical positioning just in time (rather than x milliseconds late as in traditional prediction). The clients are not affected at all, since they all receive the positional information of their opponents with the exact same latency as in traditional prediction.
No matter which algorithm you pick, you may want to add cheat-protection if you're releasing a major title. If you are, I suggest adding encryption against stooge bots (for instance an XOR stream cipher where the "keystream is generated by a pseudo-random number generator") and simple sanity checks against cracks. Some developers also implement algorithms to check that the binaries are intact (to reduce risk of cracking) or to ensure that the user isn't running a debugger (to reduce risk of a crack being developed), but those are more debatable.
If you're just making a smaller indie game, that may only be played by some few thousand players, don't bother implementing any anti-cheat algorithms until 1) you need them; or 2) the user base grows.
we have implemented a multiplayer snake game based on a mandatory server and remote players that make predictions. Every 150ms (in most cases) the server sends back a message containing all the consolidated movements sent by each remote player. If remote client movements arrive late to the server, he discards them. The client the will replay last movement.
Check out Networking education topics at the XNA Creator's Club website. It delves into topics such as network architecture (peer to peer or client/server), Network Prediction, and a few other things (in the context of XNA of course). This may help you find the answers you're looking for.
http://creators.xna.com/education/catalog/?contenttype=0&devarea=19&sort=1
You could try imposing latency to all your clients, depending on the average latency in the area. That way the client can try to work around the latency issues and it will feel similar for most players.
I'm of course not suggesting that you force a 500ms delay on everyone, but people with 50ms can be fine with 150 (extra 100ms added) in order for the gameplay to appear smoother.
In a nutshell; if you have 3 players:
John: 30ms
Paul: 150ms
Amy: 80ms
After calculations, instead of sending the data back to the clients all at the same time, you account for their latency and start sending to Paul and Amy before John, for example.
But this approach is not viable in extreme latency situations where dialup connections or wireless users could really mess it up for everybody. But it's an idea.
Related
I am researching about Peer-To-Peer network architecture for games.
What i have read from multiples sources is that Peer-To-Peer model makes it easy for people to hack. Sending incorrect data about your game character, whether it is your wrong position or the amount of health point you have.
Now I have read that one of the things to make Peer-To-Peer more secure is to put an anti-cheat system into your game, which controls some thing like: how fast has someone moved from spot A to spot B, or controls if someones health points did not change drastically without a reason.
I have also read about Lockstep, which is described as a "handshake" between all the clients in Peer-to-Peer network, where clients promise not to do certain things, for instance "move faster than X or not to be able to jump higher than Y" and then their actions are compared to the rules set in the "handshake".
To me this seems like an anti-cheat system.
What I am asking in the end is: What is Lockstep in Peer-To-Peer model, is it an Anti-Cheat system or something else and where should this system be placed in Peer-To-Peer. In every players computer or could it work if it is not in all of the players computer, should this system control the whole game, or only a subset?
Lockstep was designed primarily to save on bandwidth (in the days before broadband).
Question: How can you simulate (tens of) thousands of units, distributed across multiple systems, when you have only a vanishingly small amount of bandwidth (14400-28800 baud)?
What you can't do: Send tens of thousands of positions or deltas, every tick, across the network.
What you can do: Send only the inputs that each player makes, for example, "Player A orders this (limited size) group ID=3 of selected units to go to x=12,y=207".
However, the onus of responsibility now falls on each client application (or rather, on developers of P2P client code) to transform those inputs into exactly the same gamestate per every logic tick. Otherwise you get synchronisation errors and simulation failure, since no peer is authoritative. These sync errors can result from a great deal more than just cheaters, i.e. they can arise in many legitimate, non-cheating scenarios (and indeed, when I was a young man in the '90s playing lockstepped games, this was a frequent frustration even over LAN, which should be reliable).
So now you are using only a tiny fraction of the bandwidth. But the meticulous coding required to be certain that clients do not produce desync conditions makes this a lot harder to code than an authoritative server, where non-sane inputs or gamestate can be discarded by the server.
Cheating: It is easy to see things you shouldn't be able to see: every client has all the simulation data available. It is very hard to modify the gamestate without immediately crashing the game.
I've accidentally stumbled across this question in google search results, and thought I might as well answer years later. For future generations, you know :)
Lockstep is not an anti-cheat system, it is one of the common p2p network models used to implement online multiplayer in games (most notably in strategy games). The base concept is fairly straightforward:
The game simulation is split into fairly short time frames.
After each frame players collect input commands from that frame and send those commands over the network
Once all the players receive the commands from all the other players, they apply them to their local game simulation during the next time frame.
If simulation is deterministic (as it should be for lockstep to work), after applying the commands all the players will have the same world state. Implementing the determinism right is arguably the hardest part, especially for cross-platform games.
Being a p2p model lockstep is inherently weak to cheaters, since there is no agent in the network that can be fully trusted. As opposed to, for example, server-authoritative network models, where developer can trust a privately-owned server that hosts the game. Lockstep does not offer any special protection against cheaters by itself, but it can certainly be designed to be less (or more) vulnerable to cheating.
Here is an old but still good write-up on lockstep model used in Age of Empires series if anyone needs a concrete example.
I was receiving UDP data packets in real-time around every 200ms from the server, and from time to time I got packets in the delay of even a couple of seconds.
I have put my UDP pings (from client-side) to 200ms and it made my network a lot more responsive, but that is all learned from experimenting.
I don't have a solid knowledge of why this worked.
I would like to understand how power conservation algorithms work on mobile devices because I think that is why this happened.
After some investigation the best thing I have found so far is
http://www.crittercism.com/2014/03/200ms-the-magical-number-for-faster-response-times/
But it has not so many details on the topic nor any reference to further read.
There's probably two ways to look at this question:
Why is '200ms' so special
Users can tell when something takes longer than about 200ms - 300ms. This might include not just the transmission time, but also the time spent by the mobile device to use that data. Practically speaking, this is an important balancing point. Assuming new data is continuously available and that the network could send an update at any time, it might be a waste of resources to be any faster than about 200ms.
https://medium.com/appdiff/magic-numbers-for-app-performance-45c7e3bc9e46
Why does adjusting ping frequency seem to have an effect
Many energy conservation algorithms do two things:
Send/request updates only when necessary, but...
Periodically let the network know you're still around
This defines another balance point, since seconds may pass between 'necessary' updates, and the network might check that you're still there before that point. By adjusting the ping, you might be indicating that "I'm still here, and want updates!" every 200ms.
https://en.wikipedia.org/wiki/Nagle%27s_algorithm
This might not explain the effect you're seeing, but these would be some factors to consider.
From a programming/network point of view, what are the reasons why there are very few/no larger scale games than 16v16?
There are some 32v32 games but these are seeming exceptions to the rule.
Quite simply, scaling is hard and/or expensive, and O(n^3) is usually a pipe dream. For a game of 2v2, and a naive communication algorithm, you need each computer to communicate with ((2*2)-1) = 3 other computers (not counting some sort of connection mediation server), which comes to ((2*2)!/2) = 12 connections altogether ; likewise, for n versus n players, each computer needs to communicate with ((n*n)-1) computers, which comes to (n!/2) connections altogether.
That becomes ridiculous rather quickly, and other approaches are needed, such as "all players communicate with a central server, which provides them with updates". That is slightly more scalable, but only up to a point. Calculating state for 64 players and communicating with them (and keeping the game state synchronized, even through short disconnections!) isn't exactly simple, especially for games where latency matters (e.g. FPS).
Having massive server farms (with huge pipes to the net) which are dedicated and customized for this, can help, but it's expensive (that's a large part of your WoW subscription); if one of the players' computers is doing this work, lags will become drastically more apparent - both for lack of processing power, bandwith, and latency.
I presume you are talking about mostly first person shooter game, as MMO game supports more than 32 players online at the same time.
From a programming/network point of view, there are no reason that most computer games are no more that 16vs16. A common server can handle the load without problem, and bandwidth wouldn't be a problem.
It's really a game issue. Game that have more than 32 players is more chaotic that nothing else.
I wish I could play music or video on one computer, and have a second computer playing the same media, synchronized. As in, I can hear both computers' speakers at the same time, and it doesn't sound funny.
I want to do this over Wi-Fi, which is slightly unreliable.
Algorithmically, what's the best approach to this problem?
EDIT 1
Whether both computers "play" the same media, or one "plays" the media and streams it to the other, doesn't matter to me.
I am certain this is a tractable problem because I once saw a demo of Wi-Fi speakers. That was 5+ years ago, so I'm figure the technology should make it easier today.
(I myself was looking for an application which did this, hoping I wouldn't have to write one myself, when I stumbled upon this question.)
overview
You introduce a bit of buffer latency and use a network time-synchronization protocol to align the streams. That is, you split the stream up into packets, and timestamp each packet with "play later at time T", where T is for example 50-100ms in the future (or more if the network is glitchy). You send (or multicast) the packets on the local network, to all computers in the chorus. The computers will all play the sound at the same time because the application clock is synced.
Note that there may be other factors like OS/driver/soundcard latency which may have to be factored into the time-synchronization protocol. If you are not too discerning, the synchronization protocol may be as simple as one computer beeping every second -- plus you hitting a key on the other computer in beat. This has the advantage of accounting for any other source of lag at the OS/driver/soundcard layers, but has the disadvantage that manual intervention is needed if the clocks become desynchronized.
hybrid manual-network sync
One way to account for other sources of latency, without constant manual intervention, is to combine this approach with a standard network-clock synchronization protocol; the first time you run the protocol on new machines:
synchronize the machines with manual beat-style intervention
synchronize the machines with a network-clock sync protocol
for each machine in the chorus, take the difference of the two synchronizations; this is the OS/driver/soundcard latency of each machine, which they each keep track of
Now whenever the network backbone changes, all one needs to do is resync using the network-clock sync protocol (#2), and subtract out the OS/driver/soundcard latencies, obviating the need for manual intervention (unless you change the OS/drivers/soundcards).
nature-mimicking firefly sync
If you are doing this in a quiet room and all machines have microphones, you do not even need manual intervention (#1), because you can have them all follow a "firefly-style" synchronizing algorithm. Many species of fireflies in nature will all blink in unison. http://tinkerlog.com/2007/05/11/synchronizing-fireflies/ describes the algorithm these fireflies use: "If a firefly receives a flash of a neighbour firefly, it flashes slightly earlier." Flashes correspond to beeps or buzzes (through the soundcard, not the mobo piezo buzzer!), and seeing corresponds to listening through the microphone.
This may be a bit awkward over very large room distances due to the speed of sound, but I doubt it'll be an issue (if so, decrease rate of beeping).
The synchronization is relative to the position of the listener relative to each speaker. I don't think the reliability of the network would have as much to do with this synchronization as it would the content of the audio stream. In order to synchronize you need to find the distance between each speaker and the listener. Find the difference between each of those values and the value for the farthest speaker. For each 1.1 feet of difference, delay each of the close speakers by 1ms. This will ensure that the audio stream reaches the listener at the same time. This all assumes an open area, as any in proximity to your scenario will generate reflections of the audio waves and create destructive interference. Objects within the area may also transmit sound at a slower speed resulting in delayed sound of their own.
I'm playing around with a native (non-web) single-player game I'm writing, and it occured to me that having a daily/weekly/all-time online highscore list (think Xbox Live Leaderboard) would make the game much more interesting, adding some (small) amount of community and competition. However, I'm afraid people would see such a feature as an invitation to hacking, which would discourage regular players due to impossibly high scores.
I thought about the obvious ways of preventing such attempts (public/private key encryption, for example), but I've figured out reasonably simple ways hackers could circumvent all of my ideas (extracting the public key from the binary and thus sending fake encrypted scores, for example).
Have you ever implemented an online highscore list or leaderboard? Did you find a reasonably hacker-proof way of implementing this? If so, how did you do it? What are your experiences with hacking attempts?
At the end of the day, you are relying on trusting the client. If the client sends replays to the server, it is easy enough to replicable or modify a successful playthrough and send that to the server.
Your best bet is to raise the bar for cheating above what a player would deem worth surmounting. To do this, there are a number of proven (but oft-unmentioned) techniques you can use:
Leave blacklisted cheaters in a honeypot. They can see their own scores, but no one else can. Unless they verify by logging in with a different account, they think they have successfully hacked your game.
When someone is flagged as a cheater, defer any account repercussions from transpiring until a given point in the future. Make this point random, within one to three days. Typically, a cheater will try multiple methods and will eventually succeed. By deferring account status feedback until a later date, they fail to understand what got them caught.
Capture all game user commands and send them to the server. Verify them against other scores within a given delta. For instance, if the player used the shoot action 200 times, but obtained a score of 200,000, but the neighboring players in the game shot 5,000 times to obtain a score of 210,000, it may trigger a threshold that flags the person for further or human investigation.
Add value and persistence to your user accounts. If your user accounts have unlockables for your game, or if your game requires purchase, the weight of a ban is greater as the user cannot regain his previous account status by simply creating a new account through a web-based proxy.
No solution is ever going to be perfect while the game is running on a system under the user's control, but there are a few steps you could take to make hacking the system more trouble. In the end, the goal can only be to make hacking the system more trouble than it's worth.
Send some additional information with the high score requests to validate one the server side. If you get 5 points for every X, and the game only contains 10 Xs, then you've got some extra hoops to make the hacker to jump through to get their score accepted as valid.
Have the server send a random challenge which must be met with a few bytes of the game's binary from that offset. That means the hacker must keep a pristine copy of the binary around (just a bit more trouble).
If you have license keys, require high scores to include them, so you can ban people caught hacking the system. This also lets you track invalid attempts as defined above, to ban people testing out the protocol before the ever even submit a valid score.
All in all though, getting the game popular enough for people to care to hack it is probably a far bigger challenge.
I honestly don't think it's possible.
I've done it before using pretty simple key encryption with a compressed binary which worked well enough for the security I required but I honestly think if somebody considers cracking your online high score table a hack it will be done.
There are some pretty sad people out there who also happen to be pretty bright unless you can get them all laid it's a lost cause.
If your game has a replay system built in, you can submit replays to the server and have the server calculate the score from the replay.
This method isn't perfect, you can still cheat by slowing down the game (if it is action-based), or by writing a bot.
I've been doing some of this with my Flash games, and it's a losing battle really. Especially for ActionScript that can be decompiled into somewhat readable code without too much effort.
The way I've been doing it is a rather conventional approach of sending the score and player name in plain text and then a hash of the two (properly salted). Very few people are determined enough to take the effort to figure that out, and the few who are would do it anyway, negating all the time you put into it.
To summarize, my philosophy is to spend the time on making the game better and just make it hard enough to cheat.
One thing that might be pretty effective is to have the game submit the score to the server several times as you are playing, sending a bit of gameplay information each time, allowing you to validate if the score is "realistic". But that might be a bit over-the-top really.
That's a really hard question.
I've never implemented such thing but here's a simple aproximmation.
Your main concern is due to hackers guessing what is it your application is doing and then sending their own results.
Well, first of all, unless your application has a great success I wouldn't be worried. Doing such thing is extremely difficult.
Encryption won't help with the problem. You see, encryption helps to protect the data on its way but it doesn't protect either of the sides of the transaction before the data is encrypted (which is where the main vulnerability may be). So if you encrypt the sure, the data will remain private but it won't be safe.
If you are really worried about it I will suggest obfuscating the code and designing the score system in a way which is not completely obvious what is doing. Here we can borrow some things from an encryption protocol. Here is an example:
Let's say the score is some number m
Compute some kind of check over the score (for example the CRC or any other system you see feet. In fact, if you just invent one, no matter how lame is it it will work better)
Obtain the private key of the user (D) from your remote server (over a secure connection obviously). You're the only one which know this key.
Compute X=m^D mod n (n being the public module of your public/private key algorithm) (that is, encrypt it :P)
As you see that's just obfuscation of another kind. You can go down that way as long as you want. For example you can lookup the nearest two prime numbers to X and use them to encrypt the CRC and send it also to the server so you'll have the CRC and the score separately and with different encryption schemes.
If you use that in conjunction with obfuscation I'd say that would be difficult to hack. Nontheless even that could be reverse engingeered, it all depends on the interest and ability of the hacker but ... seriously, what kind of freak takes so much effort to change its results on a game? (Unless is WoW or something)
One last note
Obfuscator for .NET
Obfuscator for Delphi/C++
Obfuscator for assembler (x86)
As the other answer says, you are forced to trust a potentially malicious client, and a simple deterant plus a little human monitoring is going to be enough for a small game.
If you want to get fancy, you then have to look for fraud patterns in the score data, simmular to a credit card company looking at charge data. The more state the client communicates onto your server, the potentially easier it is to find a pattern of correct or incorrect behavior via code. For example. say that the client had to upload a time based audit log of the score (which maybe you can also use to let another clients watch the top games), the server can then validate if the score log breaks any of the game rules.
In the end, this is still about making it expensive enough to discourage cheating the scoreboard. You would want a system where you can always improve the (easier to update)server code to deal with any new attacks on your validation system.
#Martin.
This is how I believe Mario Kart Wii works. The added bonus is that you can let all the other players watch how the high score holder got the high score. The funny thing about this is that if you check out the fastest "Grumble Volcano" time trail, you'll see that somebody found a shortcut that let you skip 95% of the track. I'm not sure if they still have that up as the fastest time.
You can't do it on a nontrusted client platform. In practice it is possible to defeat even some "trusted" platforms.
There are various attacks which are impossible to detect in the general case - mainly modifying variables in memory. If you can't trust your own program's variables, you can't really achieve very much.
The other techniques outlined above may help, but don't solve the basic problem of running on a nontrusted platform.
Out of interest, are you sure that people will try to hack a high score table? I have had a game online for over two years now with a trivially-crackabe high score table. Many people have played it but I have no evidence that anyone's tried to crack the high scores.
Usually, the biggest defender against cheating and hacking is a community watch. If a score seems rather suspicious, a user can report the score for cheating. And if enough people report that score, the replay can be checked by the admins for validity. It is fairly easy to see the difference between a bot an an actual player, if there's already a bunch of players playing the game in full legitimacy.
The admins must oversee only those scores that get questioned, because there is a small chance that a bunch of users might bandwagon to remove a perfectly hard-earned score. And the admins only have to view the few scores that do get reported, so it's not too much of their time, even less for a small game.
Even just knowing that if you work hard to make a bot, just to be shot down again by the report system, is a deterrent in itself.
Perhaps even encrypting the replay data wouldn't hurt, either. Replay data is often small, and encrypting it wouldn't take too much more space. And to help improve that, the server itself would try out the replay by the control log, and make sure it matches up with the score achieved.
If there's something the anti-cheat system can't find, users will find it.