Reading lots on this for my first network game, I understand the core difference of guaranteed delivery versus time-to-deliver for TCP v UDP. I've also read diametrically opposed views whether realtime games should use UDP or TCP! ;)
What no-one has covered well is how to handle the issue of a dropped packet.
TCP : Read an article using TCP for an FPS that recommended only using TCP. How would an authoritative server using TCP client input handle a packet drop and sudden epic spike in lag? Does the game just stop for a moment and then pick up where it left off? Is TCP packet loss so rare that it's not really that much of an issue and an FPS over TCP actually works well?
UDP : Another article suggested only ever using UDP. Clearly one-shot UDP events like "grenade thrown" aren't reliable enough as they won't fire some of the time. Do you have to implement a message-received, resend protocol manually? Or some other solution?
My game is a tick-based authoritative server with 1/10th second updates from the server to clients and local simulation to keep things seeming more responsive, although the question is applicable to a lot more applications.
I did a real-time TV editing system. All real-time communication was via UDP, but none-real-time used TCP as it is simpler. With the UDP we would send a state packet every frame. e.g. start video in 100 frames, 99,98,…3,2,1,0,-1,-2,-3 so even if no message gets through until -3 then the receiver would start on the 4th frame (just skipping the first 3), hoping that no one would notice, and knowing that this was better than lagging from here on in. We even added the countdown from around +¼ second (as no-one will notice), this way hardly any frames where dropped.
So in summary, we sent the same status packet every frame. It contained all real-time data about past, current, and future events.
The trick is keeping this data-set small. So instead of sending play button pressed event (there is an unbound number of these), we send the video-id, frame-number, start-mask and end-mask. (start/stop mask are frame numbers, if start-mask is positive and stop-mask is negative then show video, at frame frame-number).
Now we need to be able to start a video during another or shortly after it stops. So we consider how many consecutive video can be played at the same time. We need a slot for each, but can we reuse them immediately? If we have pressed stop, so do not know the stop mask until then, then reuse the slot will the video stop. Well there will be no slot for this video, so we should stop it. So yes we can reuse the slot immediately, as long as we use unique IDs.
Other tips: Do not send +1 events instead send current total. If two players have to update the some total, then each should have their own total, sum all totals at point of use, but never edit someone else's total.
Related
Context
We have got un unsteady transmission channel. Some packets may be lost.
Sending a single network packet in any direction (from A to B or from B to A) takes 3 seconds.
We allow a signal delay of 5 seconds, no more. So we have a 5-second buffer. We can use those 5 seconds however we want.
Currently we use only 80% of the transmission channel, so we have 1/4 more room to utilize.
The quality of the video cannot be worsened.
Problem
We need to make the quality better. How to handle lost packets?
Solution proposition
A certain thing - we cannot use TCP in this case, because when TCP detects some problems, it requests retransmission of lost data. That would mean that a packet would arrive after 9 seconds, which is more than the limit.
Therefore we need to use UDP and handle those errors ourselves. How to do it then? How to make sure that not so many packets will be lost as currently, without retransmitting them?
Its a complicated solution, but by far the best option will be to add forward error correction (FEC). This is how images are transfered form space probes where the latency is measures in minutes or hours. Its also use by cell phones where delayed packets are bad for two way communication.
A not as good, but eaiser to implement option is to use UDT. This is a UDP with tcp like retranmission library, but allows you much more controll over the protocol.
Context
We have got un unsteady transmission channel. Some packets may be lost.
Sending a single network packet in any direction (from A to B or from B to A) takes 3 seconds.
We allow a signal delay of 5 seconds, no more. So we have a 5-second buffer. We can use those 5 seconds however we want.
Currently we use only 80% of the transmission channel, so we have 1/4 more room to utilize.
The quality of the video cannot be worsened.
We need to use UDP.
Problem
We need to make the quality better. How to handle lost packets? We need to use UDP and handle those errors ourselves. How to do it then? How to make sure that not so many packets will be lost as currently (we can't guarantee 100%, so we only want it better), without retransmitting them? We can do everything, this is theory.
There are different logic's to handle these things.It depends on what application you are using. Are you doing real time video streaming? stringent requirements?
As you said you have a buffer, you can actually maintain a buffer for the packets and then send an acknowledgement for the lost packets (if you feel you can wait).
As this is video application, send acknowledgements only to the key frames. make sure that you have a key or I frame and then do interpolation at the rx side.
Look into something called forward error correction, fountain codes, luby codes. Here, you will encode the packets 1 and 2 and produce packet 3. If packet1 is lost, use packet3 and packet2 to get the packet1 back at the rx side. Basically you send redundant packets. Its little harsh on network but you get most of the data.
i need to create some statistics from packets in my network interface, but i'm concerned only for my tcp sessions. i thought i could do that with nfdump and nfsen. because i'm new to this stuff, i dont really get what nfdump defines as 'flow'.
furthermore, can i get statistics with these tools only for the tcp protocol sessions? i mean, for example, that i need to have some average duration of all the connections(srcip-srcport, dstip-dstport pairs) in a server of mine. And for this reason i need the time between the 3WH and the closing of each connection (either with [fin/ack,ack] or with [rst]). Is that possible with nfdump-nfsen?
Short answer here is no: you don't have anything in your list of software that generates netflow information. It's not going to work. Netflow collectors do not work as hard as you might like to maintain your idea of a connection - a flow is just a collection of related packets that happen during part of the collection cycle. For a long-lived session, you can expect to see a few flows.
For your application, you will do better to capture syn-ack and fin packets with tcpdump and analyse the timing of these with your favourite text processing tool.
Also, on the left side of your keyboard, you may find a key with an arrow that allows you to type capital letters.
I'm learning about the various networking technologies, specifically the protocols UDP and TCP.
I've read numerous times that games like Quake use UDP because, "it doesn't matter if you miss a position update packet for a missile or the like, because the next packet will put the missile where it needs to be."
This thought process is all well-and-good during the flight path of an object, but it's not good for when the missile reaches it's target. If one computer receives the message that the missile reached it's intended target, but that packet got dropped on a different computer, that would cause some trouble.
Clearly that type of thing doesn't really happen in games like Quake, so what strategy are they using to make sure that everyone is in sync with instantaneous type events, such as a collision?
You've identified two distinct kinds of information:
updates that can be safely missed, because the information they carry will be provided in the next update;
updates that can't be missed, because the information they carry is not part of the next regular update.
You're right - and what the games typically do is to separate out those two kinds of messages within their protocol, and require acknowledgements and retransmissions for the second type, but not for the first type. (If the underlying IP protocol is UDP, then these acknowledgements / retransmissions need to be provided at a higher layer).
When you say that "clearly doesn't happen", you clearly haven't played games on a lossy connection. A popular trick amongst the console crowd is to put a switch on the receive line of your ethernet connection so you can make your console temporarily stop receiving packets, so everybody is nice and still for you to shoot them all.
The reason that could happen is the console that did the shooting decides if it was a hit or not, and relays that information to the opponent. That ensures out of sync or laggy hit data can be deterministically decided. Even if the remote end didn't think that the shot was a hit, it should be close enough that it doesn't seem horribly bad. It works in a reasonable manner, except for what I've mentioned above. Of course, if you assume your players are not cheating, this approach works quite reasonably.
I'm no expert, but there seems to be two approaches you can take. Let the client decide if it's a hit or not (allows for cheating), or let the server decide.
With the former, if you shoot a bullet, and it looks like a hit, it will count as a hit. There may be a bit of a delay before everyone else receives this data though (i.e., you may hit someone, but they'll still be able to play for half a second, and then drop dead).
With the latter, as long as the server receives the information that you shot a bullet, it can use whatever positions it currently has to determine if there was a hit or not, then send that data back for you. This means neither you nor the victim will be aware of you hit or not until that data is sent back to you.
I guess to "smooth" it out you let the client decide for itself, and then if the server pipes in and says "no, that didn't happen" it corrects. Which I suppose could mean players popping back to life, but I reckon it would make more sense just to set their life to 0 and until you get a definitive answer so you don't have weird graphical things going on.
As for ensuring the server/client has received the event... I guess there are two more approaches. Either get the server/client to respond "Yeah, I received the event" or forget about events altogether and just think about everything in terms of state. There is no "hit" event, there's just HP before and after. Sooner or later, it'll receive the most up-to-date state.
I wish I could play music or video on one computer, and have a second computer playing the same media, synchronized. As in, I can hear both computers' speakers at the same time, and it doesn't sound funny.
I want to do this over Wi-Fi, which is slightly unreliable.
Algorithmically, what's the best approach to this problem?
EDIT 1
Whether both computers "play" the same media, or one "plays" the media and streams it to the other, doesn't matter to me.
I am certain this is a tractable problem because I once saw a demo of Wi-Fi speakers. That was 5+ years ago, so I'm figure the technology should make it easier today.
(I myself was looking for an application which did this, hoping I wouldn't have to write one myself, when I stumbled upon this question.)
overview
You introduce a bit of buffer latency and use a network time-synchronization protocol to align the streams. That is, you split the stream up into packets, and timestamp each packet with "play later at time T", where T is for example 50-100ms in the future (or more if the network is glitchy). You send (or multicast) the packets on the local network, to all computers in the chorus. The computers will all play the sound at the same time because the application clock is synced.
Note that there may be other factors like OS/driver/soundcard latency which may have to be factored into the time-synchronization protocol. If you are not too discerning, the synchronization protocol may be as simple as one computer beeping every second -- plus you hitting a key on the other computer in beat. This has the advantage of accounting for any other source of lag at the OS/driver/soundcard layers, but has the disadvantage that manual intervention is needed if the clocks become desynchronized.
hybrid manual-network sync
One way to account for other sources of latency, without constant manual intervention, is to combine this approach with a standard network-clock synchronization protocol; the first time you run the protocol on new machines:
synchronize the machines with manual beat-style intervention
synchronize the machines with a network-clock sync protocol
for each machine in the chorus, take the difference of the two synchronizations; this is the OS/driver/soundcard latency of each machine, which they each keep track of
Now whenever the network backbone changes, all one needs to do is resync using the network-clock sync protocol (#2), and subtract out the OS/driver/soundcard latencies, obviating the need for manual intervention (unless you change the OS/drivers/soundcards).
nature-mimicking firefly sync
If you are doing this in a quiet room and all machines have microphones, you do not even need manual intervention (#1), because you can have them all follow a "firefly-style" synchronizing algorithm. Many species of fireflies in nature will all blink in unison. http://tinkerlog.com/2007/05/11/synchronizing-fireflies/ describes the algorithm these fireflies use: "If a firefly receives a flash of a neighbour firefly, it flashes slightly earlier." Flashes correspond to beeps or buzzes (through the soundcard, not the mobo piezo buzzer!), and seeing corresponds to listening through the microphone.
This may be a bit awkward over very large room distances due to the speed of sound, but I doubt it'll be an issue (if so, decrease rate of beeping).
The synchronization is relative to the position of the listener relative to each speaker. I don't think the reliability of the network would have as much to do with this synchronization as it would the content of the audio stream. In order to synchronize you need to find the distance between each speaker and the listener. Find the difference between each of those values and the value for the farthest speaker. For each 1.1 feet of difference, delay each of the close speakers by 1ms. This will ensure that the audio stream reaches the listener at the same time. This all assumes an open area, as any in proximity to your scenario will generate reflections of the audio waves and create destructive interference. Objects within the area may also transmit sound at a slower speed resulting in delayed sound of their own.