Sender sends the data.
Receiver waits a couple of seconds and then calculates the throughput rate / s
Receiver sends the rate at which its receiving packets (bytes / s) to sender
Sender calculates its rate of sending packets
If the rate of sender is significantly higher, reduce it to match receiving rate.
Alternatively, a more advanced approach:
Sender starts sending at a predefined min rate (eg. 1kb / s)
Receiver sends the calculated receiving rate back to sender.
If the receiving rate is the same as sending rate (taking latency into account) increase the rate by a set pct (eg. rate * 2)
Keep doing this until the sending rate becomes higher than receiving rate.
Keep monitoring the rates to account for changes in bandwidth increase / reduce rate if needed.
Could this work if you were to implement your own UDP congestion control algorithm?
Sure, it is feasible.
Now, would this give you the expected result, or is it sound...
I think you're trying to address a problem that has been studied and standardized by the very smart peoples of IETF. I'd advise you to take a look at RTP/RTCP which sit on top of UDP, would it be only to understand why it's a tough problem and grab some ideas.
https://en.wikipedia.org/wiki/RTP_Control_Protocol
"The primary function of RTCP is to provide feedback on the quality of service (QoS) in media distribution by periodically sending statistics information to participants in a streaming multimedia session."
https://en.wikipedia.org/wiki/Real-time_Transport_Protocol
The fact that it's main use case is audio/video streaming is, I think, not so important: the point of RTCP is to provide feedback among particpants about a UDP stream of data[*].
Be warned that:
These are complicated protocols, because the problem at hand is indeed complicated.
IIRC, RTCP does not define what the sender side should do with these QoS reports. It just formalize the way these reports are exchanged from side to side.
[*]: Well, not enterily true since in A/V, synchronization is an important aspect ("send/receive in a timely manner"), whereas what you're trying to do is "go as fast as possible, yet not too fast".
Related
I came across the concept of window size when browsing gRPC's dial options. Because gRPC uses HTTP/2 underneath, I dug this article up, which describes:
Flow control window is just nothing more than an integer value indicating the buffering capacity of the receiver. Each sender maintains a separate flow control window for each stream and for the overall connection.
If this is the window size gRPC is talking about and I understand this correctly. This is for HTTP/2 to maintain multiple concurrent stream within the same connection. Basically a number that's advertised to the sender about how much data the receiver wants the sender to send next. For control flow reasons, the connection puts different stream's data among different windows in serial.
My question is/are: is the window all or nothing? Meaning if my window size is n bytes, the stream won't send any data until it's accumulated at least n bytes? More generally, how do I maximize the performance of my stream if I maintain only one stream? I assume a bigger window size would help avoid overheads but increase risk for data loss?
Meaning if my window size is n bytes, the stream won't send any data until it's accumulated at least n bytes?
No.
The sender can send any number of bytes less than or equal to n.
More generally, how do I maximize the performance of my stream if I maintain only one stream?
For just one stream, just use the max possible value, 2^31-1.
Furthermore, you want to configure the receiver to send WINDOW_UPDATE frames soon enough, so that the sender always has a large enough flow control window that allows it to never stop sending.
One important thing to note is that the configuration of the max flow control window is related to the memory capacity of the receiver.
Since HTTP/2 is multiplexed, the implementation must continue to read data until the flow control window is exhausted.
Using the max flow control window, 2 GiB, means that the receiver needs to be prepared to buffer at least up to 2 GiB of data, until the application decides to consume that data.
In other words: reading the data from the network by the implementation, and consuming that data by the application may happen at different speeds; if reading is faster than consuming, the implementation must read the data and accumulate it aside until the application can consume it.
When the application consumes the data, it tells the implementation how many bytes were consumed, and the implementation may send a WINDOW_UPDATE frame to the sender, to enlarge the flow control window again, so the sender can continue to send.
Note that implementations really want to apply backpressure, i.e. wait for applications to consume the data before sending WINDOW_UPDATEs back to the sender.
If the implementation (wrongly) acknowledges consumption of data before passing it to the application, then it is open to memory blow-up, as the sender will continue to send, but the receiver is forced to accumulate it aside until the host memory of the receiver is exhausted (assuming the application is slower to consume data than the implementation to read data from the network).
Given the above, a single connection, for the max flow control window, may require up to 2 GiB of memory.
Imagine 1024 connections (not that many for a server), and you need 2 TiB of memory.
Also consider that for such large flow control windows, you may hit TCP congestion (head of line blocking) before the flow control window is exhausted.
If this happens, you are basically back to the TCP connection capacity, meaning that HTTP/2 flow control limits never trigger because the TCP limits trigger before (or you are otherwise limited by bandwidth, etc.).
Another consideration to make is that you want to avoid that the sender exhausts the flow control window and therefore is forced to stall and stop sending.
For a flow control window of 1 MiB, you don't want to receive 1 MiB of data, consume it and then send back a WINDOW_UPDATE of 1 MiB, because otherwise the client will send 1 MiB, stall, receive the WINDOW_UPDATE, send another 1 MiB, stall again, etc. (see also how to use Multiplexing http2 feature when uploading).
Historically, small flow control windows (as the one suggested in the specification of 64 KiB) were causing super-slow downloads in browsers, that quickly realized that they needed to tell servers that their flow control window was large enough so that the server would not stall the downloads.
Currently, Firefox and Chrome set it at 16 MiB.
You want to feed the sender with WINDOW_UPDATEs so it never stalls.
This is a combination of how fast the application consumes the received data, how much you want to "accumulate" the number of consumed bytes before sending the WINDOW_UPDATE (to avoid sending WINDOW_UPDATE too frequently), and how long it takes for the WINDOW_UPDATE to go from receiver to sender.
The GATT architecture of BLE lends itself to small fixed pieces of data (20 bytes max per characteristic). But in some cases, you end up wanting to “stream” some arbitrary length of data, that is greater than 20 bytes. For example, a firmware upgrade, even if you know its slow.
I’m curious what scheme others have used if any, to “stream” data (even if small and slow) over BLE characteristics.
I’ve used two different schemes to date:
One was to use a control characteristic, where the receiving device notify the sending device how much data it had received, and the sending device then used that to trigger the next write (I did both with_response, and without_response) on a different characteristic.
Another scheme I did recently, was to basically chunk the data into 19 byte segments, where the first byte indicates the number of packets to follow, when it hits 0, that clues the receiver that all of the recent updates can be concatenated and processed as a single packet.
The kind of answer I'm looking for, is an overview of how someone with experience has implemented a decent schema for doing this. And can justify why what they did is the best (or at least better) solution.
After some review of existing protocols, I ended up designing a protocol for over-the-air update of my BLE peripherals.
Design assumptions
we cannot predict stack behavior (protocol will be used with all our products, whatever the chip used and the vendor stack, either on peripheral side or on central side, potentially unknown yet),
use standard GATT service,
avoid L2CAP fragmentation,
assume packets get queued before TX,
assume there may be some dropped packets (even if stacks should not),
avoid unnecessary packet round-trips,
put code complexity on central side,
assume 4.2 enhancements are unavailable.
1 implies 2-5, 6 is a performance requirement, 7 is optimization, 8 is portability.
Overall design
After discovery of service and reading a few read-only characteristics to check compatibility of device with image to be uploaded, all upload takes place between two characteristics:
payload (write only, without response),
status (notifiable).
The whole firmware image is sent in chunks through the payload characteristic.
Payload is a 20-byte characteristic: 4-byte chunk offset, plus 16-byte data chunk.
Status notifications tell whether there is an error condition or not, and next expected payload chunk offset. This way, uploader can tell whether it may go on speculatively, sending its chunks from its own offset, or if it should resume from offset found in status notification.
Status updates are sent for two main reasons:
when all goes well (payloads flying in, in order), at a given rate (like 4Hz, not on every packet),
on error (out of order, after some time without payload received, etc.), with the same given rate (not on every erroneous packet either).
Receiver expects all chunks in order, it does no reordering. If a chunk is out of order, it gets dropped, and an error status notification is pushed.
When a status comes in, it acknowledges all chunks with smaller offsets implicitly.
Lastly, there is a transmit window on the sender side, where many successful acknowledges flying allow sender to enlarge its window (send more chunks ahead of matching acknowledge). Window is reduced if errors happen, dropped chunks probably are because of a queue overflow somewhere.
Discussion
Using "one way" PDUs (write without response and notification) is to avoid 6. above, as ATT protocol explicitly tells acknowledged PDUs (write, indications) must not be pipelined (i.e. you may not send next PDU until you received response).
Status, containing the last received chunk, palliates 5.
To abide 2. and 3., payload is a 20-byte characteristic write. 4+16 has numerous advantages, one being the offset validation with a 16-byte chunk only involves shifts, another is that chunks are always page-aligned in target flash (better for 7.).
To cope with 4., more than one chunk is sent before receiving status update, speculating it will be correctly received.
This protocol has the following features:
it adapts to radio conditions,
it adapts to queues on sender side,
there is no status flooding from target,
queues are kept filled, this allows the whole central stack to use every possible TX opportunity.
Some parameters are out of this protocol:
central should enforce short connection interval (try to enforce it in the updater app);
slave PHY should be well-behaved with slave latency (YMMV, test your vendor's stack);
you should probably compress your payload to reduce transfer time.
Numbers
With:
15% compression,
a device connected with connectionInterval = 10ms,
a master PHY limiting every connection event to 4-5 TX packets,
average radio conditions.
I get 3.8 packets per connection event on average, i.e. ~6 kB/s of useful payload after packet loss, protocol overhead, etc.
This way, upload of a 60 kB image is done in less than 10 seconds, the whole process (connection, discovery, transfer, image verification, decompression, flashing, reboot) under 20 seconds.
It depends a bit on what kind of central device you have.
Generally, Write Without Response is the way to stream data over BLE.
Packets being received out-of-order should not happen since BLE's link layer never sends the next packet before it the previous one has been acknowledged.
For Android it's very easy: just use Write Without Response to send all packets, one after another. Once you get the onCharacteristicWrite you send the next packet. That way Android automatically queues up the packets and it also has its own mechanism for flow control. When all its buffers are filled up, the onCharacteristicWrite will be called when there is space again.
iOS is not that smart however. If you send a lot of Write Without Response packets and the internal buffers are full, iOS will silently drop new packets. There are two ways around this, either implement some (maybe complex) protocol for the peripheral notifying the status of the transmission, like Nipos answer. An easier way however is to send each 10th packet or so as a Write With Response, the rest as Write Without Response. That way iOS will queue up all packets for you and not drop the Write Without Response packets. The only downside is that the Write With Response packets require one round-trip. This scheme should nevertheless give you high throughput.
What's the difference between flow control and congestion control in TCP?
This question can be broken down into two parts:
What is the overall purpose of flow and congestion control?
How is the task accomplished?
According to Wikipedia, TCP flow control relies on the window size reported in an ACK message. Congestion control also relies on acknowledgement messages. I would like to know what the difference is between the two goals, and how they work.
As to part 1, super general overview:
Flow control is controlled by the receiving side. It ensures that the sender only sends what the receiver can handle. Think of a situation where someone with a fast fiber connection might be sending to someone on dialup or something similar. The sender would have the ability to send packets very quickly, but that would be useless to the receiver on dialup, so they would need a way to throttle what the sending side can send. Flow control deals with the mechanisms available to ensure that this communication goes smoothly.
Congestion control is a method of ensuring that everyone across a network has a "fair" amount of access to network resources, at any given time. In a mixed-network environment, everyone needs to be able to assume the same general level of performance. A common scenario to help understand this is an office LAN. You have a number of LAN segments in an office all doing their thing within the LAN, but then they may all need to go out over a WAN link that is slower than the constituent LAN segments. Picture having 100mb connections within the LAN that ultimately go out through a 5mb WAN link. Some kind of congestion control would need to be in place there to ensure there are no issues across the greater network.
As to part 2:
If this is an interview-prep question, as you said above, I would consider taking some time to read up on TCP/IP in general. Don't use Wikipedia. RTFM! This is VERY much worth your time. You could argue that this is the most important protocol holding up most of the modern internet.
Things to read about for Flow Control: stop and wait, sliding window, PAUSE frames.
Things to read about for Congestion Control: QoS (Quality-of-Service), retransmission policies, windowing policies.
Beyond that, you can search for any particular vendor implementations (Cisco, etc..)
Flow Control: Sender will send enough data that can be accommodated at the receiver end.
Congestion Control: Sender will reduce the amount of sent packets to avoid overflowing the router's buffer(Queue).
Flow Control:
It makes sure that the sender does not overload the receiver.
It's a local phenomenon, unlike congestion control.
It's generally initiated by the sender.
Congestion control:
It makes sure that the network is able to handle the load of packets.
It's a global phenomenon and affects every host connected with that network.
It's initiated by the router.
Congestion control is a global issue – involves every router and host within the subnet
Flow control, that is scoped from point to point, involves just sender and receiver.
Flow control is mainly done on the receiver side, to adjust how much data the sender is injecting into the network; congestion control is mainly done on the sender side, trying to sense congestion on the network by the timing of ACK-packets, to adjust the volume of data sent to the corresponding situation.
Congestion control:In addition to preventing the router's buffer from overflowing it also deals with two other important factors
Fairness: Starvation shouldn't occur for any host connected to the network.Although the terminology is way more complex.
Efficiency: The links should be utilised to their maximum capacity such that it doesn't cause congestion.
Flow Control is done at the receiver side. If the sender sends packets greater than the receiver's buffer size, overflow occurs at the receiver's buffer. To avoid this overflow at the receiver side, there is a windowing technique used at the sender side.
Congestion control is done at the sender side. This is a global phenomenon. This happens at the router. Router's buffer overflows when many senders try to push more packets through the same link.
Flow Control:
When the sender buffer is full then we prevent the source from sending the data so that data should not be dropped.
Receiver buffer is full in this case.
It can be achieved easily by sliding window protocol.
Congestion Control
When we start transmitting the data from source then it reaches the destination with the help of the network. Congestion control preventing the source so the data should not be dropped by the router in the network.
This issue is related to the queue of the router
It is more complicated to achieve because router gets different packets from the different source connected to its network.
The Wikipedia article on TCP indicates that the IP packets transporting TCP segments can sometimes go lost, and that TCP "requests retransmission of lost data".
What exactly are the rules for requesting retransmission of lost data? At what time frequency are the retransmission requests performed? Is there an upper bound on the number? Is there functionality for the client to indicate to the server to forget about the whole TCP segment for which part went missing when the IP packet went missing?
What exactly are the rules for requesting retransmission of lost data?
The receiver does not request the retransmission. The sender waits for an ACK for the byte-range sent to the client and when not received, resends the packets, after a particular interval.
This is ARQ (Automatic Repeat reQuest). There are several ways in which this is implemented.
Stop-and-wait ARQ
Go-Back-N ARQ
Selective Repeat ARQ
are detailed in the RFC 3366.
At what time frequency are the retransmission requests performed?
The retransmissions-times and the number of attempts isn't enforced by the standard. It is implemented differently by different operating systems, but the methodology is fixed. (One of the ways to fingerprint OSs perhaps?)
The timeouts are measured in terms of the RTT (Round Trip Time) times. But this isn't needed very often due to Fast-retransmit which kicks in when 3 Duplicate ACKs are received.
Is there an upper bound on the number?
Yes there is. After a certain number of retries, the host is considered to be "down" and the sender gives up and tears down the TCP connection.
Is there functionality for the client to indicate to the server to forget about the whole TCP segment for which part went missing when the IP packet went missing?
The whole point is reliable communication. If you wanted the client to forget about some part, you wouldn't be using TCP in the first place. (UDP perhaps?)
There's no fixed time for retransmission. Simple implementations estimate the RTT (round-trip-time) and if no ACK to send data has been received in 2x that time then they re-send.
They then double the wait-time and re-send once more if again there is no reply. Rinse. Repeat.
More sophisticated systems make better estimates of how long it should take for the ACK as well as guesses about exactly which data has been lost.
The bottom-line is that there is no hard-and-fast rule about exactly when to retransmit. It's up to the implementation. All retransmissions are triggered solely by the sender based on lack of response from the receiver.
TCP never drops data so no, there is no way to indicate a server should forget about some segment.
I have noticed that viewing images or websites that are hosted on US servers (Im in europe) is considerably slower. The main reason would be the latency because of the distance.
But if 1 packet takes n milliseconds to be received, can't this be alleviated by sending more packets simultaneously?
Does this actually happen or are the packets sent one by one? And if yes what determines how many packets can be send simultaneously (something to do with the cable i guess)?
But if 1 packet takes n milliseconds
to be received, can't this be
alleviated by sending more packets
simultaneously?
Not in a boundless way, by TCP/IP standards, because there algorithms that determine how much can be in flight and not yet acknowledged to avoid overloading the whole network.
Does this actually happen or are the
packets sent one by one?
TCP can and does keep up to a certain amount of packets and data "in flight".
And if yes what determines how many
packets can be send simultaneously
(something to do with the cable i
guess)?
What cable? The same standards apply whether you're on cabled, wireless, or mixed sequences of connections (remember your packet goes through many routers on its way to the destination, and the sequence of router can change among packets).
You can start your study of TCP e.g. wikipedia. Your specific questions deal with congestion control algorithms and standard, Wikipedia will give you pointers to all relevant algorithms and RFCs, but the whole picture won't do you much good if you try to start studying at that spot without a lot of other understanding of TCP (e.g., its flow control concepts).
Wikipedia and similar encyclopedia/tutorial sites can only give you a summary of the summary, while RFCs are not studied to be readable, or understandable to non-experts. If you care about TCP, I'd recommend starting your study with Stevens' immortal trilogy of books (though there are many other valid ones, Stevens' are by far my personal favorites).
The issue is parallelism.
Latency does not directly affect your pipe's throughput. For instance, a dump truck across the country has terrible latency, but wonderful throughput if you stuff it full of 2TB tapes.
The problem is that your web browser can't start asking for things until it knows what to ask for. So, when you load a web page with ten images, you have to wait until the img tags arrive before you can send the request for them. So everything is perceptibly slower, not because your connection is saturated but because there's down time between making one request and the next.
A prefetcher helps alleviate this issue.
As far as "multiple packets at a time" are concerned, a single TCP connection will have many packets in transit at once, as specified by the window scaling algorithm the ends are using. But that only helps with one connection at a time...
TCP uses what's called a sliding window. Basically the amount of buffer space, X, the receiver has to re-assemble out of order packets. The sender can send X bytes past the last acknowledged byte, sequence number N, say. This way you can fill the pipe between sender and receiver with X unacknowledged bytes under the assumption that the packets will likely get there and if not the receiver will let you know by not acknowledging the missing packets. On each response packet the receiver sends a cumulative acknowledgment, saying "I've got all the bytes up to byte X." This lets it ack multiple packets at once.
Imagine a client sending 3 packets, X, Y, and Z, starting at sequence number N. Due to routing Y arrives first, then Z, and then X. Y and Z will be buffered at the destination's stack and when X arrives the receiver will ack N+ (the cumulative lengths of X,Y, and Z). This will bump the start of the sliding window allowing the client to send additional packets.
It's possible with selective acknowledgement to ack portions of the sliding window and ask the sender to retransmit just the lost portions. In the classic scheme is Y was lost the sender would have to resend Y and Z. Selective acknowledgement means the sender can just resend Y. Take a look at the wikipedia page.
Regarding speed, one thing that may slow you down is DNS. That adds an additional round-trip, if the IP isn't cached, before you can even request the image in question. If it's not a common site this may be the case.
TCP Illustrated volume 1, by Richard Stevens is tremendous if you want to learn more. The title sounds funny but the packets diagrams and annotated arrows from one host to the other really make this stuff easier to understand. It's one of those books that you can learn from and then end up keeping as a reference. It's one of my 3 go-to books on networking projects.
The TCP protocol will try to send more and more packets at a time, up to a certain amount (I think), until it starts noticing that they're dropping (the packets die in router/switch land when their Time To Live expires) and then it throttles back. This way it determines the size of the window (bandwidth) that it can use. If the sending host is noticing from its end a lot of dropped packets, then the receiving host is just going to see it as a slow connection. It could very well be blasting you with data, you're just not seeing it.
I guess parallel packets transmission is also possible(ya.. there can be limiit on No. of packets to be send at a time)..U will get more information about the packet transmission from topics::> message switching,packet switching ,circuit switching & virtual circuit packet switching...