How many concurrent read and write request can firebase support. Can it support 50k write and 50k read concurrent requests. What will be the average response time for both read and write at this load. Assume that the data size is not huge, say only 50k records are there and each request will be reading or writing single record only.
There is a physical limit to how much data can be written to a disk in a certain time interval (you fail to mention any time interval in your question). Firebase's limits are based on the physical limits combined with logic to keep the shared service responsive for everyone. These numbers change regularly, so we don't disclose them.
Once you reach the limit, writes will be queued and processed in turn. So any peak in write throughput will be buffered. You can detect when your writes are being buffered by using a completion listener. For buffered writes you'll see an increase in the time between when you call set() and when the completion listener fires.
For this specific case, I'd recommend setting up a small jsbin and simply testing it. A loop that writes 50k nodes should be simple enough:
var ref = firebase.database().ref();
for (var i=0; i < 50000; i++) {
ref.child(i).set(true).then(function(ref) {
// this is when the write has completed
});
}
You'll want to add some performance measurement logic in there, probably using performance.now().
Related
Context & goal
I need to periodically create snapshots of cosmosDB partitions. That is:
export all documents from a single CosmosDB partition. Ca 100-10k doc per partition, 1KB-200KB each doc, entire partition JSON usually <50M)
each document must be handled separately, with id known.
Host the process in Azure function app, using consumption plan (so mem/CPU/duration matters).
And run this for thousands of partitions..
Using Microsoft.Azure.Cosmos v3 C# API.
What I've tried
I can skip deserialization using the Container.*StreamAsync() tools in API, and avoid parsing the document contents. This should notably reduce the CPU/Mem need also avoids accidentally touching the documents to be exported with serialization roundtrip. The tricky part is how to combine it with having 10k documents per partition.
Query individually x 10k
I could query item ids per partition using SQL and just send send separate ReadItemStreamAsync(id) requests.
This skips deserialization, still have ids, I could control how many docs are in memory at given time, etc.
It would work, but it smells as too chatty, sending 1+10k requests to CosmosDB per partition, which is a lot = millions of requests.. Also, by my experience SQL-querying large documents would usually be RU-wise cheaper than loading those documents by point reads which would add up in this scale. It would be nice to be able to pull N docuents with a single (SQL query) request..
Query all as stream x 1.
There is Container.GetItemQueryStreamIterator() for which I could just pass in select * from c where c.partiton = #key. It would be simpler, use less RUs, I could control batch size with MaxItemsCount, it sends just a minimal number or requests to cosmos (query+continuations). All is good, except ..
.. it would return a single JSON array for all documents in batch and I would need to deserialize it all to split it into individual documents and mapping to their ids. Defeating the purpose of loading them as Stream.
Similarly, ReadManyItemsStreamAsync(..) would return the items as single response stream.
Question
Does the CosmosDB API provide a better way to download a lot of individual raw JSON documents without deserializing?
Preferably with having some control over how much data is being buffered in client.
While I agree that designing the solution around streaming documents with change feed is promising and might have better scalability and cost-effect on cosmosDB side, but to answer the original question ..
The chattiness of solution "Query individually x 10k" could be reduced with Bulk mode.
That is:
Prepare a bulk CosmosClient with AllowBulkExecution option
query document ids to export (select c.id from c where c.partition = #key)
(Optionally) split the ids to batches of desired size to limit the number of documents loaded in memory.
For each batch:
Load all documents in batch concurrently using ReadItemStreamAsync(id, partition), this avoids deserialization but retains link to id.
Write all documents to destination before starting next batch to release memory.
Since all reads are to the same partition, then bulk mode will internally merge the underlying requests to CosmosDB, reducing the network "chattiness" and trading this for some internal (hidden) complexity and slight increase in latency.
It's worth noting that:
It is still doing the 1+10k queries to cosmosDB + their RU cost. It's just compacted in network.
batching ids and waiting on batch completion is required as otherwise Bulk would send all internal batches concurrently (See: Bulk support improvements for Azure Cosmos DB .NET SDK). Or don't, if you prefer to max out throughput instead and don't care about memory footprint. In this case the partitions are smallish enough so it does not matter much.
Bulk has a separate internal batch size. Most likely its best to use the same value. This seems to be 100, which is a rather good chunk of data to process anyway.
Bulk may add latency to requests if waiting for internal batch to fill up
before dispatching (100ms). Imho this is largely neglible in this case and could be avoided by fully filling the internal Bulk batch bucket if possible.
This solution is not optimal, for example due to burst load put on CosmosDB, but the main benefit is simplicity of implementation, and the logic could be run in any host, on-demand, with no infra setup required..
There isn't anything out of the box that provides a means to doing on-demand, per-partition batch copying of data from Cosmos to blob storage.
However, before looking at other ways you can do this as a batch job, another approach you may consider, is to stream your data using Change Feed from Cosmos to blob storage. The reason is that, for a database like Cosmos, throughput (and cost) is measured on a per-second basis. The more you can amortize the cost of some set of operations over time, the less expensive it is. One other major thing I should point out too is, the fact that you want to do this on a per-partition basis means that the amount of throughput and cost required for the batch operation will be a combination of throughput * the number of physical partitions for your container. This is because when you increase throughput in a container, the throughput is allocated evenly across all physical partitions, so if I need 10k RU additional throughput to do some work on data in one container with 10 physical partitions, I will need to provision 100k RU/s to do the same work for the same amount of time.
Streaming data is often a less expensive solution when the amount of data involved is significant. Streaming effectively amortizes cost over time reducing the amount of throughput required to move that data elsewhere. In scenarios where the data is being moved to blob storage, often when you need the data is not important because blob storage is very cheap (0.01 USD/GB)
compared to Cosmos (0.25c USD/GB)
As an example, if I have 1M (1Kb) documents in a container with 10 physical partitions and I need to copy from one container to another, the amount of RU needed to do the entire thing will be 1M RU to read each document, then approximately 10 RU (with no indexes) to write it into another container.
Here is the breakdown for how much incremental throughput I would need and the cost for that throughput (in USD), if I ran this as a batch job over that period of time. Keep in mind that Cosmos DB charges you for the maximum throughput per hour.
Complete in 1 second = 11M RU/s $880 USD * 10 partitions = $8800 USD
Complete in 1 minute = 183K RU/s $14 USD * 10 partitions = $140 USD
Complete in 10 minutes = 18.3K $1/USD * 10 partitions = $10 USD
However, if I streamed this job over the course of a month, the incremental throughput required would be, only 4 RU/s which can be done without any additional RU at all. Another benefit is that it is usually less complex to stream data than to handle as a batch. Handling exceptions and dead-letter queuing are easier to manage. Although because you are streaming, you will need to first look up the document in blob storage and then replace it due to the data being streamed over time.
There are two simple ways you can stream data from Cosmos DB to blob storage. The easiest is Azure Data Factory. However, it doesn't really give you the ability to capture costs on a per logical partition basis as you're looking to do.
To do this you'd need to write your own utility using change feed processor. Then within the utility, as you read in and write each item, you can capture the amount of throughput to read the data (usually 1 RU/s) and can calculate the cost of writing it to blob storage based upon the per unit cost for whatever your monthly hosting cost is for the Azure Function that hosts the process.
As I prefaced, this is only a suggestion. But given the amount of data and the fact that it is on a per-partition basis, may be worth exploring.
As mentioned in Firestore usage and limits, maximum sustained write rate to a document is 1 per second. Is it also valid for batch writes, or does it already respect this limit?
I know it is a soft limit, but according to Best practices for Cloud firestore:
"Pay special attention to the 1 write per second limit for documents and the limit of 1,000,000 concurrent connections per database. These are soft limits that Cloud Firestore does not stop you from exceeding. However, going over these limits might affect performance, depending on your total read and write rates."
For example, is the following code block problematic, or batch writes handle it accordingly?
const testRef = firestore.collection("testCollection").doc("testDoc");
const testRef2 = firestore.collection("testCollection").doc("testDoc2");
const batch = db.batch();
batch.update(testRef, {
test1: "test1",
});
batch.update(testRef, {
test22: "test22",
});
batch.update(testRef, {
test33: "test33",
});
batch.update(testRef2, {
test444: "test444",
});
batch.update(testRef2, {
test5555: "test5555",
});
await batch.commit();
As the documentation says, the 1 write to a document per second is not a hard limit that is enforced. It's instead merely the physical limit of how frequently the data in the document (and the indexes) can be updated.
If you perform sustained writes more frequently than the limit, you may see delays in how long it takes to commit a write.
There is no difference here batched writes. They're merely a bunch of writes that happen in a single context, but it doesn't affect this limit in any meaningful way.
That said, a single batch like you have in your code is not going to cause problems either way.
I need to feed the db with something like 10k items, I don't need to rush and can/want stay below the 25wcu free plan.
It will take something like 6.5minutes (10000requests/25requests/sec).
Here is the question. I will loop on a json to feed the base do I have to handle the number of request by second myself or can I push to the max and it will be queued ?
I read that I may just have an error message (400?) when I exceed the limit can I just brutally retry the failed requests (eventually making more fail) until my 10k items are put in the db ?
tldr; => best way/strategy to feed the base knowing there is a limit of calls/sec
ps: it's run from a lambda idk if it matters.
Your calculation is a little bit off unless you do this constantly, because AWS actually allows you to burst a little bit (docs):
DynamoDB provides some flexibility in your per-partition throughput
provisioning by providing burst capacity. Whenever you're not fully
using a partition's throughput, DynamoDB reserves a portion of that
unused capacity for later bursts of throughput to handle usage spikes.
DynamoDB currently retains up to 5 minutes (300 seconds) of unused
read and write capacity. During an occasional burst of read or write
activity, these extra capacity units can be consumed quickly—even
faster than the per-second provisioned throughput capacity that you've
defined for your table.
Since 300 (seconds) * 25 (WCU) = 7500 this leaves you with about 7.5k items until it will actually throttle.
Afterwards just responding to the ProvisionedThroughputExceeded error by retrying later is fine - but make sure to add a small delay between retries (e.g. 1 second) as you know that it takes time for the new WCU to flow into the tocken bucket. Immediately retrying and hammering the API is not a kind way to use the service and might look like a DoS attack.
You can also write the items in Batches to reduce the amount of network requests as network throughput is also limited in Lambda. This makes handling the ProvisionedThroughputExceeded error slightly more cumbersome, because you need to inspect the response which items failed to write, but will probably be a net positive.
I would like to use firebase to load a 2D map in my site. There will be dynamic load of fields when user scroll map and also show changed map-fields.
But i am interested what is more efficient.
Read list of values even if i need only about 50% of loaded fields (e.g. 100 loaded fields)
geoRef.startAt(null, start).endAt(null, end).on('value', callback);
maybe better:
geoRef.startAt(null, start).endAt(null, end).once('value', callback);
geoRef.startAt(null, start).endAt(null, end).on('child_changed', callback);
or read a lot of single value (e.g. 50x)
geoRef.child(valueX).child(valueY).on('value', callback);
These reads will be triggerd for every user scroll. So that there will be a lot of 50x vs 1x(50%) reads.
Thanks
It's impossible to answer this question specifically without details. Are we talking 1000 records that are each 10 bytes or 1 million records that are each 5MB?
Data size and network bandwidth are the consideration here, not the number of connections. Firebase holds a socket opened to the server, so the overhead of establishing TCP connections is not a concern (as it would be with multiple HTTP requests), although the time it takes a request to return from the server (latency) is.
This leaves only two considerations: how much data and how many records.
For instance, if my system contains 1002 records and I want 1000 of them, and each is 1KB in size, it's going to be faster to simply request them all at once (since this requires only the latency of waiting for one response from the server). But if I want 10 of them, requesting them separately would likely be faster.
Even more ideal would be to segment them using priorities or split them cleverly into multiple paths by category, time frame, or another context. Then I can retrieve only segments of the data as a single transaction.
For example:
/messages/today
/messages/yesterday
/messages/all_messages
Now, assuming today is measured in hundreds and the payload is 1KB, I can just grab this whole list and iterate it client side any time I'd like--not worth the energy to grab them individually. If this is my common use case, perfect.
And assuming all_messages is measured in the millions of records, each about 1KB, then to grab 100 messages from here, I'll naturally gravitate to snagging each one individually.
I want an extremely efficient TCP client to send google protocol buffer messages. I have been using the Netty library to develop a server/client.
In tests the server seems to be able to handle up to 500k transactions per second, without to many problems, but the client tends to peak around 180k transactions per second.
I have based my client on the examples provided in the Netty documentation, but the difference is I just want to send the message and forget, I don't want a response (which most of the examples get). Is there anyway to optimize my client, so that I can achieve a higher TPS ?
Should my client maintain multiple channels, or should I be able to achieve a higher throughput than this with a single channel?
1) If the client is only interested in sending, not in receiving, you can always disable reading from channel like below
channel.setReadable(false);
2) You can increase the throughput very easily by having multiple client channels per client, and also it can scale too.
3) and you can do following tweaks to improve the performance in general (for read/ write)
Its better to have a SEDA like pipline by adding a EXecutionHandler with OrderdMemoryAwareThreadPoolExecutor, (with min, max channel memory with optimal value)
bootstrap.setPipelineFactory(new ChannelPipelineFactory() {
#Override
public ChannelPipeline getPipeline() throws Exception {
return Channels.pipeline(
executionHandler1,//sharable
new MessageDecoderHandler(),
new MessageEncoderHandler(),
executionHandler2,//sharable
new BusinessLogicHandler1(),
new BusinessLogicHandler2());
}
});
Setting the writeBufferHighWaterMark of the channel to optimal value (Make sure that setting a big value will not create congestion)
bootstrap.setOption("writeBufferHighWaterMark", 10 * 64 * 1024);
Setting the SO_READ, SO_WRITE buffer size
bootstrap.setOption("sendBufferSize", 1048576);
bootstrap.setOption("receiveBufferSize", 1048576);
Enabling the TCP No delay
bootstrap.setOption("tcpNoDelay", true);
I am not sure if "tcpNoDelay" helps to improve the throughput. Delay is there to improve the performance. None the less, I tried it and saw that the throughput actually fell more than 90%.