I'm working on Flutter and Firebase Application.
I saw some documents about Firebase that Firebase need transaction control to process lot's of firestore documents update.(==> many users)
However I'm wondering about just document reading and writing.
Is transaction also needed for just lot's of trafic of reading or just lot's of writing new documents?
Also when just one application(admin app) is updating or deleting document and lot's of other users are reading that documents, is transaction also needed?
A transaction needed when multiple users may be updated the same document at (almost) the same time in a way that may product conflicting updated. In such scenarios you use a transaction to prevent the concurrent writes from producing a conflict.
A slight variant of this is Firestore's batched write, which you can use when you want to update multiple documents atomically, but don't to first read any data to determine the new value in those documents.
If there is no chance of conflicts in your writes, you don't need to use a transaction or batched write, and using them will likely actually hurt performance.
So in your scenario, if there's only a single concurrent write coming from the admin app, you won't need transactions. If you have a specific use-case where you are not sure though, it's always best to share that specific use-case and the implementation code with us.
Related
i've been searching for what is the concurrent users limit for the cloud firestore spark plan but couldn't find it.
https://firebase.google.com/docs/firestore/quotas
It did said 1.000.000 concurrent users limit, but did not mention whether it is for the spark plan or the blaze plan. I've also tried searching answer elswhere, but did not find it answered specifically (with a source).
Help would be appreciated, thank you.
Per the Cloud Firestore pricing information (which Firebase uses):
When you use Firestore, you are charged for the following:
The number of documents you read, write, and delete.
The amount of storage that your database uses, including overhead for metadata and indexes.
The amount of network bandwidth that you use.
There is also no mention of any connection limits on Firebase's pricing page or the quotas documentation that you linked.
Unlike the Realtime Database, Cloud Firestore does not charge on a per-connection basis.
This video series also covers the ins and outs of Firebase products and is well worth sitting through.
Think of Cloud Firestore like a folder on your computer, which can contain thousands of little text files, similar to how documents in Cloud Firestore are stored. Users can update them with little chance of collision and grabbing a single document file would only require feeding 1s and 0s back to the requestor. This is why you are charged for network bandwidth rather than by individual connection.
In comparison, the RTDB was similar to one large JSON text file, with many people all trying to update it at once. Because parsing this text file on the server side was required to read and write data from it, it required compute resources to be able to do so. For this reason (among others), the number of connections the RTDB manager processes handled on behalf of spark plans were rate-limited to prevent abuse.
Hey so with my current feed database design, I am using Redis for the cache for super-fast reads, which are routed through my Google Cloud Functions. The Redis database handles all post data and timeline updates, which is great and all, but I forgot one of the most considerable caveats to this. Firebase Firestore only permits one document write per second, meaning that if I have a document that stores the post data (post_id, user_id, content, like_count), the like_count would be impossible to track with the possibility for many likes per second. Does anyone have any solutions to this?
You can shard your counter among multiple documents and query them in aggregate as needed.
You can also try Cloud Tasks queue to smooth out the write frequency. It will add considerable complexity to the system, but is really the only genericized way in GCP to manage the rate of some work. This might not work out the way you need, however.
If you use Cloud Tasks, your task will need to be configured with a rate limit, and it will have to deliver the document data to write to yet another function or other HTTP endpoint that will perform the write.
I have a use case where I log each activity open count in a firestore document (activity_name->field, and count->value)... So I wanted to know, when the user is offline and each of his activity navigation is stored in firestore cache, as soon as the user gets online and firebase sdk syncs the changes to the main database, does firestore record the synced changes as a single write or it sees the various individual field changes since the last change and record as multiple writes?
The writes are queued up in the client and delivered individually, so there will be a cost of one write for each document that was written offline.
The important issue here is not so much the billing as it is the evaluation of security rules. It's entirely possible a series of 5 writes might actually only result in 4 successful writes and 1 failure due to the violation of a security rule. If those writes were actually compressed into a single write, that would potentially cause everything to fail, which would be undesirable (you'd likely want as many writes to succeed as possible).
My users can create documents (let's say tasks) in a subcollection with a bunch of security rules checking for authentication, permissions and data validity. They can even select multiple tasks and copy them in the same collection.
Now, a regular user will likely create at most a hundred tasks at once, but what if someone with bad intentions manage to obtain my database credentials, authenticate and try to create a huge number of valid documents programmatically? This will result in Firestore scaling without problems and an unexpected surprise in my Firebase billing.
This is my first concern, but I'm also thinking about the possibility to limit a collection size for other reasons, and it would be at the same time a solution for the problem described.
I read about techniques to count documents in a collection described in the Firestore documentation, but I did not found a solution.
Keeping a counter on a doc field updated with a transaction in a cloud function would be inefficient in my case. Distributed counters increase the complexity of my data model a bit, and also I would not know how to properly read those counters in security rules for every task creation, and even if that would be an efficient solution.
Does anyone has suggestions?
I believe the way for a person to gain read/write access to your database would be to either to hack Google servers, in which case no one is safe and it doesn't really matter what you do, or to guess the exact name of your collections and documents.
As for the latter case, what I have done in my project is that for each collection and document I have used the name I wanted plus random 10-char Strings (including all kinds of chars and numbers. For example Users-x5NfaS1jCb) which kind of serve as independent, separate passwords every step of the way. This, at least, makes it difficult to guess the name of the collections and documents.
(Just like mentioned in the question) If using authentication does not cause any complications for you project, you can use it to further raise the security of your database by limiting access to users authenticating through your app only.
I guess (have never tried it) you can make use of Firebase Functions to limit the number of documents available in any given collection based on the criteria you want. This function will be invoked every time an event in created in the database.
If by "obtain my database credentials", you mean finding the username and password to your Firebase account, well it doesn't really matter what you do again. If they know what they are doing, they can take so many advantages that this particular issue will be the least of your problems.
All in all, if you ask me, your database is safe unless either someone guesses your collection and document names, or gains access to your Firebase account.
These are the only things I can think of for now. I'll try to update my answer later.
Unfortunately when using the amazing Firebase Realtime Database (ie, traditional Firebase), and the Cloud Functions thereof
There's no concept really of lockup available, other than the base transaction concept. (Which is awesome as far as it goes.) For example you can't do a say read, delete, insert.
We haven't user the new Firestore in a project yet; I'm wondering if it solves that particular problem?
This would make it tremendously useful for things like, well almost anything really, transactional game currencies, logic, etc.
Is this an advantage of Firestore?
Transactions in Firestore are more flexible than those in Realtime Database. With Realtime Database transactions, you had to choose a single location in that transaction, and you could only modify children under that location. All clients has to be using transactions to safely modify that transaction.
With Firestore transactions, you can transact using any arbitrary set of documents across any set of collections in your database, and you have atomicity on changes made to those documents. You're not obliged to choose just one collection or just one document.
There is no such thing as a "lock" in either product. Locks are not provided because they're difficult to manage correctly (avoiding deadlock) while also being scalable to millions of concurrent writers.