Why does Firestore require read operations to be executed before write operations? I understand that in general reads should be checked in transactions, whether some other transaction is also accessing data but is there some more in-depth explanation?
If you need to take some actions according to the value of a particular field, then you can use transactions. This means that the value of that field is read first and right after that is updated according to your logic. This is because a transaction absolutely requires round trip communications with the server in order to ensure that the code inside the transaction completes successfully.
If you don't care about what exists in the particular field and you want to increment a counter for example, then you can simply use increment(your_number). This operation doesn't require a round trip with the Firebase servers, it simply just increments the value.
Related
If every document in a collection is a user resource that is limited, how can you ensure the user does not go over their assigned limit?
My first thought was to take advantage of the Firestore triggers to avoid building a real backend, but the triggers sometimes fire more than once even if the inputed data has not changed. I was comparing the new doc to the old doc and taking action if certain keys did not match but if GCP fires the same function twice I get double the result. In this case incrementing or decrementing counts.
The Firestore docs state:
Events are delivered at least once, but a single event may result in multiple function invocations. Avoid depending on exactly-once mechanics, and write idempotent functions.
So in my situation the only solution I can think of is saving the event id's somewhere and ensuring they did not fire already. Or even worse doing a read on each call to count the current docs and adjust them accordingly (increasing read costs).
Whats a smart way to approach this?
If reinvocations (which while possible are quite uncommon) are a concern for your use-case, you could indeed store the ID of the invocation event or something less frequent, like (depending on the use-case) the source document ID.
I am implementing an auction process in Firestore and so far, everything works fine. Each auction item has a deadline and until the time runs out, user are able to bid on the item. The user with the highest bid wins.
However, I wonder how I can handle multiple users bidding at the same time for an item. At the moment I have a collection ("auction_item") which saves the highest bid and the user accordingly. When a user bids in the app, I make sure that the bid is higher than the current highest bid and then save the user and the latest bid.
I fear that multiple users bidding at the same time will cause multiple bids with the same amount in the collection. How do I prevent that in Firestore?
Thank you!
What you're looking for is to support an atomic operation to ensure that the writing of the bid from any of the users is done always with up-to-date and consistent data.
Firestore supports this kind of operation through transactions:
Cloud Firestore supports atomic operations for reading and writing data. In a set of atomic operations, either all of the operations succeed, or none of them are applied.
Transactions: a transaction is a set of read and write operations on one or more documents.
Please note that when using transactions the following applies:
Read operations must come before write operations.
A function calling a transaction (transaction function) might run more than once if a concurrent edit affects a document that the transaction reads.
Transaction functions should not directly modify application state.
Transactions will fail when the client is offline.
Depending on the scenario, a transaction might have a performance impact, as the function might run more than once in case of concurrent edits.
I'm developing a Flutter App and I'm using the Firebase services. I'd like to stick only to using transactions as I prefer consistency over simplicity.
await Firestore.instance.collection('user').document(id).updateData({'name': 'new name'});
await Firestore.instance.runTransaction((transaction) async {
transaction.update(Firestore.instance.collection('user').document(id), {'name': 'new name'});
});
Are there any (major) downsides to transactions? For example, are they more expensive (Firebase billing, not computationally)? After all there might be changes to the data on the Firestore database which will result in up to 5 retries.
For reference: https://firebase.google.com/docs/firestore/manage-data/transactions
"You can also make atomic changes to data using transactions. While
this is a bit heavy-handed for incrementing a vote total, it is the
right approach for more complex changes."
https://codelabs.developers.google.com/codelabs/flutter-firebase/#10
With the specific code samples you're showing, there is little advantage to using a transaction. If your document update makes a static change to a document, without regard to its existing data, a transaction doesn't make sense. The transaction you're proposing is actually just a slower version of the update, since it has to round-trip with the server twice in order to make the change. A plain update just uses a single round trip.
For example, if you want to append data to a string, two clients might overwrite each other's changes, depending on when they each read the document. Using a transaction, you can be sure that each append is going to take effect, no matter when the append was executed, since the transaction will be retried with updated data in the face of concurrency.
Typically, you should strive to get your work done without transactions if possible. For example, prefer to use FieldValue.increment() outside of a transaction instead of manually incrementing within a transaction.
Transactions are intended to be used when you have changes to make to a document (or, typically, multiple documents) that must take the current values of its fields into account before making the final write. This prevents two clients from clobbering each others' changes when they should actually work in tandem.
Please read more about transactions in the documentation to better understand how they work. It is not quite like SQL transactions.
Are there any (major) downsides to transactions?
I don't know any downsides.
For example, are they more expensive (Firebase billing, not computationally)?
No, a transaction costs like any other write operaton. For example, if you create a transaction to increase a counter, you'll be charged with only one write operation.
I'm not sure I understand your last question completely but if a transaction fails, Cloud Firestore retries the transaction for sure.
I am use Firestore and try to remove race condition in Flutter app by use transaction.
I have subcollection where add 2 document maximum.
Race condition mean more than 2 document may be add because client code is use setData. For example:
Firestore.instance.collection(‘collection').document('document').collection('subCollection’).document(subCollectionDocument2).setData({
‘document2’: documentName,
});
I am try use transaction to make sure maximum 2 document are add. So if collection has been change (For example new document add to collection) while transaction run, the transaction will fail.
But I am read docs and it seem transaction use more for race condition where set field in document, not add document in subcollection.
For example if try implement:
Firestore.instance.collection(‘collection').document('document').collection('subCollection').runTransaction((transaction) async {
}),
Give error:
error: The method 'runTransaction' isn't defined for the class 'CollectionReference'.
Can transaction be use for monitor change to subcollection?
Anyone know other solution?
Can transaction be use for monitor change to subcollection?
Transactions in Firestore work by a so-called compare-and-swap operation. In a transaction, you read a document from the database, determine its current state, and then set its new state based on that. When you've done that for the entire transaction, you send the whole package of current-state-and-new-state documents to the server. The server then checks whether the current state in the storage layer still matches what your client started with, and if so it commits the new state that you specified.
Knowing this, the only way it is possible to monitor an entire collection in a transaction is to read all documents in that collection into the transaction. While that is technically possible for small collections, it's likely to be very inefficient, and I've never seen it done in practice. Then again, for just the two documents in your collection it may be totally feasible to simply read them in the transaction.
Keep in mind though that a transaction only ensures consistent data, it doesn't necessarily limit what a malicious user can do. If you want to ensure there are never more than two documents in the collection, you should look at a server-side mechanism.
The simplest mechanism (infrastructure wise) is to use Firestore's server-side security rules, but I don't think those will work to limit the number of documents in a collection, as Doug explained in his answer to Limit a number of documents in a subcollection in firestore rules.
The most likely solution in that case is (as Doug also suggests) to use Cloud Functions to write the documents in the subcollection. That way you can simply reject direct writes from the client, and enforce any business logic you want in your Cloud Functions code, which runs in a trusted environment.
Does Google Cloud Datastore support atomic operations, for counters and such? I don't see anything about in the documentation and I noticed that updates are basically getting the whole object, updating a field, then sending the whole object back which seems like it would make atomic operations impossible. https://cloud.google.com/datastore/docs/concepts/entities#Datastore_Updating_an_entity
Not directly, but you can simulate an atomic increment operation using transactions. In the simplest case, you use a single entity and increment with a transactional read+write. Such a transaction will fail if there is a concurrent increment, so the counter does not scale well. Instead, you can operate on n entities with keys in the form of [('Counter', 'MyCounterX'), ('Elt', 'k')] for some number k, and property 'Count':
To increment, pick a random number between 1 and n, and attempt a transactional read+write. If the key doesn't exist, write a new entity on the given key with count=1. If the transaction fails, you can retry with a new random number. You could also include logic for the application to increase n on the fly it it starts hitting contention frequently.
To retrieve the count, do an ancestor query using root [('Counter', 'MyCounterX')] with a projection on the Count property, and then add up all the counts.
You can see code which implements this in the second block here.
Pop your updates in an ordered queue (for example Google Pub/Sub) and have a single processor/service that is the only service to have write permissions. The advantage of this method is you can then chose the persistence platform based on the data held and querying needs rather than atomic/acid requirements.
A simpler (and cheaper) option is to use a Background Function which is triggered by Pub/Sub so a nominated serverless function receives messages from Pub/Sub and you just concentrate on writing the increment logic.