Can anyone shed some light on how billing works in Firestore (Datastore mode) when a transaction is rolled back? How many writes is one going to be billed if one batch deletes 500 entities (DeleteMulti) then rolls back the transaction?
From a billing/quota perspective, operations are tracked exactly the same in a transaction regardless of whether the transaction succeeds and is committed, or rolled back.
This means that if a you attempts to create 5 items, then delete 2 existing ones, then create one more, they will be charged for 6 write operations and 2 deletes. This will be the case if the whole transaction succeeds, or if the transaction fails on any of the individual operations, even if those operations are never committed and do not appear in the data after the transaction has been rolled back.
So answering your question, you would be billed for 500 deletes, regardless of whether the transaction succeeds or is rolled back.
Related
For example, I have 10 documents in my collection. 10 requests come in near the same second and will run the same query. They each will start their own transaction that will try and read 1 document, and then delete that document. Given the firestore documentation with document contention, it is made to seem like contention errors happen when more than one transaction occurs on the same document X amount of times (it is not documented how many times).
Cloud Firestore resolves data contention by delaying or failing one of the operations.
https://cloud.google.com/firestore/docs/transaction-data-contention
However, in this case since 1 of those transactions committed, I am assuming the other 9 that tried to operate on that same document, will retry because the document from the query was "changed" and couldn't commit. Then the next 9 transactions will try to do the same thing, but on another document, and this will continue until all requests finished deleting 1 document and there are no more active transactions.
Would the retry rules of these transactions that kept getting retried, be ABORTED due to contention, even though it's been a different document each time? Or would these transactions just keep getting delayed and retried because the contention is happening on different documents on each attempt?
According to the documentation, the transaction will retry a "finite number of times". This number is dependent on how the SDK itself is configured, which may be different for various SDK platforms and versions. It doesn't matter which contended document(s) caused the retry. The max number of retries is absolute for that transaction in order to avoid excessive work.
Newer versions of the SDK allow configuration of the number of retries (e.g. Android 24.4.0 lets you specify TransactionOptions).
After reading the documentation, there is one thing that is not completely clear to me, I am hoping someone can clarify.
Obviously if you have two read-then-write transactions that occur concurrently, one of them will fail. However, if you have one read-then-write transaction in progress, and then another read occurs that was not part of a transaction, will that other non transactional read cancel the transaction, i.e. A transaction to read-then-write on a payment/transaction record should not be cancelled by the non transactional "Get all payment data" report. Is that correct?
Yes, that's correct. Non-transactional reads and read-only transactions do not contend with read-then-write transactions.
From following document: https://cloud.google.com/datastore/docs/concepts/transactions
What would happen if transaction fails with no explicit rollback defined? For example, if we're performing put() operation on value arguments.
The document states that transaction should be idempotent, what does this mean with respect to put() operation? It is not clear how idempotency is applied in this context.
How do we detect failure if failure from commit is not reliable according to the documentation?
We are seeing some symptoms where put() against value argument is sometimes partially saving the data. Note we do not have explicit rollback defined.
As you may already know, Datastore transactions are guaranteed to be atomic, which means that it applies the all-or-nothing principle; either all operations succeed or they all fail. This ensures that the data in your database remains consistent over time.
Now, regardless whether you execute put or any other operation in your transaction, your implementation of the code should always ensure that your transaction has either successfully commited or rolled back. This means that if you aren't fully sure whether the commit succeeded, you should explicitly issue a rollback.
However, there may be some exceptions where a commit might fail, and this doesn't necessarily mean that no data was written to your database. The documentation even points out that "you can receive errors in cases where transactions have been committed."
The simple way to detect transaction failures would be to add a try/catch block in your code for when an Exception (failed transactional operation) or DatastoreException (errors related to Datastore - failed commit) are thrown. I believe that you may already have an answer in this Stackoverflow post about this particular question.
A good practice is to make your transactions idempotent whenever possible. In other words, if you're executing a transaction that includes a write operation put() to your database, if this operation were to fail and needed to be retried, the end result should ideally remain the same.
A real world example can be - you're trying to transfer some money to your friend; the transaction consists of withdrawing 20 USD from your bank account and depositing this same amount into your friend's bank account. If the transaction were to fail and had to be retried, the transaction should still operate with the same amount of money (20 USD) as the final result.
Keep in mind that the Datastore API doesn't retry transactions by default, but you can add your own retry logic to your code, as per the documentation.
In summary, if a transaction is interrupted and your logic doesn't handle the failure accordingly, you may eventually see inconsistencies in the data of your database.
I'm curious about the behavior of the locks that are performed when doing server side transactions on Cloud Firestore as mentioned in this video: https://www.youtube.com/watch?time_continue=750&v=dOVSr0OsAoU
My transaction will be reading multiple documents and placing locks on them. My question is do these locks restrict all access to the documents - including concurrent reads from client code that isn't part of a transaction? Or do they only restrict writes?
If they do restrict reads is there any way around this - it could lead to severe slowdown in the app I'm working on.
Also in the case that a transaction tries to lock documents that are already locked - what is the retry pattern - how often does it retry, and is there an exponential backoff?
Thanks!
My transaction will be reading multiple documents and placing locks on them.
A transaction operation is first reading the value of a property within a document in order to perform the write operation. So it requires round trip communications with server in order to ensure that the code inside the transaction completes successfully.
My question is do these locks restrict all access to the documents - including concurrent reads from client code that isn't part of a transaction?
The answer is no, concurrent users can read the content of the document even if you perform a write operation using a transaction.
Also in the case that a transaction tries to lock documents that are already locked - what is the retry pattern - how often does it retry, and is there an exponential backoff?
According to the official documentation regarding Firestore transactions, a transaction can fail only the following cases:
The transaction contains read operations after write operations. Read operations must always come before any write operations.
The transaction read a document that was modified outside of the transaction. In this case, the transaction automatically runs again. The transaction is retried a finite number of times.
The transaction exceeded the maximum request size of 10 MiB.
Transaction size depends on the sizes of documents and index entries modified by the transaction. For a delete operation, this includes the size of the target document and the sizes of the index entries deleted in response to the operation.
A failed transaction returns an error and does not write anything to the database. You do not need to roll back the transaction; Cloud Firestore does this automatically.
I understand that when you roll back an Oracle DB to a savepoint, any savepoints marked after that original savepoint are erased, but are all committed transactions since that savepoint also rolled back? Is it a complete flashback of the database? I'm assuming it is, just want to clarify. Many thanks.
If you commit your transaction then all savepoints prior to that are invalidated, so "all committed transactions since that savepoint also rolled back" doesn't really make sense. You can't ever roll back a commit.
From the docs:
Using ROLLBACK with the TO SAVEPOINT clause performs the following operations:
Rolls back just the portion of the transaction after the savepoint. It does not end the transaction.
Erases all savepoints created after that savepoint. The named savepoint is retained, so you can roll back to the same savepoint multiple times. Prior savepoints are also retained.
The use of the "all transactions" in your question is confusing. Transactions and savepoints only apply to a single session - any transactions in other sessions are unaffected by any rollbacks you do in your session.
Rolling back your session to a savepoint does erase "any savepoints marked after that original savepoint" in that same session, which are therefore within the same transaction. It does not affect any transactions or savepoints in any other sessions, and your current transaction is still active.
So no, it is not a complete flashback of the database. Tools exist to do that, but savepoints are a completely different thing.
Read more about transactions and savepoints.