All of my firestore transactions fail when I want to get document.
I've tried getting other files changed rules to be public. I've found out that when i use if checking it seems like get function returned data.
val currentUserDocument = firebaseFirestore.collection("user").document(firebaseAuth.currentUser!!.uid)
val classMemberDocument = firebaseFirestore.collection("class").document(remoteClassID).collection("member").document(firebaseAuth.currentUser!!.uid)
firebaseFirestore.runTransaction { transaction ->
val userSnapshot = transaction.get(currentUserDocument)
val isInClass = userSnapshot.getBoolean("haveRemoteClass")!!
val classID = userSnapshot.getString("remoteClassID")!!
if (isInClass == true && classID == remoteClassID) {
transaction.update(currentUserDocument, "haveRemoteClass", false)
transaction.update(currentUserDocument, "remoteClassID", "")
transaction.delete(classMemberDocument)
} else {
throw FirebaseFirestoreException("You aren't in this class!", FirebaseFirestoreException.Code.ABORTED)
}
null
}
This typically means that the data that you're using in the transaction is seeing a lot of contention.
Each time you run a transaction, Firebase determines the current state of all documents you use in the transaction, and sends that state and the new state of those documents to the server. If the documents that you got were changed between when the transaction started and when the server gets it, it rejects the transaction and the client retries.
For the client to fail like this, it has to retry more often than is reasonable. Consider reducing the scope of your transaction to cover fewer documents, or find another way to reduce contention (such as the approach outlined for distributed counters).
Related
This question already has an answer here:
How to return failed task result in continuation task?
(1 answer)
Closed 2 years ago.
I'm writing my first app in Kotlin and am using Firestore & Firebase Storage. In the process of deleting a document, I want to delete all files in Storage that the document references (as it is the only reference to them in my case). If the Storage delete fails, I want to abort the document delete, in order to avoid orphan files in my Storage. I also want to do everything in "one Task", to allow showing a progress bar properly. My simplified code looks like this:
fun deleteItem(id: String): Task<Void>? {
val deleteTask = deleteTaleMedia(id)
continueWithTaskOrInNew(deleteTask) { task ->
if (task?.isSuccessful != false) { ... }
}
}
fun deleteItemMedia(id: String): Task<Void>? =
getItem(id)?.continueWithTask { task ->
if (task.isSuccessful)
task.result?.toObject(ItemModel::class.java)?.let { deleteFiles(it.media) }
else ???
}
fun deleteFiles(filesList: List<String>): Task<Void>? {
var deleteTask: Task<Void>? = null
for (file in filesList) deleteTask = continueWithTaskOrInNew(deleteTask) { task ->
if (task?.isSuccessful != false) deleteFile(file)
else task
}
return task
}
fun deleteFile(fileName: String) = Firebase.storage.getReferenceFromUrl(fileName).delete()
fun getItem(id: String): Task<DocumentSnapshot>? {
val docRef = userDocRef?.collection(COLLECTION_PATH)?.document(id)
return docRef?.get()
?.addOnCompleteListener { ... }
}
fun <ResultT, TContinuationResult> continueWithTaskOrInNew(
task: Task<ResultT>?,
continuation: (Task<ResultT>?) -> Task<TContinuationResult>?
) = task?.continueWithTask { continuation.invoke(task) } ?: continuation.invoke(null)
data class ItemModel(
#DocumentId val id: String = "",
var title: String = "",
var media: List<String> = listOf()
)
My problem comes in deleteItemMedia function (the "???" at the end). In case the get task failed, I want to return a task that will tell my deleteItem function to abort deletion (task.isSuccessful == false). I cannot return the get task itself (replace "???" with "task" in code), because it's type (Task<DocumentSnapshot>) differs from the type of the delete task (Task<Void>). I cannot return null, as null is returned in the case of no media at all, which is a valid case for me (document should be deleted). Is there a way to create a new "failed Task"?
In the process of deleting a document I want to delete all files in Storage that the document references (as it is the only reference to them in my case).
There is no API that's doing that. You have to perform both delete operations yourself.
I also want to do everything in "one Task", to allow showing a progress bar properly.
Unfortunately, this is not possible in a single go. If you think of an atomic operation, this also not possible because none of the Firebase services support this kind of cross-product transactional operations. What you need to do is, get the document, get the references to the files in the Storage, delete the document and as soon as the delete operation is complete, delete the files. You can definitely reduce the risk by trying to roll-back the data from the client, but you cannot do them atomic, in "one Task". However, at some point in time, there will be an Exception that the client can't rollback.
If the Storage delete fails, I want to abort the document delete, in order to avoid orphan files in my Storage.
To avoid that, first, try not to have incomplete data. For instance, when you read the document and you get the corresponding Storage URLs, don't blindly assume that all those files actually exist. A file can unavailable for many reasons (was previously deleted, for some reasons the service is unavailable, etc.)
Another approach might be to use Cloud Functions for Firebase, so you can delete the desired document, and use onDelete function to delete the corresponding files from the Storage. Meaning, when document delete fails, the files from the Storage won't be deleted. If the operation to delete the document is successful, the Cloud Function will be triggered and the images will be deleted from the Storage. This approach will drastically reduce the chances of having failures between the document delete operation and the deletion of the files from Storage, but it doesn't eliminate that chance 100%.
Besides that, the most common approach to avoid failures is to make your code as robust as you possibly can against failure and do frequent database cleanups.
To get state I can use Vault, but what about transactions? How I can get them, for example, by txHash? Is it possible to do this by CordaRPCOps, there is internalVerifiedTransactionsSnapshot method, but it is deprecated now.
First, note that as of Corda 3, there are no stability guarantees regarding the behaviour of any method to retrieve a transaction or its dependencies. In particular, we cannot guarantee that the set of transactions retrieved will not change across Corda versions.
This is because in future versions of Corda, nodes will likely only exchange transaction chains in SGX-encrypted form. These transaction chains will then be verified inside an SGX enclave on the node. This will prevent nodes from seeing the contents of the transactions they are verifying (see the blogpost here: https://www.corda.net/2017/06/corda-sgx-privacy-update/). This may even go so far as to only allow nodes to see certain parts of the transactions they are signing.
Ways to retrieve transactions as of Corda 3
1. Using CordaRPCOps.internalVerifiedTransactionsSnapshot
If you are interacting with the node via RPC, CordaRPCOps.internalVerifiedTransactionsSnapshot returns a list of all recorded transactions.
If you only wanted to get a single transaction and you knew its hash, you could write:
val transactions = cordaRPCOps.internalVerifiedTransactionsSnapshot()
val signedTransaction = transactions
.find { it.id == transactionHash }
?: throw IllegalArgumentException("Unknown transaction hash.")
Note that the transactions returned are of type SignedTransaction. This form does not contain the transaction's attachments or inputs (only the attachment hashes and input state references).
To retrieve a transaction's attachments via RPC, you could write:
val transactions = cordaRPCOps.internalVerifiedTransactionsSnapshot()
val signedTransaction = transactions
.find { it.id == transactionHash }
?: throw IllegalArgumentException("Unknown transaction hash.")
val attachmentHashes = signedTransaction.tx.attachments
val attachmentStreams = attachmentHashes.map { hash -> cordaRPCOps.openAttachment(hash) }
And to retrieve a transaction's inputs via RPC, you could write:
val transactions = cordaRPCOps.internalVerifiedTransactionsSnapshot()
val signedTransaction = transactions
.find { it.id == transactionHash }
?: throw IllegalArgumentException("Unknown transaction hash.")
val inputStateRefs = signedTransaction.inputs
val inputStates = inputStateRefs.map { stateRef ->
val transaction = transactions.find { it.id == stateRef.txhash }
?: throw IllegalArgumentException("Unknown transaction hash.")
transaction.tx.outputStates[stateRef.index]
}
2. Using the ServiceHub
If you are in a situation where you have access to the node's ServiceHub (e.g. within a flow or a Corda service), you can use serviceHub.validatedTransactions.track().snapshot to get all transactions, and serviceHub.validatedTransactions.getTransaction(transactionHash) to get a specific transaction by hash.
Note that the transactions returned are of type SignedTransaction. This form does not contain the transaction's attachments or inputs (only the attachment hashes and input state references).
To convert the SignedTransaction to a LedgerTransaction (where the attachments and inputs are resolved), you could write:
val signedTransaction = serviceHub.validatedTransactions.getTransaction(transactionHash)
val ledgerTransaction = signedTransaction.toLedgerTransaction(serviceHub)
3. By connecting to the node's database
You can connect directly to the SQL database backing the node, and retrieve the transactions using an SQL query.
That's right, although please note that the ServiceHub and SQL approaches are basically the same thing as the deprecated RPC and may also stop working in future (or not, depending on how we manage the transition to an encrypted ledger).
There are other approaches you can use. For instance you could aggregate the bits of history you care about up into the latest version of the state. This also lets you restrict the view of the history once SGX lands.
The first solution (Using CordaRPCOps.internalVerifiedTransactionsSnapshot) is really slow.
It is exist one more way to get transaction history and it is pretty effective.
You can do it by using rpcOps.vaultQueryBy
fun transaction(transactionId: String): List<Vault.Page<ContractState>> {
// get jdbc connection (you may simplify it within cordapp)
val jt = jt()
// get all states of transaction
val output_indexes = jt.queryForList("SELECT OUTPUT_INDEX FROM VAULT_STATES WHERE transaction_id = '$transactionId'", Int::class.java)
val transactionHash = SecureHash.parse(transactionId)
// get Rpc connection
val rpcOps = initialiseNodeRPCConnection()
val transactionStates = output_indexes.map {
val constraintTypeCriteria = QueryCriteria.VaultQueryCriteria(status = Vault.StateStatus.ALL, stateRefs = listOf(StateRef(transactionHash, it)))
rpcOps.vaultQueryBy<ContractState>(constraintTypeCriteria)
}
return transactionStates
}
I have some ContractState, and there are two commands that can 'delete' the state (mark it as historic, with no new state to replace it) - let's say 'Delete' and 'Revoke', which have different real-world consequences.
I can still see the historic states in the vault, right? How can I determine which command deleted the state? I suppose I could add some enum to the state: 'Active|Deleted|Revoked', and then move the state from S(Active) -> S(Deleted|Revoked) -> Historic. But that seems clunky.
In theory, you could determine which command was used to consume the state by checking the transaction that consumed the state.
However, there is currently no non-deprecated API for viewing the contents of the node's transaction storage as a node owner. This is because in a future version of Corda, we expect transaction resolution to occur within a secure guard extension (SGX). This will affect which transactions are visible, and Corda therefore cannot commit to a stable API for viewing the contents of the node's transaction storage.
If you're willing to use the deprecated internalVerifiedTransactionsSnapshot/internalVerifiedTransactionsFeed API, you could do something like:
val (_, vaultUpdates) = proxy.vaultTrackBy<ContractState>()
vaultUpdates.toBlocking().subscribe { update ->
update.produced.forEach { stateAndRef ->
val newStateTxId = stateAndRef.ref.txhash
val transactions = proxy.internalVerifiedTransactionsSnapshot()
val transaction = transactions.find { transaction -> transaction.id == newStateTxId }!!
val commands = transaction.tx.commands
}
}
An alternative would be to add a status field to the state, and updating the status field instead of marking it as consumed when "deleting" it. For example:
class MyState(val expired: Boolean): ContractState {
override val participants = TODO()
}
We're using Firebase DB together with RxSwift and are running into problems with transactions. I don't think they're related to the combination with RxSwift but that's our context.
Im observing a data in Firebase DB for any value changes:
let child = dbReference.child(uniqueId)
let dbObserverHandle = child.observe(.value, with: { snapshot -> () in
guard snapshot.exists() else {
log.error("empty snapshot - child not found in database")
observer.onError(FirebaseDatabaseConsumerError(type: .notFound))
return
}
//more checks
...
//read the data into our object
...
//finally send the object as Rx event
observer.onNext(parsedObject)
}, withCancel: { _ in
log.error("could not read from database")
observer.onError(FirebaseDatabaseConsumerError(type: .databaseFailure))
})
No problems with this alone. Data is read and observed without any problems. Changes in data are propagated as they should.
Problems occur as soon as another part of the application modifies the data that is observer with a transaction:
dbReference.runTransactionBlock({ (currentData: FIRMutableData) -> FIRTransactionResult in
log.debug("begin transaction to modify the observed data")
guard var ourData = currentData.value as? [String : AnyObject] else {
//seems to be nil data because data is not available yet, retry as stated in the transaction example https://firebase.google.com/docs/database/ios/read-and-write
return TransactionResult.success(withValue: currentData)
}
...
//read and modify data during the transaction
...
log.debug("complete transaction")
return FIRTransactionResult.success(withValue: currentData)
}) { error, committed, _ in
if committed {
log.debug("transaction commited")
observer(.completed)
} else {
let error = error ?? FirebaseDatabaseConsumerError(type: .databaseFailure)
log.error("transaction failed - \(error)")
observer(.error(error))
}
}
The transaction receives nil data at first try (which is something you should be able to handle. We just just call
return TransactionResult.success(withValue: currentData)
in that case.
But this is propagated to the observer described above. The observer runs into the "empty snapshot - child not found in database" case because it receives an empty snapshot.
The transaction is run again, updates the data and commits successfully. And the observer receives another update with the updated data and everything is fine again.
My questions:
Is there any better way to handle the nil-data during the transaction than writing it to the database with FIRTransactionResult.success
This seems to be the only way to complete this transaction run and trigger a re-run with fresh data but maybe I'm missing something-
Why are we receiving the empty currentData at all? The data is obviously there because it's observed.
The transactions seem to be unusable with that behavior if it triggers a 'temporary delete' to all observers of that data.
Update
Gave up and restructured the data to get rid of the necessity to use transactions. With a different datastructure we were able to update the dataset concurrently without risking data corruption.
I'm unable to create an index. My Gremlin code is as follows:
usernameProperty = mgmt.getPropertyKey('username')
usernameIndex = mgmt.buildIndex('byUsernameUnique', Vertex.class).addKey(usernameProperty).unique().buildCompositeIndex()
mgmt.setConsistency(usernameIndex, ConsistencyModifier.LOCK)
mgmt.commit()
Shortly after I receive two errors:
18:04:57 ERROR com.thinkaurelius.titan.graphdb.database.management.ManagementLogger - Evicted [1#0a00009d2537-ip-10-0-0-1572] from cache but waiting too long for transactions to close. Stale transaction alert on: [standardtitantx[0x6549ce71]]
18:04:57 ERROR com.thinkaurelius.titan.graphdb.database.management.ManagementLogger - Evicted [1#0a00009d2537-ip-10-0-0-1572] from cache but waiting too long for transactions to close. Stale transaction alert on: [standardtitantx[0x2a2815cc], standardtitantx[0x025dc2c0]]
The status of the index is stuck at INSTALLED:
usernameIndex.getIndexStatus(usernameProperty)
==>INSTALLED
I read that a failed instance could cause the issue, but a check of running instances shows just one:
mgmt.getOpenInstances()
==>0a00009d3011-ip-10-0-0-1572(current)
I've also tried issuing a REGISTER_INDEX action, which also gets evicted from the transaction cache with a similar error message:
mgmt.updateIndex(usernameIndex, SchemaAction.REGISTER_INDEX).get()
mgmt.commit()
I've also tried restarting the server multiple times.
It seems like the registration process is simply timing out, causing an "eviction" from the transaction cache. I've waited 48 hours just to be sure it wasn't a slow process. Normal reads, writes, and associated commits to Titan do seem to be working correctly, I'm just not able to create this index. I'm stuck, is there something else I can try? Is there a way to extend the timeout on that transaction?
I'm running Titan 1.0.0 using a DynamoDB backend (setup with the AWS provided CloudFormation template).
EDIT:
Here is the complete command I'm pasting into Gremlin with the addition of the awaitGraphStatus step suggested by #M-T-A:
mgmt = graph.openManagement();
usernameIndex = mgmt.getPropertyKey('usernameIndex');
mgmt.buildIndex('byUsername',Vertex.class).addKey(usernameIndex).unique().buildCompositeIndex();
// I have tried with and without a commit here: mgmt.commit();
mgmt.awaitGraphIndexStatus(graph, 'byUsername').status(SchemaStatus.REGISTERED).timeout(10, java.time.temporal.ChronoUnit.MINUTES).call();
This results in the following error:
java.lang.NullPointerException
at com.thinkaurelius.titan.graphdb.database.management.GraphIndexStatusWatcher.call(GraphIndexStatusWatcher.java:52)
at com.thinkaurelius.titan.graphdb.database.management.GraphIndexStatusWatcher.call(GraphIndexStatusWatcher.java:18)
at java_util_concurrent_Callable$call.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:110)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:114)
at groovysh_evaluate.run(groovysh_evaluate:3)
at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:215)
at org.codehaus.groovy.tools.shell.Interpreter.evaluate(Interpreter.groovy:69)
at org.codehaus.groovy.tools.shell.Groovysh.execute(Groovysh.groovy:185)
at org.codehaus.groovy.tools.shell.Shell.leftShift(Shell.groovy:119)
at org.codehaus.groovy.tools.shell.ShellRunner.work(ShellRunner.groovy:94)
I will also note that the routine to disable and delete an index is also failing.
mgmt = graph.openManagement()
theIndex = mgmt.getGraphIndex('byUsername')
mgmt.updateIndex(theIndex, SchemaAction.DISABLE_INDEX).get()
mgmt.commit()
mgmt.awaitGraphIndexStatus(graph, 'byUsername').status(SchemaStatus.DISABLED).timeout(10, java.time.temporal.ChronoUnit.MINUTES).call();
m = graph.openManagement()
i = m.getGraphIndex('byUsername')
m.updateIndex(i, SchemaAction.REMOVE_INDEX).get()
m.commit()
19:26:26 ERROR com.thinkaurelius.titan.graphdb.database.management.ManagementLogger - Evicted [1#ac1f3fa810472-ip-172-31-63-1681] from cache but waiting too long for transactions to close. Stale transaction alert on: [standardtitantx[0x2314cd97], standardtitantx[0x39f8adc0], standardtitantx[0x09de1b85]]
EDIT 2:
Attempting to create a new property key and index within the same transaction DOES WORK! But does this mean that I can't create indexes on existing property keys??
graph.tx().rollback();
mgmt = graph.openManagement();
indexName = 'byUsernameTest2';
propertyKeyName = 'testPropertyName2';
propertyKey = mgmt.makePropertyKey(propertyKeyName).dataType(String.class).cardinality(Cardinality.SINGLE).make();
mgmt.buildIndex(indexName,Vertex.class).addKey(propertyKey).buildCompositeIndex();
mgmt.commit();
graph.tx().commit();
mgmt.awaitGraphIndexStatus(graph, indexName).status(SchemaStatus.REGISTERED).timeout(10, java.time.temporal.ChronoUnit.MINUTES).call();
mgmt.commit();
After a pause, this results in:
This management system instance has been closed
Trying to fetch the new index results in:
mgmt = graph.openManagement();
index = mgmt.getGraphIndex('byUsernameTest2');
propkey = mgmt.getPropertyKey('testPropertyName2');
index.getIndexStatus(propkey);
==>ENABLED
If you're running multiple Titan instances, you should be aware that they need to coordinate before the index will become available.
Furthermore, there are various subtleties around transaction management and under what circumstances transactions will be left open; I believe Titan 1.0.0 doesn't have the latest from TinkerPop in that regard. Have you tried creating the index immediately after boot?
Finally, the index creation process is different depending on whether or not the property keys being indexed have been used before. Are you attempting to index new keys, or existing ones?
You need to wait for the Titan and DynamoDB to get the index registered. You could do so by:
ManagementSystem.awaitGraphIndexStatus(graph, propertyKeyIndexName)
.status(SchemaStatus.REGISTERED)
.timeout(10, java.time.temporal.ChronoUnit.MINUTES) // set timeout to 10 min
.call();
The default timeout is usually not long enough, so you could increase it to 10 minutes, it usually does the trick with Dynamo backened.
Only when the index is in REGISTERED state, you could perform a reindex. ONce the reindex is done, you need to wait until it's ENABLED. by reusing the code sample above and changing the state to ENABLED.
For more info, see the docs.
Edit
Let me share the code that works with me on Berkeley and Dynamo DB backends all the time.
graph.tx().rollback(); //Never create new indexes while a transaction is active
TitanManagement mgmt=graph.openManagement();
PropertyKey propertyKey=getOrCreatePropertyKeyIfNotExist(mgmt, propertyKeyName);
String indexName = makePropertyKeyIndexName(propertyKey);
if (mgmt.getGraphIndex(indexName)==null) {
mgmt.buildIndex(indexName, Vertex.class).addKey(propertyKey).buildCompositeIndex();
mgmt.commit(); // you MUST commit mgmt
graph.tx().commit(); // and commit the transaction too
ManagementSystem.awaitGraphIndexStatus(graph, indexName).status(SchemaStatus.REGISTERED).call();
}else { // already defined.
mgmt.rollback();
graph.tx().rollback();
}
private static PropertyKey getOrCreatePropertyKeyIfNotExist(TitanManagement mgmt, String s) {
PropertyKey key = mgmt.getPropertyKey(s);
if (key != null)
return key;
else
return mgmt.makePropertyKey(s).dataType(String.class).make();
}
private static String makePropertyKeyIndexName(PropertyKey pk) {
return pk.name() + Tokens.INDEX_SUFFIX;
}
From the error that I saw, it seems like Titan couldn't get the index which means you're waiting for the index that is not even defined. Have a look at the line that causes the error here.
Make sure you're passing the right index name to the awaitGraphIndexStatus.