I'd like to run the following algorithm (it's more like javascript pseudocode)
const transaction = datastore.transaction();
await transaction.run();
const parentKey = createKey(namespace, kind) // note that I leave the ID th be generated
await transaction.save(ancestorKey, parentEntity);
const childKey = createKey(namepsace, kind, parentId, parentKind) // ???
await transaction.save (ChildKey, childEntity);
await transaction.commit();
How can I know the parentId since the initial save of parentEntity is not yet commited?
I'd like to run this into a single transaction, is this achievable?
No, this is not possible due to the datastore's transaction isolation and consistency (emphasis mine):
This consistent snapshot view also extends to reads after writes
inside transactions. Unlike with most databases, queries and gets
inside a Cloud Datastore transaction do not see the results of
previous writes inside that transaction. Specifically, if an entity is
modified or deleted within a transaction, a query or lookup returns
the original version of the entity as of the beginning of the
transaction, or nothing if the entity did not exist then.
Depending on why you actually need such sequence to be done transactionally you might be able to achieve something somehow equivalent this way:
create the parent transactionally
in the same transaction also create and transactionally enqueue a push task queue passing it the parent's key as parameter - the task will be enqueued only if/when the transaction succeeds
in the task handler (also made transactional) create the child entity - guaranteed to only happen once
Note that not all GAE environments support such scheme due to limited push task queue support.
Related
I'm relying on Firebase Firestore offline capabilities, so I'm not using await on my queries as stated on the Access Data Offline Firebase doc. I'm expecting that when I write something I'll get an immediate reflection on my read stream, however, I'm only getting an update when the server/remote has been updated. Basically:
Update something in the DB. Note, I'm not using await
_db.doc(parentDoc).collection(DocInnerCollection).doc(childDoc).update({
"name": value,
});
I expect my listeners to be updated immediately. Note I've set the includeMetadataChanges to true as stated in the above doc.
_db.doc(parentDoc)
.collection(DocInnerCollection)
.orderBy('start_date', 'desc')
.limitToLast(1)
.snapshots(includeMetadataChanges: true)
.map((snapshot) {
print(snapshot.metadata.isFromCache)
});
However, I get no such update and instead I only get an update when the server has been updated.
You're requesting only one document with .limitToLast(1), yet are not providing a sort order for your query. This essentially means that you'll get a random document from your collection, and the chances of that being the newly updated document are close to zero.
If you want the latest (not just last) document, you need some ordering criteria to determine what latest means. Typically you'd do this by:
Adding a lastUpdated field to your documents, and setting that to firebase.firestore.FieldValue.serverTimestamp().
Ordering your query on that timestamp with orderBy('lastUpdated', 'desc').
And then limiting to the first result with limit(1).
What would a Cosmos stored procedure look like that would set the PumperID field for every record to a default value?
We are needing to do this to repair some data, so the procedure would visit every record that has a PumperID field (not all docs have this), and set it to a default value.
Assuming a one-time data maintenance task, arguably the simplest solution is to create a single purpose .NET Core console app and use the SDK to query for the items that require changes, and perform the updates. I've used this approach to rename properties, for example. This works for any Cosmos database and doesn't require deploying any stored procs or otherwise.
Ideally, it is designed to be idempotent so it can be run multiple times if several passes are required to catch new data coming in. If the item count is large, one could optionally use the SDK operations to scale up throughput on start and scale back down when finished. For performance run it close to the endpoint on an Azure Virtual Machine or Function.
For scenarios where you want to iterate through every item in a container and update a property, the best means to accomplish this is to use the Change Feed Processor and run the operation in an Azure function or VM. See Change Feed Processor to learn more and examples to start with.
With Change Feed you will want to start it to read from the beginning of the container. To do this see Reading Change Feed from the beginning.
Then within your delegate you will read each item off the change feed, check it's value and then call ReplaceItemAsync() to write back if it needed to be updated.
static async Task HandleChangesAsync(IReadOnlyCollection<MyType> changes, CancellationToken cancellationToken)
{
Console.WriteLine("Started handling changes...");
foreach (MyType item in changes)
{
if(item.PumperID == null)
{
item.PumperID = "some value"
//call ReplaceItemAsync(), etc.
}
}
Console.WriteLine("Finished handling changes.");
}
I am new to firebase and noSQL databases. I've read the docs and watched instructional videos but there is one concept I haven't fully grasped.
Why is it that querying the database requires a listener such as .on("value")?
Since these listeners are triggered whenever there is a change of sorts (nodes created, edited, children created) shouldn't there be a more direct way of getting the data from the db? Such as
ref.orderBy("age"). equalTo(30).get()
A method to just get what's in there at the time he instruction is executed, without having to listen to some sort of event?
In SQL it's not like you have to wait for something to change in your db to make this query work:
SELECT * FROM TABLE WHERE X == Y
PS: I know .once() also exists, but my question is more about: if my db never changed, how would I be able to query it and always get the same query result/snapshot?
You didn't define a platform so I will use this Swift pseudo-code example. Suppose we want to get a list of all users, one time.
let usersRef = my_firebase.child("users")
usersRef.observeSingleEvent(by: .value, with: { snapshot in
//all users are returned to the app in the snapshot
// .value means 'give me everything in the specified node'
// so then we can iterate over the snapshot to get each users data
}
You can call the above code any time to get the list of users and it does not add an observer (listener) to the node. It's a one-shot.
Whereas, if we want to be notified of users that are added
let usersRef = my_firebase.child("users")
usersRef.observe(by: .childAdded, with: { snapshot in
//upon first calling this, each user will be sent to the app in the snapshot
// and after that, any user that's added
}
The above code attaches an observer (listener) to the users node and whenever a user is added, it's provided to the app via the snapshot.
Note the use of .observeSingleEvent vs .observe and .value to get everything in the node vs .childAdded to get a specific node.
I'm trying to update the same document which triggered an onUpdate cloud function, with a read value from the same collection.
This is in a kind of chat app made in Flutter, where the previous response to an inquiry is replicated to the document now being updated, for easier showing in the app.
The code does work, however when a user quickly responds to two separate inquiries, they both read the same latest response thus setting the same previousResponse. This must be down to the asynchronous nature of flutter and/or the cloud function, but I can't figure out where to await or if there's a better way to make the function, so it is never triggering the onUpdate for the same user, until a previous trigger is finished.
Last part also sound a bit like a bad idea.
So far I tried sticking the read/update in a transaction, however that only seems to work for the single function call, and not when they're asynchronous.
Also figured I could fix it, by reading the previous response in a transaction on the client, however firebase doesn't allow reading from a collection in a transaction, when not using the server API.
async function setPreviousResponseToInquiry(
senderUid: string,
recipientUid: string,
inquiryId: string) {
return admin.firestore().collection('/inquiries')
.where('recipientUid', '==', recipientUid)
.where('senderUid', '==', senderUid)
.where('responded', '==', true)
.orderBy('modified', 'desc')
.limit(2)
.get().then(snapshot => {
if (!snapshot.empty &&
snapshot.docs.length >= 2) {
return admin.firestore()
.doc(`/inquiries/${inquiryId}`)
.get().then(snap => {
return snap.ref.update({
previousResponse: snapshot.docs[1].data().response
})
})
}
})
}
I see three possible solutions:
Use a transaction on the server, which ensures that the update you write must be based on the version of the data you read. If the value you write depends on the data that trigger the Cloud Function, you may need to re-read that data as part of the transaction.
Don't use Cloud Functions, but run all updates from the client. This allows you to use transactions to prevent the race condition.
If it's no possible to use a transaction, you may have to include a custom version number in both the upstream data (the data that triggers the write), and the fanned out data that you're updating. You can then use security rules to ensure that the downstream data can only be written if its version matches the current upstream data.
I'd consider/try them in the above order, as they gradually get more involved.
To get state I can use Vault, but what about transactions? How I can get them, for example, by txHash? Is it possible to do this by vaultService.queryBy(criteria) ?
Since internalVerifiedTransactionsSnapshot method is deprecated now, any ways to retrieve a specific transaction by using txhash as of Corda 4?
Inside of the node you can call:
serviceHub.validatedTransactions.getTransaction(hash)
Via rpc I think you can do this:
proxy.stateMachineRecordedTransactionMappingSnapshot().map { it.transactionId }.first { it == hash }
But a better solution would be to create a flow that takes in a hash, calls the first snippet and returns the transaction.