Problem with saving data in Transaction in Firebase Firestore for Flutter - firebase

I have a problem with transactions in my web application created in Flutter. For database I use Firebase Firestore where I save documents via transaction.
Dependency:
cloud_firestore: 3.1.1
StudentGroup is my main document. It has 4 stages and each of them has 3-5 tasks. (Everything is in 1 document). I have to store game timer, so every 10 seconds I make an request to save time for current stage. (Every stage has different timer). I have a problem with saving task, because "Sometimes" when 2 requests are made in the same time, then I get some weird state manipulation.
Task is updated and "isFinished" is set to true
Timer is updated to correct value (with this update somehow previous task update is lost, "isFinished" is set to false
This is how I save task.
Future<Result> saveTask({required String sessionId, required String studentGroupId,
required Task task}) async {
print("trying to save task <$task>.");
try {
return await _firebaseFirestore.runTransaction((transaction) async {
final studentGroupRef = _getStudentGroupDocumentReference(
sessionId: sessionId,
studentGroupId: studentGroupId
);
final sessionGroupDoc = await studentGroupRef.get();
if (!sessionGroupDoc.exists) {
return Result.error("student group not exists");
}
final sessionGroup = StudentGroup.fromSnapshot(sessionGroupDoc);
sessionGroup.game.saveTask(task);
transaction.set(studentGroupRef, sessionGroup.toJson());
})
.then((value) => taskFunction(true))
.catchError((error) => taskFunction(false));
} catch (error) {
return Result.error("Error couldn't save task");
}
}
This is how I save my time
Future<Result> updateTaskTimer({required String sessionId,
required String studentGroupId, required Duration duration}) async {
print("trying to update first task timer");
try {
return await _firebaseFirestore.runTransaction((transaction) async {
final studentGroupRef = _getStudentGroupDocumentReference(
sessionId: sessionId,
studentGroupId: studentGroupId
);
final sessionGroupDoc = await studentGroupRef.get();
if (!sessionGroupDoc.exists) {
return Result.error("student group not exists");
}
final sessionGroup = StudentGroup.fromSnapshot(sessionGroupDoc);
switch (sessionGroup.game.gameStage) {
case GameStage.First:
sessionGroup.game.stages.first.duration = duration.inSeconds;
break;
case GameStage.Second:
sessionGroup.game.stages[1].duration = duration.inSeconds;
break;
case GameStage.Third:
sessionGroup.game.stages[2].duration = duration.inSeconds;
break;
case GameStage.Fourth:
sessionGroup.game.stages[3].duration = duration.inSeconds;
break;
case GameStage.Fifth:
sessionGroup.game.stages[4].duration = duration.inSeconds;
break;
}
transaction.set(
studentGroupRef,
sessionGroup.toJson(),
SetOptions(merge: true)
);
print("Did I finish task 4? ${sessionGroup.game.stages.first.tasks[3].isFinished}");
})
.then((value) => timerFunction(true))
.catchError((error) => timerFunction(false));
} catch (error) {
return Result.error("Error couldn't update task timer");
}
}
timerFunction and taskFunction print some messages in console and return Result.error or Result.success (for now it returns bool)
I don't know If I am doing something wrong with Firebase Firestore Transaction. I would like to have atomic operations for reading and writing data.

Transactions ensure atomicity - which means that if the transaction succeeds then all the reads and writes occur in a non-overlapping way with other transactions. This prevents the type of problem you are describing.
But this doesn't work if you spread your reads and writes over multiple transactions. In particular, it looks to me like you are writing a task which was obtained from outside the transaction. Instead, you should use ids or similar to track which documents you need to update, then do a read and a write inside the transaction.Alternatively firebase also provides Batched Writes, which specify the specific properties you want to update. These will ensure that any other properties are not changed.For batch writes example you can refer to the link

Related

Wait for Cloud Function to be finished

Is there any way to wait for a Cloud Function, that was triggered by a Firestore document write, to finish?
Context:
My app has groups. Owners can invite other users to a group via an invite code. Users can write themselves as member of a group if they have the right invite code. They do this by writing the groups/{groupId}/members/{userId} document that contains their profile info.
To make reading more efficient, this info is copied to array members in the groups/{groupId} document by a Cloud Function.
The Cloud Function that does that is triggered by the document write. It is usually finished after a couple of seconds, but there's no predictable execution time and it might take a bit longer if it is a cold start.
After the user has joined the group, I forward them to the groups view in my app which reads the group document. In order for the view to render correctly, the membership info needs to be available. So I would like to forward AFTER the Cloud Function has finished.
I found no way to track the execution of a Cloud Function that was triggered by a Firestore document write.
A fellow developer recommended to just poll the groups/{groupId} document until the info is written and then proceed but this doesn't seem like a clean solution to me.
Any ideas how this could be done better?
Is it possible to get a promise that resolves after the Cloud Function has finished? Is there a way to combine a Firestore document write and a Cloud Function execution into one transaction?
Thanks for the hints, I came up with the following ways to deal with the problem. The approach depends on if/when the user is allowed to read a document:
A) User is member and leaves the group > at the start of the transaction they are allowed to read the group > the moment they can't read anymore confirms that the membership was successfully revoked:
async function leaveGroup (groupId) {
await deleteDoc(doc(db, 'groups', groupId, 'members', auth.currentUser.uid))
// Cloud Function removes the membership info
// from the group doc...
await new Promise((resolve) => {
const unsubscribeFromSnapshot = onSnapshot(
doc(db, 'groups', groupId),
() => { }, // success callback
() => { // error callback
// membership info is not in the group anymore
// > user can't read the doc anymore
// > transaction was successful
// read access was revoked > transaction was successful:
unsubscribeFromSnapshot()
resolve()
}
)
})
}
B) User is not a member and wants to join the group > at the start of the transaction they are allowed to read the group > the moment they can read the group confirms that the membership was successfully confirmed (this is a simplified version that does not check the invite code):
async function joinGroup (groupId) {
try {
await setDoc(
doc(db, 'groups', groupId, 'members', auth.currentUser.uid),
{
userId: auth.currentUser.uid,
userDisplayName: auth.currentUser.displayName
}
)
// Cloud Function adds the membership
// information to the group doc ...
await new Promise((resolve) => {
let maxRetries = 10
const interval = setInterval(async () => {
try {
const docSnap = await getDoc(doc(db, 'groups', groupId))
if (docSnap.data().members.includes(auth.currentUser.uid)) {
// membership info is in the group doc
// > transaction was successful
clearInterval(interval)
resolve()
}
} catch (error) {
if (maxRetries < 1) {
clearInterval(interval)
}
}
maxRetries--
}, 2000)
})
}
Note: I went with polling here, but similar to what #samthecodingman suggested, another solution could be that the Cloud Function confirms the membership by writing back to the members document (which the user can always read) and you listen to snapshot changes on this document.
C) Most straightforward way: someone else (the group owner) removes a member from the group > they have read access through the whole transaction > directly listen to snapshot changes:
async function endMembership (groupId, userId) {
await deleteDoc(doc(db, 'groups', groupId, 'members', userId))
// Cloud Function removes the membership info
// from the group doc...
await new Promise((resolve) => {
const unsubscribe = onSnapshot(doc(db, 'groups', groupId), (doc) => {
if (!doc.data().members.includes(userId)) {
// membership info is not in the group doc anymore
// > transaction was successful
unsubscribe()
resolve()
}
})
})
}
In any case you should do proper error handling that covers other causes. I left them out to demonstrate how to use the error handlers when waiting for gaining/loosing read access.

Firestore Transactions is not handling race condition

Objective
User on click a purchase button on the web frontend, it will send a POST request to the backend to create a purchase order. First, it will check the number of available stocks. If available is greater than 0, reduce available by 1 and then create the order.
The setup
Backend (NestJS) queries the Firestore for the latest available value, and reduce available by 1. For debugging, I will return the available value.
let available;
try {
await runTransaction(firestore, async (transaction) => {
const sfDocRef = doc(collection(firestore, 'items_available'), documentId);
const sfDoc = await transaction.get(sfDocRef);
if (!sfDoc.exists()) {
throw 'Document does not exist!';
}
const data = sfDoc.data();
available = data.available;
if(available>0){
transaction.update(sfDocRef, {
available: available-1,
});
}
});
} catch (e) {
console.log('Transaction failed: ', e);
}
return { available };
My stress test setup
Our goal is to see all API requests having different available value, this would mean that Firestore Transactions is reducing the value even though there are multiple requests coming in.
I wrote a simple multi-threaded program that queries the backend's create order API, it will query the available value and return the available value. This program will save the available value returned for each API request.
The stress test performed is about 10 transactions per second, as I have 10 concurrent processes querying the backend. Each process will http.get 20 queries:
const http = require('http');
function call(){
http.get('http://localhost:5000/get_item_available', res => {
let data = [];
res.on('data', chunk => {
data.push(chunk);
});
res.on('end', () => {
console.log('Response: ', Buffer.concat(data).toString());
});
}).on('error', err => {
console.log('Error: ', err.message);
});
}
for (var i=0; i<20; i++){
call();
}
The problem
Unfortunately, the available values I got from the requests contains repeated values, that is, having same available values instead of having unique available values.
What is wrong? Isn't Firestore Transactions meant to handle race conditions? Any suggestions on what I could change to handle multiple requests hitting the server and return a new value for each request?
You have a catch clause to handle when the transaction fails, but then still end up returning a value to the caller return { available }. In that situation you should return an error to the caller.

2 updates within one Firestore transaction

I am new to Firestore transaction, and would like to update a document field based the current data of the document.
My planned transaction is given below:
const cityRef = db.collection('cities').doc('SF');
try {
await db.runTransaction(async (t) => {
const doc = await t.get(cityRef);
let status = doc.data().myStatus;
if (status == "one") {
throw "err";
} else {
// run some function - next status is based on the return
let output = await someFunction();
if (output) {
await t.update(cityRef, { myStatus: "two" });
return output;
} else {
await t.update(cityRef, { myStatus: "three" });
return output;
}
}
});
console.log("transaction successful");
} catch (err) {
console.log("Alreadu updated");
output = "one";
return output;
}
My queries are given below:
As per the documentation I have returned the data after update, however it does not seem to be working as expected.
Can we have 2 updates within one single transaction (both are updating the same field in the firestore)?
Thank you
You make the following clarification in the comments above:
someFunction() does some processing on other firestore
collection/documents (not the one I am updating) and returns either
true or false.
As you read in the doc on Transactions, "Read operations must come before write operations". If you want to read some docs in the transaction, you need to use the get() method of the Transaction, like you did with the first document. You cannot call a function that is using other Firestore methods like the get() method of a DocumentReference.

Is there a way to prevent having to await an async method returning a stream?

We currently have a method that returns a Future<Stream<Position>> just because internally we have to await the result of a method returning a Future before we can call another method that returns the Stream<Position> which we are actually interested in. Here is the code:
Future<Stream<Position>> getPositionStream(
[LocationOptions locationOptions = const LocationOptions()]) async {
PermissionStatus permission = await _getLocationPermission();
if (permission == PermissionStatus.granted) {
if (_onPositionChanged == null) {
_onPositionChanged = _eventChannel
.receiveBroadcastStream(
Codec.encodeLocationOptions(locationOptions))
.map<Position>(
(element) => Position._fromMap(element.cast<String, double>()));
}
return _onPositionChanged;
} else {
_handleInvalidPermissions(permission);
}
return null;
}
So what happens here is:
We await the _getLocationPermission() method so that we can test if the user grants us permission to access to the location services on their device (Android or iOS);
If the user grants us permission we return a Stream<Position> which will update every time the device registers a location change.
I have the feeling we can also handle this without doing an await and returning a Future. Something along the lines of:
Manually create and return an instance of the Stream<Position> class;
Handle the logic of checking the permissions and calling the _eventChannel.receiveBroadcastStream in the then() method of the Future<PermissionStatus> returned from the _getLocationPermission() method (so we don't have to await it);
Copy the events send on the stream from the _eventChannel.receiveBroadcastStream onto the earlier created (and returned) stream.
Somehow this seems to be possible, but also includes some overhead in managing the stream and make sure it closes and is cleaned up correctly during the live cycle of the plugin or when the user unsubscribes pass through the events to the _eventChannel etc.
So I guess the question would be, what would be the best way to approach this situation?
You can write the code as an async* function, which will return a Stream and still allows await in the body:
Stream<Position> getPositionStream(
[LocationOptions locationOptions = const LocationOptions()]) async* {
PermissionStatus permission = await _getLocationPermission();
if (permission == PermissionStatus.granted) {
if (_onPositionChanged == null) {
_onPositionChanged = _eventChannel
.receiveBroadcastStream(
Codec.encodeLocationOptions(locationOptions))
.map<Position>(
(element) => Position._fromMap(element.cast<String, double>()));
}
yield* _onPositionChanged;
} else {
_handleInvalidPermissions(permission);
}
}
Alternatively, if you are using a non-async function, you can also use StreamCompleter from package:async.
It allows you to return a Stream now, even if you only get the real stream later. When that happens, you "complete" the StreamCompleter with the real stream, and the original stream will behave as if it was the real stream.

How to implement transactions in Meteor Method calls

Suppose I have 2 collections "PlanSubscriptions" and "ClientActivations". I am serially doing a insert on both the collections.
Later one depends on previous one, if any of the transaction fails then the entire operation must rollback.
How can I achieve that in Meteor 1.4?
Since MongoDB doesn't support atomicity, you will have to manage it with Method Chaining.
You can write a method, say, transaction where you will call PlanSubscriptions.insert(data, callback). Then in the callback function you will call ClientActivations.insert(data, callback1) if the first insertion is success and in callback1 return truthy if second insertion is succes, otherwise falsy. If the first insertion returns error you don't need to do anything, but if the second insertion returns error then remove the id got from the insertion in first collection.
I can suggest following structure:
'transaction'(){
PlanSubscriptions.insert(data, (error, result)=>{
if(result){
// result contains the _id
let id_plan = result;
ClientActivations.insert(data, (error, result)=>{
if(result){
// result contains the _id
return true;
}
else if(error){
PlanSubscriptions.remove(id_plan);
return false;
}
})
}
else if(error){
return false;
}
})
}
There is no way to do that in Meteor, since mongodb is not an ACID-compliant database. It has a single-document update atomicity, but not a multiple-document one, which is your case with the two collections.
From the mongo documentation:
When a single write operation modifies multiple documents, the modification of each document is atomic, but the operation as a whole is not atomic and other operations may interleave.
A way to isolate the visibility of your multi-document updates is available, but it's probably not what you need.
Using the $isolated operator, a write operation that affects multiple documents can prevent other processes from interleaving once the write operation modifies the first document. This ensures that no client sees the changes until the write operation completes or errors out.
An isolated write operation does not provide “all-or-nothing” atomicity. That is, an error during the write operation does not roll back all its changes that preceded the error.
However, there are a couple of libraries which try to tackle the problem at the app-level. I recommend taking a look at fawn
In your case, where you have exactly two dependent collections, it's possible to take advantage of the two phase commits technique. Read more about it here: two-phase-commits
Well I figured it out myself.
I added a package babrahams:transactions
At server side Meteor Method call, I called tx Object that is globally exposed by the package. The overall Server Side Meteor.method({}) looks like below.
import { Meteor } from 'meteor/meteor';
import {PlanSubscriptions} from '/imports/api/plansubscriptions/plansubscriptions.js';
import {ClientActivations} from '/imports/api/clientactivation/clientactivations.js';
Meteor.methods({
'createClientSubscription' (subscriptionData, clientActivationData) {
var txid;
try {
txid = tx.start("Adding Subscription to our database");
PlanSubscriptions.insert(subscriptionData, {tx: true})
ClientActivations.insert(activation, {tx: true});
tx.commit();
return true;
} catch(e){
tx.undo(txid);
}
return false;
}
});
With every insert I had added {tx : true}, this concluded it to be a apart of transaction.
Server Console Output:
I20170523-18:43:23.544(5.5)? Started "Adding Subscription to our database" with
transaction_id: vdJQvFgtyZuWcinyF
I20170523-18:43:23.547(5.5)? Pushed insert command to stack: vdJQvFgtyZuWcinyF
I20170523-18:43:23.549(5.5)? Pushed insert command to stack: vdJQvFgtyZuWcinyF
I20170523-18:43:23.551(5.5)? Beginning commit with transaction_id: vdJQvFgtyZuWcinyF
I20170523-18:43:23.655(5.5)? Executed insert
I20170523-18:43:23.666(5.5)? Executed insert
I20170523-18:43:23.698(5.5)? Commit reset transaction manager to clean state
For more Information you can goto link : https://github.com/JackAdams/meteor-transactions
NOTE: I am using Meteor 1.4.4.2
Just sharing this link for future readers:
https://forums.meteor.com/t/solved-transactions-with-mongodb-meteor-methods/48677
import { MongoInternals } from 'meteor/mongo';
// utility async function to wrap async raw mongo operations with a transaction
const runTransactionAsync = async asyncRawMongoOperations => {
// setup a transaction
const { client } = MongoInternals.defaultRemoteCollectionDriver().mongo;
const session = await client.startSession();
await session.startTransaction();
try {
// running the async operations
let result = await asyncRawMongoOperations(session);
await session.commitTransaction();
// transaction committed - return value to the client
return result;
} catch (err) {
await session.abortTransaction();
console.error(err.message);
// transaction aborted - report error to the client
throw new Meteor.Error('Database Transaction Failed', err.message);
} finally {
session.endSession();
}
};
import { runTransactionAsync } from '/imports/utils'; // or where you defined it
Meteor.methods({
async doSomething(arg) {
// remember to check method input first
// define the operations we want to run in transaction
const asyncRawMongoOperations = async session => {
// it's critical to receive the session parameter here
// and pass it to every raw operation as shown below
const item = await collection1.rawCollection().findOne(arg, { session: session });
const response = await collection2.rawCollection().insertOne(item, { session: session });
// if Mongo or you throw an error here runTransactionAsync(..) will catch it
// and wrap it with a Meteor.Error(..) so it will arrive to the client safely
return 'whatever you want'; // will be the result in the client
};
let result = await runTransactionAsync(asyncRawMongoOperations);
return result;
}
});

Resources