When doing multiple updates with batch, the page is getting stuck in a loop - firebase

I am trying to update multiple documents using batch() and commit().
The intended document are getting updated but the page goes still and I cannot click or go back or refresh the page.
However, when I console log the output message its going on a loop recursively.
let users_DB = db.collection( `users/${ profileEmail }/details` )
let currentDate = new Date()
users_DB.onSnapshot( ( querySnapshot ) =>
{
var batch = firebase.firestore().batch()
querySnapshot.forEach( ( doc ) =>
{
let dbDate = new Date( doc.data().dateInMills * 1000 )
if ( dbDate >= currentDate )
{
batch.update( users_DB.doc( doc.id ), {
'fees': parseInt( fees.value ),
'timeStamp': firebase.firestore.FieldValue.arrayUnion( firebase.firestore.Timestamp.fromDate( new Date() ) ),
} )
}
batch.commit().then(() => {console.log( 'changes made.' );})
} )
} )
How do I ensure avoid this behavior?

Because you are using the CollectionReference#onSnapshot method, you are listening for any changes to the data that matches the query. When you invoke batch.commit(), in addition to writing the new data to the server, all listeners will be notified that their data has changed - including your listener that's doing the update. Because you add a new element to the timeStamp field on every iteration, your code recursively calls itself.
To correct this, switch to a one time data retrieval using CollectionReference#get instead.
const users_DB = db.collection(`users/${profileEmail}/details`);
const currentDate = new Date();
users_DB
.get()
.then((querySnapshot) => {
const batch = db.batch(); // use db instead of firebase.firestore() for consistency
querySnapshot.forEach(doc => {
const dbDate = new Date(doc.get("dateInMills") * 1000); // dateInMills isn't in milliseconds?
if (dbDate >= currentDate) {
batch.update(doc.ref, {
'fees': parseInt( fees.value ),
'timeStamp': firebase.firestore.FieldValue.arrayUnion(firebase.firestore.Timestamp.fromDate(new Date())),
});
}
}
return batch.commit(); // this is a no-op if no changes are needed
})
.then(() => console.log("changes made"));
However, you can refine this with a query (which is how I would approach it):
const nowMS = Date.now(),
newData = {
fees: parseInt(fees.value),
timeStamp: firebase.firestore.FieldValue.arrayUnion(firebase.firestore.Timestamp.fromMillis(nowMS))
};
db.collection(`users/${profileEmail}/details`)
.where("dateInMills", ">=", nowMS / 1000)
.get()
.then(qs => {
const batch = db.batch();
qs.forEach(docSnap => {
batch.update(docSnap.ref, newData);
});
return batch.commit();
})
.then(() => console.log("changes made"));

Related

Firestore: How to run a batch write statement in a forEach() loop?

I am learning about Firestore's batch writes method and it looks really neat. Almost async-like! However, I am needing some help figuring out how to run a batch statement when doing a forEach() on a query.
My use case is that when a user deletes a post, I need to also "clean up" and update/delete other items associated with that post. That could be all bookmarks users have created for this post, likes, etc.
Here is an example of a deletePost function. How do you run a batch statement on the bookmarksQuery and usersAnswerQuery queries?
async deletePost(post) {
const response = confirm('Delete this post?')
const batch = this.$fire.firestore.batch()
if (response === true && this.userProfile.uid === this.post.uid) {
try {
const postRef = this.$fire.firestore
.collection(`users/${post.uid}/posts`)
.doc(this.post.id)
const answerRef = this.$fire.firestore
.collection('answers')
.doc(this.post.commentIdWithAnswer)
const usersAnswerQuery = await this.$fire.firestore
.collectionGroup('answers')
.where('id', '==', this.post.commentIdWithAnswer)
.get()
const bookmarksQuery = await this.$fire.firestore
.collectionGroup('bookmarks')
.where('id', '==', this.post.id)
.get()
batch.update(postRef, {
published: false,
deleted: true,
updatedAt: this.$fireModule.firestore.FieldValue.serverTimestamp()
})
bookmarksQuery.forEach((doc) => doc.ref.delete()) //<---- how to add this to batch?
usersAnswerQuery.forEach((doc) => doc.ref.delete()) //<---- how to add this to batch?
batch.delete(answerRef)
await batch.commit()
// To do: delete all user 'likes' associated with this post
alert('Post successfully deleted!')
} catch (error) {
console.error('error deleting post.', error)
}
} else {
return null
}
}
To add a document deletion to the batch, you would use WriteBatch#delete() like you have done for answerRef using:
// prepare the batch
const batch = firebase.firestore().batch();
// add each doc's deletion to the batch
docs.forEach((doc) => batch.delete(doc.ref));
// commit the changes
await batch.commit();
While the above approach works fine, a batched write has a limit of 500 operations. As you will likely hit this limit on popular posts while tidying up bookmarks, answers and likes, we need to handle this case. We can achieve this by tracking the number of operations you've added into the batch and create a new batch each time you reach the limit.
// prepare the batch
let currentBatch = firebase.firestore().batch();
let currentBatchSize = 0;
const batches = [ currentBatch ];
// add each doc's deletion to the batch
docs.forEach((doc) => {
// when batch is too large, start a new one
if (++currentBatchSize >= 500) {
currentBatch = firebase.firestore.batch();
batches.push(currentBatch);
currentBatchSize = 1;
}
// add operation to batch
currentBatch.delete(doc.ref);
})
// commit the changes
await Promise.all(batches.map(batch => batch.commit()));
Other things I've noticed in your current code:
deletePost has an inconsistent return type of Promise<void | null> - consider returning a Promise<boolean> (to indicate success, because you are handling errors in your function)
You ask for user confirmation before checking whether the post can actually be deleted by the current user - you should check first
Silently fails to delete another user's post, instead of showing an error (this should also be enforced by security rules)
Silently fails to delete the post, without showing a message to the user
You have a large if block followed by a tiny else block, you should flip it so you can "fail-fast" and not need to indent most of the code.
Applying the solution plus these other changes gives:
async deletePost(post) {
if (this.userProfile.uid !== this.post.uid) {
alert("You can't delete another user's post.");
return false; // denied
}
const response = confirm('Delete this post?')
if (!response)
return false; // cancelled
try {
const postRef = this.$fire.firestore
.collection(`users/${post.uid}/posts`)
.doc(this.post.id)
const answerRef = this.$fire.firestore
.collection('answers')
.doc(this.post.commentIdWithAnswer)
const usersAnswerQuery = await this.$fire.firestore
.collectionGroup('answers')
.where('id', '==', this.post.commentIdWithAnswer)
.get()
const bookmarksQuery = await this.$fire.firestore
.collectionGroup('bookmarks')
.where('id', '==', this.post.id)
.get()
let currentBatch = this.$fire.firestore.batch();
const batches = [currentBatch];
currentBatch.update(postRef, {
published: false,
deleted: true,
updatedAt: this.$fireModule.firestore.FieldValue.serverTimestamp()
});
currentBatch.delete(answerRef);
let currentBatchSize = 2;
const addDocDeletionToBatch = (doc) => {
if (++currentBatchSize >= 500) {
currentBatch = this.$fire.firestore.batch();
batches.push(currentBatch);
currentBatchSize = 1;
}
currentBatch.delete(doc.ref);
}
bookmarksQuery.forEach(addDocDeletionToBatch)
usersAnswerQuery.forEach(addDocDeletionToBatch)
// TODO: delete all user 'likes' associated with this post
// commit changes
await Promise.all(batches.map(batch => batch.commit()));
alert('Post successfully deleted!')
return true;
} catch (error) {
console.error('error deleting post.', error)
alert('Failed to delete post!');
return false;
}
}
Note: If you use the standard comments // TODO and // FIXME, you can make use of many tools that recognise and highlight these comments.
Do as follows. Do not forget the 500 docs limit for a batched write (which includes deletions).
async deletePost(post) {
const response = confirm('Delete this post?')
const batch = this.$fire.firestore.batch()
if (response === true && this.userProfile.uid === this.post.uid) {
try {
// ...
batch.update(postRef, {
published: false,
deleted: true,
updatedAt: this.$fireModule.firestore.FieldValue.serverTimestamp()
})
bookmarksQuery.forEach((doc) => batch.delete(doc.ref))
usersAnswerQuery.forEach((doc) => batch.delete(doc.ref))
batch.delete(answerRef)
await batch.commit()
// To do: delete all user 'likes' associated with this post
alert('Post successfully deleted!')
} catch (error) {
console.error('error deleting post.', error)
}
} else {
return null
}
}

Firestore transaction failing inside forEach loop

I am trying to update several Firestore documents, based on the result of a third-party service inside a transaction. Problem is, I am getting the following error:
Error: Cannot modify a WriteBatch that has been committed.
Here is my code:
export default async function debitDueTransactions(context: any) {
const now = new Date().getTime();
return db.runTransaction(async (transaction: FirebaseFirestore.Transaction) => {
const chargesToCaptureRef = db.collection(`charges_to_capture`)
.where('dateToCapture', '>=', now)
.where('dateToCapture', '<=', (now + 86400000))
.where('captureResult', '==', null);
return transaction.get(chargesToCaptureRef).then((chargeToCaptureQuerySnap: FirebaseFirestore.QuerySnapshot) => {
chargeToCaptureQuerySnap.forEach(async (doc: FirebaseFirestore.QueryDocumentSnapshot) => {
const chargeToCapture = <ChargeToCapture>doc.data();
chargeToCapture.id = doc.id;
let errorKey = null;
// Calling third party service here, waiting response
const captureResult = await captureCharge(chargeToCapture.chargeId).catch((error: any) => {
errorKey = error.code ? error.code : 'unknown_error';
});
transaction.update(doc.ref, { captureResult: captureResult, errorKey: errorKey });
});
return new Promise((resolve) => { resolve(); });
})
});
}
Can't get what I am doing wrong, any idea ?
As you can see from the API documentation, transaction.get() only accepts a DocumentReference type object. You're passing it a Query object. A Firestore transaction isn't capable of transacting on a Query. If you want to transact on all the documents returned from a Query, you should perform the query before the transaction, then use transaction.get() on each DocumentReference individually.

How to update document in firebase cloud function

In my cloud function I want to update my document from 'dashboard' collection when a new student added to 'students' collection.
const getActiveStudents = () => {
return db.collection('/students/').where('status', '==', true).get().then(
snapshot => {
let studentsCount = snapshot.docs.length;
db.collection('/dashboard/').where('type', '==', 'students').get().then(
result => {
if (result.docs.length === 0) {
db.collection('dashboard').add({
count: studentsCount,
type: 'students',
label: 'Active students'
});
}else {
result.docs[0].ref.update({
count: studentsCount,
type: 'students',
label: 'Active students'
});
}
return result;
}
).catch(error => {
console.log(error);
});
return snapshot;
}
).catch(error => {
console.log(error);
})
}
exports.onChangesInStudents = functions.firestore.document('/students/{studentId}').onWrite(event => {
getActiveStudents();
return;
});
When I add a new student, instead of updating document it adds a new document to my 'dashboard' collection.
How should I organize my code in order to properly update the quantity of students.
as #Doug mentioned, iterating over the entire collection is too heavy. instead you can stream the query results and iterate over keys, using query.stream().
to access and update a single field in a document, first retrieve the document by its ID with doc(), then use update() while specifying the field.
here's an example of implementation based on your scenario.
package.json
{
"dependencies": {
"firebase-admin": "^6.5.1",
"firebase-functions": "^2.1.0"
}
}
index.js
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp(functions.config().firebase);
const studentsRef = admin.firestore().collection('students');
const dashboardRef = admin.firestore().collection('dashboard');
exports.addStudent = functions.firestore
.document('students/{studentId}')
.onCreate((snap, context) => {
var newStudent = snap.data();
console.log('New student in collection: ', newStudent);
var activeCount = 0;
studentsRef.where('status', '==', true).select().stream()
.on('data', () => {
++activeCount;
}).on('end', () => {
dashboardRef.where('type', '==', 'students').get()
.then(querySnap => {
if (querySnap.docs[0].data().count == activeCount){
console.log('No new active student: ', querySnap.docs[0].data());
} else {
console.log('New active count: ', activeCount);
console.log('Student Dashboard before update: ', querySnap.docs[0].id, '=>', querySnap.docs[0].data());
dashboardRef.doc(querySnap.docs[0].id).update({
count: activeCount
});
console.log('Active student count updated: ', querySnap.docs[0].data().count, '=>', activeCount);
};
});
});
return null
});
gcloud
gcloud functions deploy addStudent \
--runtime nodejs8 \
--trigger-event providers/cloud.firestore/eventTypes/document.create \
--trigger-resource "projects/[PROJECT_ID]/databases/(default)/documents/students/{studentId}"
When a function is triggered, you might want to get data from a document that was updated, or get the data prior to update.
You can get the prior data by using change.before.data(), which contains the document snapshot before the update.
Similarly, change.after.data() contains the document snapshot state after the update.
Node.js
exports.updateUser = functions.firestore
.document('users/{userId}')
.onUpdate((change, context) => {
// Get an object representing the current document
const newValue = change.after.data();
// ...or the previous value before this update
const previousValue = change.before.data();
//...therefore update the document as.
admin.firestore().collection('user').doc(docId).update(snapshot.after.data());
});
Reference:-
https://firebase.google.com/docs/functions/firestore-events

Firebase Cloud Functions set value operation deletes the trigger originating node

Hello, above is my simple data structure in firebase's realtime db. I'm working on a simple cloud function that will listen to update in a user node 'score' property and update the 'averageScore' field that sits higher up the hierarchy.
Here's my onWrite callback:
.onWrite((change, context) => {
if (!change.before.exists() || !change.after.exists()) {
return null;
}
const beforeScore = parseFloat(change.before.val()['score']);
const afterScore = parseFloat(change.after.val()['score']);
const userRef = change.after.ref;
var promises = [
userRef.parent.parent.child('userCount').once('value'),
userRef.parent.parent.child('averageScore').once('value')
];
return userRef.transaction(() => {
return Promise.all(promises).then((snapshots) => {
const userCount = snapshots[0].val();
const averageScore = snapshots[1].val();
const currentAverage = (( ( averageScore * userCount ) - beforeScore + afterScore ) / userCount ).toFixed(2);
return userRef.parent.parent.child('averageScore').set(currentAverage);
});
});
});
If I update userId 1234's score, the averageScore field is updated correctly per this code. However, the whole user Id node 1234 gets DELETED following this update. This is quite a head scratcher and hoping to get some insight from the community on what I might be doing wrong.
Cheers.
.onWrite((change, context) => {
if ( !change.before.exists() || !change.after.exists()) {
return null;
}
const beforeScore = parseFloat(change.before.val()['score']);
const afterScore = parseFloat(change.after.val()['score']);
const crowdStatsRef = change.after.ref.parent.parent.child('crowdStats');
return Promise.all( [
crowdStatsRef.child('userCount').once('value'),
crowdStatsRef.child('averageScore').once('value')
]).then((snapshots) => {
return crowdStatsRef.transaction((crowdStatsNode) => {
if (crowdStatsNode) {
const userCount = snapshots[0].val();
const averageScore = snapshots[1].val();
const currentAverage = (( ( averageScore * userCount ) - beforeScore + afterScore ) / userCount ).toFixed(2);
crowdStatsNode.score = parseFloat(currentAverage);
}
return crowdStatsNode;
}, (error, committed, snapshot) => {
if (error) {
console.error(error);
}
});
});
});
Misunderstood how transactions worked. The object you're locking onto must be returned in the callback function. Also, a null check in that callback function is essential here.
Examples are noted here:
https://firebase.googleblog.com/2016/01/keeping-our-promises-and-callbacks_76.html

firestore cloud functions onCreate/onDelete sometimes immediately triggered twice

I have observed this behavior occasionally with both onCreate and onDelete triggers.
Both the executions happened for the same document created in firestore. There's only one document there so I don't understand how it could trigger the handler twice. the handler itself is very simple:
module.exports = functions.firestore.document('notes/{noteId}').onCreate((event) => {
const db = admin.firestore();
const params = event.params;
const data = event.data.data();
// empty
});
this doesn't happen all the time. What am I missing?
See the Cloud Firestore Triggers Limitations and Guarantees:
Delivery of function invocations is not currently guaranteed. As the
Cloud Firestore and Cloud Functions integration improves, we plan to
guarantee "at least once" delivery. However, this may not always be
the case during beta. This may also result in multiple invocations
for a single event, so for the highest quality functions ensure that
the functions are written to be idempotent.
There is a Firecast video with tips for implementing idempotence.
Also two Google Blog posts: the first, the second.
Based on #saranpol's answer we use the below for now. We have yet to check if we actually get any duplicate event ids though.
const alreadyTriggered = eventId => {
// Firestore doesn't support forward slash in ids and the eventId often has it
const validEventId = eventId.replace('/', '')
const firestore = firebase.firestore()
return firestore.runTransaction(async transaction => {
const ref = firestore.doc(`eventIds/${validEventId}`)
const doc = await transaction.get(ref)
if (doc.exists) {
console.error(`Already triggered function for event: ${validEventId}`)
return true
} else {
transaction.set(ref, {})
return false
}
})
}
// Usage
if (await alreadyTriggered(context.eventId)) {
return
}
In my case I try to use eventId and transaction to prevent onCreate sometimes triggered twice
(you may need to save eventId in list and check if it exist if your function actually triggered often)
const functions = require('firebase-functions')
const admin = require('firebase-admin')
const db = admin.firestore()
exports = module.exports = functions.firestore.document('...').onCreate((snap, context) => {
const prize = 1000
const eventId = context.eventId
if (!eventId) {
return false
}
// increment money
const p1 = () => {
const ref = db.doc('...')
return db.runTransaction(t => {
return t.get(ref).then(doc => {
let money_total = 0
if (doc.exists) {
const eventIdLast = doc.data().event_id_last
if (eventIdLast === eventId) {
throw 'duplicated event'
}
const m0 = doc.data().money_total
if(m0 !== undefined) {
money_total = m0 + prize
}
} else {
money_total = prize
}
return t.set(ref, {
money_total: money_total,
event_id_last: eventId
}, {merge: true})
})
})
}
// will execute p2 p3 p4 if p1 success
const p2 = () => {
...
}
const p3 = () => {
...
}
const p4 = () => {
...
}
return p1().then(() => {
return Promise.all([p2(), p3(), p4()])
}).catch((error) => {
console.log(error)
})
})
Late to the party, I had this issue but having a min instance solved the issue for me
Upon looking #xaxsis attached screenshot, my function took almost the amount of time about 15 seconds for the first request and about 1/4 of that for the second request

Resources