I'm trying to keep track of the number of documents in collections and the number of users in my Firebase project. I set up some .create triggers to update a stats document using increment, but sometimes the .create functions trigger multiple times for a single creation event. This happens with both Firestore documents and new users. Any ideas?
const functions = require('firebase-functions');
const admin = require('firebase-admin');
const firestore = require('#google-cloud/firestore')
admin.initializeApp();
const db = admin.firestore()
/* for counting documents created */
exports.countDoc = functions.firestore
.document('collection/{docId}')
.onCreate((change, context) => {
const docId = context.params.docId
db.doc('stats/doc').update({
'docsCreated': firestore.FieldValue.increment(1)
})
return true;
});
/* for counting users created */
exports.countUsers = functions.auth.user().onCreate((user) => {
db.doc('stats/doc').update({
'usersCreated': firestore.FieldValue.increment(1)
})
return true;
});
Thanks!
There is some advice on how to achieve your functions' idempotency.
There are FieldValue.arrayUnion() & FieldValue.arrayRemove() functions which safely remove and add elements to an array, without duplicates or errors if the element being deleted is nonexistent.
You can make array fields in your documents called 'users' and 'docs' and add there data with FieldValue.arrayUnion() by triggered functions. With that approach you can retrieve the actual sizes on the client side by getting users & docs fields and calling .size() on it.
You should expect that a background trigger could possibly be executed multiple times per event. This should be very rare, but not impossible. It's part of the guarantee that Cloud Functions gives you for "at-least-once execution". Since the internal infrastructure is entirely asynchronous with respect to the execution of your code on a dedicated server instance, that infrastructure might not receive the signal that your function finished successfully. In that case, it triggers the function again in order to ensure delivery.
It's recommended that you write your function to be idempotent in order to handle this situation, if it's important for your app. This is not always a very simple thing to implement correctly, and could also add a lot of weight to your code. There are also many ways to do this for different sorts of scenarios. But the choice is yours.
Read more about it in the documentation for execution guarantees.
Related
Here is the situation:
I have collections 'lists', 'stats', and 'posts'.
From frontend, there is a scenario where the user uploads a content. The frontend function creates a document under 'lists', and after the document is created, it creates another document under 'posts'.
I have a CF that listens to creation of a document under 'lists' and create a new document under 'stats'.
I have a CF that listens to creation of a document under 'posts' and update the document created under 'stats'.
The intended order of things to happen is 2->3->4. However, apparently, step 4 is triggered before step 3, and so there is no relevant document under 'stats' to update, thus throwing an error.
Is there a way to make the function wait for the document creation under 'stats' and update only after it is created? I thought about using setTimeout() for the function in step 4, but guess there might be a better way.
Below is the code that I am using for steps 3 and 4. Can someone advise? Thanks!
//This listens to a creation of a document under 'lists' and creates a new document
//with the same document ID under 'stats'.
exports.statsCreate = functions.firestore
.document('lists/{listid}').onCreate((snap,context)=>{
const listidpath=snap.ref.path;
const pathfinder=listidpath.split('/');
const listid=pathfinder[pathfinder.length-1];
return db.collection('stats').doc(listid).set({
postcount:0,
})
})
//This listens to a creation of a document under 'posts' and updates the corresponding
// document under 'stats'. There is a field under 'posts' with the list ID to make this possible.
// How do I make sure the update operation happens only after the document is actually there?
exports.statsUpdate = functions.firestore
.document('posts/{postid}').onCreate((snap,context)=>{
const data=snap.data();
return db.collection('stats').doc(data.listid).update({
postcount:admin.firestore.FieldValue.increment(1)
})
})
I can see at least two "easy" solutions:
Solution #1: In your front end, set a listener to the to-be-created stat document (with onSnapshot()), and only create the post document when the stat one has been created. Note however that this solution will not work if the user does not have read access right to the posts collection.
Solution #2: Use the "retry on failure" option for background Cloud Functions. Within your statsUpdate Cloud Function you intentionally throw an exception if the stat doc is not found => The CF will be retried until the stat doc is created.
A third solution would be to use a Callable Cloud Function, called from your front-end. This Callable Cloud Function would write the three docs in the following order: list, stat and post. Then the statsUpdate Cloud Function would be triggered in the background (or you could include its business logic in the Callable Cloud Function as well).
One of the drawbacks of this solution is that the Cloud Function may encounter some cold start effect. In this case, from an end-user perspective, the process may take more time than the abonne solutions. However note that you can specify a minimum number of container instances to be kept warm and ready to serve requests.
PS: Note that in the statsCreate CF, you don't need to extract the listid with:
const listidpath=snap.ref.path;
const pathfinder=listidpath.split('/');
const listid=pathfinder[pathfinder.length-1];
Just do:
const listid = context.params.listid;
The context parameter provides information about the Cloud Function's execution.
Does anyone know how to limit an array so new items get pushed in and old ones are discarded in the same write?
I'm guessing this isn't possible but it sure would be handy.
// * Store notification
// Users collection
const usersCollection = db.collection('users').doc(uid).collection('notifications').doc();
// Write this notification to the database as well
await usersCollection.update({
count: admin.firestore.FieldValue.increment,
notifications: admin.firestore.FieldValue.arrayUnion({
'symbol': symbol,
'companyname': companyname,
'change': priceDifference,
'changeDirection': directionOperatorHandler,
'updatedPrice': symbolLatestPrice,
'timestamp': currentTimestamp,
})
});
Written in Typescript
Alternatively, I was thinking of running a scheduled cloud function every week to go through and trim down the arrays based on the timestamp.
The reason I'm using an array to store my notifications is because I'm expecting a lot of writes.
There is no simple configuration for this. Your code should implement your requirements by:
Reading the document
Modifying the array in memory
Checking that the size is within limits
Writing the document back
I'm trying to update the same document which triggered an onUpdate cloud function, with a read value from the same collection.
This is in a kind of chat app made in Flutter, where the previous response to an inquiry is replicated to the document now being updated, for easier showing in the app.
The code does work, however when a user quickly responds to two separate inquiries, they both read the same latest response thus setting the same previousResponse. This must be down to the asynchronous nature of flutter and/or the cloud function, but I can't figure out where to await or if there's a better way to make the function, so it is never triggering the onUpdate for the same user, until a previous trigger is finished.
Last part also sound a bit like a bad idea.
So far I tried sticking the read/update in a transaction, however that only seems to work for the single function call, and not when they're asynchronous.
Also figured I could fix it, by reading the previous response in a transaction on the client, however firebase doesn't allow reading from a collection in a transaction, when not using the server API.
async function setPreviousResponseToInquiry(
senderUid: string,
recipientUid: string,
inquiryId: string) {
return admin.firestore().collection('/inquiries')
.where('recipientUid', '==', recipientUid)
.where('senderUid', '==', senderUid)
.where('responded', '==', true)
.orderBy('modified', 'desc')
.limit(2)
.get().then(snapshot => {
if (!snapshot.empty &&
snapshot.docs.length >= 2) {
return admin.firestore()
.doc(`/inquiries/${inquiryId}`)
.get().then(snap => {
return snap.ref.update({
previousResponse: snapshot.docs[1].data().response
})
})
}
})
}
I see three possible solutions:
Use a transaction on the server, which ensures that the update you write must be based on the version of the data you read. If the value you write depends on the data that trigger the Cloud Function, you may need to re-read that data as part of the transaction.
Don't use Cloud Functions, but run all updates from the client. This allows you to use transactions to prevent the race condition.
If it's no possible to use a transaction, you may have to include a custom version number in both the upstream data (the data that triggers the write), and the fanned out data that you're updating. You can then use security rules to ensure that the downstream data can only be written if its version matches the current upstream data.
I'd consider/try them in the above order, as they gradually get more involved.
I'm using Firestore at beta version with Cloud Functions. In my app I need to trigger a function that listens for an onCreate event at /company/{id}/point/{id} and performs an insert (collection('event').add({...}))
My problem is: Cloud Functions with Firestore require an idempotent function. I don't know how to ensure that if my function triggers two times in a row with the same event, I won't add two documents with the same data.
I've found that context.eventId could handle that problem, but I don't recognize a way to use it.
exports.creatingEvents = functions.firestore
.document('/companies/{companyId}/points/{pointId}')
.onCreate((snap, context) => {
//some logic...
return db.doc(`eventlog/${context.params.companyId}`).collection('events').add(data)
})
Two things:
First check your collection to see if a document has a property with the value of context.eventId in it. If it exists, do nothing in the function.
If a document with the event id doesn't already exist, put the value of context.eventId in a property in the document that you add.
This should prevent multiple invocations of the function from adding more than one document for a given event id.
Why not set the document (indexing by the event id from your context) instead of creating it? This way if you write it twice, you'll just overwrite rather than create a new record.
https://firebase.google.com/docs/firestore/manage-data/add-data
This approach makes the write operation idempotent.
It's a known issue that firebase doesn't have easy way to count items. I'm planning to create an app that relies heavily on counts and other aggregates. I fear creating this app's counters with the rules as suggested here will be incredibly complex and hard to maintain.
So I thought about this pattern:
I will keep a server that will listen to all items entered in the database and this server will update all counters and aggregates. The server will hold the UID of a special admin that only he can update counters.
This way, users will not have to download entire nodes in order to get a count, plus I won't have to deal with issues that arise from maintaining counters by clients.
Does this pattern make sense? Am I missing something?
Firebase has recently released Cloud Functions. As mentioned on the documentation:
Cloud Functions is a hosted, private, and scalable Node.js environment
where you can run JavaScript code.
With Cloud Functions, you don't need to create your own Server. You can simply write JavaScript functions and upload it to Firebase. Firebase will be responsible for triggering functions whenever an event occurs.
For example, let's say you want to count the number of likes in a post. You should have a structure similar to this one:
{
"Posts" : {
"randomKey" : {
"likes_count":5,
"likes" : {
"userX" : true,
"userY" : true,
"userZ" : true,
...
}
}
}
}
And your JavaScript function would be written like this:
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp(functions.config().firebase);
// Keeps track of the length of the 'likes' child list in a separate attribute.
exports.countlikes = functions.database.ref('/posts/$postid/likes').onWrite(event => {
return event.data.ref.parent().child('likes_count').set(event.data.numChildren());
});
This code increases the likes_count variable every time there is a new write on the likes node.
This sample is available on GitHub.