I have observed this behavior occasionally with both onCreate and onDelete triggers.
Both the executions happened for the same document created in firestore. There's only one document there so I don't understand how it could trigger the handler twice. the handler itself is very simple:
module.exports = functions.firestore.document('notes/{noteId}').onCreate((event) => {
const db = admin.firestore();
const params = event.params;
const data = event.data.data();
// empty
});
this doesn't happen all the time. What am I missing?
See the Cloud Firestore Triggers Limitations and Guarantees:
Delivery of function invocations is not currently guaranteed. As the
Cloud Firestore and Cloud Functions integration improves, we plan to
guarantee "at least once" delivery. However, this may not always be
the case during beta. This may also result in multiple invocations
for a single event, so for the highest quality functions ensure that
the functions are written to be idempotent.
There is a Firecast video with tips for implementing idempotence.
Also two Google Blog posts: the first, the second.
Based on #saranpol's answer we use the below for now. We have yet to check if we actually get any duplicate event ids though.
const alreadyTriggered = eventId => {
// Firestore doesn't support forward slash in ids and the eventId often has it
const validEventId = eventId.replace('/', '')
const firestore = firebase.firestore()
return firestore.runTransaction(async transaction => {
const ref = firestore.doc(`eventIds/${validEventId}`)
const doc = await transaction.get(ref)
if (doc.exists) {
console.error(`Already triggered function for event: ${validEventId}`)
return true
} else {
transaction.set(ref, {})
return false
}
})
}
// Usage
if (await alreadyTriggered(context.eventId)) {
return
}
In my case I try to use eventId and transaction to prevent onCreate sometimes triggered twice
(you may need to save eventId in list and check if it exist if your function actually triggered often)
const functions = require('firebase-functions')
const admin = require('firebase-admin')
const db = admin.firestore()
exports = module.exports = functions.firestore.document('...').onCreate((snap, context) => {
const prize = 1000
const eventId = context.eventId
if (!eventId) {
return false
}
// increment money
const p1 = () => {
const ref = db.doc('...')
return db.runTransaction(t => {
return t.get(ref).then(doc => {
let money_total = 0
if (doc.exists) {
const eventIdLast = doc.data().event_id_last
if (eventIdLast === eventId) {
throw 'duplicated event'
}
const m0 = doc.data().money_total
if(m0 !== undefined) {
money_total = m0 + prize
}
} else {
money_total = prize
}
return t.set(ref, {
money_total: money_total,
event_id_last: eventId
}, {merge: true})
})
})
}
// will execute p2 p3 p4 if p1 success
const p2 = () => {
...
}
const p3 = () => {
...
}
const p4 = () => {
...
}
return p1().then(() => {
return Promise.all([p2(), p3(), p4()])
}).catch((error) => {
console.log(error)
})
})
Late to the party, I had this issue but having a min instance solved the issue for me
Upon looking #xaxsis attached screenshot, my function took almost the amount of time about 15 seconds for the first request and about 1/4 of that for the second request
Related
Assume there is a collection of users and each user is associated with accounts, which are kept in a separate collection. For each account there is a balance which is updated periodically by some external means (e.g. the http trigger below). I need to be able to query for the user's total balance across all of her accounts.
I added onUpdate trigger which gets called everytime an account changes and updates the total accordingly. However, it seems that there is some race condition e.g. when two accounts get updated around the same time: after onUpdate is called for the first account and updates the total balance, it is still not updated when onUpdate is called for the second account. I'm guessing I need to somehow use "transaction" for the bookkeeping but not sure how.
const data = {
'users/XXX': {
email: "a#b.com",
balance: 0
},
"accounts/YYY": {
title: "Acc1",
userID: "XXX"
balance: 0
},
"accounts/ZZZ": {
title: "Acc2",
userID: "XXX"
balance: 0
}
};
exports.updateAccounts = functions.https.onRequest((request, response) => {
admin.firestore().collection('accounts').get().then((accounts) => {
accounts.forEach((account) => {
return admin.firestore().collection('accounts').doc(account.id).update({balance:
WHATEVER});
})
response.send("Done");
});
exports.updateAccount = functions.firestore
.document('accounts/{accountID}')
.onUpdate((change, context) => {
const userID = change.after.data().userID;
admin.firestore().doc("users/"+userID).get().then((user) => {
const new_balance = change.after.data().balance;
const old_balance = change.before.data().balance;
var user_balance = user.data().balance + new_balance - old_balance;
admin.firestore().doc("users/"+userID).update({balance: user_balance});
});
});
By looking at your code we can see several parts of it that could lead to incorrect results. It is not possible, without thoroughly testing and reproducing your problem, to be sure at 100% that correcting them will totally solve your problem but it is most probably the cause of the problems.
HTTP Cloud Function:
With the forEach() loop you are calling several asynchronous operations (the update() method) but you don't wait that all these asynchronous operations are completed before sending back the response. You should do as follows, using Promise.all() to wait all the asynchronous methods are completed before sending the response:
exports.updateAccounts = functions.https.onRequest((request, response) => {
const promises = [];
admin.firestore().collection('accounts').get()
.then(accounts => {
accounts.forEach((account) => {
promises.push(admin.firestore().collection('accounts').doc(account.id).update({balance: WHATEVER}));
return Promise.all(promises);
})
.then(() => {
response.send("Done");
})
.catch(error => {....});
});
onUpdate background triggered Cloud Function
There you need to correctly return the Promises chain in order to indicate to the platform when the Cloud Function is complete. The following should do the trick:
exports.updateAccount = functions.firestore
.document('accounts/{accountID}')
.onUpdate((change, context) => {
const userID = change.after.data().userID;
return admin.firestore().doc("users/"+userID).get() //Note the return here. (Note that in the HTTP Cloud Function we don't need it! see the link to the video series below)
.then(user => {
const new_balance = change.after.data().balance;
const old_balance = change.before.data().balance;
var user_balance = user.data().balance + new_balance - old_balance;
return admin.firestore().doc("users/"+userID).update({balance: user_balance}); //Note the return here.
});
});
I would suggest that you watch the 3 videos about "JavaScript Promises" from the Firebase video series: https://firebase.google.com/docs/functions/video-series/. They explain all the key points that were corrected above.
At first sight, it seems that if you modify, in the updateAccounts Cloud Function, several account documents that share the same user you will indeed need to implement the user balance update in a transaction, as several instances of the updateAccount Cloud Function may be triggered in parallel. The doc on Transactions is here.
Update:
You could implement a Transaction as follows in the updateAccounts Cloud Function (untested):
exports.updateAccount = functions.firestore
.document('accounts/{accountID}')
.onUpdate((change, context) => {
const userID = change.after.data().userID;
const userRef = admin.firestore().doc("users/" + userID);
return admin.firestore().runTransaction(transaction => {
// This code may get re-run multiple times if there are conflicts.
return transaction.get(userRef).then(userDoc => {
if (!userDoc.exists) {
throw "Document does not exist!";
}
const new_balance = change.after.data().balance;
const old_balance = change.before.data().balance;
var user_balance = userDoc.data().balance + new_balance - old_balance;
transaction.update(userRef, {balance: user_balance});
});
}).catch(error => {
console.log("Transaction failed: ", error);
return null;
});
});
In addition to what #Renaud Tarnec covered in their answer, you may also want to consider the following approaches:
Batched Write
In your updateAccounts function, you are writing many pieces of data at once, if any one of these fail, you may end up with a database that contains a mix of correctly updated data and data that had failed to be updated.
To solve this, you can use a batched write to write the data atomically where all new data is updated successfully or none of your data is written leaving your database in a known state.
exports.updateAccounts = functions.https.onRequest((request, response) => {
const db = admin.firestore();
db.collection('accounts')
.get()
.then((qsAccounts) => { // qs -> QuerySnapshot
const batch = db.batch();
qsAccounts.forEach((accountSnap) => {
batch.update(accountSnap.ref, {balance: WHATEVER});
})
return batch.commit();
})
.then(() => response.send("Done"))
.catch((err) => {
console.log("Error whilst updating balances via HTTP Request:", err);
response.status(500).send("Error: " + err.message)
});
});
Splitting the counters
Instead of storing a single "balance" in your document, it may instead be desirable (based on what you are trying to do) to store each account's balance in the user's document.
"users/someUser": {
...,
"balances": {
"accountId1": 10,
"accountId4": -20,
"accountId23": 5
}
}
If you need the cumulative balance, just add them together on the client. If you need to remove a balance, simply delete it's entry in the user document.
exports.updateAccount = functions.firestore
.document('accounts/{accountID}')
.onUpdate((change, context) => {
const db = admin.firestore();
const accountID = context.params.accountID;
const newData = change.after.data();
const accountBalance = newData.balance;
const userID = newData.userID;
return db.doc("users/"+userID)
.get()
.then((userSnap) => {
return db.doc("users/"+userID).update({["balances." + accountID]: accountBalance});
})
.then(() => console.log(`Successfully updated account #${accountID} balance for user #${userID}`))
.catch((err) => {
console.log(`Error whilst updating account #${accountID} balance for user #${userID}`, err);
throw err;
});
});
Trying to read a pushToken from a given user in the users collection (after an update operation on another collection) returns undefined
exports.addDenuncia = functions.firestore
.document('Denuncias/{denunciaID}')
.onWrite((snap, context) => {
const doc = snap.after.data()
const classificadoId = doc.cid
const idTo = doc.peerId
db.collection('Classificados').doc(classificadoId)
.update({
aprovado: false
})
.then(r => {
getToken(idTo).then(token => {
// sendMsg...
})
}).catch(updateErr => {
console.log("updateErr: " + updateErr)
})
async function getToken(id) {
let response = "getTokenResponse"
console.log("id in getToken: " + id)
return db.collection('users').doc(id).get()
.then(user => {
console.log("user in getToken: " + user.data())
response = user.data().pushToken
})
.catch(e => {
console.log("error get userToken: " + e)
response = e
});
return response
}
return null
});
And this is from the FB console log:
-1:43:33.906 AM Function execution started
-1:43:36.799 AM Function execution took 2894 ms, finished with status: 'ok'
-1:43:43.797 AM id in getToken: Fm1RwJaVfmZoSgNEFHq4sbBgoEh1
-1:43:49.196 AM user in getToken: undefined
-1:43:49.196 AM error get userToken: TypeError: Cannot read property 'pushToken' of undefined
-1:43:49.196 AM returned token: undefined
And we can see in this screenshot from the db that the doc does exist:
Hope someone can point me to what I'm doing wrong here.
added screenshot of second example of #Renaud as deployed:
As Doug wrote in his comment, you need to "return a promise from the top level function that resolves when all the async work is complete". He also explains that very well in the official video series: https://firebase.google.com/docs/functions/video-series/ (in particular the 3 videos titled "Learn JavaScript Promises"). You should definitely watch them, highly recommended!
So, the following modifications to your code should work (untested):
exports.addDenuncia = functions.firestore
.document('Denuncias/{denunciaID}')
.onWrite(async (snap, context) => { // <- note the async keyword
try {
const doc = snap.after.data()
const classificadoId = doc.cid
const idTo = doc.peerId
await db.collection('Classificados').doc(classificadoId)
.update({
aprovado: false
});
const userToSnapshot = await db.collection('users').doc(idTo).get();
const token = userToSnapshot.data().pushToken;
await sendMsg(token); // <- Here you should take extra care to correctly deal with the asynchronous character of the sendMsg operation
return null; // <-- This return is key, in order to indicate to the Cloud Function platform that all the asynchronous work is done
} catch (error) {
console.log(error);
return null;
}
});
Since you use an async function in your code, I've used the async/await syntax but we could very well write it by chaining the promises with the then() method, as shown below.
Also, I am not sure, in your case, that it adds any value to put the code that gets the token in a function (unless you want to call it from other Cloud Functions but then you should move it out of the addDenuncia Cloud Function). That's why it has been replaced by two lines of code within the main try block.
Version with chaining promises via the then() method
In this version we chain the different promises returned by the asynchronous methods with the then() method. Compared to the async/await version above, it shows very clearly what means "to return a promise from the top level function that resolves when all the asynchronous work is complete".
exports.addDenuncia = functions.firestore
.document('Denuncias/{denunciaID}')
.onWrite((snap, context) => { // <- no more async keyword
const doc = snap.after.data()
const classificadoId = doc.cid
const idTo = doc.peerId
return db.collection('Classificados').doc(classificadoId) // <- we return a promise from the top level function
.update({
aprovado: false
})
.then(() => {
return db.collection('users').doc(idTo).get();
})
.then(userToSnapshot => {
if {!userToSnapshot.exists) {
throw new Error('No document for the idTo user');
}
const token = userToSnapshot.data().pushToken;
return sendMsg(token); // Again, here we make the assumption that sendMsg is an asynchronous function
})
.catch(error => {
console.log(error);
return null;
})
});
I have an app that loads some tricks. In the first iteration (next, next, play) the subscription works properly. In the second iteration (end, next, next, play) it doesn't load the tricks anymore.
The Observable is correct, it is from firebase that the tricks don't provide.
Here is some code:
console.log('did enter');
this.startDate = Date.now();
this.params = this.paramsService.get();
console.log(this.params);
console.log(this.trickService.getTricks());
var myTricks = this.trickService.getTricks()
myTricks.subscribe(tricks =>{
console.log(tricks);
this.tricksCollection = db.collection<Trick>('tricks');
this.tricks = this.tricksCollection.snapshotChanges().pipe(
map(actions => {
return actions.map(a => {
const data = a.payload.doc.data();
const id = a.payload.doc.id;
return { id, ...data };
});
})
);
}
getTricks() {
console.log(this.tricks);
return this.tricks;
}```
Anyone got a clue, I would appreciate,
Alex
as far as I know, background trigger cloud function should return a promise,right? but what if I want to do nothing in the one of my execution path ?
export const updateDataWhenUserUnattendTheEvent = functions.firestore
.document('events/{eventId}/Attendee/{userId}')
.onDelete((snap, context) => {
const eventID = context.params.eventId
const eventRef = snap.ref.firestore.collection('events').doc(eventID)
const db = admin.firestore()
return db.runTransaction(async t => {
const doc = await t.get(eventRef)
if (doc) {
const eventRankPoint = doc.data().rankPoint
let eventCapacity = doc.data().capacity
return t.update(eventRef,{
isFullCapacity : false,
capacity : eventCapacity + 1,
rankPoint: eventRankPoint - 1
})
} else {
// what should I write in here? empty promise?
return new Promise()
}
})
})
I want to my function worked only if the document is exist. so what should I do ? I write new Promise but .... I don't know what to do actually. thanks in advance
You can just return null if there's no asynchronous work to perform in some code path of your functions. You only truly need a promise if it tracks some async work.
Alternatively, you could return a promise that's resolved immediately with Promise.resolve(null)
Because db.runTransaction is an async function it will return a Promise all the time.
You can drop the else statement and the method will perform as expected because runTransaction will return Promise<void> which is a valid response for Cloud Functions
export const updateDataWhenUserUnattendTheEvent = functions.firestore
.document('events/{eventId}/Attendee/{userId}')
.onDelete((snap, context) => {
const eventID = context.params.eventId;
const eventRef = snap.ref.firestore.collection('events').doc(eventID);
const db = admin.firestore();
return db.runTransaction(async t => {
const doc = await t.get(eventRef);
if (doc) {
const eventRankPoint = doc.data().rankPoint;
let eventCapacity = doc.data().capacity ;
return t.update(eventRef,{
isFullCapacity : false,
capacity : eventCapacity + 1,
rankPoint: eventRankPoint - 1
});
}
});
});
You can also make the onDelete function async which means you can force it to always return a Promise - the below is valid and will exit the function correctly.
export const updateDataWhenUserUnattendTheEvent = functions.firestore
.document('events/{eventId}/Attendee/{userId}')
.onDelete(async (snap, context) => {
// Do Nothing
return;
});
Currently, I am running two cloud functions. One adds information to posts while another deletes old posts, as such :
exports.reveal = functions.database.ref('/reveals/{postIDthatWasRevealed}/revealed').onUpdate((change, context) => {
const revealedValue = change.after.val()
if (revealedValue === true) {
var updates = {}
const postID = context.params.postIDthatWasRevealed
console.log(postID)
return admin.firestore().collection('posters').doc(postID).get().then(snapshot => {
const value = snapshot.data()
console.log(value)
// console.log(value)
const posterID = value.posterID
const posterName = value.posterName
const profileImage = value.profileImage
const postKey = value.key
return admin.database().ref('/convoID/' + postID).once('value', (snapshot) => {
if (snapshot.exists()) {
const convoIDCollection = snapshot.val()
for (var child in convoIDCollection) {
const convoID = child
updates["/conversations/"+convoID+"/information/reciever/Name"] = posterName
updates["/conversations/"+convoID+"/information/reciever/profileImage"] = profileImage
updates["/conversations/"+convoID+"/key"] = postKey
}
}
const currentTime = Date.now()
const addedTime = currentTime + 172800000
const batch = admin.firestore().batch()
const postFireStoreRef = admin.firestore().collection('posts').doc(postID)
batch.update(postFireStoreRef,{"revealedDate": currentTime})
batch.update(postFireStoreRef,{"timeOfDeletion": addedTime})
batch.update(postFireStoreRef,{"information": {"posterID":posterID,"posterName":posterName,"profileImage":profileImage} })
batch.update(postFireStoreRef,{"key":postKey})
return batch.commit(), admin.database().ref().update(updates)
})
})
}
else {
return null
}
})
^That post adds information to a post when it gets enough likes. The posts are stored in firestore, and when a firebase node associated with the post gets enough likes, some information will be downloaded appended to the firestore post entity. In theory, it should not even take one read as the data from the firestore entity is never downloaded, merely modified. The second function running is as follows :
exports.hourly_job = functions.pubsub.topic('hourly-tick').onPublish((change,context) => {
const currentTime = Date.now()
const getPostsForDate = admin.firestore().collection('posts').where('timeOfDeletion', '<', currentTime)
return getPostsForDate.get().then(snapshot => {
const updates = {}
const batch = admin.firestore().batch()
snapshot.forEach((doc) => {
var key = doc.id
console.log(key)
const convos = database().ref('/convoID/' + key).once('value', (snapshot) => {
if (snapshot.exists){
for (var child in snapshot) {
const convoID = child
console.log(child+"shit")
updates["conversations/" + value] = null
}
}
})
updates["/convoID/"+ key] = null
updates["/reveals/" + key] = null
updates["/postDetails/" + key] = null
const postFireStoreRef = admin.firestore().collection('posts').doc(key)
const posterRef = admin.firestore().collection('posters').doc(key)
batch.delete(postFireStoreRef)
batch.delete(posterRef)
})
return admin.database().ref().update(updates), batch.commit()
})
})
Each minute, this queriers firestore for old posts. At most, it may return two to three posts, leading to a few reads. However, after testing these functions out for an hour, the Google App Engine Quota shows Ten Thousand Reads while I was expecting close to twenty to fifty. Additionally, the entire day the functions had only been deployed such that they ran only 87 times. Are these functions not optimized? Is there a way to monitor where the read operations are coming from?
Edit : It seems that each time I am triggering the deletion function (actually changing the timestamp of a post such that it will be deleted when the. deletion function is called every minute) my reads increase by a couple of hundred...