A firebase cloud function has been implemented to be triggered every time the value of price changes, the code is as follows:
export const orderListener = functions.firestore
.document("/users/{userId}/Order/{orderId}")
.onUpdate(async (change, context) => {
const { userId, orderId } = context.params;
const price = change.after.data().price;
try {
if (price !== 0) {
const userDoc = await admin
.firestore()
.collection(`/users/${userId}/Personal information`)
.doc("Personal Information")
.get();
const { fcmToken } = userDoc.data()!;
functions.logger.log({
userId,
fcmToken,
orderId,
price,
});
//RETURN
return admin.messaging().sendToDevice(fcmToken, {
notification: {
title: "Title of the message",
body: `${orderId}'s price > 0`,
},
});
} else {
return null
}
} catch (e) {
functions.logger.error(e.message);
//RETURN
return null
This code has been deployed in the firebase function, and I tried to test it by changing the value of price, and monitoring execution by looking at firebase console.
Expectation:
As soon as I change the value of price in my firestore database, it executes almost immediately (within 5s).
Result:
The function executes just fine and was logged on the firebase console, but there is a 50 minutes delay in execution after I updated the price value (Function execution was only logged 50mins later). It was tested by someone else but does not seem to have this issue.
Question:
Any potential error contained within this code? Is it likely to be some error with the firebase setup, settings, etc...?
Edit: Thanks to the contribution of Dharmaraj I have updated the above code with return statements. The problem however still persists.
You should consider two things here:
1 - Cold Start of functions. If a functions is update or hasen't been used for a while it will kind of "shut down" or "go to sleep". When it gets triggered again it has a "cold start" that takes a while. It kind of boots the function from the "sleep"
2 - The loggs are not in real time. The loggs are not shown in realtime. They also have some lag and deppending if you are watching them on the Firebase Console or GCP console you may need to refresh them manually.
50 min is way to much for a cloud function. I belive that you maybe just haben't seen the log as you expected it. It also could be that you test device was offline while trying it and it synced later. Without more information of your code, device and how you tested it it's hard for us to tell what exactly it would be but the mentioned szenarios are the most likely ones.
Related
I'm looking for a way to prevent writing more than a given limit of documents to a (sub)collection in a given periode.
For example: Messenger A is not allowed to write more then 1000 Messages per 24 hours.
This should be done in the context of an Firebase Function API endpoint because it's called by third parties.
The endpoint
app.post('/message', async function(req:any, res:any) {
// get the messenger's API key
const key = req.query.key
// if no key was provided, return error
if(!key) {
res.status(400).send('Please provide a valid key')
return
}
// get the messenger by the provided key
const getMessengerResult = await admin.firestore().collection('messengers')
.where('key', '==', key).limit(1).get()
// if there is no result the messenger is not valid, return error
if (getMessengerResult.empty){
res.status(400).send('Please provide a valid key')
return
}
// get the messenger from the result
const messenger = getMessengerResult.docs[0]
// TODO: check if messenger hasn't reached limit of 1000 messages per 24 hours
// get message info from the body
const title:String = req.body.title
const body: String = req.body.body
// add message
await messenger.ref.collection('messages').add({
'title':title,
'body':body,
'timestamp': Timestamp.now()
})
// send response
res.status(201).send('The notification has been created');
})
One thing I've tried was the following piece of code in place of the TODO::
// get the limit message and validate its timestamp
const limitMessageResult = await messenger.ref.collection('messages')
.orderBy('timestamp',"desc").limit(1).offset(1000).get()
if(!limitMessageResult.empty) {
const limitMessage = limitMessageResult.docs[0]
const timestamp: Timestamp = limitMessage.data()['timestamp']
// create a date object for 24 hours ago
const 24HoursAgo = new Date()
24HoursAgo.setDate(24HoursAgo.getDate()-1)
if(24HoursAgo < timestamp.toDate()) {
res.status(405).send('You\'ve exceeded your messages limit, please try again later!')
return
}
}
This code works, but there is a big BUT. The offset does indeed skip the 1000 results, but Firebase will still charge you for it! So every time the messenger tries to add 1 message, 1000+ are read... and that's costly.
So I need a better (cheaper) way to do this.
One thing I've come up with, but haven't yet tried would be adding an index/counter field to a message that increases by 1 every message.
Then instead of doing:
const limitMessageResult = await messenger.ref.collection('messages')
.orderBy('timestamp',"desc").limit(1).offset(1000).get()
I could do something like:
const limitMessageResult = await messenger.ref.collection('messages')
.where('index','==', currentIndex-1000).limit(1).get()
But I was wondering if that would be a save way.
For example, what would happen if there are multiple request at the same time.
I would first need to get the current index from the last message and add the new message with index+1. But could two requests read, and thus write the same index? Or could this be handled with transactions?
Or is there a totally different way to solve my problem?
I have a strong aversion against using offset() in my server-side code, precisely because it makes it seem like it's skipping documents, where it's actually reading-and-discarding them.
The simplest way I can think of to implement your maximum-writes-per-day count is to keep a writes-per-day counter for each messenger, that you then update whenever they write a message.
For example, you could do the following whenever you write a message:
await messenger.ref.collection('messages').add({
'title':title,
'body':body,
'timestamp': Timestamp.now()
})
const today = new Date().toISOString().substring(0, 10); // e.g. "2020-04-11"
await messenger.ref.set({
[today]: admin.firestore.FieldValue.increment(1)
}, { merge: true })
So this adds an additional field to your messenger document for each day, whee it then keeps a count of the number of messages that messenger has written for that day.
You'd then use this count instead of your current limitMessageResult.
const messageCount = (await messenger.get()).data()[today] || 0;
if (messageCount < 1000) {
... post the message and increase the counter
}
else {
... reject the message, and return a message
}
Steps left to do:
You'll want to secure write access to the counter fields, as the messenger shouldn't be able to modify these on their own.
You may want to clean out older message counts periodically, if you're worried about the messenger's document becoming too big. I prefer to leave these types of counters, as they give an opportunity to provide some stats cheaply if needed.
I'm using firebase admin sdk in my cloud functions and I'm getting error randomly in some executions when trying to get a user by uid .
let userRecord = await admin.auth().getUser(userId);
The error details are:
{"error":{"code":400,"message":"TOO_MANY_ATTEMPTS_TRY_LATER",
"errors":[{ "message":"TOO_MANY_ATTEMPTS_TRY_LATER",
"domain":"global","reason":"invalid"}]
}
}
My cloud function executes on a real time database write and can be triggered for multiple users. In total I have 4 auth function calls in one execution first is the above one, second call is to again get user by uid or email, third call is generateEmailVerificationLink and the last call is generatePasswordResetLink.
I have checked the rate limits in documentation for auth but there is no mention of rate limit for these operation. Also the error TOO_MANY_ATTEMPTS_TRY_LATER was only mentioned in REST API for sign up with email password.
If this error is due to rate limit what should I change to prevent this error given these 4 calls are necessary for the operation needed on database write?.
EDIT:
I have identified the actual call which is throwing too many attempts error. The calls auth().generateEmailVerificationLink() and auth().generatePasswordResetLink() throw this error when called too many times.
I called these two in loop with 100 iterations and waited for the promises. The first executions finishes without any errors i.e. 200 requests. But starting second execution as soon as the first one ends will throw the error of too many attempts. So I think these two calls have limit. Now I'm trying to reduce these calls and reuse the link information. Other calls like getUserByEmail works fine.
let promises = [];
let auth = admin.auth();
let hrstart = process.hrtime()
for (let i = 0; i < 100; i++) {
promises.push(auth.getUserByEmail("user email"));
promises.push(auth.generateEmailVerificationLink("user email", {url: `https://app.firebaseapp.com/path`}));
promises.push(auth.generatePasswordResetLink("user email", {url: `https://app.firebaseapp.com/path`}));
}
Promise.all(promises)
.then(value => {
let hrend = process.hrtime(hrstart);
console.log(hrend);
// console.log(value)
});
The error was specifically in the operation auth.createEmailLink. This function has following limit: 20QPS/I.P address where QPS is (query per second). This limit can be increased by submitting the use case to Firebase.
I got this information from firebase support after submitting my issue.
Link to my github issue: https://github.com/firebase/firebase-admin-node/issues/458
I was way under 20QPS but was receiving this exception. In fact, it would always throw the TOO_MANY_ATTEMPTS_TRY_LATER exception on the 2nd attempt.
It turned out to be usage of FirebaseAuth.DefaultInstance instead of instantiating a static instance thusly:
In class definition:
private readonly FirebaseApp _firebase;
In class constructor:
_firebase = FirebaseAdmin.FirebaseApp.Create();
In function:
var auth = FirebaseAuth.GetAuth(_firebase);
var actionCodeSettings = new ActionCodeSettings()
{
...
};
var link = await auth.GenerateEmailVerificationLinkAsync(email, actionCodeSettings);
return link;
In addition to the answer mentioned in https://stackoverflow.com/a/54782967/5515861, I want to add another solution if you found this issue while trying to create custom email verification.
Inspired by the response in this GitHub isssue https://github.com/firebase/firebase-admin-node/issues/458#issuecomment-933161448 .
I am also seeing this issue. I have not ran admin.auth().generateEmailVerificationLink in over 24hrs (from anywhere else or any user at all) and called it just now only one time (while deployed in the prod functions environment) and got this 400 TOO_MANY_ATTEMPTS_TRY_LATER error ...
But, the client did also call the Firebase.auth.currentUser.sendEmailVerification() method around same time (obviously different IP).
Could that be the issue?
My solution to this issue is by adding a retry. e.g.
exports.sendWelcomeEmail = functions.runWith({failurePolicy: true}).auth.user().onCreate(async (user) => {
functions.logger.log("Running email...");
const email = user.email;
const displayName = user.displayName;
const link = await auth.generateEmailVerificationLink(email, {
url: 'https://mpj.io',
});
await sendWelcomeEmail(email, displayName, link);
});
The .runWith({failurePolicy: true}) is key.
It s giving you an error because your cloud functions/backend call the generateEmailVerificationLink while at the same time the default behaviour of the Firebase is also doing the same and it is counted as 20QPS. It some weird Google Rate Limit accounting rule. So my solution is just to add a retry.
The Downside is, it is calling twice, so if the call is billable, it might be billable twice.
I build react native app with firebase & firestore.
what I'm looking to do is, when user open app, to insert/update his status to 'online' (kind of presence system), when user close app, his status 'offline'.
I did it with firebase.database.onDisconnect(), it works fine.
this is the function
async signupAnonymous() {
const user = await firebase.auth().signInAnonymouslyAndRetrieveData();
this.uid = firebase.auth().currentUser.uid
this.userStatusDatabaseRef = firebase.database().ref(`UserStatus/${this.uid}`);
this.userStatusFirestoreRef = firebase.firestore().doc(`UserStatus/${this.uid}`);
firebase.database().ref('.info/connected').on('value', async connected => {
if (connected.val() === false) {
// this.userStatusFirestoreRef.set({ state: 'offline', last_changed: firebase.firestore.FieldValue.serverTimestamp()},{merge:true});
return;
}
await firebase.database().ref(`UserStatus/${this.uid}`).onDisconnect().set({ state: 'offline', last_changed: firebase.firestore.FieldValue.serverTimestamp() },{merge:true});
this.userStatusDatabaseRef.set({ state: 'online', last_changed: firebase.firestore.FieldValue.serverTimestamp() },{merge:true});
// this.userStatusFirestoreRef.set({ state: 'online',last_changed: firebase.firestore.FieldValue.serverTimestamp() },{merge:true});
});
}
after that, I did trigger to insert data into firestore(because I want to work with firestore), this is the function(works fine, BUT it takes 3-4 sec)
module.exports.onUserStatusChanged = functions.database
.ref('/UserStatus/{uid}').onUpdate((change,context) => {
const eventStatus = change.after.val();
const userStatusFirestoreRef = firestore.doc(`UserStatus/${context.params.uid}`);
return change.after.ref.once("value").then((statusSnapshot) => {
return statusSnapshot.val();
}).then((status) => {
console.log(status, eventStatus);
if (status.last_changed > eventStatus.last_changed) return status;
eventStatus.last_changed = new Date(eventStatus.last_changed);
//return userStatusFirestoreRef.set(eventStatus);
return userStatusFirestoreRef.set(eventStatus,{merge:true});
});
});
then after that, I want to calculate the online users in app, so I did trigger when write new data to node of firestore so it calculate the size of online users by query.(it works fine but takes 4-7 sec)
module.exports.countOnlineUsers = functions.firestore.document('/UserStatus/{uid}').onWrite((change,context) => {
console.log('userStatus')
const userOnlineCounterRef = firestore.doc('Counters/onlineUsersCounter');
const docRef = firestore.collection('UserStatus').where('state','==','online').get().then(e=>{
let count = e.size;
console.log('count',count)
return userOnlineCounterRef.update({count})
})
return Promise.resolve({success:'added'})
})
then into my react native app
I get the count of online users
this.unsubscribe = firebase.firestore().doc(`Counters/onlineUsersCounter`).onSnapshot(doc=>{
console.log('count',doc.data().count)
})
All the operations takes about 12 sec. it's too much for me, it's online app
my firebase structure
what I'm doing wrong? maybe there is unnecessary function or something?
My final goals:
minimize time operation.
get online users count (with listener-each
change, it will update in app)
update user status.
if there are other way to do that, I would love to know.
Cloud Functions go into a 'cold start' mode, where they take some time to boot up. This is the only reason I can think of that it would take that long. Stack Overflow: Firebase Cloud Functions Is Very Slow
But your cloud function only needs to write to Firestore on log out to
catch the case where your user closes the app. You can write to it directly on log in from your client
with auth().onAuthStateChange().
You could also just always read who is logged in or out directly from the
realtime database and use Firestore for the rest of your data.
You can rearrange your data so that instead of a 'UserStatus' collection you have an 'OnlineUsers' collection containing only online users, kept in sync by deleting the documents on log out. Then it won't take a query operation to get them. The query's impact on your performance is likely minimal, but this would perform better with a large number of users.
The documentation also has a guide that may be useful: Firebase Docs: Build Presence in Cloud Firestore
I want to check user's online status, in realtime database I used to check this with the help of onDisconnect(), but now I've shifted to firestore and can't find any similar method in that.
According to this onDisconnect:
The onDisconnect class is most commonly used to manage presence in applications where it is useful to detect how many clients are connected and when other clients disconnect.
To be able to use presence in firestore, you need to connect firestore with realtime firebase(no other way).
Please check this for more info:
https://firebase.google.com/docs/firestore/solutions/presence
NOTE: This solution is not especially efficient
Off the top of my head (read: I haven't thought through the caveats), you could do something like this:
const fiveMinutes = 300000 // five minutes, or whatever makes sense for your app
// "maintain connection"
setInterval(() => {
userPresenceDoc.set({ online: new Date().getTime() })
}, fiveMinutes)
then, on each client...
userPresenceDoc.onSnapshot(doc => {
const fiveMinutesAgo = new Date().getTime() - fiveMinutes
const isOnline = doc.data().online > fiveMinutesAgo
setUserPresence(isOnline)
})
You'd probably want the code checking for presence to use an interval a little more than the interval used by the code maintaining the connection to account for network lag, etc.
A NOTE ABOUT COST
So, obviously, there could be significant lag between when someone disconnects and when that disconnection is recognized by other clients. You can decrease that lag time by increasing the frequency of writes to Firestore, and thus increasing your costs. Running the numbers, I came out with the following costs for different intervals, assuming a single client connection running continuously for a month:
Interval Cost/User/Month
----------------------------
10m $0.007776
5m $0.015552
1m $0.07776
10s $0.46656
5s $0.93312
1s $4.6656
Going with an interval of one second is pretty pricy, at $46,656 a month for a system with 10,000 users who leave the app open all month long. An interval of 10 minutes with the same number of users would only cost $77.76 a month. A more reasonable interval of one minute, 10,000 users, and only four hours of app usage per day per user, rings in at $129.60 / month.
There is no equivalent. The Firestore SDK currently doesn't have presence management like the Realtime Database SDK.
Instead, you might want to use the Realtime Database onDisconnect() in conjunction with Cloud Functions to kick off some work when the client disconnects from RTDB. You would be assuming that that your app probably also lost its connection to Firestore at the same time.
Try this, but this method kind of hackish because we cannot use onDisconnected in firestore. As far as I know, realtime database use secure WebSocket technology, so thats why onDisconnected have on it
But you can use realtime database that can be implemented in cloud functions
to update the firestore data,
functions.database.ref('users/{userId}').onUpdate()
Somewhere in clientside:
firebase.database()
.ref('.info/connected')
.on('value', async (snap) => {
if (snap.val() === true) {
// Update the online status in RTDB
await firebase.database()
.ref(`users/${credentials.user.uid}/`)
.set({
online_status: true,
start_online: firebase.database.ServerValue.TIMESTAMP
});
// OnDisconnect
firebase.database()
.ref(`users/${credentials.user.uid}/`)
.onDisconnect()
.set({
online_status: false,
last_online: firebase.database.ServerValue.TIMESTAMP
});
}
});
in cloudfunctions (this will be trigger on update)
export const onUserOnlineStatusChanged = functions.database.ref('users/{userId}').onUpdate((event: functions.Change<functions.database.DataSnapshot>, context: functions.EventContext) => {
return event.after.ref.once('value')
.then((dataSnapshot) => dataSnapshot.val()) // Get the latest value from the Firebase Realtime database
.then((value: any) => {
// Update the value from RTDB to Firestore
console.log('value.online_status', value.online_status);
if (value.online_status == true) {
// Set the value to the firestore
admin.firestore()
.collection('users_info')
.doc(context.params.userId) // Get document by the userId / Or use .where
.set({
online_status: value.online_status,
updated_at: new Date
}, {
mergeFields: [
'online_status',
'updated_at'
]
});
// Add code if necessary (when the online_status is true)
} else if (value.online_status == false) {
// Set the value to the firestore
admin.firestore()
.collection('users_info')
.doc(context.params.userId) // Get document by the userId / Or use .where
.set({
online_status: value.online_status,
updated_at: new Date
}, {
mergeFields: [
'online_status',
'updated_at'
]
});
// Add code if necessary (when the online_status is false)
}
});
});
It would take about 1 or 2 seconds to update from cloud function to firestore
There is not a direct way to do this thing but this trick helped me achieve this disconnect listener.
window.addEventListener("beforeunload", async function (e) { e.preventDefault(); await firestoreRef.doc("doc-ref").update({ online: false }); });
I have a cloud function that is triggered when an account is created. This function populates a few fields in the user's Firebase Database entry. One of the fields is named 'accntActive', and triggers a second cloud function to do some other tasks.
In the second cloud function that gets triggered on a change of the accntActive field, the first thing I do is read the value of accntActive to check if it was changed to the value 'true'. What I am seeing is that about 50% of the time, the cloud function crashes when I try to use this value, saying it is undefined/null. Up until a week ago this always worked fine, 100% of the time, so the code itself has not changed. When I examine the user's DB entry at a later point in time, 100% of the time it has been populated correctly.
I am wondering if there is some race condition, where the DB cloud function is triggered before the value that got written is stable, and thus cannot be read immediately. This would seem a bit ridiculous, given the entire point of the DB cloud function is to respond to the value that is being changed.
The code for the second function:
exports.myActiveFxn = functions.database.ref('/users/{uid}/accntStatus/active')
.onWrite(event => {
const accntActive = event.data.val();
const userID = event.params.uid;
console.log(accntActive.toString()) //This throws an exception since accntActive is sometimes undefined
In the above, note that the userID variable is always stable. I've even tried re-reading the value once inside the function, and I get the same result:
admin.database().ref('/users/'+userID+'/accntStatus/active').once('value').then(function (snapshot) {
var accntActive = snapshot.val();
console.log(accntActive.toString());
}).then(function() {
Does anybody have any insight into what I can do to fix this? Thank you in advance!
Edit: Here is the code for the first function
exports.authUserCreated = functions.auth.user().onCreate(function(event) {
var UID = event.data.uid;
admin.database().ref('/users/'+UID).set({
email: event.data.email,
accntStatus: {
active: "true",
key: randomstring.generate(22)
},
uid: UID
});
return;
});
In your function, you aren't returning a promise that resolves when the work is complete. If you don't do that, strange and inconsistent things may happen. Please read the documentation on async programming with Cloud Functions.
You should return from the function the promise that's returned by the set() method on the database ref.
exports.authUserCreated = functions.auth.user().onCreate(function(event) {
var UID = event.data.uid;
return admin.database().ref('/users/'+UID).set({
email: event.data.email,
accntStatus: {
active: "true",
key: randomstring.generate(22)
},
uid: UID
});
});