My firebase collection that I'm trying to query is literally empty. Zero documents inside. Even then, when querying it, I'm getting this error
I'm doing this inside a LAMBDA function on AWS. I know that the query works because it does return results sometimes but it's very random. Mostly it's just coming up with this error
Here are my lambda logs with following ENV variables turned on
GRPC_TRACE=all
GRPC_VERBOSITY=DEBUG
I have even tried this as I found this online somewhere but it didn't make any difference
db.settings({
clientConfig: {
interfaces: {
'google.firestore.v1.Firestore': {
methods: {
RunQuery: {
timeout_millis: 5 * 60 * 1000
}
}
}
}
}
});
Here's what my query code looks like
let snap = await db.collection('notifications').where("siteID", "==", msg.siteId).where("procCode", "==", code).where("aptNum", "==", msg.affectedRows[0].after.AptNum).get();
Here's the output I get eventually. Not even in the same lambda execution but in a separate one which is also strange.
I figured out what my issue was here. I was calling the Firebase API inside an async function but the caller of that function wasn't "await"ing that call which led to this. It all works now.
Related
A firebase cloud function has been implemented to be triggered every time the value of price changes, the code is as follows:
export const orderListener = functions.firestore
.document("/users/{userId}/Order/{orderId}")
.onUpdate(async (change, context) => {
const { userId, orderId } = context.params;
const price = change.after.data().price;
try {
if (price !== 0) {
const userDoc = await admin
.firestore()
.collection(`/users/${userId}/Personal information`)
.doc("Personal Information")
.get();
const { fcmToken } = userDoc.data()!;
functions.logger.log({
userId,
fcmToken,
orderId,
price,
});
//RETURN
return admin.messaging().sendToDevice(fcmToken, {
notification: {
title: "Title of the message",
body: `${orderId}'s price > 0`,
},
});
} else {
return null
}
} catch (e) {
functions.logger.error(e.message);
//RETURN
return null
This code has been deployed in the firebase function, and I tried to test it by changing the value of price, and monitoring execution by looking at firebase console.
Expectation:
As soon as I change the value of price in my firestore database, it executes almost immediately (within 5s).
Result:
The function executes just fine and was logged on the firebase console, but there is a 50 minutes delay in execution after I updated the price value (Function execution was only logged 50mins later). It was tested by someone else but does not seem to have this issue.
Question:
Any potential error contained within this code? Is it likely to be some error with the firebase setup, settings, etc...?
Edit: Thanks to the contribution of Dharmaraj I have updated the above code with return statements. The problem however still persists.
You should consider two things here:
1 - Cold Start of functions. If a functions is update or hasen't been used for a while it will kind of "shut down" or "go to sleep". When it gets triggered again it has a "cold start" that takes a while. It kind of boots the function from the "sleep"
2 - The loggs are not in real time. The loggs are not shown in realtime. They also have some lag and deppending if you are watching them on the Firebase Console or GCP console you may need to refresh them manually.
50 min is way to much for a cloud function. I belive that you maybe just haben't seen the log as you expected it. It also could be that you test device was offline while trying it and it synced later. Without more information of your code, device and how you tested it it's hard for us to tell what exactly it would be but the mentioned szenarios are the most likely ones.
I'm using firebase admin sdk in my cloud functions and I'm getting error randomly in some executions when trying to get a user by uid .
let userRecord = await admin.auth().getUser(userId);
The error details are:
{"error":{"code":400,"message":"TOO_MANY_ATTEMPTS_TRY_LATER",
"errors":[{ "message":"TOO_MANY_ATTEMPTS_TRY_LATER",
"domain":"global","reason":"invalid"}]
}
}
My cloud function executes on a real time database write and can be triggered for multiple users. In total I have 4 auth function calls in one execution first is the above one, second call is to again get user by uid or email, third call is generateEmailVerificationLink and the last call is generatePasswordResetLink.
I have checked the rate limits in documentation for auth but there is no mention of rate limit for these operation. Also the error TOO_MANY_ATTEMPTS_TRY_LATER was only mentioned in REST API for sign up with email password.
If this error is due to rate limit what should I change to prevent this error given these 4 calls are necessary for the operation needed on database write?.
EDIT:
I have identified the actual call which is throwing too many attempts error. The calls auth().generateEmailVerificationLink() and auth().generatePasswordResetLink() throw this error when called too many times.
I called these two in loop with 100 iterations and waited for the promises. The first executions finishes without any errors i.e. 200 requests. But starting second execution as soon as the first one ends will throw the error of too many attempts. So I think these two calls have limit. Now I'm trying to reduce these calls and reuse the link information. Other calls like getUserByEmail works fine.
let promises = [];
let auth = admin.auth();
let hrstart = process.hrtime()
for (let i = 0; i < 100; i++) {
promises.push(auth.getUserByEmail("user email"));
promises.push(auth.generateEmailVerificationLink("user email", {url: `https://app.firebaseapp.com/path`}));
promises.push(auth.generatePasswordResetLink("user email", {url: `https://app.firebaseapp.com/path`}));
}
Promise.all(promises)
.then(value => {
let hrend = process.hrtime(hrstart);
console.log(hrend);
// console.log(value)
});
The error was specifically in the operation auth.createEmailLink. This function has following limit: 20QPS/I.P address where QPS is (query per second). This limit can be increased by submitting the use case to Firebase.
I got this information from firebase support after submitting my issue.
Link to my github issue: https://github.com/firebase/firebase-admin-node/issues/458
I was way under 20QPS but was receiving this exception. In fact, it would always throw the TOO_MANY_ATTEMPTS_TRY_LATER exception on the 2nd attempt.
It turned out to be usage of FirebaseAuth.DefaultInstance instead of instantiating a static instance thusly:
In class definition:
private readonly FirebaseApp _firebase;
In class constructor:
_firebase = FirebaseAdmin.FirebaseApp.Create();
In function:
var auth = FirebaseAuth.GetAuth(_firebase);
var actionCodeSettings = new ActionCodeSettings()
{
...
};
var link = await auth.GenerateEmailVerificationLinkAsync(email, actionCodeSettings);
return link;
In addition to the answer mentioned in https://stackoverflow.com/a/54782967/5515861, I want to add another solution if you found this issue while trying to create custom email verification.
Inspired by the response in this GitHub isssue https://github.com/firebase/firebase-admin-node/issues/458#issuecomment-933161448 .
I am also seeing this issue. I have not ran admin.auth().generateEmailVerificationLink in over 24hrs (from anywhere else or any user at all) and called it just now only one time (while deployed in the prod functions environment) and got this 400 TOO_MANY_ATTEMPTS_TRY_LATER error ...
But, the client did also call the Firebase.auth.currentUser.sendEmailVerification() method around same time (obviously different IP).
Could that be the issue?
My solution to this issue is by adding a retry. e.g.
exports.sendWelcomeEmail = functions.runWith({failurePolicy: true}).auth.user().onCreate(async (user) => {
functions.logger.log("Running email...");
const email = user.email;
const displayName = user.displayName;
const link = await auth.generateEmailVerificationLink(email, {
url: 'https://mpj.io',
});
await sendWelcomeEmail(email, displayName, link);
});
The .runWith({failurePolicy: true}) is key.
It s giving you an error because your cloud functions/backend call the generateEmailVerificationLink while at the same time the default behaviour of the Firebase is also doing the same and it is counted as 20QPS. It some weird Google Rate Limit accounting rule. So my solution is just to add a retry.
The Downside is, it is calling twice, so if the call is billable, it might be billable twice.
So I have a cloud function that is triggered each time a transaction is liked/unliked. This function increments/decrements the likesCount. I've used firestore transactions to achieve the same. I think the problem is the Code inside the Transaction block is getting executed multiple times, which may be correct as per the documentation.
But my Likes count are being updated incorrectly at certain times.
return firestore.runTransaction(function (transaction) {
return transaction.get(transRef).then(function (transDoc) {
let currentLikesCount = transDoc.get("likesCount");
if (event.data && !event.data.previous) {
newLikesCount = currentLikesCount == 0 || isNaN(currentLikesCount) ? 1 : transDoc.get("likesCount") + 1;
} else {
newLikesCount = currentLikesCount == 0 || isNaN(currentLikesCount) ? 0 : transDoc.get("likesCount") - 1;
}
transaction.update(transRef, { likesCount: newLikesCount });
});
});
Anyone had similar experience
Guys finally found out the cause for this unexpected behaviour.
Firestore isn't suitable for maintaining counters if your application is going to be traffic intensive. They have mentioned it in their documentation. The solution they suggest is to use a Distributed counter.
Many realtime apps have documents that act as counters. For example,
you might count 'likes' on a post, or 'favorites' of a specific item.
In Cloud Firestore, you can only update a single document about once
per second, which might be too low for some high-traffic applications.
https://cloud.google.com/firestore/docs/solutions/counters
I wasn't convinced with that approach as it's too complex for a simple use case, which is when I stumbled across the following blog
https://medium.com/evenbit/on-collision-course-with-cloud-firestore-7af26242bc2d
These guys used a combination of Firestore + Firebase thereby eliminating their weaknesses.
Cloud Firestore is sitting conveniently close to the Firebase Realtime
Database, and the two are easily available to use, mix and match
within an application. You can freely choose to store data in both
places for your project, if that serves your needs.
So, why not use the Realtime database for one of its strengths: to
manage fast data streams from distributed clients. Which is the one
problem that arises when trying to aggregate and count data in the
Firestore.
Its not correct to say that Firestore is an upgrade to the Realtime database (as it is advertised) but a different database with different purposes and both can and should coexist in a large scale application. That's my thought.
It might have something to do with what you're returning from the function, as you have
return transaction.get(transRef).then(function (transDoc) { ... })
And then another return inside that callback, but no return inside the inner-most nested callback. So it might not be executing the transaction.update. Try removing the first two return keywords and add one before transaction.update:
firestore.runTransaction(function (transaction) {
transaction.get(transRef).then(function (transDoc) {
let currentLikesCount = transDoc.get("likesCount");
if (event.data && !event.data.previous) {
newLikesCount = currentLikesCount == 0 || isNaN(currentLikesCount) ? 1 : transDoc.get("likesCount") + 1;
} else {
newLikesCount = currentLikesCount == 0 || isNaN(currentLikesCount) ? 0 : transDoc.get("likesCount") - 1;
}
return transaction.update(transRef, { likesCount: newLikesCount });
});
});
Timeouts
First of all, check your Cloud Functions logs to see if you get any timeout messages.
Function execution took 60087 ms, finished with status: 'timeout'
If so, sort out your function so that it returns a Promise.resolve(). And shows
Function execution took 344 ms, finished with status: 'ok'
Idempotency
Secondly, write your data so that the function is idempotent. When your function runs, write a value to the document that you are reading. You can then check if that value exists before running the function again.
See this example for ensuring that functions are only run once.
I have an issue with my cloud functions where in all my database events all return empty. For example, in the following event the event.data.val() would return null. I am doing an update operation and have tested the update by testing the cloud function using the shell as well as after deploying.
export const createSubscription = functions.database.ref('/users/{userId}/subscription').onWrite( event => {
if(!event.data.val()) {
return;
}
});
But I can easily hook into the auth.user() events like the following and receive the data.
export const createStripeUser = functions.auth.user().onCreate(event => {
const user = event.data;
});
Edit: Passing data into the collection for example like the one below on the emulator console
createSubscription({
testKey: 'testValue'
})
or the following on from my frontend
db.ref(`/users/23213213213/subscription`).update({ testKey: 'testValue'});
would return null on the function.
DougStevenson is correct. For the .onCreate() you would be doing it correct wit a myDatabaseFunction('new_data').
With the .onWrite() you need to pass in the before and after like my example below.
You may have got stuck were I did. Note that I have a curly bracket around the before and after the final JSON. It didn't work properly without them.
addComment({before: {"comment":"before comment","role":"guest"}, after:{"comment":"After Comment","role":"guest"}})
Hope my example helps a bit more than a generic string parameter.
Good Luck!
I use Cloud Functions for Firebase for some server-side-tasks. Using the database-trigger onWrite() I experience some unexpected behaviour.
exports.doStuff = functions.database.ref('/topic/{topicId}/new').onWrite((event) => {
// If data, then continue...
if (event.data.val()){
// doStuff
//
} else {
console.log("started, but no content!");
}
When new data is added to the specified folder the function is started at least once without any new content ("started, but no content!" is logged to the console). Sometimes even two, three or four times. Then it's run again, automatically (a couple of seconds later) and this time everything works as expected.
EDIT:
The code that writes to the specified node is as follows:
functionA(topicId){
return this.db.object('/topic/'+topicId+'/new').update({
timestamp: firebase.database.ServerValue.TIMESTAMP
});
}
The timestamp is only set once. So before that operation is called, the node new does not exist. So there is no edit or delete. However, this means, by calling the above function first the node new is created, some miliseconds later timestamp and then the value for timestamp. Does Firebase call the onWrite() function for each of these events?
Does this make sense to anybody? Any idea how to make sure, that the function is only executed, when there is really new data available?
onWrite() has a new format, since version 1.0 of Firebase SDK
Your original code was based on tutorials or help from an earlier, beta, version of Firebase.
exports.doStuff = functions.database.ref('/topic/{topicId}/new')
.onWrite((event) => {
if (event.data.val()){
// do stuff
}
}
However since version 1.0, the first parameter in the onWrite function is a "change" object which has two properties: before and after. You want the after. You can get it as follows:
exports.doStuff = functions.database.ref('/topic/{topicId}/new')
.onWrite((change) => {
if (change.after.val()){
// do stuff
}
}
Source: https://firebase.google.com/docs/reference/functions/functions.Change