I have a Realtime DB in Firebase and have setup an agent in Google Cloud's Dialogflow. This agent agent is fetching data about bus route names. The end user is asked for a bus number and the agent should get relevant info based on that route number. I can call the database but only for a set bus number.
So for example below I can pull in bus info for 100 based on having the snapshot.child set to 100. But I want the snapshot.child to change based on the askBus parameter from Dialogflow. Any suggestions?
function handleBus(agent) {
const bus = agent.parameters.bus;
agent.add(`Thank you...`);
return admin.database().ref('Routes').once("value").then((snapshot) => {
var routeInfo = snapshot.child('100/route_desc').val();
var routeName = snapshot.child('100/route_long_name').val();
agent.add(`Bus info is ` + routeInfo + ' and is called ' + routeName);
In general, the best way to handle this is to reference the node of the bus number as part of setting up the path to the query. Getting it once you have the result is certainly possible, but means you're pulling in a lot more data than you need to for each query.
But there are a few ways to do this.
The one most similar to how you're doing it now is to generate a string that includes the route number. This example shows how to do it using a back-quote, which is available in the most recent JavaScript, or you can just do string concatenation:
function handleBus(agent) {
const bus = agent.parameters.bus;
agent.add(`Thank you...`);
return admin.database().ref('Routes').once("value").then((snapshot) => {
var routeInfo = snapshot.child(`${bus}/route_desc`).val();
var routeName = snapshot.child(`${bus}/route_long_name`).val();
agent.add(`Bus info is ` + routeInfo + ' and is called ' + routeName);
But if you're just looking for the information from that route, you can setup the reference to the database to include the route, get the entire result and its value, and then treat this as a JavaScript object.
function handleBus(agent) {
const bus = agent.parameters.bus;
agent.add(`Thank you...`);
return admin.database().ref('Routes').child(bus).once("value").then((snapshot) => {
var route = snapshot.val();
var routeInfo = route['route_desc'];
var routeName = route['route_long_name'];
agent.add(`Bus info is ` + routeInfo + ' and is called ' + routeName);
As an aside, I want to point out that you're using Promises perfectly. That is a trap many people fall into, and you've done a good job querying the value through a Promise, handling it as part of Promise fulfillment, and returning a Promise in your handler.
In the webhook use async call to firebase to fetch the bus information.
Fetch the parameter value.
Access Firebase DB.
Fetch information based on parameter using async call.
Use promise to reply back with the correct response. See this for responding via promise.
Promise would be used inside your Firebase function when it fetches the DB information.
Related
I'm currently developing an app using Firebase.
My Firestore Database looks like below:
Once the user passes the Firebase authentication procedure, I'm creating a user document with a field "Phone:" which contains his phone number. Basically, everyone who is gonna using the app will be listed in the database.
And here is my challenge:
I'm using the plugin easy_contact_picker to store all the contacts of the users device to a List.
How can I find out whether the users contacts are using the app or whether they are listed in the database?
My goal is create a contact List Widget which shows me all my contacts. But those contacts which are using the app or which are listed in the database, should be highlighted or marked particularly.
Which is the best way to realize that if we consider that millions of users (to minimize computing power)are listed in the database?
Anyone has an idea?
Thanks a lot
First of all try to awoid giving everyone access to read all users. That is something most ppl do when handling such a problem. The do it because the query over all users won't work if you don't give the rights to read all of them.
Because of security reasons I would move the logic for checking if a user exists into callable function (not a http function!). That way you can call it inside of your app and check for a single user or multiple of them in an array. That would depend how your frontend would handle it.
Very importand would be to store all phone numbers in the absolute same format. That way you could query for them. Regardless of the number of users you could always find a specific one like here:
var citiesRef = db.collection("users");
var query = citiesRef.where("Phone", "==", "+4912345679");
The numbers need to be absolutely the same without any emtpy spaces - chars and the +49 or 0049 also needs to be the same.
You could create two callable funcitons. One to check if a single user exists in your app and another where you send an array of phone numbers and you get an array back. The cloud function can use Promise.all to performe such queries in parallel so you get your responce quite fast.
I'm using a similar approach to add users in my app as admins to specific groups where you just enter the email of the user and if he is in the app he will be added. I not he get's an invitation on the email to join the App.
With the help of Tarik's answer, Ayrix and I came up with the following solution.
Important: Read Tarik's answer for more information.
Client: callable_compare_contacts.dart
import 'package:cloud_functions/cloud_functions.dart';
Future<List<Object>> getMembersByPhoneNumber(List<String> allPhoneNumbers) async {
HttpsCallable callable = FirebaseFunctions.instance.httpsCallable('membersByPhoneNumber');
final results = await callable.call(<String, dynamic>{'allPhoneNumbers': allPhoneNumbers});
return results.data;
}
Server: index.js
const functions = require("firebase-functions");
const admin = require("firebase-admin");
if (admin.apps.length === 0) {
admin.initializeApp({
credential: admin.credential.applicationDefault(),
});
}
exports.membersByPhoneNumber = functions.https.onCall((data, context) => {
return new Promise((resolve, reject) => {
if (!data || !data.allPhoneNumbers.length) return resolve([]);
const phoneNumbers = data.allPhoneNumbers;
// TODO: different scope? move vars for future use
const db = admin.firestore();
const collectionRef = db.collection("User");
let batches = [];
// because of wrong eslint parsing (dirty)
batches = [];
while (phoneNumbers.length) {
// firestore limits batches to 10
const batch = phoneNumbers.splice(0, 10);
// add the batch request to to a queue
batches.push(
new Promise((response) => {
collectionRef.where("Phone", "in", [...batch]).get()
.then((results) =>
response(results.docs.map(function(result) {
return result.data().Phone;
} )));
})
);
}
// response / return to client
Promise.all(batches).then(function(content) {
// console.log("content.flat()");
// console.log(content.flat());
return resolve(content.flat());
});
});
});
Note: This is our first callable/cloud function .. so Suggestions for changes are welcome.
I'm looking for a way to prevent writing more than a given limit of documents to a (sub)collection in a given periode.
For example: Messenger A is not allowed to write more then 1000 Messages per 24 hours.
This should be done in the context of an Firebase Function API endpoint because it's called by third parties.
The endpoint
app.post('/message', async function(req:any, res:any) {
// get the messenger's API key
const key = req.query.key
// if no key was provided, return error
if(!key) {
res.status(400).send('Please provide a valid key')
return
}
// get the messenger by the provided key
const getMessengerResult = await admin.firestore().collection('messengers')
.where('key', '==', key).limit(1).get()
// if there is no result the messenger is not valid, return error
if (getMessengerResult.empty){
res.status(400).send('Please provide a valid key')
return
}
// get the messenger from the result
const messenger = getMessengerResult.docs[0]
// TODO: check if messenger hasn't reached limit of 1000 messages per 24 hours
// get message info from the body
const title:String = req.body.title
const body: String = req.body.body
// add message
await messenger.ref.collection('messages').add({
'title':title,
'body':body,
'timestamp': Timestamp.now()
})
// send response
res.status(201).send('The notification has been created');
})
One thing I've tried was the following piece of code in place of the TODO::
// get the limit message and validate its timestamp
const limitMessageResult = await messenger.ref.collection('messages')
.orderBy('timestamp',"desc").limit(1).offset(1000).get()
if(!limitMessageResult.empty) {
const limitMessage = limitMessageResult.docs[0]
const timestamp: Timestamp = limitMessage.data()['timestamp']
// create a date object for 24 hours ago
const 24HoursAgo = new Date()
24HoursAgo.setDate(24HoursAgo.getDate()-1)
if(24HoursAgo < timestamp.toDate()) {
res.status(405).send('You\'ve exceeded your messages limit, please try again later!')
return
}
}
This code works, but there is a big BUT. The offset does indeed skip the 1000 results, but Firebase will still charge you for it! So every time the messenger tries to add 1 message, 1000+ are read... and that's costly.
So I need a better (cheaper) way to do this.
One thing I've come up with, but haven't yet tried would be adding an index/counter field to a message that increases by 1 every message.
Then instead of doing:
const limitMessageResult = await messenger.ref.collection('messages')
.orderBy('timestamp',"desc").limit(1).offset(1000).get()
I could do something like:
const limitMessageResult = await messenger.ref.collection('messages')
.where('index','==', currentIndex-1000).limit(1).get()
But I was wondering if that would be a save way.
For example, what would happen if there are multiple request at the same time.
I would first need to get the current index from the last message and add the new message with index+1. But could two requests read, and thus write the same index? Or could this be handled with transactions?
Or is there a totally different way to solve my problem?
I have a strong aversion against using offset() in my server-side code, precisely because it makes it seem like it's skipping documents, where it's actually reading-and-discarding them.
The simplest way I can think of to implement your maximum-writes-per-day count is to keep a writes-per-day counter for each messenger, that you then update whenever they write a message.
For example, you could do the following whenever you write a message:
await messenger.ref.collection('messages').add({
'title':title,
'body':body,
'timestamp': Timestamp.now()
})
const today = new Date().toISOString().substring(0, 10); // e.g. "2020-04-11"
await messenger.ref.set({
[today]: admin.firestore.FieldValue.increment(1)
}, { merge: true })
So this adds an additional field to your messenger document for each day, whee it then keeps a count of the number of messages that messenger has written for that day.
You'd then use this count instead of your current limitMessageResult.
const messageCount = (await messenger.get()).data()[today] || 0;
if (messageCount < 1000) {
... post the message and increase the counter
}
else {
... reject the message, and return a message
}
Steps left to do:
You'll want to secure write access to the counter fields, as the messenger shouldn't be able to modify these on their own.
You may want to clean out older message counts periodically, if you're worried about the messenger's document becoming too big. I prefer to leave these types of counters, as they give an opportunity to provide some stats cheaply if needed.
I'm using firebase admin sdk in my cloud functions and I'm getting error randomly in some executions when trying to get a user by uid .
let userRecord = await admin.auth().getUser(userId);
The error details are:
{"error":{"code":400,"message":"TOO_MANY_ATTEMPTS_TRY_LATER",
"errors":[{ "message":"TOO_MANY_ATTEMPTS_TRY_LATER",
"domain":"global","reason":"invalid"}]
}
}
My cloud function executes on a real time database write and can be triggered for multiple users. In total I have 4 auth function calls in one execution first is the above one, second call is to again get user by uid or email, third call is generateEmailVerificationLink and the last call is generatePasswordResetLink.
I have checked the rate limits in documentation for auth but there is no mention of rate limit for these operation. Also the error TOO_MANY_ATTEMPTS_TRY_LATER was only mentioned in REST API for sign up with email password.
If this error is due to rate limit what should I change to prevent this error given these 4 calls are necessary for the operation needed on database write?.
EDIT:
I have identified the actual call which is throwing too many attempts error. The calls auth().generateEmailVerificationLink() and auth().generatePasswordResetLink() throw this error when called too many times.
I called these two in loop with 100 iterations and waited for the promises. The first executions finishes without any errors i.e. 200 requests. But starting second execution as soon as the first one ends will throw the error of too many attempts. So I think these two calls have limit. Now I'm trying to reduce these calls and reuse the link information. Other calls like getUserByEmail works fine.
let promises = [];
let auth = admin.auth();
let hrstart = process.hrtime()
for (let i = 0; i < 100; i++) {
promises.push(auth.getUserByEmail("user email"));
promises.push(auth.generateEmailVerificationLink("user email", {url: `https://app.firebaseapp.com/path`}));
promises.push(auth.generatePasswordResetLink("user email", {url: `https://app.firebaseapp.com/path`}));
}
Promise.all(promises)
.then(value => {
let hrend = process.hrtime(hrstart);
console.log(hrend);
// console.log(value)
});
The error was specifically in the operation auth.createEmailLink. This function has following limit: 20QPS/I.P address where QPS is (query per second). This limit can be increased by submitting the use case to Firebase.
I got this information from firebase support after submitting my issue.
Link to my github issue: https://github.com/firebase/firebase-admin-node/issues/458
I was way under 20QPS but was receiving this exception. In fact, it would always throw the TOO_MANY_ATTEMPTS_TRY_LATER exception on the 2nd attempt.
It turned out to be usage of FirebaseAuth.DefaultInstance instead of instantiating a static instance thusly:
In class definition:
private readonly FirebaseApp _firebase;
In class constructor:
_firebase = FirebaseAdmin.FirebaseApp.Create();
In function:
var auth = FirebaseAuth.GetAuth(_firebase);
var actionCodeSettings = new ActionCodeSettings()
{
...
};
var link = await auth.GenerateEmailVerificationLinkAsync(email, actionCodeSettings);
return link;
In addition to the answer mentioned in https://stackoverflow.com/a/54782967/5515861, I want to add another solution if you found this issue while trying to create custom email verification.
Inspired by the response in this GitHub isssue https://github.com/firebase/firebase-admin-node/issues/458#issuecomment-933161448 .
I am also seeing this issue. I have not ran admin.auth().generateEmailVerificationLink in over 24hrs (from anywhere else or any user at all) and called it just now only one time (while deployed in the prod functions environment) and got this 400 TOO_MANY_ATTEMPTS_TRY_LATER error ...
But, the client did also call the Firebase.auth.currentUser.sendEmailVerification() method around same time (obviously different IP).
Could that be the issue?
My solution to this issue is by adding a retry. e.g.
exports.sendWelcomeEmail = functions.runWith({failurePolicy: true}).auth.user().onCreate(async (user) => {
functions.logger.log("Running email...");
const email = user.email;
const displayName = user.displayName;
const link = await auth.generateEmailVerificationLink(email, {
url: 'https://mpj.io',
});
await sendWelcomeEmail(email, displayName, link);
});
The .runWith({failurePolicy: true}) is key.
It s giving you an error because your cloud functions/backend call the generateEmailVerificationLink while at the same time the default behaviour of the Firebase is also doing the same and it is counted as 20QPS. It some weird Google Rate Limit accounting rule. So my solution is just to add a retry.
The Downside is, it is calling twice, so if the call is billable, it might be billable twice.
I'm trying to send back simple value from firebase but error appearing like this
mycode is :
exports.getTotalPrice = functions.https.onRequest((req, res) => {
admin.database().ref('carresult').once('value').then(function(snapshot) {
var totalPrice = snapshot.val().price;
res.status(200).send(totalPrice);
});
});
ps. In error 65000 is the value I need it to send back.
The Express documentation for res.send([body]) indicates:
The body parameter can be a Buffer object, a String, an object, or an
Array
In your database, /carresult/price is likely stored as a number, making totalPrice an invalid parameter to send(). Your options are to store it as a String convert it to a String before passing to send(), or leave it a number and send it back as a property of an object: send({price: totalPrice}).
exports.getTotalPrice = functions.https.onRequest((req, res) => {
admin.database().ref('carresult').once('value').then(function(snapshot) {
var totalPrice = snapshot.val().price;
res.status(200).send(String(totalPrice)); // <= ADDED String()
});
});
Also note that performing a database read (asynchronous) in an HTTPS function is risky, as Frank van Puffelen explains in this answer:
Note that this is a tricky pattern. The call to the database happens
asynchronously and may take some time to complete. While waiting for
that, the HTTP function may time out and be terminated by the Google
Cloud Functions system...As a general rule I'd recommend using a Firebase Database SDK or its REST API to access the database and not rely on a HTTP function as middleware.
I'm using the .push method on firebase to write new records. I'd like to save the key where the new record is saved to the record itself at the id key. Currently, I do this in 2 operations, first push the record and then update using the ref returned. Can I do this in 1 write? Does it not matter?
If you invoke the Firebase push() method without arguments it is a pure client-side operation.
var newRef = ref.push(); // this does *not* call the server
You can then add the key() of the new ref to your item:
var newItem = {
name: 'anauleau'
id: newRef.key()
};
And write the item to the new location:
newRef.set(newItem);
There's no method to do this in one operation. However, it typically does not matter, because you can always get the push id from the .key() method on the DataSnapshot.
But, there's nothing wrong either about storing the push id. So you coul create a function on the Firebase prototype.
Firebase.prototype.pushWithId = function pushWithid(data) {
var childRef = this.push();
data.key = childRef.key();
childRef.update(data); // or .set() depending on your case
return childRef;
};
var ref = new Firebase('<my-firebase-app>');
ref.pushWithId({ name: 'Alice' });
Take caution with modifying the prototype of functions you do not own. In this case, you'll likely be fine. This method does little, and there's not much of a chance that the Firebase SDK gains a .pushWithId() method.