I build react native app with firebase & firestore.
what I'm looking to do is, when user open app, to insert/update his status to 'online' (kind of presence system), when user close app, his status 'offline'.
I did it with firebase.database.onDisconnect(), it works fine.
this is the function
async signupAnonymous() {
const user = await firebase.auth().signInAnonymouslyAndRetrieveData();
this.uid = firebase.auth().currentUser.uid
this.userStatusDatabaseRef = firebase.database().ref(`UserStatus/${this.uid}`);
this.userStatusFirestoreRef = firebase.firestore().doc(`UserStatus/${this.uid}`);
firebase.database().ref('.info/connected').on('value', async connected => {
if (connected.val() === false) {
// this.userStatusFirestoreRef.set({ state: 'offline', last_changed: firebase.firestore.FieldValue.serverTimestamp()},{merge:true});
return;
}
await firebase.database().ref(`UserStatus/${this.uid}`).onDisconnect().set({ state: 'offline', last_changed: firebase.firestore.FieldValue.serverTimestamp() },{merge:true});
this.userStatusDatabaseRef.set({ state: 'online', last_changed: firebase.firestore.FieldValue.serverTimestamp() },{merge:true});
// this.userStatusFirestoreRef.set({ state: 'online',last_changed: firebase.firestore.FieldValue.serverTimestamp() },{merge:true});
});
}
after that, I did trigger to insert data into firestore(because I want to work with firestore), this is the function(works fine, BUT it takes 3-4 sec)
module.exports.onUserStatusChanged = functions.database
.ref('/UserStatus/{uid}').onUpdate((change,context) => {
const eventStatus = change.after.val();
const userStatusFirestoreRef = firestore.doc(`UserStatus/${context.params.uid}`);
return change.after.ref.once("value").then((statusSnapshot) => {
return statusSnapshot.val();
}).then((status) => {
console.log(status, eventStatus);
if (status.last_changed > eventStatus.last_changed) return status;
eventStatus.last_changed = new Date(eventStatus.last_changed);
//return userStatusFirestoreRef.set(eventStatus);
return userStatusFirestoreRef.set(eventStatus,{merge:true});
});
});
then after that, I want to calculate the online users in app, so I did trigger when write new data to node of firestore so it calculate the size of online users by query.(it works fine but takes 4-7 sec)
module.exports.countOnlineUsers = functions.firestore.document('/UserStatus/{uid}').onWrite((change,context) => {
console.log('userStatus')
const userOnlineCounterRef = firestore.doc('Counters/onlineUsersCounter');
const docRef = firestore.collection('UserStatus').where('state','==','online').get().then(e=>{
let count = e.size;
console.log('count',count)
return userOnlineCounterRef.update({count})
})
return Promise.resolve({success:'added'})
})
then into my react native app
I get the count of online users
this.unsubscribe = firebase.firestore().doc(`Counters/onlineUsersCounter`).onSnapshot(doc=>{
console.log('count',doc.data().count)
})
All the operations takes about 12 sec. it's too much for me, it's online app
my firebase structure
what I'm doing wrong? maybe there is unnecessary function or something?
My final goals:
minimize time operation.
get online users count (with listener-each
change, it will update in app)
update user status.
if there are other way to do that, I would love to know.
Cloud Functions go into a 'cold start' mode, where they take some time to boot up. This is the only reason I can think of that it would take that long. Stack Overflow: Firebase Cloud Functions Is Very Slow
But your cloud function only needs to write to Firestore on log out to
catch the case where your user closes the app. You can write to it directly on log in from your client
with auth().onAuthStateChange().
You could also just always read who is logged in or out directly from the
realtime database and use Firestore for the rest of your data.
You can rearrange your data so that instead of a 'UserStatus' collection you have an 'OnlineUsers' collection containing only online users, kept in sync by deleting the documents on log out. Then it won't take a query operation to get them. The query's impact on your performance is likely minimal, but this would perform better with a large number of users.
The documentation also has a guide that may be useful: Firebase Docs: Build Presence in Cloud Firestore
Related
A firebase cloud function has been implemented to be triggered every time the value of price changes, the code is as follows:
export const orderListener = functions.firestore
.document("/users/{userId}/Order/{orderId}")
.onUpdate(async (change, context) => {
const { userId, orderId } = context.params;
const price = change.after.data().price;
try {
if (price !== 0) {
const userDoc = await admin
.firestore()
.collection(`/users/${userId}/Personal information`)
.doc("Personal Information")
.get();
const { fcmToken } = userDoc.data()!;
functions.logger.log({
userId,
fcmToken,
orderId,
price,
});
//RETURN
return admin.messaging().sendToDevice(fcmToken, {
notification: {
title: "Title of the message",
body: `${orderId}'s price > 0`,
},
});
} else {
return null
}
} catch (e) {
functions.logger.error(e.message);
//RETURN
return null
This code has been deployed in the firebase function, and I tried to test it by changing the value of price, and monitoring execution by looking at firebase console.
Expectation:
As soon as I change the value of price in my firestore database, it executes almost immediately (within 5s).
Result:
The function executes just fine and was logged on the firebase console, but there is a 50 minutes delay in execution after I updated the price value (Function execution was only logged 50mins later). It was tested by someone else but does not seem to have this issue.
Question:
Any potential error contained within this code? Is it likely to be some error with the firebase setup, settings, etc...?
Edit: Thanks to the contribution of Dharmaraj I have updated the above code with return statements. The problem however still persists.
You should consider two things here:
1 - Cold Start of functions. If a functions is update or hasen't been used for a while it will kind of "shut down" or "go to sleep". When it gets triggered again it has a "cold start" that takes a while. It kind of boots the function from the "sleep"
2 - The loggs are not in real time. The loggs are not shown in realtime. They also have some lag and deppending if you are watching them on the Firebase Console or GCP console you may need to refresh them manually.
50 min is way to much for a cloud function. I belive that you maybe just haben't seen the log as you expected it. It also could be that you test device was offline while trying it and it synced later. Without more information of your code, device and how you tested it it's hard for us to tell what exactly it would be but the mentioned szenarios are the most likely ones.
I'm currently developing an app using Firebase.
My Firestore Database looks like below:
Once the user passes the Firebase authentication procedure, I'm creating a user document with a field "Phone:" which contains his phone number. Basically, everyone who is gonna using the app will be listed in the database.
And here is my challenge:
I'm using the plugin easy_contact_picker to store all the contacts of the users device to a List.
How can I find out whether the users contacts are using the app or whether they are listed in the database?
My goal is create a contact List Widget which shows me all my contacts. But those contacts which are using the app or which are listed in the database, should be highlighted or marked particularly.
Which is the best way to realize that if we consider that millions of users (to minimize computing power)are listed in the database?
Anyone has an idea?
Thanks a lot
First of all try to awoid giving everyone access to read all users. That is something most ppl do when handling such a problem. The do it because the query over all users won't work if you don't give the rights to read all of them.
Because of security reasons I would move the logic for checking if a user exists into callable function (not a http function!). That way you can call it inside of your app and check for a single user or multiple of them in an array. That would depend how your frontend would handle it.
Very importand would be to store all phone numbers in the absolute same format. That way you could query for them. Regardless of the number of users you could always find a specific one like here:
var citiesRef = db.collection("users");
var query = citiesRef.where("Phone", "==", "+4912345679");
The numbers need to be absolutely the same without any emtpy spaces - chars and the +49 or 0049 also needs to be the same.
You could create two callable funcitons. One to check if a single user exists in your app and another where you send an array of phone numbers and you get an array back. The cloud function can use Promise.all to performe such queries in parallel so you get your responce quite fast.
I'm using a similar approach to add users in my app as admins to specific groups where you just enter the email of the user and if he is in the app he will be added. I not he get's an invitation on the email to join the App.
With the help of Tarik's answer, Ayrix and I came up with the following solution.
Important: Read Tarik's answer for more information.
Client: callable_compare_contacts.dart
import 'package:cloud_functions/cloud_functions.dart';
Future<List<Object>> getMembersByPhoneNumber(List<String> allPhoneNumbers) async {
HttpsCallable callable = FirebaseFunctions.instance.httpsCallable('membersByPhoneNumber');
final results = await callable.call(<String, dynamic>{'allPhoneNumbers': allPhoneNumbers});
return results.data;
}
Server: index.js
const functions = require("firebase-functions");
const admin = require("firebase-admin");
if (admin.apps.length === 0) {
admin.initializeApp({
credential: admin.credential.applicationDefault(),
});
}
exports.membersByPhoneNumber = functions.https.onCall((data, context) => {
return new Promise((resolve, reject) => {
if (!data || !data.allPhoneNumbers.length) return resolve([]);
const phoneNumbers = data.allPhoneNumbers;
// TODO: different scope? move vars for future use
const db = admin.firestore();
const collectionRef = db.collection("User");
let batches = [];
// because of wrong eslint parsing (dirty)
batches = [];
while (phoneNumbers.length) {
// firestore limits batches to 10
const batch = phoneNumbers.splice(0, 10);
// add the batch request to to a queue
batches.push(
new Promise((response) => {
collectionRef.where("Phone", "in", [...batch]).get()
.then((results) =>
response(results.docs.map(function(result) {
return result.data().Phone;
} )));
})
);
}
// response / return to client
Promise.all(batches).then(function(content) {
// console.log("content.flat()");
// console.log(content.flat());
return resolve(content.flat());
});
});
});
Note: This is our first callable/cloud function .. so Suggestions for changes are welcome.
I want to check user's online status, in realtime database I used to check this with the help of onDisconnect(), but now I've shifted to firestore and can't find any similar method in that.
According to this onDisconnect:
The onDisconnect class is most commonly used to manage presence in applications where it is useful to detect how many clients are connected and when other clients disconnect.
To be able to use presence in firestore, you need to connect firestore with realtime firebase(no other way).
Please check this for more info:
https://firebase.google.com/docs/firestore/solutions/presence
NOTE: This solution is not especially efficient
Off the top of my head (read: I haven't thought through the caveats), you could do something like this:
const fiveMinutes = 300000 // five minutes, or whatever makes sense for your app
// "maintain connection"
setInterval(() => {
userPresenceDoc.set({ online: new Date().getTime() })
}, fiveMinutes)
then, on each client...
userPresenceDoc.onSnapshot(doc => {
const fiveMinutesAgo = new Date().getTime() - fiveMinutes
const isOnline = doc.data().online > fiveMinutesAgo
setUserPresence(isOnline)
})
You'd probably want the code checking for presence to use an interval a little more than the interval used by the code maintaining the connection to account for network lag, etc.
A NOTE ABOUT COST
So, obviously, there could be significant lag between when someone disconnects and when that disconnection is recognized by other clients. You can decrease that lag time by increasing the frequency of writes to Firestore, and thus increasing your costs. Running the numbers, I came out with the following costs for different intervals, assuming a single client connection running continuously for a month:
Interval Cost/User/Month
----------------------------
10m $0.007776
5m $0.015552
1m $0.07776
10s $0.46656
5s $0.93312
1s $4.6656
Going with an interval of one second is pretty pricy, at $46,656 a month for a system with 10,000 users who leave the app open all month long. An interval of 10 minutes with the same number of users would only cost $77.76 a month. A more reasonable interval of one minute, 10,000 users, and only four hours of app usage per day per user, rings in at $129.60 / month.
There is no equivalent. The Firestore SDK currently doesn't have presence management like the Realtime Database SDK.
Instead, you might want to use the Realtime Database onDisconnect() in conjunction with Cloud Functions to kick off some work when the client disconnects from RTDB. You would be assuming that that your app probably also lost its connection to Firestore at the same time.
Try this, but this method kind of hackish because we cannot use onDisconnected in firestore. As far as I know, realtime database use secure WebSocket technology, so thats why onDisconnected have on it
But you can use realtime database that can be implemented in cloud functions
to update the firestore data,
functions.database.ref('users/{userId}').onUpdate()
Somewhere in clientside:
firebase.database()
.ref('.info/connected')
.on('value', async (snap) => {
if (snap.val() === true) {
// Update the online status in RTDB
await firebase.database()
.ref(`users/${credentials.user.uid}/`)
.set({
online_status: true,
start_online: firebase.database.ServerValue.TIMESTAMP
});
// OnDisconnect
firebase.database()
.ref(`users/${credentials.user.uid}/`)
.onDisconnect()
.set({
online_status: false,
last_online: firebase.database.ServerValue.TIMESTAMP
});
}
});
in cloudfunctions (this will be trigger on update)
export const onUserOnlineStatusChanged = functions.database.ref('users/{userId}').onUpdate((event: functions.Change<functions.database.DataSnapshot>, context: functions.EventContext) => {
return event.after.ref.once('value')
.then((dataSnapshot) => dataSnapshot.val()) // Get the latest value from the Firebase Realtime database
.then((value: any) => {
// Update the value from RTDB to Firestore
console.log('value.online_status', value.online_status);
if (value.online_status == true) {
// Set the value to the firestore
admin.firestore()
.collection('users_info')
.doc(context.params.userId) // Get document by the userId / Or use .where
.set({
online_status: value.online_status,
updated_at: new Date
}, {
mergeFields: [
'online_status',
'updated_at'
]
});
// Add code if necessary (when the online_status is true)
} else if (value.online_status == false) {
// Set the value to the firestore
admin.firestore()
.collection('users_info')
.doc(context.params.userId) // Get document by the userId / Or use .where
.set({
online_status: value.online_status,
updated_at: new Date
}, {
mergeFields: [
'online_status',
'updated_at'
]
});
// Add code if necessary (when the online_status is false)
}
});
});
It would take about 1 or 2 seconds to update from cloud function to firestore
There is not a direct way to do this thing but this trick helped me achieve this disconnect listener.
window.addEventListener("beforeunload", async function (e) { e.preventDefault(); await firestoreRef.doc("doc-ref").update({ online: false }); });
I am developing a PWA, which displays a list of transactions (transaction is an object with ~10 fields). I am using firestore for storage and realtime updates and I have also enabled persistance.
I want my application to have all the data in memory and I want to take care of displaying only necessary information myself (e.g. using virtual scrolling for transaction list). Due to this reason I listen to the whole collection (a.k.a the transactions).
At the start of the app, I want to make sure the data is loaded so I use one time cache query to get the transactions. I would expect the query to be nearly instantaneous, but on laptop it takes around ~1 second to get the initial data (and I also have another collection which I fetch from cache and this resolves after ~2 seconds after transactions request). For mobile it takes around ~9seconds (loading on mobile, loading on laptop)
I want my app to feel instantaneous, but I takes a few seconds until the data is in place. Note, that I am not doing any advanced queries (I just want to load the data to memory).
Am I doing something wrong? I have read Firestore docs, but I don't think the amount of data that I have in cache should cause such bad performance.
UPDATE: Even if I limit the initial query to just load 20 documents. It still takes around ~2 seconds to retrieve them.
UPDATE 2: The code looks like this:
export const initializeFirestore = (): Thunk => (dispatch) => {
const initialQueries: Array<Promise<unknown>> = []
getQueries().forEach((query) => {
const q = query.createFirestoneQuery()
initialQueries.push(
q
.get({
source: 'cache',
})
.then((snapshot) =>
dispatch(firestoneChangeAction(query, snapshot, true)),
),
)
q.onSnapshot((change) => {
dispatch(firestoneChangeAction(query, change))
})
})
console.log('Now I am just waiting for initial data...')
return Promise.all(initialQueries)
}
You may be interested by the smart approach presented by Firebase engineers during the "Faster web apps with Firebase" Session of the Firebase Summit 2019 (You can watch the video here: https://www.youtube.com/watch?v=DHbVyRLkX4c).
In a nutshell, their idea is to use the Firestore REST API to make the first query to the database (which does not need to download any SDK), and in parallel, dynamically import the Web SDK in order to use it for the subsequent queries.
The github repository is here: https://github.com/hsubox76/fireconf-demo
I paste below the content of the key js file (https://github.com/hsubox76/fireconf-demo/blob/master/src/dynamic.js) for further reference.
import { firebaseConfigDynamic as firebaseConfig } from "./shared/firebase-config";
import { renderPage, logPerformance } from "./shared/helpers";
let firstLoad = false;
// Firestore REST URL for "current" collection.
const COLLECTION_URL =
`https://firestore.googleapis.com/v1/projects/exchange-rates-adcf6/` +
`databases/(default)/documents/current`;
// STEPS
// 1) Fetch REST data
// 2) Render data
// 3) Dynamically import Firebase components
// 4) Subscribe to Firestore
// HTTP GET from Firestore REST endpoint.
fetch(COLLECTION_URL)
.then(res => res.json())
.then(json => {
// Format JSON data into a tabular format.
const stocks = formatJSONStocks(json);
// Measure time between navigation start and now (first data loaded)
performance && performance.measure("initialDataLoadTime");
// Render using initial REST data.
renderPage({
title: "Dynamic Loading (no Firebase loaded)",
tableData: stocks
});
// Import Firebase library.
dynamicFirebaseImport().then(firebase => {
firebase.initializeApp(firebaseConfig);
firebase.performance(); // Use Firebase Performance - 1 line
subscribeToFirestore(firebase);
});
});
/**
* FUNCTIONS
*/
// Dynamically imports firebase/app, firebase/firestore, and firebase/performance.
function dynamicFirebaseImport() {
const appImport = import(
/* webpackChunkName: "firebase-app-dynamic" */
"firebase/app"
);
const firestoreImport = import(
/* webpackChunkName: "firebase-firestore-dynamic" */
"firebase/firestore"
);
const performanceImport = import(
/* webpackChunkName: "firebase-performance-dynamic" */
"firebase/performance"
);
return Promise.all([appImport, firestoreImport, performanceImport]).then(
([dynamicFirebase]) => {
return dynamicFirebase;
}
);
}
// Subscribe to "current" collection with `onSnapshot()`.
function subscribeToFirestore(firebase) {
firebase
.firestore()
.collection(`current`)
.onSnapshot(snap => {
if (!firstLoad) {
// Measure time between navigation start and now (first data loaded)
performance && performance.measure("realtimeDataLoadTime");
// Log to console for internal development
logPerformance();
firstLoad = true;
}
const stocks = formatSDKStocks(snap);
renderPage({
title: "Dynamic Loading (Firebase now loaded)",
tableData: stocks
});
});
}
// Format stock data in JSON format (returned from REST endpoint)
function formatJSONStocks(json) {
const stocks = [];
json.documents.forEach(doc => {
const pathParts = doc.name.split("/");
const symbol = pathParts[pathParts.length - 1];
stocks.push({
symbol,
value: doc.fields.closeValue.doubleValue || 0,
delta: doc.fields.delta.doubleValue || 0,
timestamp: parseInt(doc.fields.timestamp.integerValue)
});
});
return stocks;
}
// Format stock data in Firestore format (returned from `onSnapshot()`)
function formatSDKStocks(snap) {
const stocks = [];
snap.forEach(docSnap => {
if (!docSnap.data()) return;
const symbol = docSnap.id;
const value = docSnap.data().closeValue;
stocks.push({
symbol,
value,
delta: docSnap.data().delta,
timestamp: docSnap.data().timestamp
});
});
return stocks;
}
You're not doing anything wrong. The query will take as much time as it needs to finish. This is why many sites use a loading indicator.
For the first query in your app, it's going to include the time it takes to fully initialize the SDK, which might involve asynchronous work beyond more than just the query itself. Also bear in mind that reading and sorting data from local disk isn't necessarily "fast", and that for larger amounts of documents, the local disk cache read might even be more expensive than the time it would take the fetch the same documents over the network.
Since we don't have any indication of how many documents you have, and how much total data you're trying to transfer, and the code you're using for this, all we can do is guess. But there's really not much you can do to speed up the initial query, other than perhaps limiting the size of the result set.
If you think that what you're experiencing is a bug, then please file a bug report on GitHub.
I want to check user's online status, in realtime database I used to check this with the help of onDisconnect(), but now I've shifted to firestore and can't find any similar method in that.
According to this onDisconnect:
The onDisconnect class is most commonly used to manage presence in applications where it is useful to detect how many clients are connected and when other clients disconnect.
To be able to use presence in firestore, you need to connect firestore with realtime firebase(no other way).
Please check this for more info:
https://firebase.google.com/docs/firestore/solutions/presence
NOTE: This solution is not especially efficient
Off the top of my head (read: I haven't thought through the caveats), you could do something like this:
const fiveMinutes = 300000 // five minutes, or whatever makes sense for your app
// "maintain connection"
setInterval(() => {
userPresenceDoc.set({ online: new Date().getTime() })
}, fiveMinutes)
then, on each client...
userPresenceDoc.onSnapshot(doc => {
const fiveMinutesAgo = new Date().getTime() - fiveMinutes
const isOnline = doc.data().online > fiveMinutesAgo
setUserPresence(isOnline)
})
You'd probably want the code checking for presence to use an interval a little more than the interval used by the code maintaining the connection to account for network lag, etc.
A NOTE ABOUT COST
So, obviously, there could be significant lag between when someone disconnects and when that disconnection is recognized by other clients. You can decrease that lag time by increasing the frequency of writes to Firestore, and thus increasing your costs. Running the numbers, I came out with the following costs for different intervals, assuming a single client connection running continuously for a month:
Interval Cost/User/Month
----------------------------
10m $0.007776
5m $0.015552
1m $0.07776
10s $0.46656
5s $0.93312
1s $4.6656
Going with an interval of one second is pretty pricy, at $46,656 a month for a system with 10,000 users who leave the app open all month long. An interval of 10 minutes with the same number of users would only cost $77.76 a month. A more reasonable interval of one minute, 10,000 users, and only four hours of app usage per day per user, rings in at $129.60 / month.
There is no equivalent. The Firestore SDK currently doesn't have presence management like the Realtime Database SDK.
Instead, you might want to use the Realtime Database onDisconnect() in conjunction with Cloud Functions to kick off some work when the client disconnects from RTDB. You would be assuming that that your app probably also lost its connection to Firestore at the same time.
Try this, but this method kind of hackish because we cannot use onDisconnected in firestore. As far as I know, realtime database use secure WebSocket technology, so thats why onDisconnected have on it
But you can use realtime database that can be implemented in cloud functions
to update the firestore data,
functions.database.ref('users/{userId}').onUpdate()
Somewhere in clientside:
firebase.database()
.ref('.info/connected')
.on('value', async (snap) => {
if (snap.val() === true) {
// Update the online status in RTDB
await firebase.database()
.ref(`users/${credentials.user.uid}/`)
.set({
online_status: true,
start_online: firebase.database.ServerValue.TIMESTAMP
});
// OnDisconnect
firebase.database()
.ref(`users/${credentials.user.uid}/`)
.onDisconnect()
.set({
online_status: false,
last_online: firebase.database.ServerValue.TIMESTAMP
});
}
});
in cloudfunctions (this will be trigger on update)
export const onUserOnlineStatusChanged = functions.database.ref('users/{userId}').onUpdate((event: functions.Change<functions.database.DataSnapshot>, context: functions.EventContext) => {
return event.after.ref.once('value')
.then((dataSnapshot) => dataSnapshot.val()) // Get the latest value from the Firebase Realtime database
.then((value: any) => {
// Update the value from RTDB to Firestore
console.log('value.online_status', value.online_status);
if (value.online_status == true) {
// Set the value to the firestore
admin.firestore()
.collection('users_info')
.doc(context.params.userId) // Get document by the userId / Or use .where
.set({
online_status: value.online_status,
updated_at: new Date
}, {
mergeFields: [
'online_status',
'updated_at'
]
});
// Add code if necessary (when the online_status is true)
} else if (value.online_status == false) {
// Set the value to the firestore
admin.firestore()
.collection('users_info')
.doc(context.params.userId) // Get document by the userId / Or use .where
.set({
online_status: value.online_status,
updated_at: new Date
}, {
mergeFields: [
'online_status',
'updated_at'
]
});
// Add code if necessary (when the online_status is false)
}
});
});
It would take about 1 or 2 seconds to update from cloud function to firestore
There is not a direct way to do this thing but this trick helped me achieve this disconnect listener.
window.addEventListener("beforeunload", async function (e) { e.preventDefault(); await firestoreRef.doc("doc-ref").update({ online: false }); });