I'm having issues implementing a counter using the firestore DB.
I have a collection of documents where the document has a field that is updated at some point.
Collection
Object
Field:true
Object
Field:false
Object
Field:true
I need to count all the fields where the value is equals to true. The app is updated from multiple devices at the same time.
How can I accomplish this? Do I need to use the distributed counters approach o just with a cloud function would be enough
Cloud Firestore has a limit of one write per document/second. I would not advise having multiple users writing to the same document at the same time. You should definitely use the Distributed Counters method.
Use cloud functions. simple example:
// database in the 'posts' collection
{
title: "My great post",
categories: {
"technology": true,
"opinion": false,
"cats": true
}
Cloud functions script:
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp({
credential: admin.credential.applicationDefault()
});
const docRef = admin.firestore().collection('post')
.where('categories', '==', true)
.get()
.then((result) => {
res.send(result.size()); //return the count
}).catch(function(error){
console.log("got an error",error);
})
exports.countFunction = functions.https.onRequest(...);
Related
I want to set in every document of a collection, a specific field to 0. How can I do this?
To do in dart:
Geting all documents and update fields:
QuerySnapshot snapshot = await FirebaseFirestore.instance
.collection("collectionName").doc("documentName").get();
snapshot.docs.forEach((doc) {
FirebaseFirestore.instance.collection("collectionName").doc(doc.id).update({
"fieldName": 0,
});
});
But I recommend using the firebase admin for this type of activity, because if there are many documents you can exceed the limits of its use in Firestore, stay tuned! With firebase admin you can use different types of languages, with python for example, and update everything at once (Batched writes), in a single activity.
Example in Python (Firebase admin):
reference = db.collection(u'themes')
query_snapshot = reference.where(u'collections', u'array_contains', u'reflections').stream()
batch = db.batch()
for doc in query_snapshot:
doc_data = doc.to_dict()
new_data = {}
new_data["keywords_pt"] = generate_keywords(doc_data[u"title-pt"].lower())
new_data["keywords_es"] = generate_keywords(doc_data[u"title-es"].lower())
new_data["keywords_en"] = generate_keywords(doc_data[u"title-en"].lower())
batch.update(reference.document(doc.id), new_data)
batch.commit()
For more information:
Firebase admin: https://firebase.google.com/docs/database/admin/start
Batched writes: https://firebase.google.com/docs/firestore/manage-data/transactions#batched-writes
I build react native app with firebase & firestore.
what I'm looking to do is, when user open app, to insert/update his status to 'online' (kind of presence system), when user close app, his status 'offline'.
I did it with firebase.database.onDisconnect(), it works fine.
this is the function
async signupAnonymous() {
const user = await firebase.auth().signInAnonymouslyAndRetrieveData();
this.uid = firebase.auth().currentUser.uid
this.userStatusDatabaseRef = firebase.database().ref(`UserStatus/${this.uid}`);
this.userStatusFirestoreRef = firebase.firestore().doc(`UserStatus/${this.uid}`);
firebase.database().ref('.info/connected').on('value', async connected => {
if (connected.val() === false) {
// this.userStatusFirestoreRef.set({ state: 'offline', last_changed: firebase.firestore.FieldValue.serverTimestamp()},{merge:true});
return;
}
await firebase.database().ref(`UserStatus/${this.uid}`).onDisconnect().set({ state: 'offline', last_changed: firebase.firestore.FieldValue.serverTimestamp() },{merge:true});
this.userStatusDatabaseRef.set({ state: 'online', last_changed: firebase.firestore.FieldValue.serverTimestamp() },{merge:true});
// this.userStatusFirestoreRef.set({ state: 'online',last_changed: firebase.firestore.FieldValue.serverTimestamp() },{merge:true});
});
}
after that, I did trigger to insert data into firestore(because I want to work with firestore), this is the function(works fine, BUT it takes 3-4 sec)
module.exports.onUserStatusChanged = functions.database
.ref('/UserStatus/{uid}').onUpdate((change,context) => {
const eventStatus = change.after.val();
const userStatusFirestoreRef = firestore.doc(`UserStatus/${context.params.uid}`);
return change.after.ref.once("value").then((statusSnapshot) => {
return statusSnapshot.val();
}).then((status) => {
console.log(status, eventStatus);
if (status.last_changed > eventStatus.last_changed) return status;
eventStatus.last_changed = new Date(eventStatus.last_changed);
//return userStatusFirestoreRef.set(eventStatus);
return userStatusFirestoreRef.set(eventStatus,{merge:true});
});
});
then after that, I want to calculate the online users in app, so I did trigger when write new data to node of firestore so it calculate the size of online users by query.(it works fine but takes 4-7 sec)
module.exports.countOnlineUsers = functions.firestore.document('/UserStatus/{uid}').onWrite((change,context) => {
console.log('userStatus')
const userOnlineCounterRef = firestore.doc('Counters/onlineUsersCounter');
const docRef = firestore.collection('UserStatus').where('state','==','online').get().then(e=>{
let count = e.size;
console.log('count',count)
return userOnlineCounterRef.update({count})
})
return Promise.resolve({success:'added'})
})
then into my react native app
I get the count of online users
this.unsubscribe = firebase.firestore().doc(`Counters/onlineUsersCounter`).onSnapshot(doc=>{
console.log('count',doc.data().count)
})
All the operations takes about 12 sec. it's too much for me, it's online app
my firebase structure
what I'm doing wrong? maybe there is unnecessary function or something?
My final goals:
minimize time operation.
get online users count (with listener-each
change, it will update in app)
update user status.
if there are other way to do that, I would love to know.
Cloud Functions go into a 'cold start' mode, where they take some time to boot up. This is the only reason I can think of that it would take that long. Stack Overflow: Firebase Cloud Functions Is Very Slow
But your cloud function only needs to write to Firestore on log out to
catch the case where your user closes the app. You can write to it directly on log in from your client
with auth().onAuthStateChange().
You could also just always read who is logged in or out directly from the
realtime database and use Firestore for the rest of your data.
You can rearrange your data so that instead of a 'UserStatus' collection you have an 'OnlineUsers' collection containing only online users, kept in sync by deleting the documents on log out. Then it won't take a query operation to get them. The query's impact on your performance is likely minimal, but this would perform better with a large number of users.
The documentation also has a guide that may be useful: Firebase Docs: Build Presence in Cloud Firestore
I'm using the firestore of firebase and I want to iterate through the whole collection. Is there something like:
db.collection('something').forEach((doc) => {
// do something
})
Yes, you can simply query the collection for all its documents using the get() method on the collection reference. A CollectionReference object subclasses Query, so you can call Query methods on it. By itself, a collection reference is essentially an unfiltered query for all of its documents.
Android: Query.get()
iOS/Swift: Query.getDocuments()
JavaScript: Query.get()
In each platform, this method is asynchronous, so you'll have to deal with the callbacks correctly.
See also the product documentation for "Get all documents in a collection".
db.collection("cities").get().then(function(querySnapshot) {
querySnapshot.forEach(function(doc) {
// doc.data() is never undefined for query doc snapshots
console.log(doc.id, " => ", doc.data());
});
});
If you know that there aren't too many docs in the collection (e.g. thousands or millions) then you can just use collectionRef.get() as described in the top-voted answer here and explained in Firebase docs.
However, in many cases, a collection can contain large numbers of documents that you can't just "get" at once, as your program's memory usage will explode. In these cases you need to implement a different traversal logic that will go through the entire collection by batches. You also need to ensure that you don’t miss any documents or process any of them multiple times.
This is why I wrote Firecode, an open-source Node.js library that solves precisely this problem. It is an extremely light, robust, well-typed, and well-documented library that provides you with configurable traverser objects that walk you through a given collection.
You can find the Github repo here and the docs site here. Also, here's a short snippet that shows you how you would traverse a users collection with Firecode.
const usersCollection = firestore().collection('users');
const traverser = createTraverser(usersCollection);
const { batchCount, docCount } = await traverser.traverse(async (batchDocs, batchIndex) => {
const batchSize = batchDocs.length;
await Promise.all(
batchDocs.map(async (doc) => {
const { email, firstName } = doc.data();
await sendEmail({ to: email, content: `Hello ${firstName}!` });
})
);
console.log(`Batch ${batchIndex} done! We emailed ${batchSize} users in this batch.`);
});
console.log(`Traversal done! We emailed ${docCount} users in ${batchCount} batches!`);
I'm trying to model "memberships" with Firestore. The idea is that there are companies, users and then memberships.
The memberships collection stores a reference to a company and to a user, as well as a role as a string, e..g admin or editor.
How would I query to get the users with a certain role for a company?
This is what I currently have with some basic logging.
const currentCompanyID = 'someid';
return database
.collection('memberships')
.where('company', '==', database.doc(`companies/${currentCompanyID}`))
.where('role', '==', 'admin')
.get()
.then(snap => {
snap.forEach(function(doc) {
console.log(doc.id, ' => ', doc.data());
const data = doc.data();
console.log(data.user.get());
});
})
.catch(error => {
console.error('Error fetching documents: ', error);
});
data.user.get() returns a promise to the user, but I'd have to do that for every user which seems inefficient?
What would be the best way to approach this?
Your code is close to what you want, but there are two issues:
Your where() clause can't compare a field with a document reference, because Firestore is a classic denormalized datastore. There aren't ways to strongly guarantee that one document refers to another. You'll need to store document IDs and maintain consistency yourself. (Example below).
Queries actually return a QuerySnapshot, which includes all the docs that result from a query. So you're not getting one document at a time — you'll get all the ones that match. (See code below)
So a corrected version that fits the spirit of what you want:
const currentCompanyID = '8675309';
const querySnapshot = await database
.collection('memberships')
.where('companyId', '==', currentCompanyID)
.where('role', '==', 'admin')
.get(); // <-- this promise, when awaited, pulls all matching docs
await Promise.all(querySnapshot.map(async snap => {
const data = doc.data();
const user = await database
.collection('users')
.doc(data.userId)
.get();
console.log(doc.id, ' => ', data);
console.log(user);
});
There isn't a faster way on the client side to fetch all the users that your query refers to at once -- it's part of the trouble of trying to use a denormalized store for queries that feel much more like classic relational database queries.
If this ends up being a query you run often (i.e. get users with a certain role within a specific company), you could consider storing membership information as part of the user doc instead. That way, you could query the users collection and get all the matching users in one shot.
Within my Google Firebase Firstore database I would like to gather aggregative data such as how many documents a collection has. Since Firestore does not provide aggregative queries I'm attempting to write a cloud function that will increment a field every time a document is added to the database which will contain the number of documents a collection has.
The problem I'm having is I cannot for the life of me figure out how to grab documents from Firestore within a cloud function using nodejs.
Here's what I'm doing:
At the top of my index.js file I configure the admin SDK and what have you like this:
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp(functions.config().firebase);
Then for my cloud function I do this:
exports.createPost = functions.firestore
.document('posts/{post_id}')
.onCreate(event => {
// Get the post object
var post = event.data.data();
var senderID = post["sender_id"]; // This is not null
// Access to user's document and the main ledger document
const userDocRef = admin.database().ref('/users').orderByChild("user_id").equalTo(senderID).once('value')
const ledgerDocRef = admin.database().ref('/posts/ledger').once('value');
return Promise.all([userDocRef, ledgerDocRef]).then(snapshot => {
const user = snapshot[0].val();
const ledger = snapshot[1].val();
console.log("user => "+user); // Logs: user => null
console.log("ledger => "+ledger); // Logs: ledger => null
const userPostCount = user["user_name"];
const globalPostCount = ledger["global_post_count"] + 1;
const userUpdate = user.update({"post_count" : userPostCount});
const ledgerUpdate = ledger.update({"global_post_count" : globalPostCount});
return Promise.all([userUpdate, ledgerUpdate]);
});
});
I end up with the error:
TypeError: Cannot read property 'global_post_count' of null
at Promise.all.then.snapshot
Which I figure means something is wrong with my query but I don't know what. Both the users and posts are root level collections.
Im also getting a warning that says:
Billing account not configured. External network is not accessible and
quotas are severely limited.
From what I've read online, I don't think that effects it but I thought it was worth noting.
Please help.
Looks like you've written a Firestore trigger, but are then reaching into Realtime Database for queries:
const userDocRef = admin.database().ref('/users').orderByChild("user_id").equalTo(senderID).once('value')
const ledgerDocRef = admin.database().ref('/posts/ledger').once('value');
If your RTDB is empty, these queries will also end up empty.
To query Firestore, you need to be using admin.firestore() instead of admin.database(). Firestore has a mostly different API (via the Cloud SDK I just linked) than RTDB, but they are similar in some ways.