I am use cloud Firestore to store users data and my question is:
Is it possible to know how many documents are stored in specified collection. If possible write
Firebase Firestore almost always return entire collection, including all its documents. Hence, you just count the retrieved data. The code would depend on the language/framework you use, but all methods are well described #official docs, if you have managed to retrieve data, you should be able to just count its items. Let say you need a service for Angular2+ using AngularFire2 and you like to use valueChanges() method. Once you map the retrieved data, you count its items. The code might look like this (ideally, you should add collection/document interfaces as well ):
import { Injectable } from '#angular/core';
import { AngularFirestore } from 'angularfire2/firestore';
#Injectable()
export class YourService {
constructor( private _fireStore: AngularFirestore ) { }
getSomething = () => {
const collection_reference = this._fireStore.collection('logs');
const collection_data = collection_reference.valueChanges();
collection_data.map( data => {
const number_of_documents = data.length;
} ).subscribe((query) => {});
}
}
There are no built-in aggregation queries in Cloud Firestore. So to get the count of documents in a collection, you will indeed either have to retrieve all those documents (as Emocuc answered) or keep the count of the number of documents in the database somewhere.
With the second approach you update a counter in the database, any time you add or remove a document. This adds some work to the write operations, but after to show the number of documents you simply can read the counter (instead of reading all documents). This kind of trade-off (more complex writes for a faster/simpler read) is very common in NoSQL database and is one of the reasons they can scale to so many readers.
The Cloud Firestore documentation has some examples of these sort of aggregation operations that I recommend you check out. While the samples are more complex than what you need, they are well documented and should be a useful starting point.
Related
Does anyone know how to limit an array so new items get pushed in and old ones are discarded in the same write?
I'm guessing this isn't possible but it sure would be handy.
// * Store notification
// Users collection
const usersCollection = db.collection('users').doc(uid).collection('notifications').doc();
// Write this notification to the database as well
await usersCollection.update({
count: admin.firestore.FieldValue.increment,
notifications: admin.firestore.FieldValue.arrayUnion({
'symbol': symbol,
'companyname': companyname,
'change': priceDifference,
'changeDirection': directionOperatorHandler,
'updatedPrice': symbolLatestPrice,
'timestamp': currentTimestamp,
})
});
Written in Typescript
Alternatively, I was thinking of running a scheduled cloud function every week to go through and trim down the arrays based on the timestamp.
The reason I'm using an array to store my notifications is because I'm expecting a lot of writes.
There is no simple configuration for this. Your code should implement your requirements by:
Reading the document
Modifying the array in memory
Checking that the size is within limits
Writing the document back
I'm trying to keep track of the number of documents in collections and the number of users in my Firebase project. I set up some .create triggers to update a stats document using increment, but sometimes the .create functions trigger multiple times for a single creation event. This happens with both Firestore documents and new users. Any ideas?
const functions = require('firebase-functions');
const admin = require('firebase-admin');
const firestore = require('#google-cloud/firestore')
admin.initializeApp();
const db = admin.firestore()
/* for counting documents created */
exports.countDoc = functions.firestore
.document('collection/{docId}')
.onCreate((change, context) => {
const docId = context.params.docId
db.doc('stats/doc').update({
'docsCreated': firestore.FieldValue.increment(1)
})
return true;
});
/* for counting users created */
exports.countUsers = functions.auth.user().onCreate((user) => {
db.doc('stats/doc').update({
'usersCreated': firestore.FieldValue.increment(1)
})
return true;
});
Thanks!
There is some advice on how to achieve your functions' idempotency.
There are FieldValue.arrayUnion() & FieldValue.arrayRemove() functions which safely remove and add elements to an array, without duplicates or errors if the element being deleted is nonexistent.
You can make array fields in your documents called 'users' and 'docs' and add there data with FieldValue.arrayUnion() by triggered functions. With that approach you can retrieve the actual sizes on the client side by getting users & docs fields and calling .size() on it.
You should expect that a background trigger could possibly be executed multiple times per event. This should be very rare, but not impossible. It's part of the guarantee that Cloud Functions gives you for "at-least-once execution". Since the internal infrastructure is entirely asynchronous with respect to the execution of your code on a dedicated server instance, that infrastructure might not receive the signal that your function finished successfully. In that case, it triggers the function again in order to ensure delivery.
It's recommended that you write your function to be idempotent in order to handle this situation, if it's important for your app. This is not always a very simple thing to implement correctly, and could also add a lot of weight to your code. There are also many ways to do this for different sorts of scenarios. But the choice is yours.
Read more about it in the documentation for execution guarantees.
I am building chat application somewhat like whatsapp. I want to show registered app users list from user's device contact list while creating new group. Now in order to do that I have to compare each and every contact number with firebase firestore users. And any normal user can have more than 500 contacts in device. And moreover firestore has limitation to for querying the db so I can not compare more than one number at a time, the whole process takes almost 6-7 minutes as well as each read operation costs financially.
How can I overcome with this situation, or what is the better way to deal with this particular scenario?
You can store the contacts of the user on device and only send them to firestore as backup. You can then sync your local database with firestore on app start.
The operations you need are not possible to be robust in firebase. Even then if you want to do a search in firebase data, you need to use 3rd party search solution like Elastic Search with your firebase data to perform complex searching.
For local database you can use Room library: https://developer.android.com/topic/libraries/architecture/room
For using Elastic Search with Firebase have a look at this utility Flashlight: https://github.com/FirebaseExtended/flashlight .
The OP requested a structure and some code (Swift, Firebase Database) as a solution. I will present two options
If you want to use a Firebase Query to see if the phone numbers exist, a possible stucture would be
users
uid_0
contact_name: "Larry"
contact_phone: "111-222-3333"
uid_1
contact_name: "Joe"
contact_phone: "444-555-6666"
and then the swift code to query for existing numbers
let phoneNumbers = ["111-222-3333","444-555-6666"] //an array of numbers to look for
let myQueryRef = self.ref.child("users")
for contactPhone in phoneNumbers {
let queryRef = myQueryRef.queryOrdered(byChild: "contact_phone").queryEqual(toValue: contactPhone)
queryRef.observeSingleEvent(of: .childAdded, with: { snapshot in
if snapshot.exists() {
print("found \(contactPhone)") //or add to array etc
}
})
}
Having queries in a tight loop like this is generally not recommended but it usually works fine for me with low iterations. However, queries have a lot more overhead than .observers.
IMO, a better and considerably faster option is to keep a node of just phone numbers. Then iterate over the ones you are looking for and use .observe to see if that node exists.
phone_numbers
111-222-3333: true
444-555-6666: true
and then the code to see if the ones from the array exist
let phoneNumbers = ["111-222-3333","444-555-6666"] //an array of numbers to look for
let phoneNumberRef = self.ref.child("phone_numbers")
for contactPhone in phoneNumbers {
let ref = phoneNumberRef.child(contactPhone)
ref.observeSingleEvent(of: .value, with: { snapshot in
if snapshot.exists() {
print("found \(contactPhone)")
}
})
}
In testing, this second solution is must faster than the first solution.
I'm following along the with the basic AngularFire2 docs, and the general format seems to be:
const items = af.database.list('/items');
// to get a key, check the Example app below
items.update('key-of-some-data', { size: newSize });
My confusion is that in the source code, it seems as though calling database.list() grabs all the data at the listed url (line 114 here)
Can anyone help clarify how that works? If it does indeed grab all the data, is there a better way of getting a reference without doing that? Or should I just reference each particular URL individually?
Thanks!
When you create an AngularFire2 list, it holds an internal Firebase ref - accessible via the list's public $ref property.
The list is an Observable - which serves as the interface for reading from the database - and includes some additional methods for writing to the database: push, update and remove.
In the code in your question, you are only calling the update method and are not subscribing to the observable, so no data is loaded from the database into memory:
const items = af.database.list('/items');
// to get a key, check the Example app below
items.update('key-of-some-data', { size: newSize });
It's only when a subscription to the observable is made that listeners for value and the child_... events are added to the ref and the list builds and maintains an internal array that's emitted via the observable. So if you are only calling the methods that write to the database, it won't be loading any data.
The AngularFire2 object is implemented in a similar manner.
Since Firebase security rules cannot be used to filter children, what's the best way to structure data for efficient queries in a basic multi-user application? I've read through several guides, but they seem to break down when scaled past the examples given.
Say you have a basic messaging application like WhatsApp. Users can open chats with other groups of users to send private messages between themselves. Here's my initial idea of how this could be organized in Firebase (a bit similar to this example from the docs):
{
users: {
$uid: {
name: string,
chats: {
$chat_uid : true,
$chat2_uid: true
}
}
},
chats: {
$uid: {
messages: {
message1: 'first message',
message2: 'another message'
}
}
}
}
Firebase permissions could be set up to only let users read chats that are marked true in their user object (and restrict adding arbitrarily to the chats object, etc).
However this layout requires N+1 selects for several common scenarios. For example: to build the home screen, the app has to first retrieve the user's chats object, then make a get request for each thread to get its info. Same thing if a user wants to search their conversations for a specific string: the app has to run a separate request for every chat they have access to in order to see if it matches.
I'm tempted to set up a node.js server to run root-authenticated queries against the chats tree and skip the client-side firebase code altogether. But that's defeating the purpose of Firebase in the first place.
Is there a way to organize data like this using Firebase permissions and avoid the N+1 select problem?
It appears that n+1 queries do not necessarily need to be avoided and that Firebase is engineered specifically to offer good performance when doing n+1 selects, despite being counter-intuitive for developers coming from a relational database background.
An example of n+1 in the Firebase 2.4.2 documentation is followed by a reassuring message:
// List the names of all Mary's groups
var ref = new Firebase("https://docs-examples.firebaseio.com/web/org");
// fetch a list of Mary's groups
ref.child("users/mchen/groups").on('child_added', function(snapshot) {
// for each group, fetch the name and print it
String groupKey = snapshot.key();
ref.child("groups/" + groupKey + "/name").once('value', function(snapshot) {
System.out.println("Mary is a member of this group: " + snapshot.val());
});
});
Is it really okay to look up each record individually? Yes. The Firebase protocol uses web sockets, and the client libraries do a great deal of internal optimization of incoming and outgoing requests. Until we get into tens of thousands of records, this approach is perfectly reasonable. In fact, the time required to download the data (i.e. the byte count) eclipses any other concerns regarding connection overhead.