Normally, we use StreamProvider to deal with the data from FireStore in Flutter project like this:
// I have a collection of Customer in my DataService
Stream<List<Client>> streamCustomers() {
return Firestore.instance.collection('customers').snapshots().map((list) =>
list.documents.map((doc) => Customer.fromMap(doc.data, doc.documentID)).toList());
}
This is the Provider of stream data:
Expanded(
child: StreamProvider<List<Customer>>.value(
value: _dataSvc.streamCustomers(),
child: CustomerListWidget(),
),
);
This is where the stream data consumed:
final _customers = Provider.of<List<Customer>>(context);
return Container(
child: _customers == null? Text('Loading...') : _buildList(context, _customers),
);
I'll show all the customer data in CustomerListWidget. Because the data of Customer collection is very big (more the 10,000 - 50,000 records). Apparently this is not efficient solution. I'm wondering normally what kind of the practical solutions are used to deal with this scenario in Flutter/Firestore project?
P.S.: Pagination is definitely one of my possible choice. But there are some issues because I'll apply some filters to the data. For example, I have to query to Firestore every time when the filter criterio has been changed which will cause the data usage increased. And also it seems I can only use getDocuments() instead of snapshot to get the Stream data.
I think using Pagination is which better fits for you. Implementing pagination in Flutter using Firebase Cloud Firestore database can improve your app performance and reduce bandwidth usage (which is one of your biggest concerns). There are some articles about this.
Related
I have seen a few articles written online by people pertaining to be able to do this, but they only tell you how to do this with a specific, controlled list of images where they also know all the filenames beforehand.
There is also this "answer" posted here: Flutter - Get all images from firebase storage which does not actually resolve this issue at all as it suggests a .listAll() method for the recommended plugin, but there is no such method as .listAll() using the suggested plugin.
I need to be able to not know how many images are in Firebase storage, or what they might be called, just to return everything stored there.
UPDATE:
So because Firebase is full of so many limitations that it hardly qualifies as a database at all, it seems we may have to keep the images in Firebase Storage and a reference list of these in a Firebase Realtime Database Document.
I am stuck on the actual implementation of this however, as I am not even sure firstly how best to go about this. What I am attempting to do is store all the Storage image URLs in an array (if this is not the best way to do this, let me know!):
Firebase Structure
collection -> document -> fields
userData profileImages URLs (array)
My first issue is that I don't know how to append new data to the existing array, it seems to just overwrite it each time I add a new item so that I only ever have one string in the array in the database:
Firestore.instance.collection('userData').document('profileImages').updateData({
'URLs': _uploadedFileURL,
});
Then after this I am also not sure how to actually retrieve the full array of data later when I need it:
Container(
child: StreamBuilder(
stream: Firestore.instance.collection('userData').document('profileImages').snapshots(),
builder: (BuildContext context, AsyncSnapshot<DocumentSnapshot> snapshot) {
if (!snapshot.hasData) {
return LoadingAnimationBasic();
}
if (snapshot.data.data == null) {
return LoadingAnimationBasic();
} else {
return ListView(
shrinkWrap: true,
children: _buildProfileGallery(snapshot),
);
}
},
),
),
And then the function:
_buildProfileGallery(AsyncSnapshot<DocumentSnapshot> snapshot) {
int test = snapshot.data["URLs"].length;
print('URLs in List: ' + test.toString());
return snapshot.data.data.map(???);
}
I have no idea what to put as the parameters of this map, as the hint text is insane:
MapEntry(K2, V2> f(String key, V value)
I can't even begin to guess what this means.
Am I on the right track? Am I on the right planet?
The ability to list files was added to Firebase's client-side SDKs for Android, iOS and JavaScript last year, but has not yet landed in the FlutterFire bindings.
The answer to the question you linked, refers to a pull request on the FlutterFire open-source project that adds this functionality. So while somebody wrote code to allow listing of files in Flutter too, this code hasn't been added to a release so far. So the only way to use that code now, is to build your own version of the FlutterFire library for firebase_storage.
Without the ability to list files in the Firebase Storage API, you'll have to revert back to what everyone did before this feature was added: keeping a list of the files in a secondary location, such as the Firebase Realtime Database or Cloud Firestore.
Does anyone know how to limit an array so new items get pushed in and old ones are discarded in the same write?
I'm guessing this isn't possible but it sure would be handy.
// * Store notification
// Users collection
const usersCollection = db.collection('users').doc(uid).collection('notifications').doc();
// Write this notification to the database as well
await usersCollection.update({
count: admin.firestore.FieldValue.increment,
notifications: admin.firestore.FieldValue.arrayUnion({
'symbol': symbol,
'companyname': companyname,
'change': priceDifference,
'changeDirection': directionOperatorHandler,
'updatedPrice': symbolLatestPrice,
'timestamp': currentTimestamp,
})
});
Written in Typescript
Alternatively, I was thinking of running a scheduled cloud function every week to go through and trim down the arrays based on the timestamp.
The reason I'm using an array to store my notifications is because I'm expecting a lot of writes.
There is no simple configuration for this. Your code should implement your requirements by:
Reading the document
Modifying the array in memory
Checking that the size is within limits
Writing the document back
I have a Flutter project that uses Cloud Firestore. In one of my widgets, I have a StreamBuilder that ideally reads snapshots from the database. In theory, my API for reading from a remote server should be abstract enough to swap Firestore out with a different implementation.
class Database {
Stream<QuerySnapshot> get snapshots => Firestore.instance.collection('entires').snapshots();
Stream<List<String>> get entries => snapshots.map((snapshot) => snapshot.documents.map((document) => document.data['name']).toList());
}
If my StreamBuilder uses snapshots, then AsyncSnapshot<QuerySnapshot> has data (hasData returns true).
If my StreamBuilder uses entries, then AsyncSnapshot<List<String>> will never have data (hasData returns false)---even if I successfully printed out what the data is (my return result is a populated list of strings).
I hope to keep my Database interface free of Firestore. So, my question is: why does StreamBuilder's AsyncSnapshot return nothing even if I have data?
It seems that the issue revolves around snapshot() sending data off immediately. I managed to create a work around by creating a class that wraps my Firestore collection.
This wrapper has its own StreamController<T> where T is the non-Firestore (abstract across multiple backend implementations) data type that I want to return.
The wrapper will listen on the necessary Firestore snapshots on construction. The latest snapshot result is cached. My StreamBuilder can then use both the internal StreamController.stream and the cached snapshot data for its construction.
StreamBuilder(
stream: _wrapper.stream,
initialData: _wrapper.latestData,
)
This works for now, but has the side effect of continuously listening on snapshots indefinitely. I'll have to update my wrapper to stop listening on snapshots when not necessary (no subscribers) and re-listen when there are new subscribers, or simply use a mapping of the snapshot stream.
I am building chat application somewhat like whatsapp. I want to show registered app users list from user's device contact list while creating new group. Now in order to do that I have to compare each and every contact number with firebase firestore users. And any normal user can have more than 500 contacts in device. And moreover firestore has limitation to for querying the db so I can not compare more than one number at a time, the whole process takes almost 6-7 minutes as well as each read operation costs financially.
How can I overcome with this situation, or what is the better way to deal with this particular scenario?
You can store the contacts of the user on device and only send them to firestore as backup. You can then sync your local database with firestore on app start.
The operations you need are not possible to be robust in firebase. Even then if you want to do a search in firebase data, you need to use 3rd party search solution like Elastic Search with your firebase data to perform complex searching.
For local database you can use Room library: https://developer.android.com/topic/libraries/architecture/room
For using Elastic Search with Firebase have a look at this utility Flashlight: https://github.com/FirebaseExtended/flashlight .
The OP requested a structure and some code (Swift, Firebase Database) as a solution. I will present two options
If you want to use a Firebase Query to see if the phone numbers exist, a possible stucture would be
users
uid_0
contact_name: "Larry"
contact_phone: "111-222-3333"
uid_1
contact_name: "Joe"
contact_phone: "444-555-6666"
and then the swift code to query for existing numbers
let phoneNumbers = ["111-222-3333","444-555-6666"] //an array of numbers to look for
let myQueryRef = self.ref.child("users")
for contactPhone in phoneNumbers {
let queryRef = myQueryRef.queryOrdered(byChild: "contact_phone").queryEqual(toValue: contactPhone)
queryRef.observeSingleEvent(of: .childAdded, with: { snapshot in
if snapshot.exists() {
print("found \(contactPhone)") //or add to array etc
}
})
}
Having queries in a tight loop like this is generally not recommended but it usually works fine for me with low iterations. However, queries have a lot more overhead than .observers.
IMO, a better and considerably faster option is to keep a node of just phone numbers. Then iterate over the ones you are looking for and use .observe to see if that node exists.
phone_numbers
111-222-3333: true
444-555-6666: true
and then the code to see if the ones from the array exist
let phoneNumbers = ["111-222-3333","444-555-6666"] //an array of numbers to look for
let phoneNumberRef = self.ref.child("phone_numbers")
for contactPhone in phoneNumbers {
let ref = phoneNumberRef.child(contactPhone)
ref.observeSingleEvent(of: .value, with: { snapshot in
if snapshot.exists() {
print("found \(contactPhone)")
}
})
}
In testing, this second solution is must faster than the first solution.
I am use cloud Firestore to store users data and my question is:
Is it possible to know how many documents are stored in specified collection. If possible write
Firebase Firestore almost always return entire collection, including all its documents. Hence, you just count the retrieved data. The code would depend on the language/framework you use, but all methods are well described #official docs, if you have managed to retrieve data, you should be able to just count its items. Let say you need a service for Angular2+ using AngularFire2 and you like to use valueChanges() method. Once you map the retrieved data, you count its items. The code might look like this (ideally, you should add collection/document interfaces as well ):
import { Injectable } from '#angular/core';
import { AngularFirestore } from 'angularfire2/firestore';
#Injectable()
export class YourService {
constructor( private _fireStore: AngularFirestore ) { }
getSomething = () => {
const collection_reference = this._fireStore.collection('logs');
const collection_data = collection_reference.valueChanges();
collection_data.map( data => {
const number_of_documents = data.length;
} ).subscribe((query) => {});
}
}
There are no built-in aggregation queries in Cloud Firestore. So to get the count of documents in a collection, you will indeed either have to retrieve all those documents (as Emocuc answered) or keep the count of the number of documents in the database somewhere.
With the second approach you update a counter in the database, any time you add or remove a document. This adds some work to the write operations, but after to show the number of documents you simply can read the counter (instead of reading all documents). This kind of trade-off (more complex writes for a faster/simpler read) is very common in NoSQL database and is one of the reasons they can scale to so many readers.
The Cloud Firestore documentation has some examples of these sort of aggregation operations that I recommend you check out. While the samples are more complex than what you need, they are well documented and should be a useful starting point.