I have a social network app.
I need to query the last Posts ordered by date from any people that i follow.
My working query so far is :
Query getRecentFriendsPost(List<String> followings, int postedDate) {
return getUsersCollection()
.where("uid", whereIn: followings)
.where("postedAt", isGreaterThanOrEqualTo: postedDate)
.orderBy("dateCreated", descending: true)
.limit(16);
}
This doesn't answer my needs unfortunately.
The followings list contains the userIDs that i follow, and the length can grow until thousands. Firestore support only 10 items in whereIn clause.
Second problem, if the length is 1000, it will cost 100x10 queries to split the followings list into block of 10 uids.
This is very expensive and slow.
I must use the whole followings list to be sure to not miss the most recents posts.
How can i perform such a query with Firestore ?
The followings list allows me to query the User object associated to the Post, if a Post is retrieved by the query.
Should i go for RealTime Database in such situation ? (If i should even keep Firebase products)
I did not find a better architecture either, if anyone knows how to improve this it would be much appreciated.
I have migrated my database to MongoDB and i am able to do so in one query.
Performances are not impacted before passing an array of 6000 values, after that it starts giving results in more than 100ms which remains perfectly fine for my use case.
Related
I am new in Flutter - Firestore
I am learning flutter with firebase and creating a sample dating app
I have a list of users that I get in a stream and display it using List view
Firestore.instance.collection('users').snapshots()
I have learnt to filter this like so
.where((user) => user.age < settings.agemax && user.age > settings.agemin))
and all this works.
I also have a subcollection called shortlist (list of users that current user has shortlisted) that I get using,
Firestore.instance.collection('users').document(uid).collection('shortlist').snapshots()
Now I am trying to redefine my first query GetUsers with filters based on following
How do I exclude shortlisted users that I am fetching in a stream from all users stream
Similarly would also need to filter out "matched users" and "Blocked / declined" users as well !
I believe my question is how do I query Users Collection and exclude records with uid's that contained in a Shortlist subcollection. I am planning to use the same logic for matches and blocked !? am I on the right track ?
also ... do I need to refetch all records when a users shortlists/matches/blocks someone, as the stream would change or is there a way to remove that one record from the listview without rebuilding, may be I should separate this question in two.
If I understand correctly you are looking for the (just introduced) not-in operator, so I recommend also checking out this blog post.
I expect that this operator hasn't landed in the Flutter libraries yet, as that may take some time. I recommend checking the upcoming releases to see when it lands, or checking/filing an issue on the repo.
Until then, there's no way to exclude results from a query, so you will have to exclude the items from the stream of results in your application code.
I tried my best with the search function and not really getting the results I´m searching for.
At the moment I try to replicate a functionality for swiping profiles on a card stack. The profiles are loaded over a Firebase Firestore backend. The problem is, due to the geoFireStore query, I get with every load within a specific radius a whole bunch of documents and I would like to reduce the read amount directly in the query.
So what I'm trying to achieve is, that when I swiped a profile once (left or right) I never want to read it in the query again. In addition, to save reads, I don't want to do this client sided, it should be done in the query directly.
Firestore JSON structure at the moment:
Users (collection)
UIDs [documents]
name
active
match {map}
If the match map doesn't contain the own UID (if exist is null) then read in query, else when own UID exist in other user profile under the map match (boolean: true or false for liked or disliked) then don't show as query result.
My stream is built this way:
return geo
.collection(
collectionRef:
friendrCollection.where("active", isEqualTo: true).where("match.$uid", isNull: true))
.within(
center: geoFirePoint,
radius: suchRadius,
field: 'position',
strictMode: false)
.map(_friendrsGPSListFromSnapshot);
}
I am working on this for 3 weeks now and not getting the results I want :D
-Should I stay with firestore or better the real-time-database?
-How do I need to structure the data to get a "except" query working?
Right now it is like:
Collection: Users --> Documents: UIDs --> Match{ownUID}
Thanks in advance
Andreas :)
I still not managed to find a proper solution to reduce the number of unnecessary profile reads inside the firestore query.
Has anyone an idea to handle this part?
The only way I found was to lookup existing matches on the client side (extra firestore collection), but that means that I always have to load the whole bundle of profiles and throwing them away afterward :(
Thank you!
Andreas
I am rendering the following view using Firebase. So basically the search is powered by a Firebase query.
I am using the following code:
Query query = FirebaseUtils.buildQuery(
fireStore, 'customers', filters, lastDocument, documentLimit);
print("query =" + query.toString());
QuerySnapshot querySnapshot = await query.getDocuments();
print("Got reply from firestore. No of items =" + querySnapshot.documents.length.toString());
Questions:
If the user hits the same query, again and again, it still hits the server. I checked this by using doc.metadata.isFromCache and it always returns false.
Will using query snapshots help in reduce no of reads for this search query? I guess no. As the user is changing the query again and again.
Any other way to limit the number of reads?
If the user hits the same query, again and again, it still hits the server. I checked this by using doc.metadata.isFromCache and it always returns false.
If you are online, it will always return false and that's the expected behavior since the listener is always looking for changes on the server. If you want to force the retrieval of the data from the cache while you are online, then you should explicitly specify this to Firestore by adding Source.CACHE to your get() call. If you're offline, it will always return true.
Will using query snapshots help in reduce no of reads for this search query? I guess no. As the user is changing the query again and again.
No, it won't. What does a query snapshot represent? It's basically an object that contains the results of your query. However, if you perform a query, "again and again", as long as it's the same query and nothing has changed on the server, then you will not be charged with any read operations. This is happening because the second time you perform the query, the results are coming from the cache. If you perform each time a new search, you'll always be billed with a number of read operations that are equal with the number of elements that are returned by your query. Furthermore, if you create new searches and the elements that are returned are already in your cache, then you'll be billed with a read operation only for the new ones.
Any other way to limit the number of reads?
The simplest method to limit the results of a query is to use a limit() call and pass as an argument the number of elements you want your query to return:
limit(10)
I have an app that returns a list of health foods. There will be approximately 10000-20000 foods (documents) in the product collection.
These foods are queried by multiple fields using arrayContains. This may be categories, subcategories and when the user searches in the search bar it is an arrayContains on the keywords array.
With so many products I plan to paginate the results of query as I get the documents. The issue is that I need to know the amount of results to display the total of results to the user.
I have read that for a query you are charged one read and then if you get the documents then they are further charged per document. Is there a way of getting the number of results for a query without getting all the documents.
I have seen this answer here:
Get size of the query in Firestore
But in this example they say to use a counter which doesn't seem practical as I am using a query on keyword when the user searches and I am using a mixture of categories, subcategories when the user filters.
Thanks
With so many products I plan to paginate the results of query as I get the documents.
That's a very good decision since getting 10000-20000 foods (documents) at once is not an option. Reason one is the cost, it will be quite expensive and second is that you'll get an OutOfMemoryError when trying to load such enormous amount of data.
The issue is that I need to know the amount of results to display the total of results to the user.
There is no way in Firestore so you can know in advance the size of the result set.
Is there a way of getting the number of results for a query without getting all the documents.
No, you have to page through all the results that are returned by your query to get the total size.
But in this example they say to use a counter which doesn't seem practical as I am using a query on keyword when the user searches
That's correct, that solution doesn't fit your needs since it solves the problem of storing the number of all documents in a collection and not the number of documents that are returned by a query. As far as I know, it's just not scalable to provide that information, in the way this cloud hosted, NoSQL, realtime database needs to "massively scale".
For any future lurker, a "solution" to this problem is to paginate results with a cursor until the query doesn't return any more documents. When the query snapshot is empty, return undefined for your cursor and handle from there:
const LIMIT = 100
const offset = req.query.offset
const results = db.collection(COLLECTION)
.offset(offset)
.limit(LIMIT)
.get()
const docs = results.docs.map(doc => doc.data())
res.status(200).send({
data: docs,
// Return the next offset or undefined if no docs are returned anymore.
offset: docs.length > 0 ? offset + LIMIT : undefined
})
I want to get the total number of the documents that exist in firestore.
I don't want to get the data only the total number of inside Products collection I have 200.000 items is that possible with Angular 4-5, not angular.js
Can someone expert tell me how I can achieve that ??
My code so far and is not work
get_total_messages() {
this.messages_collection = this.afs.collection<MessageEntity>('messages');
return this.messages_collection.snapshotChanges();
}
End this is how I try to get the data but is not what I want;
this.firebase_Service.get_total_messages().subscribe( data => {
console.log(data);
});
There is no API to get the count of the number of documents in a Firestore collection. This means that the only ways to get the count are:
Get all documents and count them client-side.
Store the count as a separate property and update that as you add/remove documents.
Both approaches are quite common in NoSQL databases, with the second of course being a lot more efficient as the number of documents grows.
Firebase provides a sample of using Cloud Functions to keep a counter. While this sample is written for the Firebase Realtime Database, it can easily be modified to work on Cloud Firestore too.
Firestore also provides documentation on running aggregation queries and running distributed counters. Both seem slightly more involved than the first sample I linked though.
this.firebase_Service.get_total_messages().subscribe( data=>this.totalnumber=data.length);
//now, you can get total number of messages
luckily , i've solved somehow using the code,
try this, and it works well .
this.db.collection('User').valueChanges()
.subscribe( result => {
console.log(result.length);
})