Just implemented "startAfter" for the first time. However, a bit of searching around and I cannot find an answer to an issue I have.
How best to handle when there are no more documents to load when using "start after".
Should I grab the collections total document length each time I fire a "load more" function. Check total vs current and if they are equal, disable the load more function?
If so, how do I find the total count of the collections documents?
Should I grab the collections total document length each time I fire a "load more" function.
No. When we create a pagination algorithm, we are always interested to load data in smaller chunks. In Firestore, we are always also interested in the size of each new chunk that we get. So basically, if the new chunk has the size that is smaller than the limit we have set, for example, 10 items per page, then that's the moment when you should disable "load more function".
So the idea is that you can request pages of data of a certain size. Here's a helpful method named limit(X) that can help you achieve this. So you can perform a query and get a page of size X. Simply continue that query with another page of size X, and so on. So you start on page 1, then progress through the pages using the corresponding methods where you should specify which document was the last one on the page and then continue to get the next page until there are no elements left.
If so, how do I find the total count of the collections documents?
Isn't about the number of documents in a collection that you have to worry it's about the number of the items you get in the next page.
What you can do is save the last visible document to a variable, like so
lastVisible = querySnapshot.docs[querySnapshot.docs.length - 1]
Related
In my app, a user can log in, then a list of their "pages" (a collection of documents) is retrieved, and their "selectedPageId" is retrieved. This way the UI can show a list of page names, and the content of the selected page.
This means three requests have to happen, each waiting for the other:
log in
get list of pages
get selected page id
Is there a way to make all this happen on the server at once so I can make a single request, which I assume would make my UI render more quickly?
As Dharmaraj mentioned in his comment, "Firebase Auth will still be required on client side". However, is you use the default Auth state persistence (i.e. firebase.auth.Auth.Persistence.LOCAL) once a user is signed in he/she doesn't need to sign in the next times ("the state will be persisted even when the browser window is closed. An explicit sign out is needed to clear that state"). So signing in is not really a problem IMO.
For the Firestore queries: Executing the two queries back-to-back with the second one returning only one document should not take a lot of extra time compare to executing only the first query (which fetches the pages list).
Having said that, one possible approach would be to save, for each user, a Firestore document containing the list of the user's pages plus the user's selectedPageId. You can maintain this page with a Cloud Function which, for example, mirrors the list (documents in a collection) with an array in this Firebase doc. However, you need to take care to not reach the maximum size for a document i.e. 1MiB (hence my question on the number of fields displayed for a page).
One possible variation in this case would be to initially present, in your first app page, a limited number (e.g. 25) of pages of the list, with a "Show more" button.
Finally, note that using a Cloud Function as mentioned in the above comments will very most probably increase the response time compare to standard queries executed via the SDK.
I'm working on a Flutter application that basically allows users to place orders to restaurants then go and pick-up those orders.
A restaurant has a List of MenuGroups and each group has a List of ExtraIngredients and List of MenuItems.
A MenuItem has several variants with different prices also List of Ingredients that come with that item and ExtraIngridients that can be added.
Currently, in firestore I've a collection called restaurants and each restaurant has a List of MenuGroups. Is there a way to make this more efficient
For example, is it better to do the menuGroups as a subcollection in the document?
Also to implement an order queue number system (first order starts from 1 goes to 99 then goes back to 1)
Is it better to store that in a variable in restaurant document(Whenever there is a new order there will be 1 read to get the current number than 1 write to increase that number and also after reaching 99 to set it back to 1)
or in the order document itself (Now each order has an extra field 1 read to get the last order's number and the new order will be written all together so there is no extra write operation just for the queue number)
There is no certain way to answer this. But there are some rules I would say to tackle this efficiently.
Put Data in a same document if you want to show it together. (Not too big, neither too small)
Put data in collections, when you want to search an individual piece of that data, or you have a database to grow.
Use map if you want to search a parameter based on that data.
use map if you want to store related data ( Like delivery addresses of the user).
Document write doesn't count on data you wrote, no matter if you increase your order counter value by 1, or change the whole document, it will count as one write.
In the Project Tracker template there is a feature where statistics for a certain project's items are displayed. You can filter the project items, but the statistics will only show those statistics for all the project items, i.e. the filters do not affect the statistics.
I would like to add the feature of filters affecting those statistics in a similar implementation of mine. My current solution passes the keys of those project items (affected by filters, too) to a calculated data source, which then calculates the statistics using those item keys, essentially applying the filters used in the page.
My issue is that my calculations are restricted by the query page size. For example, if I apply filters that limit the number of items to 15 records, but the page size is 10 records, I will only have statistics of those first 10 items, which is not useful. I'd need to have the statistics on all the records that are left after filtering.
One way to solve this would be to get rid of the query page size and leave it at 0. However, similarly to the Project Tracker template, I'm displaying the project items on the page in a table, and if I do that, the page becomes too heavy.
How can I circumvent the query page size? I'm thinking I could
limit the items displayed in the page by some other way than query page size (i.e. hiding items from the UI)
use a different datasource for the statistics, but in some way copying the filters used in the datasource that is displaying the project items
Both of these ways I could think of, I can't seem to implement. I don't know how I could hide items from the UI to make it less heavy, as query page size pretty much does it. I have also tried copying the filters from a datasource to a similar one, but that does not seem to work.
EDIT: I might have come up with a way to solve this myself, but I still need to implement it. Now I'm using the page size restricted Items Ds to apply the filters on, and the statistics are build from this data source. If instead, I use a non-restricted Ds called AllItems, and apply the filters on it, and then pass the item keys to a page size restricted Ds (to show the items in UI) AND to a calculated Ds (for the stats). Will make a response once I've verified it works.
I solved the issue myself.
To produce (refreshable) statistics subject to filtering but not restricted by page size, I used the following datasource structure:
In this structure, the filters practically flow to the data sources below, as they are passed the ItemKeys that fit the filter. What this completes is that my Statistics (used in piegraphs etc.) can be filtered dynamically and account for all the records that fit the filter, while the UI does not get crowded over too many records, as the data source used in UI has a query page size limitation.
I'm trying to find a way to properly desing my Data Model with Firestore. I'm looking for something similar to what Tinder does, showing you people that you have'nt swiped yet, based on your location.
So I ended up with something like :
A User1 has an array of "met people"
A "Haven't yet met user"/ User2 his also a User with the same document model
They all belong in the same "Users" collection
I want to query all the users that this User1 haven't swiped yet
I know that you can't do something like "array_not_contains" or "!=" because all fields that you query need to be indexed.
So I wonder, is this possible to model data to make it work, or the only solution is to drop Firebase because this kind of query is not possible at all?
One alternative can be to store in a collection all the relationships (with theirs status) between all users. But that also means that whenever a user signup, I have to create as many documents as I have users that's really ugly and make a enormous numbers of documents.
EDIT:
Thanks again for your answer and sorry for my late answer.
There is no need to create a new database call since you already got all the users from that area in the first place.
Not If have a large response set, I will limit to a number. (5 in the example below).
And even If I don't limit the number, in the next db call, how I can know that new peoples has been added and how to retrieve only those.
I will not remove them from Users Collection has they can be show to others users.
P.S: I forget User4 in Users Collection pictures.
For User 1, get 5 first matchs, remove existing ones, show User5.
For User2, get 5 first matchs, remove existing ones, show User4, User5.
After users choices, Users are added to their list. Users Collection stay the same.
For User 1, get 5 first matchs, remove existing ones, nothing to show, even if I have a User 6, 7.
To fix that I launch a second query get the new ones but, more the user use the app more query I may need to do to try to display to him existing user in his area.
Maybe I've misunderstood what you named "initial list", for me it is the list object retrieve from my db containing all users (with limit).
EDIT 2:
You can check the answers of Alex Mamo to know how to query documents that are not exist in an array possible.
Let's me explain my use case and why I think, that won't work.
I want to be able to search all users next to me, for trying to do that in Firebase, I store Geopoint. Geopoint can't be really use for now out of the box with Firebase, so I user Geofirestore in a Cloud Function.
I store and update user Geopoints based on theirs locations, so this means user location change by time.
I limit the numbers of Users return by this function.
In my initial state I retrieve users next to me (User1), I get 3 an 4.
Let's say that I store last checked userId to use it later as a cursor for my query (User 4).
Now my geopoint change, and the users in this area changes too.
I request next bunch of users next to me, and I use my previous userId/document to "startAfter" (more on this
here), see the image below, that's won't work.
If I use the cursor (User4), I'll take 5, but not 2, because in the return list, if I order by Id, 2 will be before 4.
Worse, like below, if the return list may not even have user 4 in it, the cursor will be pointless.
My example is a bit simplified and does not take in account what is described in the first answer and my first edit (limited subset of users, data design).
A possible database structure for your app might be:
Firestore-root
|
--- users (collection)
|
--- uid (document)
|
--- acceptedUsers: ["uidOne", "uidTwo"]
|
--- declinedUsers: ["uidThree", "uidFour"]
|
--- //Other user properties
The mechanism is simple. When you first want to show a user profile to the current (authenticated) user, you have to create a query that will return all users (in user area). According to the user decision, you need to add the corresponding uid in either the acceptedUsers array or in declinedUsers array. Once you want to show another users, use the same query but this time, you need to make an extra operation. Once the query returns the users within user location, add all those users to a list. Compare the list that is coming from the database with your exting arrays and remove all the users from both arrays. In this way you'll have a list that contains only users that the actual user didn't see. This extra step is needed to make sure the id of the user does not exist in one of those arrays. In the end, simply choose a random user from the list and show the details to the user. That's it!
One alternative can be to store in a collection all the relationships (with theirs status) between all users. But that also means that whenever a user signup, I have to create as many documents as I have users.....that's really ugly and make a enormous numbers of documents.
This is not an option. This means that you need to write each time a user joins your app an enormous amount of data, which will be very costly. Since everything in Firestore is about the number of read and writes, I think you should think again about this approach. Please see Firestore usage and limits.
Edit:
Let's consider the initial list of users that has 10 records. With other words, all the users within that area are 10. You say that 7 users are already seen, that makes the list contain only the 3 remaining users.
So I display the 3, (or I do another request to get some more) and he check the 3.
Yes, you should display those 3 users and then remove them one by one from the initial list. There is no need to create a new database call since you already got all the users from that area in the first place. Once the list remains empty, you should display a message to the user that in that particular area are no more users to swipe.
When will create another database call?
Only when needed. Which means that you create another call once new users enter that area. Let's say 3 new users are new, you get a list now of 3 user and use the same algorithm.
More my user use the app more it’s difficult to show people that he haven’t seen, because his list become bigger.
If you think that the arrays will grow more than a document can hold, then you should consider storing the users in a collection and not in an array. So in this case, the problem is that the documents have limits. So there are some limits when it comes to how much data you can put into a document. According to the official documentation regarding usage and limits:
Maximum size for a document: 1 MiB (1,048,576 bytes)
As you can see, you are limited to 1 MiB total of data in a single document. When we are talking about storing text (uids), you can store pretty much but as your array getts bigger, be careful about this limitation.
But if you'll stay within this limits, which I personally think you'll do, you have nothing to worry about.
Edit2:
Not If have a large response set, I will limit to a number. (5 in the example below). And even If I don't limit the number, in the next db call, how I can know that new peoples has been added and how to retrieve only those.
I will not remove them from Users Collection has they can be show to others users.
If you have large amount of data (many users in a single area), yes it's good idea to limit the results, but a much better idea would be to load the data in smaller chunks. In short, get 5 users, remove one by one till the list has zero users, load other 5 users and so on. This can be made using my answer from the following post:
Is there a way to paginate queries by combining query cursors using FirestoreRecyclerAdapter?
The initial list, is the list that you are getting when you first query the database. In this case, the initial list will contain 5 users.
I recently got more results than I bargained for when testing my cordapp with the vault.trackby function and blew the default page specification limit.
So to confirm my understanding of how the page specification works with trackby in relation to the initial snapshot, here's a scenario and a couple of questions.
Lets say I have a 1000 states in the vault that meet my query criteria.
So I call vault.trackby with a page specification where the pageNumber is 1 and the page size is 250
I assume I will get the first 250 results with the initial snapshot, is this correct?
Will the remaining 750 results comeback with the updates? Or will only new updates come back. Leaving the other 750 states only accessible by querying again using trackBy or queryBy, page 2,3,4 etc.
I hope that makes sense.
Thanks
Specific answers to your questions:
1) Correct
2) Only new updates will be streamed back, and you would need to issue another queryBy (do not use trackBy again) explicitly asking for page 2, 3, 4.
However, beware that subsequent queries (eg. for pages 2, 3, 4) may include new results - including already streamed updates from the initial trackBy query - as each snapshot query is unique at a specific moment in time.
The intended usage pattern of trackBy is to obtain a bounded snapshot (defined by the associated Query Criteria) that fits in a single page, and thus guarantee atomicity of streaming updates (eg. no duplicates).
A recommended usage pattern is to:
a) first issue a snapshot (queryBy) query using some bounded criteria (eg. all entries older than 24 hours), followed by a
b) second streaming (trackBy) query using a constrained bounded criteria (eg. all entries in the last 24 hours)
So in summary, the pagination semantic of trackBy should be used with caution and with the understanding that subsequent page requests may contain different results (based on any CRUD operations effected since the original trackBy query)