I'm using flutter and firebase. I use pagination, max 5 documents per page. How do I know if there are more documents left to get from a firestore collection. I want to use this information to enable/disable a next page button presented to the user.
limit: 5 (5 documents each time)
orderBy: "date" (newest first)
startAfterDocument: latestDocument (just a variable that holds the latest document)
This is how I fetch the documents.
collection.limit(5).orderBy("date", descending: true).startAfterDocument(latestDocument).get()
I thought about checking if the number of docs received from firestore is equal to 5, then assume there are more docs to get. But this will not work if I there are a total of n * 5 docs in the collection.
I thought about getting the last document in the collection and store this and compare this to every doc in the batches I get, if there is a match then I know I've reach the end, but this means one excess read.
Or maybe I could keep on getting docs until I get an empty list and assume I've reached the end of the collection.
I still feel there are a much better solution to this.
Let me know if you need more info, this is my first question on this account.
There is no flag in the response to indicate there are more documents. The common solution is to request one more document than you need/display, and then use the presence of that last document as an indicator that there are more documents.
This is also what the database would have to do to include such a flag in its response, which is probably why this isn't an explicit option in the SDK.
You might also want to check the documentation on keeping a distributed count of the number of documents in a collection as that's another way to determine whether you need to enable the UI to load a next page.
here's a way to get a large data from firebase collection
let latestDoc = null; // this is to store the last doc from a query
//result
const dataArr = []; // this is to store the data getting from firestore
let loadMore = true; // this is to check if there's more data or no
const initialQuery = async () => {
const first = db
.collection("recipes-test")
.orderBy("title")
.startAfter(latestDoc || 0)
.limit(10);
const data = await first.get();
data.docs.forEach((doc) => {
// console.log("doc.data", doc.data());
dataArr.push(doc.data()); // pushing the data into the array
});
//! update latest doc
latestDoc = data.docs[data.docs.length - 1];
//! unattach event listeners if no more docs
if (data.empty) {
loadMore = false;
}
};
// running this through this function so we can actual await for the
//docs to get from firebase
const run = async () => {
// looping until we get all the docs
while (loadMore) {
console.log({ loadMore });
await initialQuery();
}
};
How to write a query like getting the data from Firebase using PHP. Like I have to get all the data which status is 1.
Here is the Code:
According to your snapshot, this should work
var userId = firebase.auth().currentUser.uid;
return firebase.database().ref('/tutor/' + userId).once('value').orderBy('approve_status')
.equalTo('1').then((snapshot) => {
console.log(snapshot.val().bio);
// Data you want...
});
Try the firebase documentation to get more idea about queries here
I have a large Firestore collection with 10,000 documents.
I want to show these documents in a table by paging and filtering the results at 25 at a time.
My idea, to limit the "reads" (and therefore the costs), was to request only 25 documents at a time (using the 'limit' method), and to load the next 25 documents at the page change.
But there's a problem. In order to show the number of pages I have to know the total number of documents and I would be forced to query all the documents to find that number.
I could opt for an infinite scroll, but even in this case I would never know the total number of results that my filter has found.
Another option would be to request all documents at the beginning and then paging and filtering using the client.
so, what is the best way to show data in this type of situation by optimizing performance and costs?
Thanks!
You will find in the Firestore documentation a page dedicated to Paginating data with query cursors.
I paste here the example which "combines query cursors with the limit() method".
var first = db.collection("cities")
.orderBy("population")
.limit(25);
return first.get().then(function (documentSnapshots) {
// Get the last visible document
var lastVisible = documentSnapshots.docs[documentSnapshots.docs.length-1];
console.log("last", lastVisible);
// Construct a new query starting at this document,
// get the next 25 cities.
var next = db.collection("cities")
.orderBy("population")
.startAfter(lastVisible)
.limit(25);
});
If you opt for an infinite scroll, you can easily know if you have reached the end of the collection by looking at the value of documentSnapshots.size. If it is under 25 (the value used in the example), you know that you have reached the end of the collection.
If you want to show the total number of documents in the collection, the best is to use a distributed counter which holds the number of documents, as explained in this answer: https://stackoverflow.com/a/61250956/3371862
Firestore does not provide a way to know how many results would be returned by a query without actually executing the query and reading each document. If you need a total count, you will have to somehow track that yourself in another document. There are plenty of suggestions on Stack Overflow about counting documents in collections.
Cloud Firestore collection count
How to get a count of number of documents in a collection with Cloud Firestore
However, the paging API itself will not help you. You need to track it on your own, which is just not very easy, especially for flexible queries that could have any number of filters.
My guess is you would be using Mat-Paginator and the next button is disabled because you cannot specify the exact length? In that case or not, a simple workaround for this is to get (pageSize +1) documents each time from the Firestore sorted by a field (such as createdAt), so that after a new page is loaded, you will always have one document in the next page which will enable the "next" button on the paginator.
What worked best for me:
Create a simple query
Create a simple pagination query
Combine both (after validating each one works separately)
Simple Pagination Query
const queryHandler = query(
db.collection('YOUR-COLLECTION-NAME'),
orderBy('ORDER-FIELD-GOES-HERE'),
startAt(0),
limit(50)
)
const result = await getDocs(queryHandler)
which will return the first 50 results (ordered by your criteria)
Simple Query
const queryHandler = query(
db.collection('YOUR-COLLECTION-NAME'),
where('FIELD-NAME', 'OPERATOR', 'VALUE')
)
const result = await getDocs(queryHandler)
Note that the result object has both query field (with relevant query) and docs field (to populate actual data)
So... combining both will result with:
const queryHandler = query(
db.collection('YOUR-COLLECTION-NAME'),
where('FIELD-NAME', 'OPERATOR', 'VALUE'),
orderBy('FIELD-NAME'),
startAt(0),
limit(50)
)
const result = await getDocs(queryHandler)
Please note that the field in the where clause and in orderBy must be the same one! Also, it is worth mentioning that you may be required to create an index (for some use cases) or that this operation will fail while using equality operators and so on.
My tip: inspect the error itself where you will find a detailed description describing why the operation failed and what should be done in order to fix it (see an example output using js client in image below)
Firebase V9 functional approach. Don't forget to enable persistence so you won't get huge bills. Don't forget to use where() function if some documents have restrictions in rules. Firestore will throw error if even one document is not allowed to read by user. In case bellow documents has to have isPublic = true.
firebase.ts
function paginatedCollection(collectionPath: string, initDocumentsLimit: number, initQueryConstraint: QueryConstraint[]) {
const data = vueRef<any[]>([]) // Vue 3 Ref<T> object You can change it to even simple array.
let snapshot: QuerySnapshot<DocumentData>
let firstDoc: QueryDocumentSnapshot<DocumentData>
let unSubSnap: Unsubscribe
let docsLimit: number = initDocumentsLimit
let queryConst: QueryConstraint[] = initQueryConstraint
const onPagination = (option?: "endBefore" | "startAfter" | "startAt") => {
if (option && !snapshot) throw new Error("Your first onPagination invoked function has to have no arguments.")
let que = queryConst
option === "endBefore" ? que = [...que, limitToLast(docsLimit), endBefore(snapshot.docs[0])] : que = [...que, limit(docsLimit)]
if (option === "startAfter") que = [...que, startAfter(snapshot.docs[snapshot.docs.length - 1])]
if (option === "startAt") que = [...que, startAt(snapshot.docs[0])]
const q = query(collection(db, collectionPath), ...que)
const unSubscribtion = onSnapshot(q, snap => {
if (!snap.empty && !option) { firstDoc = snap.docs[0] }
if (option === "endBefore") {
const firstDocInSnap = JSON.stringify(snap.docs[0])
const firstSaved = JSON.stringify(firstDoc)
if (firstDocInSnap === firstSaved || snap.empty || snap.docs.length < docsLimit) {
return onPagination()
}
}
if (option === "startAfter" && snap.empty) {
onPagination("startAt")
}
if (!snap.empty) {
snapshot = snap
data.value = []
snap.forEach(docSnap => {
const doc = docSnap.data()
doc.id = docSnap.id
data.value = [...data.value, doc]
})
}
})
if (unSubSnap) unSubSnap()
unSubSnap = unSubscribtion
}
function setLimit(documentsLimit: number) {
docsLimit = documentsLimit
}
function setQueryConstraint(queryConstraint: QueryConstraint[]) {
queryConst = queryConstraint
}
function unSub() {
if (unSubSnap) unSubSnap()
}
return { data, onPagination, unSub, setLimit, setQueryConstraint }
}
export { paginatedCollection }
How to use example in Vue 3 in TypeScript
const { data, onPagination, unSub } = paginatedCollection("posts", 8, [where("isPublic", "==", true), where("category", "==", "Blog"), orderBy("createdAt", "desc")])
onMounted(() => onPagination()) // Lifecycle function
onUnmounted(() => unSub()) // Lifecycle function
function next() {
onPagination('startAfter')
window.scrollTo({ top: 0, behavior: 'smooth' })
}
function prev() {
onPagination('endBefore')
window.scrollTo({ top: 0, behavior: 'smooth' })
}
You might have problem with knowing which document is last one for example to disable button.
I need to add current system time into child data field.
I'm using TypeScript, but this might still give you and idea how you could do it.
My code uses the event.timestamp property to get date and time:
export const onWrite = functions.database.ref('/{databaseName}/{tableName}/{key}').onCreate((event) => {
let ref = event.data.ref;
let isCreate = event.eventType == 'providers/google.firebase.database/eventTypes/ref.create';
ref.once('value').then(e => {
// Cloud functions are sometimes executed with a delay and the record might not exist anymore.
if (e.val() === null) {
return;
}
if (isCreate) {
return ref.update({
'createdDateTime': event.timestamp
});
}
});
});
The created events for clients won't include this added data yet, only a later change event does.
I'm haven't investigated yet if this can be fixed (perhaps by making use of transaction).
I saw your image description and understood u want to add system time into firebase.
If you want to do you can do that by , like below
var fb_db=firebase.database().ref('treeName');
var key=fb_db.push().key;
var updatenode={}
updatenode[key+"\"]= new Date();
fb_db.update(updatenode).then(function(){
alert("Success")
})
I have a Firebase invocation which looks like this,
ref.child(`floorPosts/${this.props.floorName}`)
.orderByChild('numberOfLikes')
.limitToLast(15)
.once('value', (snapshot) => {
var topPosts = [];
snapshot.forEach((childSnapshot) => {
var post = childSnapshot.val();
post.key = childSnapshot.key();
topPosts.unshift(post);
});
this.lastPostKeyTop = topPosts[topPosts.length - 1].key;
this.setState({
topPosts,
isLoading: false
});
});
Now what I'm trying to do, is when the user scrolls to the bottom of the page, I invoke this function.
ref.child(`floorPosts/${this.props.floorName}`)
.orderByChild('numberOfLikes')
.startAt(null, this.lastPostKeyTop)
.limitToLast(15)
.once('value', (snapshot) => {
var topPosts = [];
snapshot.forEach((childSnapshot) => {
var post = childSnapshot.val();
post.key = childSnapshot.key();
topPosts.unshift(post);
});
this.setState({
topPosts: this.state.topPosts.concat(topPosts),
isLoading: false,
});
});
However, it's concatenating the same first 15 results as it initially did. .startAt takes in a a second argument, which (I believe) is the key of the last item which was fetched from firebase the initial fetching. However, this doesn't seem to be working in my case. I have a hunch it's something to do with the snapshot.forEach code.
sorry for the incredibly late answer Tyler
The second argument to startAt() is only used to disambiguate between items that match the first argument. So you must pass in both the value of numberOfLikes to start at and the key of the last item on the first page (often referred to as the anchor item).
Also note that this means the first item of the next set of results will be the same as the last item in the previous set of results. So you'll need to retrieve one extra item.