I am relatively new to the Flutter world and need some help in efficiently querying a nested Firebase query with pagination in Flutter.
I am building an app where I have a few thousand of user selected documents in collectionA (selected from >100k documents from collectionB) and need to loop through a chunk of 10 documents (whereIn limit of 10) queried via stream. My current implementation (snippet below) takes a list of document ID's "d" and chunks in a list of 10 via quiver/iterables. The drawback of the methodology is that it reads thousands of user documents upfront before displaying it in the app. I would like to use a pagination and control the reads.
Would you suggest a solution (with a code snippet) on how to use the pagination through thousands of user selected documents?
Future<List<Article>> fbWhereGT10Article(
String collection, String field, List<dynamic> d) async {
final chunks = partition(d, 10);
// print(chunks);
final querySnapshots = await Future.wait(chunks.map(
(chunk) {
Query itemsQuery = FirebaseFirestore.instance
.collection(collection)
.where(field, whereIn: chunk);
// .orderBy('timestamp', descending: true);
return itemsQuery.get();
},
).toList());
return querySnapshots == null
? []
: await Stream.fromIterable(querySnapshots)
.flatMap((qs) =>
Stream.fromIterable(qs.docs).map((e) => Article.fromFirestore(e)))
.toList();
}
Related
In the 1st screen shot there are many documents in collection users. Each documents contains further collection jobPost and that collection contains further documents and its meta data.
What I want here is go to the every document of collection users and further subcollection jobPost and fetch all the documents.
Suppose first it should go to document 1 in collection users, in the document 1 it should fetch all the documnets in subcollection jobPost then it should go to the 2nd document of collection users and then get all the documents in the subcollection jobPost and so on. what will be the query or implementation to this technique
What you're describing is known as a collection group query, which allows you to query all collections with a specific name. Unlike what the name suggests, you can actually read all documents from all subcollections named jobPost that way with:
FirebaseFirestore.instance.collectionGroup('jobPost').get()...
When performing a query, Firestore returns either a QuerySnapshot or a DocumentSnapshot.
A QuerySnapshot is returned from a collection query and allows you to inspect the collection.
To access the documents within a QuerySnapshot, call the docs property, which returns a List containing DocumentSnapshot classes.
But subcollection data are not included in document snapshots because Firestore queries are shallow. You have to make a new query using the subcollection name to get subcollection data.
FirebaseFirestore.instance
.collection('users')
.get()
.then((QuerySnapshot querySnapshot) {
querySnapshot.docs.forEach((doc) {
FirebaseFirestore.instance
.document(doc.id)
.collection("jobPost")
.get()
.then(...);
});
});
**
if you not have field in first collection document then its shows italic thats means its delte by default otherwise if you have field in first collection document then you can access it easily so this is the best way that i share
**
static Future<List<PostSrc>> getAllFeedPosts()async
{
List<PostSrc> allPosts = [];
var query= await FirebaseFirestore.instance.collection("posts").get();
for(var userdoc in query.docs)
{
QuerySnapshot feed = await FirebaseFirestore.instance.collection("posts")
.doc(userdoc.id).collection("userPosts").get();
for (var postDoc in feed.docs ) {
PostSrc post = PostSrc.fromDoc(postDoc);
allPosts.add(post);
}
}
return allPosts;
}
I am working on a social networking app whose data model is as follows:
Here posts collection contains multiple documents with same or different tag fields, a user can follow multiple tags I already have those tags in a list I need to extract the documents where tag is equal to an element of that list and sort them according to time stamp
getPosts() async {
for (int i = 0; i < Constants.hashtags.length; i++) {
QuerySnapshot dataDoc = await Firestore.instance
.collection("posts")
.where("tag", isEqualTo: Constants.hashtags[i])
.getDocuments();
}
}
Here Constants.hashtags is list of tags that a user if following. I am able to get all documents using above code but I don't know how to merge these documents in a single and sorted way which can be further presented to user in a List.builder form.
Looking at the documentation Here,
you can see the where has an optional parameter whereIn
Query where (
dynamic field,
{dynamic isEqualTo,
dynamic isLessThan,
dynamic isLessThanOrEqualTo,
dynamic isGreaterThan,
dynamic isGreaterThanOrEqualTo,
dynamic arrayContains,
List arrayContainsAny,
List whereIn,
bool isNull}
)
And you can use it Like
QuerySnapshot dataDoc = await Firestore.instance
.collection("posts")
.where("tag", whereIn: Constants.hashtags)
.getDocuments()
This query snapshot contains list of all documents that has a tag field which is equals to Constants.hashtags
I am trying to get some random posts from Firebase. But i am unable to get random document id.
Is there any way to retrieve data from Firebase like this :-
getRandomData() async {
QuerySnapshot snapshot = await posts
.document(random.id)
.collection('userPosts')
.getDocuments();}
i am trying to say that. Now i am able to get documentID normally not in italics.so now how can i get random documentID from Firebase.
List documentIds in a list first.
var list = ['documentId1','documentId2','documentId3'];
var element = getRandomElement(list);
Then query the documentSnapshot
You can first get all documents in you collection.
Try this code:
async getMarker() {
const snapshot = await firebase.firestore().collection('userPosts').get();
const documents = [];
snapshot.forEach(doc => {
documents[doc.id] = doc.data();
});
return documents;
}
Next, from return documents you can create a list of documents id and get random numbers (document id) from this list.
The main problem here that's going to prevent you from moving forward is the fact that you don't actually have any documents nested immediately under "posts". Notice that the names of the documents are in italics. That means there isn't actually a document here at all. The reason why they are show, however, is because you have a subcollection "userPosts" nested under the path where that document ID exists.
Since you don't have any documents at all under "posts", the usual strategies to find a random document won't work at all. You're going to have to actually populate some data there to select from, or find another way to select from the data in the subcollections.
firestore()
.collection('SOURCE')
.doc(props?.route?.params?.data?.id)
.collection('userPosts')
.get()
.then(querySnapshot => {
querySnapshot.forEach(documentSnapshot => {
console.log('User ID: ', documentSnapshot.id, documentSnapshot.data());
})
})
Ofc, I know the basic way we retrieve a whole bunch of documents from a collection like so:
stream: Firestore.instance
.collection("collection_name")
.orderBy("date", descending: true) // new entries first
.snapshots(),
builder: (context, snapshot) { ....
This is fine when I'm displaying a whole bunch of data.
But I have situation where I need to do some calculations with the last added numbers, meaning I just need to get 1 document, so I what I do is sort data by date and limit by 1, then make the query for this one document like this
List<DocumentSnapshot> list;
QuerySnapshot querySnapshots;
if (await AuthHelper.checkIfUserIsLoggedIn()) {
String userId = await AuthHelper.getUserID();
querySnapshots = await Firestore.instance
.collection("collection_name")
.orderBy("date", descending: true) // new entries first, date is one the entries btw
.limit(1)
.getDocuments(); // Get Documents not as a stream
list = querySnapshots.documents; // Make snapshots into a list
return (list ?? null);
} else {
return null;
}
But is this the best way to do it ? When making this query aren't I getting the whole set of documents and discarding the rest ? Which means when this collections grows this is going get slower and slower ?
Is there a better way to retrieve the last added document ? So I can use it as list/array/map and not as stream ?
Thanks.
When you limit the query, you are only ever going to read that number of documents maximum. It's not going to read the entire collection and discard the extras.
I'm trying to model "memberships" with Firestore. The idea is that there are companies, users and then memberships.
The memberships collection stores a reference to a company and to a user, as well as a role as a string, e..g admin or editor.
How would I query to get the users with a certain role for a company?
This is what I currently have with some basic logging.
const currentCompanyID = 'someid';
return database
.collection('memberships')
.where('company', '==', database.doc(`companies/${currentCompanyID}`))
.where('role', '==', 'admin')
.get()
.then(snap => {
snap.forEach(function(doc) {
console.log(doc.id, ' => ', doc.data());
const data = doc.data();
console.log(data.user.get());
});
})
.catch(error => {
console.error('Error fetching documents: ', error);
});
data.user.get() returns a promise to the user, but I'd have to do that for every user which seems inefficient?
What would be the best way to approach this?
Your code is close to what you want, but there are two issues:
Your where() clause can't compare a field with a document reference, because Firestore is a classic denormalized datastore. There aren't ways to strongly guarantee that one document refers to another. You'll need to store document IDs and maintain consistency yourself. (Example below).
Queries actually return a QuerySnapshot, which includes all the docs that result from a query. So you're not getting one document at a time — you'll get all the ones that match. (See code below)
So a corrected version that fits the spirit of what you want:
const currentCompanyID = '8675309';
const querySnapshot = await database
.collection('memberships')
.where('companyId', '==', currentCompanyID)
.where('role', '==', 'admin')
.get(); // <-- this promise, when awaited, pulls all matching docs
await Promise.all(querySnapshot.map(async snap => {
const data = doc.data();
const user = await database
.collection('users')
.doc(data.userId)
.get();
console.log(doc.id, ' => ', data);
console.log(user);
});
There isn't a faster way on the client side to fetch all the users that your query refers to at once -- it's part of the trouble of trying to use a denormalized store for queries that feel much more like classic relational database queries.
If this ends up being a query you run often (i.e. get users with a certain role within a specific company), you could consider storing membership information as part of the user doc instead. That way, you could query the users collection and get all the matching users in one shot.