I want to do something like:
final collectionReference = Firestore.instance.collection('myCollection');
final List<String> documentList = collectionReference.getDocList();
OR
final query = collectionReference.orderBy('lastUpdated', descending: true).limit(100);
final List<String> documentList = query.getDocList();
Currently when we use get() or query.getDocuments() it will return the whole Document list and all contents along with it. But you know that we wanna optimize reads thus we make use of the 1MiB limit of each Documents. Thus we don't wanna download the whole document's contents. Rather we just need the IDs for other usage. Is there a way to do this ?
Thanks
There no way to do this with the Flutter APIs (or any of the web or mobile clients).
The only way is with backend code, where you have a method like select() (link to the nodejs API) on Query that lets you select which document fields to return. So, you could have your app call a backend to return the document IDs, but not directly in the app.
If you must query from the client, consider moving fields unnecessary for queries to documents in anther collection with the same IDs, and request them only when needed.
Related
Im trying to create one stream, that is using multiple documents references that are stored and fetched from Firebase Firestore.
Lets say I have two collection named users and documents. When user is created he gets document with his id in users collection with field named documentsHasAccessTo that is list of references to documents inside documents collection. It is important, that these documents can be located in different sub collections inside documents collection so I dont want to query whole documents and filter it, in order to save Firestore transfer and make it faster I already know paths to documents stored in documentsHasAccessTo field.
So for example, I can have user with data inside users/<user uid> document with documentsHasAccessTo field that stores 3 different document references.
I would like to achieve something like this (untested):
final userId = 'blablakfn1n21n4109';
final usersDocumentRef = FirebaseFirestore.instance.doc('users/$userId');
usersDocumentRef.snapshots().listen((snapshot) {
final references = snapshot.data()['documentsHasAccessTo'] as List<DocumentReference>;
final documentsStream = // create single query stream using all references from list
});
Keep in mind, that it would also be great, if this stream would update query if documentsHasAccessTo changes like in the example above, hence I used snapshots() on usersDocumentReferences rather than single get() fetch.
The more I think about this Im starting to believe this is simple impossible or theres a more simple and clean solution. Im open to anything.
You could use rxdart's switchMap and MergeStream:
usersDocumentRef.snapshots().switchMap((snapshot) {
final references = snapshot.data()['documentsHasAccessTo'] as List<DocumentReference>;
return MergeStream(references.map(ref) => /* do something that creates a stream */));
});
I have a nested collection in firestore that I want to make a query from it.
As you can see the first collection called 'businessUsers' and the nested one called 'campaigns',
If I make a query for a field in the 'businessUsers' it's working OK:
return FirebaseFirestore.instance.collection("businessUsers").where('xxx',
isEqualTo:filterBusinessResult ).snapshots().map(_businessListFromSnapshot);
but how can I make a query to 'campaigns' collection field?
I tried
return FirebaseFirestore.instance.collection("businessUsers").doc().collection("campaigns").where('campaignCategory', isEqualTo:filterBusinessResult ).snapshots()
.map(_businessListFromSnapshot);
but it wont work.
Its important to note, that I need all the data with 'campaignCategory' == filterBusinessResult
any idea?
It depends on whether you want to query a specific user's campaigns, or the campaigns of all users together.
If you want to query the campaigns of a specific user, you need to query under their document:
FirebaseFirestore.instance
.collection("businessUsers").doc("your user ID").
.collection("campaigns")
.where('campaignCategory', isEqualTo:filterBusinessResult)
So you have to know the ID of the businessUsers document here.
If you want to query across all campaigns subcollections are once, that is known as a collection group query and would look like this:
FirebaseFirestore.instance
.collectionGroup("campaigns")
.where('campaignCategory', isEqualTo:filterBusinessResult)
The results are going to documents from the campaigns collection only, but you can look up the parent document reference for each DocumentSnapshot with docSnapshot.reference.parent.parent.
When querying something, .doc() requires the id of the document you're trying to get. So it's failing because it doesn't know which document in the businessUsers collection you're trying to fetch a subcollection on. You probably want to use a .collectionGroup() query here. They let you query all subcollections at once (see documentation here). With FlutterFire specifically, it's going to be something like:
var snapshots = FirebaseFirestore.instance.collectionGroup("campaigns")
.where("campaignCategory", isEqualTo: filterBusinessResult)
.snapshots();
I'm working on a Flutter app where each user can create projects, and share projects with other users. I've created a 'shares' collection, where each user's ID is a document, and within that document, all project IDs that have been shared with that user are collected like so, with a boolean that represents whether or not the share has been accepted yet:
Next, I created a collection of the projects themselves, like so:
Now, I'd like to query the 'projects' collection and return only the projects that are in a given user's 'shares' list. First off, how can I get each document in the share list's ID? And secondly, is it possible to compare that ID to the contents of a List using a .where() clause?
I've been trying something like this, but to no avail:
Stream<List<Map<String, dynamic>>> getListOfProjectsForUser({#required List<String> shares}) {
var ref = _firestore.collection('projects');
return ref
.where(shares, arrayContains: ref.id)
.snapshots()
.map((QuerySnapshot snapshot) => snapshot.docs.map((DocumentSnapshot doc) => doc.data()).toList());
}
I also tried this:
Stream<List<Map<String, dynamic>>> getListOfProjectsForUser({#required List<String> shares}) {
var ref = _firestore.collection('projects');
return ref
.where(shares, arrayContains: FieldPath.documentId)
.snapshots()
.map((QuerySnapshot snapshot) => snapshot.docs.map((DocumentSnapshot doc) => doc.data()).toList());
}
Is what I'm trying to do even possible? I've been messing with this for two days and my head's exploding. Any help would be greatly appreciated. Thanks in advance.
You'll need two operations.
Read the document for the user, to determine the list of project IDs.
Perform a in query for the project documents matching those IDs. The in operator accepts up to 10 IDs, so if you have more than 10 projects you'll need multiple queries and merge the results in your application code.
var citiesRef = db.collection("projects");
citiesRef.where(FieldPath.documentId, arrayContains: ['project1id', 'project2id']);
Also see:
The FlutterFire documentation for the where(field, whereIn:) operation
The FlutterFire documentation for the FieldPath.documentId field
First off, how can I get each document in the share list's ID?
For this, you're required to actually query the entire collection. You can iterate the results to collect the IDs of each document. There is no easy way to just get a list of IDs directly from web and mobile client code. See: How to get a list of document IDs in a collection Cloud Firestore?
And secondly, is it possible to compare that ID to the contents of a List using a .where() clause?
If you have a list of document ID strings in memory that could be any length, you will need to perform a query filtering projects for "projOwner" for each individual ID. There are no SQL-like joins in Firestore, so you can't simply join the two collections together with a single query.
Here's how you do a single one - you have to call out the name of the field to filter on:
firestore
.collection("projects")
.where("projOwner", isEqualTo: id)
If you have 10 or less share IDs in the list, you can use an "in" query to find matches from projects, and it will not work with any more.
firestore
.collection("projects")
.where("projOwner", whereIn: listOfIds)
So, if you think the list could ever be larger than 10, you should just start by performing individual queries for each share ID.
if 'arrayContains' is not working try 'whereIn'.
var citiesRef = db.collection("projects");
citiesRef.where(FieldPath.documentId, whereIn: ['project1id',
'project2id']);
I need to fetch all the ids/keys of a collection in Cloud Firestore. Currently I do it like this (groovy):
ApiFuture<QuerySnapshot> snapshot = firestoreClient.database.collection(bucket).get()
List<QueryDocumentSnapshot> documents = snapshot.get().getDocuments()
for (QueryDocumentSnapshot document : documents) {
keys.add(document.id)
}
I run this on a collection which potentially has could have a lot of documents lets say 30.000 which causes a java.lang.OutOfMemoryError: Java heap space
The thing is that I don't need all the documents. As seen in my code all I need is to check which documents are in the collection (ie. a list of keys/id's), but I have not found any way to grab them with out fetching all the documents which has a huge overhead.
I using the Java Firebase Admin SDK (6.12.2).
So I'm hoping that there is a way to grab all the keys with out the overhead and without my heap maxing out.
Calling get() will get you the full documents. But you should be able to do an empty selection. From the documentation for select():
public Query select(String... fields)
Creates and returns a new Query instance that applies a field mask to the result and returns the specified subset of fields. You can specify a list of field paths to return, or use an empty list to only return the references of matching documents.
So something like this:
firestoreClient.database.collection(bucket).select(new String[0])
Also see:
How to get a list of document IDs in a collection Cloud Firestore?
the Firestore Java reference documentation for the select function
I am trying to fetch all documents whose sub-collection contain a specific document ID. Is there any way to do this?
For example, if the boxed document under 'enquiries' sub-collection exists, then I need the boxed document ID from 'books' collection. I couldn't figure out how to go backwards to get the parent document ID.
I make the assumption that all the sub-collections have the same name, i.e. enquiries. Then, you could do as follows:
Add a field docId in your enquiries document that contains the document ID.
Execute a Collection Group query in order to get all the documents with the desired docId value (Firestore.instance.collectionGroup("enquiries").where("docId", isEqualTo: "ykXB...").getDocuments()).
Then, you loop over the results of the query and for each DocumentReference you call twice the parent() methods (first time you will get the CollectionReference and second time you will get the DocumentReference of the parent document).
You just have to use the id property and you are done.
Try the following:
Firestore.instance.collection("books").where("author", isEqualTo: "Arumugam").getDocuments().then((value) {
value.documents.forEach((result) {
var id = result.documentID;
Firestore.instance.collection("books").document(id).collection("enquiries").getDocuments().then((querySnapshot) {
querySnapshot.documents.forEach((result) {
print(result.data);
});
First you need to retrieve the id under the books collection, to be able to do that you have to do a query for example where("author", isEqualTo: "Arumugam"). After retrieving the id you can then do a query to retrieve the documents inside the collection enquiries
For example, if the boxed document under 'enquiries' sub-collection exists, then I need the boxed document ID from 'books' collection.
There is no way you can do that in a single go.
I couldn't figure out how to go backwards to get the parent document ID.
There is no going back in Firestore as you probably were thinking. In Firebase Realtime Database we have a method named getParent(), which does exactly what you want but in Firestore we don't.
Queries in Firestore are shallow, meaning that it only get items from the collection that the query is run against. Firestore doesn't support queries across different collections in one go. A single query may only use the properties of documents in a single collection. So the solution to solving your problem is to perform two get() calls. The first one would be to check that document for existence in the enquiries subcollection, and if it exists, simply create another get() call to get the document from the books collection.
Renaud Tarnec's answer is great for fetching the IDs of the relevant books.
If you need to fetch more than the ID, there is a trick you could use in some scenarios. I imagine your goal is to show some sort of an index of all books associated with a particular enquiry ID. If the data you'd like to show in that index is not too long (can be serialized in less than 1500 bytes) and if it is not changing frequently, you could try to use the document ID as the placeholder for that data.
For example, let's say you wanted to display a list of book titles and authors corresponding to some enquiryId. You could create the book ID in the collection with something like so:
// Assuming admin SDK
const bookId = nanoid();
const author = 'Brandon Sanderson';
const title = 'Mistborn: The Final Empire';
// If title + author are not unique, you could add the bookId to the array
const uniquePayloadKey = Buffer.from(JSON.stringify([author, title])).toString('base64url');
booksColRef.doc(uniquePayloadKey).set({ bookId })
booksColRef.doc(uniquePayloadKey).collection('enquiries').doc(enquiryId).set({ enquiryId })
Then, after running the collection group query per Renaud Tarnec's answer, you could extract that serialized information with a regexp on the path, and deserialize. E.g.:
// Assuming Web 9 SDK
const books = query(collectionGroup(db, 'enquiries'), where('enquiryId', '==', enquiryId));
return getDocs(books).then(snapshot => {
const data = []
snapshot.forEach(doc => {
const payload = doc.ref.path.match(/books\/(.*)\/enquiries/)[1];
const [author, title] = JSON.parse(atob(details));
data.push({ author, title })
});
return data;
});
The "store payload in ID" trick can be used only to present some basic information for your child-driven search results. If your book document has a lot of information you'd like to display once the user clicks on one of the books returned by the enquiry, you may want to store this in separate documents whose IDs are the real bookIds. The bookId field added under the unique payload key allows such lookups when necessary.
You can reuse the same data structure for returning book results from different starting points, not just enquiries, without duplicating this structure. If you stored many authors per book, for example, you could add an authors sub-collection to search by. As long as the information you want to display in the resulting index page is the same and can be serialized within the 1500-byte limit, you should be good.
The (quite substantial) downside of this approach is that it is not possible to rename document IDs in Firestore. If some of the details in the payload change (e.g. an admin fixes a book titles), you will need to create all the sub-collections under it and delete the old data. This can be quite costly - at least 1 read, 1 write, and 1 delete for every document in every sub-collection. So keep in mind it may not be pragmatic for fast changing data.
The 1500-byte limit for key names is documented in Usage and Limits.
If you are concerned about potential hotspots this can generate per Best Practices for Cloud Firestore, I imagine that adding the bookId as a prefix to the uniquePayloadKey (with a delimiter that allows you to throw it away) would do the trick - but I am not certain.