I'm trying to model "memberships" with Firestore. The idea is that there are companies, users and then memberships.
The memberships collection stores a reference to a company and to a user, as well as a role as a string, e..g admin or editor.
How would I query to get the users with a certain role for a company?
This is what I currently have with some basic logging.
const currentCompanyID = 'someid';
return database
.collection('memberships')
.where('company', '==', database.doc(`companies/${currentCompanyID}`))
.where('role', '==', 'admin')
.get()
.then(snap => {
snap.forEach(function(doc) {
console.log(doc.id, ' => ', doc.data());
const data = doc.data();
console.log(data.user.get());
});
})
.catch(error => {
console.error('Error fetching documents: ', error);
});
data.user.get() returns a promise to the user, but I'd have to do that for every user which seems inefficient?
What would be the best way to approach this?
Your code is close to what you want, but there are two issues:
Your where() clause can't compare a field with a document reference, because Firestore is a classic denormalized datastore. There aren't ways to strongly guarantee that one document refers to another. You'll need to store document IDs and maintain consistency yourself. (Example below).
Queries actually return a QuerySnapshot, which includes all the docs that result from a query. So you're not getting one document at a time — you'll get all the ones that match. (See code below)
So a corrected version that fits the spirit of what you want:
const currentCompanyID = '8675309';
const querySnapshot = await database
.collection('memberships')
.where('companyId', '==', currentCompanyID)
.where('role', '==', 'admin')
.get(); // <-- this promise, when awaited, pulls all matching docs
await Promise.all(querySnapshot.map(async snap => {
const data = doc.data();
const user = await database
.collection('users')
.doc(data.userId)
.get();
console.log(doc.id, ' => ', data);
console.log(user);
});
There isn't a faster way on the client side to fetch all the users that your query refers to at once -- it's part of the trouble of trying to use a denormalized store for queries that feel much more like classic relational database queries.
If this ends up being a query you run often (i.e. get users with a certain role within a specific company), you could consider storing membership information as part of the user doc instead. That way, you could query the users collection and get all the matching users in one shot.
Related
I am trying to get some random posts from Firebase. But i am unable to get random document id.
Is there any way to retrieve data from Firebase like this :-
getRandomData() async {
QuerySnapshot snapshot = await posts
.document(random.id)
.collection('userPosts')
.getDocuments();}
i am trying to say that. Now i am able to get documentID normally not in italics.so now how can i get random documentID from Firebase.
List documentIds in a list first.
var list = ['documentId1','documentId2','documentId3'];
var element = getRandomElement(list);
Then query the documentSnapshot
You can first get all documents in you collection.
Try this code:
async getMarker() {
const snapshot = await firebase.firestore().collection('userPosts').get();
const documents = [];
snapshot.forEach(doc => {
documents[doc.id] = doc.data();
});
return documents;
}
Next, from return documents you can create a list of documents id and get random numbers (document id) from this list.
The main problem here that's going to prevent you from moving forward is the fact that you don't actually have any documents nested immediately under "posts". Notice that the names of the documents are in italics. That means there isn't actually a document here at all. The reason why they are show, however, is because you have a subcollection "userPosts" nested under the path where that document ID exists.
Since you don't have any documents at all under "posts", the usual strategies to find a random document won't work at all. You're going to have to actually populate some data there to select from, or find another way to select from the data in the subcollections.
firestore()
.collection('SOURCE')
.doc(props?.route?.params?.data?.id)
.collection('userPosts')
.get()
.then(querySnapshot => {
querySnapshot.forEach(documentSnapshot => {
console.log('User ID: ', documentSnapshot.id, documentSnapshot.data());
})
})
Using Firebase Cloud Functions I'd like to search for documents that contain a certain other document in an array of document references. My structure looks as follows;
Users
name
email
cars
ref to cars/car1 for example
ref to cars/car2 for example
Cars
registration
make
model
There are multiple users and multiple cars. I need to search for users that have a certain 'car' in their car array.
I'm trying to write this in a Cloud Function and have the following;
admin.firestore()
.collection('users')
.where('cars', 'array-contains', registration)
.get().then(doc => {
console.log("TESTING: found the user " + doc.data().email)
return
}).catch(error => {
console.error(error);
});
I know this is currently just searching for the registration string in the array. Is there anyway to search for a specific document reference. I'm using Node.js.
Working code to get all the documents that have a document reference in an array;
// Notify the owner of the car
admin.firestore()
.collection('users')
.where('cars', 'array-contains', carRef)
.get().then(snapshot => {
snapshot.forEach(doc => {
console.log("TESTING found the user " + doc.data().email);
const message = {
notification: {
body: 'Your vehicle (' + carReg + ') recieved a report. Tap here to see!',
},
token: doc.data().cloudMessagingToken
};
sendMessage(message);
});
return
}).catch(error => {
console.error("Error finding a user that has the car in their garage");
console.error(error);
});
If you want to query using reference type fields, you will need to provide a DocumentReference type object to the query. If you pass a DocumentReference to a car, the query should work. For example:
const ref = admin.firestore().collection('Cars').doc(id)
where id is the id of the document.
However, you can't search using values of fields inside the referenced document. Firestore queries only work against data in a single collection at a time. With the way you have your data organized right now, it's not possible to make a single query for all users who have references to cars with a specific registration string field. For that query, you would need to also store an array of registration strings for each user that you could query with array-contains.
Yes, this involves duplication of data, and it's called "denormalization". This is very common in nosql type databases to enable the queries you need.
I thought I read that you can query subcollections with the new Firebase Firestore, but I don't see any examples. For example I have my Firestore setup in the following way:
Dances [collection]
danceName
Songs [collection]
songName
How would I be able to query "Find all dances where songName == 'X'"
Update 2019-05-07
Today we released collection group queries, and these allow you to query across subcollections.
So, for example in the web SDK:
db.collectionGroup('Songs')
.where('songName', '==', 'X')
.get()
This would match documents in any collection where the last part of the collection path is 'Songs'.
Your original question was about finding dances where songName == 'X', and this still isn't possible directly, however, for each Song that matched you can load its parent.
Original answer
This is a feature which does not yet exist. It's called a "collection group query" and would allow you query all songs regardless of which dance contained them. This is something we intend to support but don't have a concrete timeline on when it's coming.
The alternative structure at this point is to make songs a top-level collection and make which dance the song is a part of a property of the song.
UPDATE
Now Firestore supports array-contains
Having these documents
{danceName: 'Danca name 1', songName: ['Title1','Title2']}
{danceName: 'Danca name 2', songName: ['Title3']}
do it this way
collection("Dances")
.where("songName", "array-contains", "Title1")
.get()...
#Nelson.b.austin Since firestore does not have that yet, I suggest you to have a flat structure, meaning:
Dances = {
danceName: 'Dance name 1',
songName_Title1: true,
songName_Title2: true,
songName_Title3: false
}
Having it in that way, you can get it done:
var songTitle = 'Title1';
var dances = db.collection("Dances");
var query = dances.where("songName_"+songTitle, "==", true);
I hope this helps.
UPDATE 2019
Firestore have released Collection Group Queries. See Gil's answer above or the official Collection Group Query Documentation
Previous Answer
As stated by Gil Gilbert, it seems as if collection group queries is currently in the works. In the mean time it is probably better to use root level collections and just link between these collection using the document UID's.
For those who don't already know, Jeff Delaney has some incredible guides and resources for anyone working with Firebase (and Angular) on AngularFirebase.
Firestore NoSQL Relational Data Modeling - Here he breaks down the basics of NoSQL and Firestore DB structuring
Advanced Data Modeling With Firestore by Example - These are more advanced techniques to keep in the back of your mind. A great read for those wanting to take their Firestore skills to the next level
What if you store songs as an object instead of as a collection? Each dance as, with songs as a field: type Object (not a collection)
{
danceName: "My Dance",
songs: {
"aNameOfASong": true,
"aNameOfAnotherSong": true,
}
}
then you could query for all dances with aNameOfASong:
db.collection('Dances')
.where('songs.aNameOfASong', '==', true)
.get()
.then(function(querySnapshot) {
querySnapshot.forEach(function(doc) {
console.log(doc.id, " => ", doc.data());
});
})
.catch(function(error) {
console.log("Error getting documents: ", error);
});
NEW UPDATE July 8, 2019:
db.collectionGroup('Songs')
.where('songName', isEqualTo:'X')
.get()
I have found a solution.
Please check this.
var museums = Firestore.instance.collectionGroup('Songs').where('songName', isEqualTo: "X");
museums.getDocuments().then((querySnapshot) {
setState(() {
songCounts= querySnapshot.documents.length.toString();
});
});
And then you can see Data, Rules, Indexes, Usage tabs in your cloud firestore from console.firebase.google.com.
Finally, you should set indexes in the indexes tab.
Fill in collection ID and some field value here.
Then Select the collection group option.
Enjoy it. Thanks
You can always search like this:-
this.key$ = new BehaviorSubject(null);
return this.key$.switchMap(key =>
this.angFirestore
.collection("dances").doc("danceName").collections("songs", ref =>
ref
.where("songName", "==", X)
)
.snapshotChanges()
.map(actions => {
if (actions.toString()) {
return actions.map(a => {
const data = a.payload.doc.data() as Dance;
const id = a.payload.doc.id;
return { id, ...data };
});
} else {
return false;
}
})
);
Query limitations
Cloud Firestore does not support the following types of queries:
Queries with range filters on different fields.
Single queries across multiple collections or subcollections. Each query runs against a single collection of documents. For more
information about how your data structure affects your queries, see
Choose a Data Structure.
Logical OR queries. In this case, you should create a separate query for each OR condition and merge the query results in your app.
Queries with a != clause. In this case, you should split the query into a greater-than query and a less-than query. For example, although
the query clause where("age", "!=", "30") is not supported, you can
get the same result set by combining two queries, one with the clause
where("age", "<", "30") and one with the clause where("age", ">", 30).
I'm working with Observables here and the AngularFire wrapper but here's how I managed to do that.
It's kind of crazy, I'm still learning about observables and I possibly overdid it. But it was a nice exercise.
Some explanation (not an RxJS expert):
songId$ is an observable that will emit ids
dance$ is an observable that reads that id and then gets only the first value.
it then queries the collectionGroup of all songs to find all instances of it.
Based on the instances it traverses to the parent Dances and get their ids.
Now that we have all the Dance ids we need to query them to get their data. But I wanted it to perform well so instead of querying one by one I batch them in buckets of 10 (the maximum angular will take for an in query.
We end up with N buckets and need to do N queries on firestore to get their values.
once we do the queries on firestore we still need to actually parse the data from that.
and finally we can merge all the query results to get a single array with all the Dances in it.
type Song = {id: string, name: string};
type Dance = {id: string, name: string, songs: Song[]};
const songId$: Observable<Song> = new Observable();
const dance$ = songId$.pipe(
take(1), // Only take 1 song name
switchMap( v =>
// Query across collectionGroup to get all instances.
this.db.collectionGroup('songs', ref =>
ref.where('id', '==', v.id)).get()
),
switchMap( v => {
// map the Song to the parent Dance, return the Dance ids
const obs: string[] = [];
v.docs.forEach(docRef => {
// We invoke parent twice to go from doc->collection->doc
obs.push(docRef.ref.parent.parent.id);
});
// Because we return an array here this one emit becomes N
return obs;
}),
// Firebase IN support up to 10 values so we partition the data to query the Dances
bufferCount(10),
mergeMap( v => { // query every partition in parallel
return this.db.collection('dances', ref => {
return ref.where( firebase.firestore.FieldPath.documentId(), 'in', v);
}).get();
}),
switchMap( v => {
// Almost there now just need to extract the data from the QuerySnapshots
const obs: Dance[] = [];
v.docs.forEach(docRef => {
obs.push({
...docRef.data(),
id: docRef.id
} as Dance);
});
return of(obs);
}),
// And finally we reduce the docs fetched into a single array.
reduce((acc, value) => acc.concat(value), []),
);
const parentDances = await dance$.toPromise();
I copy pasted my code and changed the variable names to yours, not sure if there are any errors, but it worked fine for me. Let me know if you find any errors or can suggest a better way to test it with maybe some mock firestore.
var songs = []
db.collection('Dances')
.where('songs.aNameOfASong', '==', true)
.get()
.then(function(querySnapshot) {
var songLength = querySnapshot.size
var i=0;
querySnapshot.forEach(function(doc) {
songs.push(doc.data())
i ++;
if(songLength===i){
console.log(songs
}
console.log(doc.id, " => ", doc.data());
});
})
.catch(function(error) {
console.log("Error getting documents: ", error);
});
It could be better to use a flat data structure.
The docs specify the pros and cons of different data structures on this page.
Specifically about the limitations of structures with sub-collections:
You can't easily delete subcollections, or perform compound queries across subcollections.
Contrasted with the purported advantages of a flat data structure:
Root-level collections offer the most flexibility and scalability, along with powerful querying within each collection.
I'm using the firestore of firebase and I want to iterate through the whole collection. Is there something like:
db.collection('something').forEach((doc) => {
// do something
})
Yes, you can simply query the collection for all its documents using the get() method on the collection reference. A CollectionReference object subclasses Query, so you can call Query methods on it. By itself, a collection reference is essentially an unfiltered query for all of its documents.
Android: Query.get()
iOS/Swift: Query.getDocuments()
JavaScript: Query.get()
In each platform, this method is asynchronous, so you'll have to deal with the callbacks correctly.
See also the product documentation for "Get all documents in a collection".
db.collection("cities").get().then(function(querySnapshot) {
querySnapshot.forEach(function(doc) {
// doc.data() is never undefined for query doc snapshots
console.log(doc.id, " => ", doc.data());
});
});
If you know that there aren't too many docs in the collection (e.g. thousands or millions) then you can just use collectionRef.get() as described in the top-voted answer here and explained in Firebase docs.
However, in many cases, a collection can contain large numbers of documents that you can't just "get" at once, as your program's memory usage will explode. In these cases you need to implement a different traversal logic that will go through the entire collection by batches. You also need to ensure that you don’t miss any documents or process any of them multiple times.
This is why I wrote Firecode, an open-source Node.js library that solves precisely this problem. It is an extremely light, robust, well-typed, and well-documented library that provides you with configurable traverser objects that walk you through a given collection.
You can find the Github repo here and the docs site here. Also, here's a short snippet that shows you how you would traverse a users collection with Firecode.
const usersCollection = firestore().collection('users');
const traverser = createTraverser(usersCollection);
const { batchCount, docCount } = await traverser.traverse(async (batchDocs, batchIndex) => {
const batchSize = batchDocs.length;
await Promise.all(
batchDocs.map(async (doc) => {
const { email, firstName } = doc.data();
await sendEmail({ to: email, content: `Hello ${firstName}!` });
})
);
console.log(`Batch ${batchIndex} done! We emailed ${batchSize} users in this batch.`);
});
console.log(`Traversal done! We emailed ${docCount} users in ${batchCount} batches!`);
I thought I read that you can query subcollections with the new Firebase Firestore, but I don't see any examples. For example I have my Firestore setup in the following way:
Dances [collection]
danceName
Songs [collection]
songName
How would I be able to query "Find all dances where songName == 'X'"
Update 2019-05-07
Today we released collection group queries, and these allow you to query across subcollections.
So, for example in the web SDK:
db.collectionGroup('Songs')
.where('songName', '==', 'X')
.get()
This would match documents in any collection where the last part of the collection path is 'Songs'.
Your original question was about finding dances where songName == 'X', and this still isn't possible directly, however, for each Song that matched you can load its parent.
Original answer
This is a feature which does not yet exist. It's called a "collection group query" and would allow you query all songs regardless of which dance contained them. This is something we intend to support but don't have a concrete timeline on when it's coming.
The alternative structure at this point is to make songs a top-level collection and make which dance the song is a part of a property of the song.
UPDATE
Now Firestore supports array-contains
Having these documents
{danceName: 'Danca name 1', songName: ['Title1','Title2']}
{danceName: 'Danca name 2', songName: ['Title3']}
do it this way
collection("Dances")
.where("songName", "array-contains", "Title1")
.get()...
#Nelson.b.austin Since firestore does not have that yet, I suggest you to have a flat structure, meaning:
Dances = {
danceName: 'Dance name 1',
songName_Title1: true,
songName_Title2: true,
songName_Title3: false
}
Having it in that way, you can get it done:
var songTitle = 'Title1';
var dances = db.collection("Dances");
var query = dances.where("songName_"+songTitle, "==", true);
I hope this helps.
UPDATE 2019
Firestore have released Collection Group Queries. See Gil's answer above or the official Collection Group Query Documentation
Previous Answer
As stated by Gil Gilbert, it seems as if collection group queries is currently in the works. In the mean time it is probably better to use root level collections and just link between these collection using the document UID's.
For those who don't already know, Jeff Delaney has some incredible guides and resources for anyone working with Firebase (and Angular) on AngularFirebase.
Firestore NoSQL Relational Data Modeling - Here he breaks down the basics of NoSQL and Firestore DB structuring
Advanced Data Modeling With Firestore by Example - These are more advanced techniques to keep in the back of your mind. A great read for those wanting to take their Firestore skills to the next level
What if you store songs as an object instead of as a collection? Each dance as, with songs as a field: type Object (not a collection)
{
danceName: "My Dance",
songs: {
"aNameOfASong": true,
"aNameOfAnotherSong": true,
}
}
then you could query for all dances with aNameOfASong:
db.collection('Dances')
.where('songs.aNameOfASong', '==', true)
.get()
.then(function(querySnapshot) {
querySnapshot.forEach(function(doc) {
console.log(doc.id, " => ", doc.data());
});
})
.catch(function(error) {
console.log("Error getting documents: ", error);
});
NEW UPDATE July 8, 2019:
db.collectionGroup('Songs')
.where('songName', isEqualTo:'X')
.get()
I have found a solution.
Please check this.
var museums = Firestore.instance.collectionGroup('Songs').where('songName', isEqualTo: "X");
museums.getDocuments().then((querySnapshot) {
setState(() {
songCounts= querySnapshot.documents.length.toString();
});
});
And then you can see Data, Rules, Indexes, Usage tabs in your cloud firestore from console.firebase.google.com.
Finally, you should set indexes in the indexes tab.
Fill in collection ID and some field value here.
Then Select the collection group option.
Enjoy it. Thanks
You can always search like this:-
this.key$ = new BehaviorSubject(null);
return this.key$.switchMap(key =>
this.angFirestore
.collection("dances").doc("danceName").collections("songs", ref =>
ref
.where("songName", "==", X)
)
.snapshotChanges()
.map(actions => {
if (actions.toString()) {
return actions.map(a => {
const data = a.payload.doc.data() as Dance;
const id = a.payload.doc.id;
return { id, ...data };
});
} else {
return false;
}
})
);
Query limitations
Cloud Firestore does not support the following types of queries:
Queries with range filters on different fields.
Single queries across multiple collections or subcollections. Each query runs against a single collection of documents. For more
information about how your data structure affects your queries, see
Choose a Data Structure.
Logical OR queries. In this case, you should create a separate query for each OR condition and merge the query results in your app.
Queries with a != clause. In this case, you should split the query into a greater-than query and a less-than query. For example, although
the query clause where("age", "!=", "30") is not supported, you can
get the same result set by combining two queries, one with the clause
where("age", "<", "30") and one with the clause where("age", ">", 30).
I'm working with Observables here and the AngularFire wrapper but here's how I managed to do that.
It's kind of crazy, I'm still learning about observables and I possibly overdid it. But it was a nice exercise.
Some explanation (not an RxJS expert):
songId$ is an observable that will emit ids
dance$ is an observable that reads that id and then gets only the first value.
it then queries the collectionGroup of all songs to find all instances of it.
Based on the instances it traverses to the parent Dances and get their ids.
Now that we have all the Dance ids we need to query them to get their data. But I wanted it to perform well so instead of querying one by one I batch them in buckets of 10 (the maximum angular will take for an in query.
We end up with N buckets and need to do N queries on firestore to get their values.
once we do the queries on firestore we still need to actually parse the data from that.
and finally we can merge all the query results to get a single array with all the Dances in it.
type Song = {id: string, name: string};
type Dance = {id: string, name: string, songs: Song[]};
const songId$: Observable<Song> = new Observable();
const dance$ = songId$.pipe(
take(1), // Only take 1 song name
switchMap( v =>
// Query across collectionGroup to get all instances.
this.db.collectionGroup('songs', ref =>
ref.where('id', '==', v.id)).get()
),
switchMap( v => {
// map the Song to the parent Dance, return the Dance ids
const obs: string[] = [];
v.docs.forEach(docRef => {
// We invoke parent twice to go from doc->collection->doc
obs.push(docRef.ref.parent.parent.id);
});
// Because we return an array here this one emit becomes N
return obs;
}),
// Firebase IN support up to 10 values so we partition the data to query the Dances
bufferCount(10),
mergeMap( v => { // query every partition in parallel
return this.db.collection('dances', ref => {
return ref.where( firebase.firestore.FieldPath.documentId(), 'in', v);
}).get();
}),
switchMap( v => {
// Almost there now just need to extract the data from the QuerySnapshots
const obs: Dance[] = [];
v.docs.forEach(docRef => {
obs.push({
...docRef.data(),
id: docRef.id
} as Dance);
});
return of(obs);
}),
// And finally we reduce the docs fetched into a single array.
reduce((acc, value) => acc.concat(value), []),
);
const parentDances = await dance$.toPromise();
I copy pasted my code and changed the variable names to yours, not sure if there are any errors, but it worked fine for me. Let me know if you find any errors or can suggest a better way to test it with maybe some mock firestore.
var songs = []
db.collection('Dances')
.where('songs.aNameOfASong', '==', true)
.get()
.then(function(querySnapshot) {
var songLength = querySnapshot.size
var i=0;
querySnapshot.forEach(function(doc) {
songs.push(doc.data())
i ++;
if(songLength===i){
console.log(songs
}
console.log(doc.id, " => ", doc.data());
});
})
.catch(function(error) {
console.log("Error getting documents: ", error);
});
It could be better to use a flat data structure.
The docs specify the pros and cons of different data structures on this page.
Specifically about the limitations of structures with sub-collections:
You can't easily delete subcollections, or perform compound queries across subcollections.
Contrasted with the purported advantages of a flat data structure:
Root-level collections offer the most flexibility and scalability, along with powerful querying within each collection.