I thought I read that you can query subcollections with the new Firebase Firestore, but I don't see any examples. For example I have my Firestore setup in the following way:
Dances [collection]
danceName
Songs [collection]
songName
How would I be able to query "Find all dances where songName == 'X'"
Update 2019-05-07
Today we released collection group queries, and these allow you to query across subcollections.
So, for example in the web SDK:
db.collectionGroup('Songs')
.where('songName', '==', 'X')
.get()
This would match documents in any collection where the last part of the collection path is 'Songs'.
Your original question was about finding dances where songName == 'X', and this still isn't possible directly, however, for each Song that matched you can load its parent.
Original answer
This is a feature which does not yet exist. It's called a "collection group query" and would allow you query all songs regardless of which dance contained them. This is something we intend to support but don't have a concrete timeline on when it's coming.
The alternative structure at this point is to make songs a top-level collection and make which dance the song is a part of a property of the song.
UPDATE
Now Firestore supports array-contains
Having these documents
{danceName: 'Danca name 1', songName: ['Title1','Title2']}
{danceName: 'Danca name 2', songName: ['Title3']}
do it this way
collection("Dances")
.where("songName", "array-contains", "Title1")
.get()...
#Nelson.b.austin Since firestore does not have that yet, I suggest you to have a flat structure, meaning:
Dances = {
danceName: 'Dance name 1',
songName_Title1: true,
songName_Title2: true,
songName_Title3: false
}
Having it in that way, you can get it done:
var songTitle = 'Title1';
var dances = db.collection("Dances");
var query = dances.where("songName_"+songTitle, "==", true);
I hope this helps.
UPDATE 2019
Firestore have released Collection Group Queries. See Gil's answer above or the official Collection Group Query Documentation
Previous Answer
As stated by Gil Gilbert, it seems as if collection group queries is currently in the works. In the mean time it is probably better to use root level collections and just link between these collection using the document UID's.
For those who don't already know, Jeff Delaney has some incredible guides and resources for anyone working with Firebase (and Angular) on AngularFirebase.
Firestore NoSQL Relational Data Modeling - Here he breaks down the basics of NoSQL and Firestore DB structuring
Advanced Data Modeling With Firestore by Example - These are more advanced techniques to keep in the back of your mind. A great read for those wanting to take their Firestore skills to the next level
What if you store songs as an object instead of as a collection? Each dance as, with songs as a field: type Object (not a collection)
{
danceName: "My Dance",
songs: {
"aNameOfASong": true,
"aNameOfAnotherSong": true,
}
}
then you could query for all dances with aNameOfASong:
db.collection('Dances')
.where('songs.aNameOfASong', '==', true)
.get()
.then(function(querySnapshot) {
querySnapshot.forEach(function(doc) {
console.log(doc.id, " => ", doc.data());
});
})
.catch(function(error) {
console.log("Error getting documents: ", error);
});
NEW UPDATE July 8, 2019:
db.collectionGroup('Songs')
.where('songName', isEqualTo:'X')
.get()
I have found a solution.
Please check this.
var museums = Firestore.instance.collectionGroup('Songs').where('songName', isEqualTo: "X");
museums.getDocuments().then((querySnapshot) {
setState(() {
songCounts= querySnapshot.documents.length.toString();
});
});
And then you can see Data, Rules, Indexes, Usage tabs in your cloud firestore from console.firebase.google.com.
Finally, you should set indexes in the indexes tab.
Fill in collection ID and some field value here.
Then Select the collection group option.
Enjoy it. Thanks
You can always search like this:-
this.key$ = new BehaviorSubject(null);
return this.key$.switchMap(key =>
this.angFirestore
.collection("dances").doc("danceName").collections("songs", ref =>
ref
.where("songName", "==", X)
)
.snapshotChanges()
.map(actions => {
if (actions.toString()) {
return actions.map(a => {
const data = a.payload.doc.data() as Dance;
const id = a.payload.doc.id;
return { id, ...data };
});
} else {
return false;
}
})
);
Query limitations
Cloud Firestore does not support the following types of queries:
Queries with range filters on different fields.
Single queries across multiple collections or subcollections. Each query runs against a single collection of documents. For more
information about how your data structure affects your queries, see
Choose a Data Structure.
Logical OR queries. In this case, you should create a separate query for each OR condition and merge the query results in your app.
Queries with a != clause. In this case, you should split the query into a greater-than query and a less-than query. For example, although
the query clause where("age", "!=", "30") is not supported, you can
get the same result set by combining two queries, one with the clause
where("age", "<", "30") and one with the clause where("age", ">", 30).
I'm working with Observables here and the AngularFire wrapper but here's how I managed to do that.
It's kind of crazy, I'm still learning about observables and I possibly overdid it. But it was a nice exercise.
Some explanation (not an RxJS expert):
songId$ is an observable that will emit ids
dance$ is an observable that reads that id and then gets only the first value.
it then queries the collectionGroup of all songs to find all instances of it.
Based on the instances it traverses to the parent Dances and get their ids.
Now that we have all the Dance ids we need to query them to get their data. But I wanted it to perform well so instead of querying one by one I batch them in buckets of 10 (the maximum angular will take for an in query.
We end up with N buckets and need to do N queries on firestore to get their values.
once we do the queries on firestore we still need to actually parse the data from that.
and finally we can merge all the query results to get a single array with all the Dances in it.
type Song = {id: string, name: string};
type Dance = {id: string, name: string, songs: Song[]};
const songId$: Observable<Song> = new Observable();
const dance$ = songId$.pipe(
take(1), // Only take 1 song name
switchMap( v =>
// Query across collectionGroup to get all instances.
this.db.collectionGroup('songs', ref =>
ref.where('id', '==', v.id)).get()
),
switchMap( v => {
// map the Song to the parent Dance, return the Dance ids
const obs: string[] = [];
v.docs.forEach(docRef => {
// We invoke parent twice to go from doc->collection->doc
obs.push(docRef.ref.parent.parent.id);
});
// Because we return an array here this one emit becomes N
return obs;
}),
// Firebase IN support up to 10 values so we partition the data to query the Dances
bufferCount(10),
mergeMap( v => { // query every partition in parallel
return this.db.collection('dances', ref => {
return ref.where( firebase.firestore.FieldPath.documentId(), 'in', v);
}).get();
}),
switchMap( v => {
// Almost there now just need to extract the data from the QuerySnapshots
const obs: Dance[] = [];
v.docs.forEach(docRef => {
obs.push({
...docRef.data(),
id: docRef.id
} as Dance);
});
return of(obs);
}),
// And finally we reduce the docs fetched into a single array.
reduce((acc, value) => acc.concat(value), []),
);
const parentDances = await dance$.toPromise();
I copy pasted my code and changed the variable names to yours, not sure if there are any errors, but it worked fine for me. Let me know if you find any errors or can suggest a better way to test it with maybe some mock firestore.
var songs = []
db.collection('Dances')
.where('songs.aNameOfASong', '==', true)
.get()
.then(function(querySnapshot) {
var songLength = querySnapshot.size
var i=0;
querySnapshot.forEach(function(doc) {
songs.push(doc.data())
i ++;
if(songLength===i){
console.log(songs
}
console.log(doc.id, " => ", doc.data());
});
})
.catch(function(error) {
console.log("Error getting documents: ", error);
});
It could be better to use a flat data structure.
The docs specify the pros and cons of different data structures on this page.
Specifically about the limitations of structures with sub-collections:
You can't easily delete subcollections, or perform compound queries across subcollections.
Contrasted with the purported advantages of a flat data structure:
Root-level collections offer the most flexibility and scalability, along with powerful querying within each collection.
I want select from Firestore collection just articles written NOT by me.
Is it really so hard?
Every article has field "owner_uid".
Thats it: I JUST want to write equivalent to "select * from articles where uid<>request.auth.uid"
TL;DR: solution found already: usages for languages/platforms: https://firebase.google.com/docs/firestore/query-data/queries#kotlin+ktx_5
EDIT Sep 18 2020
The Firebase release notes suggest there are now not-in and != queries. (Proper documentation is now available.)
not-in finds documents where a specified field’s value is not in a specified array.
!= finds documents where a specified field's value does not equal the specified value.
Neither query operator will match documents where the specified field is not present. Be sure the see the documentation for the syntax for your language.
ORIGINAL ANSWER
Firestore doesn't provide inequality checks. According to the documentation:
The where() method takes three parameters: a field to filter on, a comparison operation, and a value. The comparison can be <, <=, ==, >, or >=.
Inequality operations don't scale like other operations that use an index. Firestore indexes are good for range queries. With this type of index, for an inequality query, the backend would still have to scan every document in the collection in order to come up with results, and that's extremely bad for performance when the number of documents grows large.
If you need to filter your results to remove particular items, you can still do that locally.
You also have the option of using multiple queries to exclude a distinct value. Something like this, if you want everything except 12. Query for value < 12, then query for value > 12, then merge the results in the client.
For android it should be easy implement with Task Api.
Newbie example:
FirebaseFirestore db = FirebaseFirestore.getInstance();
Query lessQuery = db.collection("users").whereLessThan("uid", currentUid);
Query greaterQuery = db.collection("users").whereGreaterThan("uid", currentUid);
Task lessQuery Task = firstQuery.get();
Task greaterQuery = secondQuery.get();
Task combinedTask = Tasks.whenAllSuccess(lessQuery , greaterQuery)
.addOnSuccessListener(new OnSuccessListener<List<Object>>() {
#Override
public void onSuccess(List<Object> list) {
//This is the list of "users" collection without user with currentUid
}
});
Also, with this you can combine any set of queries.
For web there is rxfire
This is an example of how I solved the problem in JavaScript:
let articlesToDisplay = await db
.collection('articles')
.get()
.then((snapshot) => {
let notMyArticles = snapshot.docs.filter( (article) =>
article.data().owner_uid !== request.auth.uid
)
return notMyArticles
})
It fetches all documents and uses Array.prototype.filter() to filter out the ones you don't want. This can be run server-side or client-side.
Updating the answer of Darren G, which caused "TypeError: Converting circular structure to JSON". When we perform the filter operation, the whole firebase object was added back to the array instead of just the data. We can solve this by chaining the filter method with the map method.
let articles = []
let articlesRefs = await db.collection('articles').get();
articles = articlesRefs.docs
.filter((article) => article.data.uid !== request.auth.uid) //Get Filtered Docs
.map((article) => article.data()); //Process Docs to Data
return articles
FYI: This is an expensive operation because you will fetching all the articles from database and then filtering them locallly.
Track all user id in a single document (or two)
filter unwanted id out
Use "where in"
var mylistofidwherenotme = // code to fetch the single document where you tracked all user id, then filter yourself out
database.collection("articles").where("blogId", "in", mylistofidwherenotme)
let query = docRef.where('role','>',user_role).where('role','<',user_role).get()
This is not functioning as the "not equal" operation in firestore with string values
You can filter the array of objects within the javascript code.
var data=[Object,Object,Object] // this is your object array
var newArray = data.filter(function(el) {
return el.gender != 'Male';
});
I've found one example of recursive tree fetch but ordering does not work. What is the best way to fetch all items recursively with ordering at each level?
Model
knex.schema.createTable('items', function (table) {
table.increments();
table.string('title');
table.integer('parent');
table.integer('order');
});
var Item = bookshelf.Model.extend({
tableName: 'items',
items: function() {
return this.hasMany(Item, 'parent');
}
});
Recursive but only to certain amount of level, all would be preferred instead of specifying depth. This works but doesn't order related items correctly. Reference
Item.query({where: {parent: null}, orderBy: 'id'})
.fetchAll(
{
withRelated: ['items.items.items.items.items'],
orderBy: 'id'
});
Edit
I'm updating this with a recursive query that works for SQLite (note I'm using id for ordering because my bad choice of column name). Solution maybe to use bookshelf for crud operations and a raw query to populate objects that I need.
with recursive tc( i )
as ( select id FROM items WHERE parent IS NULL
UNION SELECT id FROM items, tc WHERE items.parent = tc.i ORDER BY id
)
select * FROM items WHERE id IN tc;
I wanna use two collections in one reactive table using meteorjs. I have two collections and I want to merge these collection in single table. I have no clues please help!!!
Take a look on reywood:publish-composite package. It allows you to publish reactive joins.
https://github.com/englue/meteor-publish-composite
When using reactive-table you can use a dynamic column which allows you to define a function for a column. You can then use this function to lookup data in another collection.
Example:
{ fieldId: 'someOtherThing', // needs to be unique in your table
key: 'otherId', // this should be the _id of the other collection you want to "join"
label: 'Some Label',
fn: function(_id){
const otherObject = OtherCollection.findOne(_id);
if ( otherObject ) return otherObject.someKey;
}
},
I have a Meteor app that pulls data in from an external api. For simplicity sake we'll say something like...
var foo = Meteor.http.call("GET", "api-endpoint-url-here");
And inserts the data into a collection...
Bar = new Mongo.Collection("bar");
Bar.insert({
Results: foo
});
The json array (e.g. foo) includes a number of individual records each of which has it's own id number and corresponding data. I'm presently using JSON.parse to establish my array and then looping through the array to create individual documents using _.each
var fooParsed = JSON.parse(foo.content)
var fooArray = fooParsed.results;
_.each(fooArray, function(records) {
Bar.insert ({
record: record
});
});
For now it's crude but that aside - I've heard using forEach is preferred for performance. Is that the general consensus and any thoughts on how to streamline this and implement such a loop in this instance?
in case it's helpful to someone else in the future - here's where this netted out
fooArray.forEach(function(item) {
Bar.insert({
_id: item.id,
description: description,
});
});