Is there a way to aggregate recurring document property in a collection from a query?
For example, a car collection contains a list of documents with carMake and carMakeModel. The property carMake can contain Honda, Ford, Toyota, and so on... while carMakeModel contains the model of the car i.e. Civic, Prius...
With that sample, is there a way to query car collection displaying only carMake and aggregating similar entries so it won't appear again on the query result?
What I'm currently doing at present is maintaining a separate carMake collection. I'm curious if there's something more efficient than that.
While it is possible to read all cars and then aggregate the makes, the problem with that is that you'll have to do it in your application code. Firestore does not have server-side aggregation capabilities built-in.
The common solution to prevent reading all cars to determine the makes (which typically is much less data), is precisely what you're doing: storing a separate document or collection with just the list of car makes.
To keep this derived list of car makes up to date, you can either write them from client-side code whenever you also write a car, or you can use Cloud Functions to trigger automatically when a car is written and then update the corresponding make.
Related
I'm looking for the most simply designed database structure for a given day, food type, food name and food price.
The part that is giving me trouble is, I feel like, the food name and food price need to be "tied together".
The pictures below are what I have experimented with, but I feel like I am over engineering the structure.
I think you need to understand the firebase ideas of collections and document better - this two level structure is often confusing to developers more familiar with SQL-like databases.
A document contains fields and potentially collections. A collection is always a collection of documents.
So in your case you might make a collection called days which contains documents 'monday','tuesday'....
the monday document might contain some other fields (the chef, the date ect) and a collection (of documents) called 'dessert'
Then each of your dessert documents might include fields for description, price etc.
So in general if you are trying to define a list of multiple similar objects your are wanting a collection of documents. If you are wanting to define fields within an object you should be looking to store them in a document.
I have a user collection. Each user has a list of collections with different names, ie "activities", "time", "distance", "sensor-a", "sensor-b", etc. The id of each document in these collections is common. So for example the all the documents with the same id are associated with each other.
Each document in the "activities" collection has properties that I would search on, while "time", "distance", and the "sensor-x" documents are for the most part a list of data points for various sensors. So for example the first element in each list correspond to each other, ie sensor-a[42] is the sensor reading at time[42], distance[42] and sensor-b[42].
So the data looks something like this:
user/100/activites/200
user/100/distance/200
user/100/time/200
user/100/sensor-a/200
user/100/sensor-b/200
user/100/activites/201
user/100/distance/201
user/100/time/201
user/100/sensor-a/201
user/100/sensor-b/201
I would like to be able to filter out a list of "activities" based on some property of the activity and then get all the documents from the other collections that have the same id as those I filtered.
I know I can create a list of ids I am interested in and then simply get each document one at a time but that does not seem very efficient.
I know that Firestore supports an in query method but that seems to be limited to a maximum of 10 entries, ie my filtered list of activities would need to be 10 or less.
I also see that Firestore has collection groups but they seem to require that all the collections have the same name. I think I could use this but I think I would need to restructure my data, probably adding a level of hierarchy, such as "activities", "distance", and "time" each have a data document that contains the current collections.
Is there any efficient way I can do this query with restructuring my data?
A single Firestore read operation can only read documents from a single collection, or from all collections with the same name. There is no way to otherwise read from multiple collections. The typically workaround is to perform a separate read for each collection, and then merge the results in your application code.
But here I would consider storing all the activity types for a single user in a single subcollection, and giving them a activityType field. One of the unique benefits of using Firestore is the performance of a query does not depend on the number of documents that it needs to consider, so there really is no penalty for combining the activities in a single subcollection.
I have two Firestore collections, Users and Posts. Below are simplified examples of what the typical document in each contains.
*Note that the document IDs in the friends subcollection are equal to the document ID of the corresponding user documents. Optionally, I could also add a uid field to the friends documents and/or the Users documents. Also, there is a reason not relevant to this question that we have friends as a subcollection to each user, but if need-be we change it into a unified root-level Friends collection.
This setup makes it very easy to query for posts, sorted chronologically, by any given user by simply looking for Posts documents whose owner field is equal to the document reference of that user.
I achieve this in iOS/Swift with the following, though we are building this app for iOS, Android, and web.
guard let uid = Auth.auth().currentUser?.uid else {
print("No UID")
return
}
let firestoreUserRef = firestore.collection("Users").document(uid)
firestorePostsQuery = firestore.collection("Posts").whereField("owner", isEqualTo: firestoreUserRef).order(by: "timestamp", descending: true).limit(to: 25)
My question is how to query Posts documents that have owner values contained in the user's friends subcollection, sorted chronologically. In other words, how to get the posts belonging to the user's friends, sorted chronologically.
For a real-world example, consider Twitter, where a given user's feed is populated by all tweets that have an owner property whose value is contained in the user's following list, sorted chronologically.
Now, I know from the documentation that Firestore does not support logical OR queries, so I can't just chain all of the friends together. Even if I could, that doesn't really seem like an optimal approach for anyone with more than a small handful of friends.
The only option I can think of is to create a separate query for each friend. There are several problems with this, however. The first being the challenges presenting (in a smooth manner) the results from many asynchronous fetches. The second being that I can't merge the data into chronological order without re-sorting the set manually on the client every time one of the query snapshots is updated (i.e., real-time update).
Is it possible to build the query I am describing, or am I going to have to go this less-than optimal approach? This seems like a fairly common query use-case, so I'll be surprised if there is not a way to do this.
The sort chronologically is easy provided you are using a Unix timestamp, e.g. 1547608677790 using the .orderBy method. However, that leaves you with a potential mountain of queries to iterate through (one per friend).
So, I think you want to re-think the data store schema.
Take advantage of Cloud Functions for Firebase Triggers. When a new post is written, have a cloud function calculate who all should see it. Each user could have an array-type property containing all unread-posts, read-posts, etc.
Something like that would be fast and least taxing.
In my firestore database, I use the same collection name in different parts of my hierarchy. For example, imagine a stackoverflow-like site with the following 2 collections
/questions/{questionId}/votes/
/questions/{questionId}/answers/{answerId}/votes/
So now I want to create an index on one of these 2 collections. I would expect firestore to require some kind of "path-with-wildcards" like I've used above to identify the data to be indexed. However, instead, they only require the collection name: in this case, "votes".
So if I put an index on "votes" does it apply to both of these collections? Is there any way to put an index on one of these collection and not the other? Is it a best practice to use unique collection names to avoid this issue?
TL;DR:
Yes. Indexes are based on the collection id. This applies to both the ones we create automatically for you on single fields, as well as the composite indexes you create manually. If they are semantically different indexes we recommend you give them unique ids, so you could use question_votes and answer_votes.
More Info
Collection id is the identifier of the collection, excluding the full path. In your case, this is votes as you've noted.
The queries we currently serve use the subset of indexes for a specific path, although we have plans in the future to allow you to do a query that spans all collections with the same collection id (the collection group). This small bit of info adds some context to why.
A second reason is there is a 200 composite index limit in the system, so if someone had a data model structured like the following, /users/{user_id}/blog_posts/{post_id}, there would be no real way for them to create composite indexes on blog_posts for more than a handful of users (not to mention the operational burden of creating new indexes for every user!)
We have an application that allows users to "follow" other users. When a user follows another, we register this data as a document within documentDB, like this:
{
"followerId": "userUUID",
"artistId": "artistUserUUID"
}
We now want to get a list of artists, ordered by the count of followers they have. So I am looking to somehow ask the DB to, based on these documents, give me back an array of artistUserUUId's, ordered by the amount of followers they have registered (as expressed in documents like the example given above).
Alternatively, we are also open to add an Array property to the document of the artistUser themselves, though even in this scenario I am still unsure how to do an ORDER BY based on the counting of a document's property (this property being an array of follower Ids).
I guess a workaround would be to add a stored procedure or trigger that will update a counter property within the artistUser document, but I'd like to validate if these is a way to implement this counting feature natively without such a trick.
Unless you denormalize the follower count into artist user documents (as you suggest), then you'll have to fetch every follower to accomplish your goal. Fetching every follower document, may or may not be prohibitive depending upon how many there are. If you fetch them only into a stored procedure rather than your actual client, it's conceptually no less efficient than an SQL GROUP_BY clause. Design your stored procedure to do the count and only returns the table of artist and counts. A robust implementation would incrementally update your output table in pages and be able to restart where it left off after a stored procedure timeout. Look at my countDocuments example stored procedure in documentdb-mock as well as my "Pattern for writing stored procedures" in the documentation for documentdb-utils for how I typically accomplish this.