I have a question about querying with Firestore.
Indeed, I am developing a mobile application of the social network type with a feed of publications from its subscribers.
However, I can't find any solution using Firestore to view posts from all my subscriptions.
Currently, I have a collection "Users" (which contains a uid and my user information), "Publications" (which lists all the publications of the application) and a document with my list of uid to who I am subscriber.
I was thinking of doing a query like "Show all publications in the collection 'Publications' where uid = one of the uid in my array of uid and order by date.
However, I haven't found a solution to do it because the solution I found is to use the IN operator which is limited to 10 items, while I can have thousands of subscriptions.
ref.orderBy('date', 'desc').where('uid', 'in', arrayOfUid).get();
Do you have an idea ?
You can't exceed the limits of the "in" query. If you need more, you will have to perform multiple queries, and merge their results in the app. There are no workarounds to this.
Related
I am trying to model 2 concepts in firestore and also associate
collection: users
key/document_id: email
document: profile info
collection: topics
key/document_id: random
document: metadata with a field indicating email of user (to use for lookups)
My goal is to
"reference" topics in users for easy lookups, but not sure how to do
it other than a sub collection.
Based on email which will be passed as part of auth, I want to have security rule to allow writes in collection only on path, field
based on email
Are both of above feasible in Firebase. Appreciate any pointers!
Preamble: There isn't ONE and only ONE correct approach in NoSQL data modelling
Your approach seems valid, however I would suggest the following adaptations:
"Reference topics in users for easy lookups":
To "reference topics in users for easy lookups" you could duplicate the list of topics in an array in the user profile. You will then be able to use array-contains (and other array membership methods) for your queries. (Note however the limitation of the in operator).
Advantage of this approach: you only need to query one document to get all the topics of a user. Possible drawback: there is a limit on the size for a document (and for a single field value) which is maximum 1 MiB (1,048,576 bytes), see the doc.
You can easily keep in sync the topics array and the topics sub-collection by combining a batched write and the arrayUnion() and arrayRemove() methods.
Use the user ID instead of the email for doc Ids and Security Rules:
Instead of using the email as the users collection document ID and using it in Security Rules, use the user ID. See the examples in the doc.
I have a users collection with a bunch of users and their details. I also have a notifications collection, that the users can query on. I expect the amount of notifications to be at least in the thousands, but probably tens of thousands over the years.
I want the users to be able to mark a notification as "seen". How would I go about this?
I have considered following options:
Add an array notificationsSeen with references to notification documents to each user document. I'm scared of hitting size limits here though, if a user has seen e.g. 50k notifications.
Add the same but as a sub-collection in users. I'm not sure how to go about this though, since I only really need one property (notification ID). Do I put the notification ID as the sub-collection doc ID and have no fields on the documents? Do I let Firestore generate a random ID and assign the notification ID as a property on the sub-collection?
Add an array seenBy with references to user documents to each notification document. Although this will allow users to see which notifications other users have seen, and I don't think I want this.
Hope you can help me out, I'm out of ideas and I am not sure how to implement the best idea I have so far (sub-collection in users), which has also been mentioned as a solution here: Firestore storage size limit how to store large arrays (but without implementation details).
The only scalable way to store an arbitrarily large list of data in Firestore is using documents in a collection. Array type fields do not scale for growing lists of data because the items will eventually exceed the 1MB size limit for a single document, which will obviously cause problems at scale.
It's OK to have a document with no fields. If all you need to do is record that a document exists in order to check its existence later in that collection, that's fine. You can use the notification ID as the document ID if you are absolutely certain that ID conforms to valid IDs in Firestore. Otherwise, you should give it a random ID, and also put the notification ID as a field in the document so that you can query for it later.
You will want to familiarize yourself with the documentation on Firestore limits, which talks about the maximum size of a document, and also the valid characters for a Firestore document ID.
1st option is not possible, as document size limitation defined by firebase, 2nd and 3rd option are possible but 2nd option is a better to implement this feature, and in 2nd option i'll prefer to set notification id as the document id in the subcollection, but setting notification id as property in document is also valid(this option is better suited for case where there are multiple documents with same id, something like posts collection, where user has multiple posts posted).
It’s possible to build a collection for each user for their notifications. You may delete a user-specific document after the notification is read. You may also add some on_snapshot targets to send out notifications after it’s been added to the collection.
This question already has answers here:
Firestore: Does querying a collection also includes their sub collection?
(2 answers)
Closed 3 years ago.
I have collection users which have a sub collection called "notes about". "Notes about" have security rules that only users with role - "manager" can access them. Now everything works nice and so on, but my problem starts when manager decides to update his name, surname or imageUrl.
Since all notes have also property called "addedBy" which contains user name, surname, id and imageUrl, I need to go through all users and all their notes about sub collection documents, so I can check each document and update it with cloud function.
So far looks like I will have to refactor my db and store "notes about" in a separate collection. But maybe there is a way to get all collection documents with all their sub collection documents in another (reasonable) way.
I would prefer not to refactor my DB but maybe that's the only solution at the moment.
What are your thoughts?
DB model:
But maybe there is a way to get all collection documents with all their sub collection documents in another (reasonable) way.
Unfortunately there isn't. Queries in Firestore are shallow, they only get items from the collection that the query is run against. There is no way to get documents from a top-level collection and other subcollections in a single query. Firestore doesn't support queries across different collections in one go. A single query may only use properties of documents in a single collection. IMO, you can go ahead by querying the database multiple times to get the desired values. Is not so slow as might think.
Another possible solution in your case might be to use collection group query. In order to use this new feature, all the subcollections shoould have the same name.
Now it's up to you to decide if you'll use more than one query to get the data, or use collection group query or even refactor the database to fit your needs.
Is there a way with Firebase or Angularfire2 to get all the docs in a collection, as well as, one or all of the immediate subcollections.
For example, if I have a collection of categories where each category has a name, as well as a subcollection of subcategories, is there a way to get all the categories with their subcategory collection?
In my head, I'd want to do something like this:
afs.collection("categories").withSubcollection("subcategories");
Another idea I was thinking of that would work but isn't possible, would be to use a wildcard in the path like this:
afs.collection("categories/{categoryId}/subcategories");
I know neither of these are possible, just wanted to see if there was a way to do what I am showing.
At the moment, I store the subcategories in an array on the category document, but documents have a maximum size, a category could theoretically have an unlimited number of subcategories.
Read operations in Firestore are always shallow, and only return document(s) from a single collection. To read documents from multiple collections, you'll need multiple read operations.
The only exception to this is when the collections you want to query have the same name, in which case you can use a collection group query.
I honestly doubt if storing the categories in a subcollection is worth the hassle. While a document has a maximum size, it's 1MB. I'd highly recommend doing the path on how many categories you can store in that, and whether that is a reasonable limit for your app.
I have two Firestore collections, Users and Posts. Below are simplified examples of what the typical document in each contains.
*Note that the document IDs in the friends subcollection are equal to the document ID of the corresponding user documents. Optionally, I could also add a uid field to the friends documents and/or the Users documents. Also, there is a reason not relevant to this question that we have friends as a subcollection to each user, but if need-be we change it into a unified root-level Friends collection.
This setup makes it very easy to query for posts, sorted chronologically, by any given user by simply looking for Posts documents whose owner field is equal to the document reference of that user.
I achieve this in iOS/Swift with the following, though we are building this app for iOS, Android, and web.
guard let uid = Auth.auth().currentUser?.uid else {
print("No UID")
return
}
let firestoreUserRef = firestore.collection("Users").document(uid)
firestorePostsQuery = firestore.collection("Posts").whereField("owner", isEqualTo: firestoreUserRef).order(by: "timestamp", descending: true).limit(to: 25)
My question is how to query Posts documents that have owner values contained in the user's friends subcollection, sorted chronologically. In other words, how to get the posts belonging to the user's friends, sorted chronologically.
For a real-world example, consider Twitter, where a given user's feed is populated by all tweets that have an owner property whose value is contained in the user's following list, sorted chronologically.
Now, I know from the documentation that Firestore does not support logical OR queries, so I can't just chain all of the friends together. Even if I could, that doesn't really seem like an optimal approach for anyone with more than a small handful of friends.
The only option I can think of is to create a separate query for each friend. There are several problems with this, however. The first being the challenges presenting (in a smooth manner) the results from many asynchronous fetches. The second being that I can't merge the data into chronological order without re-sorting the set manually on the client every time one of the query snapshots is updated (i.e., real-time update).
Is it possible to build the query I am describing, or am I going to have to go this less-than optimal approach? This seems like a fairly common query use-case, so I'll be surprised if there is not a way to do this.
The sort chronologically is easy provided you are using a Unix timestamp, e.g. 1547608677790 using the .orderBy method. However, that leaves you with a potential mountain of queries to iterate through (one per friend).
So, I think you want to re-think the data store schema.
Take advantage of Cloud Functions for Firebase Triggers. When a new post is written, have a cloud function calculate who all should see it. Each user could have an array-type property containing all unread-posts, read-posts, etc.
Something like that would be fast and least taxing.