I am trying to model 2 concepts in firestore and also associate
collection: users
key/document_id: email
document: profile info
collection: topics
key/document_id: random
document: metadata with a field indicating email of user (to use for lookups)
My goal is to
"reference" topics in users for easy lookups, but not sure how to do
it other than a sub collection.
Based on email which will be passed as part of auth, I want to have security rule to allow writes in collection only on path, field
based on email
Are both of above feasible in Firebase. Appreciate any pointers!
Preamble: There isn't ONE and only ONE correct approach in NoSQL data modelling
Your approach seems valid, however I would suggest the following adaptations:
"Reference topics in users for easy lookups":
To "reference topics in users for easy lookups" you could duplicate the list of topics in an array in the user profile. You will then be able to use array-contains (and other array membership methods) for your queries. (Note however the limitation of the in operator).
Advantage of this approach: you only need to query one document to get all the topics of a user. Possible drawback: there is a limit on the size for a document (and for a single field value) which is maximum 1 MiB (1,048,576 bytes), see the doc.
You can easily keep in sync the topics array and the topics sub-collection by combining a batched write and the arrayUnion() and arrayRemove() methods.
Use the user ID instead of the email for doc Ids and Security Rules:
Instead of using the email as the users collection document ID and using it in Security Rules, use the user ID. See the examples in the doc.
Related
I have a user collection in firestore and each user object has an array of references to "tasks" that they have applied to. Tasks is a separate collection as well and each task object has a user ref array as well.
Collection Tasks:
doc: {
name: "Do something",
time: "Time",
users: ["/users/u1", "/users/u2"]
}
Collection Users:
doc: {
name: Username,
tasks: [ "/tasks/docRef", "/tasks/anotherDoc" ]
}
I have a screen in my react-native app that lists all the tasks and when a task is clicked, it goes to a details screen that displays all the users in a list as well.
Is this the best approach to have this kind of data? Or should I have collections instead of arrays with references. I refrained from collections to prevent duplication of data but I'm not sure if they will be more efficient.
(From the comments )I wanted to inquire if this was the right approach to store the data?
1/ Should I just use uids and then query the collection to matching
those ids?
Storing uid or DocumentReferences in the arrays will not make a difference in terms of ease of querying the corresponding documents. It would only make a difference at the level of the size of the document containing this data since DocumentReference`s are longer than uids).
2/ Or create a sub-collection in the user document itself with references to the tasks?
In the NoSQL world you should not hesitate to denormalize your data.
So having the tasks list in a user doc AND having the users list in a task doc and synchronize the docs when a change is done in one of the collections is a valid approach.
HOWEVER, you may encounter a problem if you have "a lot" of tasks for a given user or a lot of users involved in a task since you may hit the maximum size for a document which is 1 MiB (I agree that you need a LOT of tasks or users :-)).
To avoid that I would advise using sub-collections. This is also the preferred approach if you plan a high frequency of changes that could cause database contention or higher latency, see the documentation section about "Designing for scale".
If the user data you want to show in a task is limited (e.g. just their name plus a button to open each user Profile based on the uid) I would keep an array of users in the task document with this limited amount of data (of course after being sure that there is no risk that a doc reaches the limit of 1 MiB). And have a sub-collection of task documents under each user doc (as advised above).
Modeling well in firestore is a bit difficult, you have to think hard about your use case.
Don't worry about data duplication, this is very common in Firestore. Remembering that you mainly pay for the number of reads and writes.
In your user array, you could keep the necessary data to save you from doing more reads on the user collection.
In your example, the user only has the username, you could keep the username and uid, saved in tasks. That way, no reads would be done on the user collection.
What if the user changes username? Use a batched writes and update all docs that contain that user.
I have an object stored in the Firestore database. Among other keys, it has a userId of the user who created it. I now want to store an email address, which is a sensitive piece of info, in the object. However, I only want this email address to be retrieved by the logged in user whose userId is equal to the userId of the object. Is it possible to restrict this using Firebase rules? Or will I need to store that email address in a /private collection under the Firebase object, apply restrictive firebase rules, and then retrieve it using my server?
TL;DR: Firestore document reads are all or nothing. Meaning, you can't retrieve a partial object from Firestore. So there is no feature at rule level that will give you granularity to restrict access to a specific field. Best approach is to create a subcollection with the sensitive fields and apply rules to it.
Taken from the documentation:
Reads in Cloud Firestore are performed at the document level. You either retrieve the full document, or you retrieve nothing. There is no way to retrieve a partial document. It is impossible using security rules alone to prevent users from reading specific fields within a document.
We solved this in two very similar approaches:
As you suggested, you can move your fields to a /private collection and apply rules there. However, this approach caused some issues for us because the /private collection is completely dettached from the original doc. Solving references implied multiple queries and extra calls to FS.
The second option -which is what the Documentation suggests also, and IMHO a bit better- is to use a subcollection. Which is pretty much the same as a collection but it keeps a hierarchical relationship with the parent coll.
From the same docs:
If there are certain fields within a document that you want to keep hidden from some users, the best way would be to put them in a separate document. For instance, you might consider creating a document in a private subcollection
NOTE:
Those Docs also include a good step-by-step on how to create this kind of structure on FS, how to apply rules to them, and how to consume the collections in various languages
is there any better way to get multiple specific data from collection in firestore?
Let's say have this collection:
--Feeds (collection)
--feedA (doc)
--comments (collection)
--commentA (doc)
users_in_conversation: [abcdefg, hijklmn, ...] //Field contains list of all user in conversation
Then, I'll need to retrieve the user data (name and avatar) from the Users collection, currently, I did 1 query per user, but it will be slow when there are many people in conversation.
What's the best way to retrieve specific users?
Thanks!
Retrieving the additional names is actually a lot faster than most developers expect, as the requests can often be pipelined over a single HTTP/2 connection. But if you're noticing performance problems, edit your question to show the code you use, the data you have, and the performance you're getting.
A common way to reduce the need to load additional documents is by duplicating data. For example, if you store the name and avatar of the user in each comment document, you won't need to look up the user profile every time you read a comment.
If you come from a background in relational databases, this sort of data duplication may be very unexpected. But it's actually quite common in NoSQL databases.
You will of course then have to consider how to deal with updates to the user profile, for which I recommend reading: How to write denormalized data in Firebase While this is for Firebase's other database, the same concepts apply to Firebase. I also in general recommend watching Getting to know Cloud Firestore.
I have tried some solution, but I think this solution is the best for the case:
When a user posts a comment, write a field of array named discussions in the user document containing the feed/post id.
When user load on a feed/post, get all user data which have its id in the user discussions (using array-contains)
it’s efficient and costs fewer transaction processes.
I have a question about querying with Firestore.
Indeed, I am developing a mobile application of the social network type with a feed of publications from its subscribers.
However, I can't find any solution using Firestore to view posts from all my subscriptions.
Currently, I have a collection "Users" (which contains a uid and my user information), "Publications" (which lists all the publications of the application) and a document with my list of uid to who I am subscriber.
I was thinking of doing a query like "Show all publications in the collection 'Publications' where uid = one of the uid in my array of uid and order by date.
However, I haven't found a solution to do it because the solution I found is to use the IN operator which is limited to 10 items, while I can have thousands of subscriptions.
ref.orderBy('date', 'desc').where('uid', 'in', arrayOfUid).get();
Do you have an idea ?
You can't exceed the limits of the "in" query. If you need more, you will have to perform multiple queries, and merge their results in the app. There are no workarounds to this.
I have a users collection with a bunch of users and their details. I also have a notifications collection, that the users can query on. I expect the amount of notifications to be at least in the thousands, but probably tens of thousands over the years.
I want the users to be able to mark a notification as "seen". How would I go about this?
I have considered following options:
Add an array notificationsSeen with references to notification documents to each user document. I'm scared of hitting size limits here though, if a user has seen e.g. 50k notifications.
Add the same but as a sub-collection in users. I'm not sure how to go about this though, since I only really need one property (notification ID). Do I put the notification ID as the sub-collection doc ID and have no fields on the documents? Do I let Firestore generate a random ID and assign the notification ID as a property on the sub-collection?
Add an array seenBy with references to user documents to each notification document. Although this will allow users to see which notifications other users have seen, and I don't think I want this.
Hope you can help me out, I'm out of ideas and I am not sure how to implement the best idea I have so far (sub-collection in users), which has also been mentioned as a solution here: Firestore storage size limit how to store large arrays (but without implementation details).
The only scalable way to store an arbitrarily large list of data in Firestore is using documents in a collection. Array type fields do not scale for growing lists of data because the items will eventually exceed the 1MB size limit for a single document, which will obviously cause problems at scale.
It's OK to have a document with no fields. If all you need to do is record that a document exists in order to check its existence later in that collection, that's fine. You can use the notification ID as the document ID if you are absolutely certain that ID conforms to valid IDs in Firestore. Otherwise, you should give it a random ID, and also put the notification ID as a field in the document so that you can query for it later.
You will want to familiarize yourself with the documentation on Firestore limits, which talks about the maximum size of a document, and also the valid characters for a Firestore document ID.
1st option is not possible, as document size limitation defined by firebase, 2nd and 3rd option are possible but 2nd option is a better to implement this feature, and in 2nd option i'll prefer to set notification id as the document id in the subcollection, but setting notification id as property in document is also valid(this option is better suited for case where there are multiple documents with same id, something like posts collection, where user has multiple posts posted).
It’s possible to build a collection for each user for their notifications. You may delete a user-specific document after the notification is read. You may also add some on_snapshot targets to send out notifications after it’s been added to the collection.