I am making a type of social media app where there is a list of conversations available for all users to see. I have implemented a feature for a blue circle to be found on unread messages:
however my implementation seems like a bad foundation to scale up on. I am looking to a better solution, if possible. The current method is as follows:
1)This app is built around a few special users, called Stingray, who can start chats.
2)the chats live in a collection called StingrayChats. One document holds a list of Chat items. There is one chat document for each stingray. All chats in a collection a queried, and combined into a single chats list on the frontend.
Next, there is a separate collection called Viewers. Similarly, there is one viewer document for each stingray. Each viewer document contains a list of ChatViewer objects, which contains:
-A string, chatId
-A list of strings id's of viewers who have viewed the chat
In the frontend, once again all viewers documents are compiled into a single list, and if a chat's ChatViewer object contains an array with the user's id, this means the user has seen the chat, and the circle will no longer appear.
the big problem with this at the moment is that it costs a large amount of unnecessary reads, in that when the array is updated a user will read it, which does not actually reflect anything in the ui if the userId is not the current user.
Is there any better implementation for something like this? To reiterate, it would need to:
Reset everytime a new message was sent in the chat
Keep track of what users have seen it
Change when a user clicks on it, if applicable
Thanks!
Related
In my app, a user can log in, then a list of their "pages" (a collection of documents) is retrieved, and their "selectedPageId" is retrieved. This way the UI can show a list of page names, and the content of the selected page.
This means three requests have to happen, each waiting for the other:
log in
get list of pages
get selected page id
Is there a way to make all this happen on the server at once so I can make a single request, which I assume would make my UI render more quickly?
As Dharmaraj mentioned in his comment, "Firebase Auth will still be required on client side". However, is you use the default Auth state persistence (i.e. firebase.auth.Auth.Persistence.LOCAL) once a user is signed in he/she doesn't need to sign in the next times ("the state will be persisted even when the browser window is closed. An explicit sign out is needed to clear that state"). So signing in is not really a problem IMO.
For the Firestore queries: Executing the two queries back-to-back with the second one returning only one document should not take a lot of extra time compare to executing only the first query (which fetches the pages list).
Having said that, one possible approach would be to save, for each user, a Firestore document containing the list of the user's pages plus the user's selectedPageId. You can maintain this page with a Cloud Function which, for example, mirrors the list (documents in a collection) with an array in this Firebase doc. However, you need to take care to not reach the maximum size for a document i.e. 1MiB (hence my question on the number of fields displayed for a page).
One possible variation in this case would be to initially present, in your first app page, a limited number (e.g. 25) of pages of the list, with a "Show more" button.
Finally, note that using a Cloud Function as mentioned in the above comments will very most probably increase the response time compare to standard queries executed via the SDK.
I have been using Firestore for a very long time. I am building an app now where scalability and keeping low costs is important. (I am using flutter)
My app has users, which have user profiles, also they can add friends and talk to them (like instagram or facebook).
I have a problem building this friends system.
My model for this friends system currently looks like this:
Users collection. Each document id = user id from auth, those docs contain data like name, username, profile picture, etc.
Friends collection. Each document id = user id from auth. For each user, those docs contain a field called: friends, which is an array with each of his friends user ids.
The model looks like:
Friends collection:
- uid:
- friends_list: [friend_uid1, friend_uid2, ...]
This is how my "backend" looks.
Now I want to show my user a list of his friends. How do I do that?
I want a list that looks like instagram, with a nice UI showing each of my user friend profile pic, name, last message, etc.
I can not find a straight forward way to do this with Firestore and queries.
Let's say I do it like this:
Get all my friends user ids in an array.
Get all their user documents using .get() for each document.
This is not doable in firestore cause it would eliminate all the querying power I have (such being able to query only for users with name "x"), I would have to fetch all users and do the query on my front-end (or in a cloud function, same thing, not scalable).
If I do this like:
Get all document using a query for all users in the Friends collection, where friends_list contains my user id.
Save from those documents only the documentID and fetch all the friends user data manually.
This comes with another problem. In Firestore there is no way of fetching a document without fetching all of its fields, so the first query which I use to get the ids only of my friends would actually give me their id + their friend list instead (cause when I query, it also gets the document id + the data), which is not good.
If I do it like:
When you add a friend, instead of just saving its uid, save its uid + data.
Now I can easily show my user his friends list nicely and do some querying on front-end.
The problem here is that now if one of my friends updates his profile photo, I need to update it in every document of all of his friends, which is very write expensive for just a little profile update.
There is also the problem of watching for more data, maybe I have another collection with Chats, and I want to show the last message of my chat with a friend, now I have to fetch the chat rooms too, which is more hard to query data that comes with all the problems that I mentioned before.
In conclusion: I don't see a good scalable way to do this kind of system within Firestore. It seems a simple system which any basic app should have, but I do not see how I can do it in a way that does not make lots of reads or read more data (or sensitive data) than it should.
What kind of model would you do for a friends system like this?
You're decribing a quintessential drawback of NoSQL Databases.
A similar example is actually given in the Get to Know Cloud Firestore series.
Like others have commented, the answer really depends on your application. And this is the assessment you'll have to do. Like which of the options is cheaper depending on the use case of the app.
For example, if you go with your third option and store the friend's user data that you'll need to populate the list. This means you'll have to implement measures to keep the integrity of the copied data whenever the user updates their information.
You can then look at the usage of your app and determine how often users change their information vs how often you would need to retrieve full users if you don't copy the data to find the cheapest method for your application.
I would appreciate some guidance on how to structure data stored within an app. While there are some reasons for the first way, I'm concerned it wouldn't be able to operate efficiently for the second case.
Simplified, the app would contain a list of Places by State. The main use case would be viewing Places within a selected State. The second use case would be that individual users could save specific Places they liked into their profile and view them all at once (showing all state Places in one list).
Option 1- Places saved in one "places" collection, which has a field of "state."
Main use: To show these places by state, the app would query where the "state" field matches the state.
Secondary use: When a user saved the place, the app would save the docID for each place into the user's profile, each of which would need to be retrieved to show the list of places.
Option 2- Have one collection per state.
Main use: To show these places by state, the app would pull all documents within the query and list them out.
Secondary use: When a user saved the place to the user's profile, the app would save the docID for each place into the user's profile, distributed across the different collections, each of which would need to be retrieved to show the list of places.
Goals:
Use the same place document to appear in both the State lists and the user's profile.
Minimize the number of calls/slowness as much as possible in the Secondary use case.
I have been reviewing Firestore data storage guidelines, but I would appreciate any thoughts from experienced developers regarding this data structure.
There is no "perfect", "the best" or "the correct" solution for structuring a Firestore database. We are usually structuring the database according to the queries that we intend to perform.
Regarding storing all the places in a single collection vs. having one collection per state, please note that there is no difference in terms of speed or costs. You'll always have to pay a number of reads that is equal to the number of documents that your query returns. However, if you need to display in your app, for example, all places of all states, then having a collection for each state, will require a separate query for each state.
Furthermore, regarding saving a list of places in a user's profile vs. storing only the IDs, it's a matter of measurement. You should measure how often the details within the places are changed. Remember that if a place is changed, then you should update that data in all places it exists. So if it's not changed so often then you can save the entire place object, otherwise, save only the ID.
I have a users collection with a bunch of users and their details. I also have a notifications collection, that the users can query on. I expect the amount of notifications to be at least in the thousands, but probably tens of thousands over the years.
I want the users to be able to mark a notification as "seen". How would I go about this?
I have considered following options:
Add an array notificationsSeen with references to notification documents to each user document. I'm scared of hitting size limits here though, if a user has seen e.g. 50k notifications.
Add the same but as a sub-collection in users. I'm not sure how to go about this though, since I only really need one property (notification ID). Do I put the notification ID as the sub-collection doc ID and have no fields on the documents? Do I let Firestore generate a random ID and assign the notification ID as a property on the sub-collection?
Add an array seenBy with references to user documents to each notification document. Although this will allow users to see which notifications other users have seen, and I don't think I want this.
Hope you can help me out, I'm out of ideas and I am not sure how to implement the best idea I have so far (sub-collection in users), which has also been mentioned as a solution here: Firestore storage size limit how to store large arrays (but without implementation details).
The only scalable way to store an arbitrarily large list of data in Firestore is using documents in a collection. Array type fields do not scale for growing lists of data because the items will eventually exceed the 1MB size limit for a single document, which will obviously cause problems at scale.
It's OK to have a document with no fields. If all you need to do is record that a document exists in order to check its existence later in that collection, that's fine. You can use the notification ID as the document ID if you are absolutely certain that ID conforms to valid IDs in Firestore. Otherwise, you should give it a random ID, and also put the notification ID as a field in the document so that you can query for it later.
You will want to familiarize yourself with the documentation on Firestore limits, which talks about the maximum size of a document, and also the valid characters for a Firestore document ID.
1st option is not possible, as document size limitation defined by firebase, 2nd and 3rd option are possible but 2nd option is a better to implement this feature, and in 2nd option i'll prefer to set notification id as the document id in the subcollection, but setting notification id as property in document is also valid(this option is better suited for case where there are multiple documents with same id, something like posts collection, where user has multiple posts posted).
It’s possible to build a collection for each user for their notifications. You may delete a user-specific document after the notification is read. You may also add some on_snapshot targets to send out notifications after it’s been added to the collection.
I have a users collection and articles collection. Every user starts with all the articles in articles collection which I store as sub-collection of the user and I only keep the articles which the user has not read. When the user reads an document from article collection, I remove it from user's sub-collection. The problem is when a new user signup, I have to fetch all the documents in articles collection and copy over to user's sub-collection which is unnecessary bandwidth usage. Is there any way to minimise it? Is my database model is good enough?
I am more familiar with firestore than firebase but I can give you some ideas until someone better comes along!
You can avoid copying the entire articles by just keeping a list of article IDs for each user that can be used to look up the article in the main list. This list could either be a sub collection or just an array in the user document.
If you were really concerned about bandwidth you could store a list of all the articles in a top level document (that would have to be updated every time you added or removed an article). This stinks a bit of duplication and makes the model more fragile (you must keep two things in sync) but would allow you just to copy this small list instead.
A different approach, if your articles tend to be read in order, is that you could combine a list of unread articles with another field that indicates the latest read article - anything after this article can be considered unread even if not in the list. A new user would then just have this new field set to 0 to indicate no articles read. This also means you wouldn't need to add new articles to all users as each user would check for any articles newer than this new field when they access your service.
Hopefully this can give you some ideas to try and play around with!