I have a Firestore data structure and a document where all my followers can see the recentPosts of mine by querying the collection of documents based on the users field of the document where querying users name is present just like below.
my question is how to share a post of others to my followers, currently i am duplicating the shared post to my recentPostsand my seperate Collection of posts documents, but what if a user deletes the post and the post was shared by million users? i have to delete all the shared posts, is there a better solution?
Given your choice in data model, having to delete the duplicated posts is pretty much the normal solution. I also don't see this as problematic, given that:
You've already written the duplicate post to all these followers to begin with, so the delete is just another write.
Deletes and other writes are relatively uncommon in most applications. If not, consider whether you should really be duplicating the data to all followers.
You could choose to implement this with a global list of deleted posts, that each client then reads. But at that point you're making the code that reads data more complex to prevent writes, which is typically not the best approach when using NoSQL databases.
Related
is there any better way to get multiple specific data from collection in firestore?
Let's say have this collection:
--Feeds (collection)
--feedA (doc)
--comments (collection)
--commentA (doc)
users_in_conversation: [abcdefg, hijklmn, ...] //Field contains list of all user in conversation
Then, I'll need to retrieve the user data (name and avatar) from the Users collection, currently, I did 1 query per user, but it will be slow when there are many people in conversation.
What's the best way to retrieve specific users?
Thanks!
Retrieving the additional names is actually a lot faster than most developers expect, as the requests can often be pipelined over a single HTTP/2 connection. But if you're noticing performance problems, edit your question to show the code you use, the data you have, and the performance you're getting.
A common way to reduce the need to load additional documents is by duplicating data. For example, if you store the name and avatar of the user in each comment document, you won't need to look up the user profile every time you read a comment.
If you come from a background in relational databases, this sort of data duplication may be very unexpected. But it's actually quite common in NoSQL databases.
You will of course then have to consider how to deal with updates to the user profile, for which I recommend reading: How to write denormalized data in Firebase While this is for Firebase's other database, the same concepts apply to Firebase. I also in general recommend watching Getting to know Cloud Firestore.
I have tried some solution, but I think this solution is the best for the case:
When a user posts a comment, write a field of array named discussions in the user document containing the feed/post id.
When user load on a feed/post, get all user data which have its id in the user discussions (using array-contains)
it’s efficient and costs fewer transaction processes.
I'm building an app using flutter and firebase and was wondering what the best firestore database structure.
I want the ability for users to post messages and then search by both the content of the post and the posters username.
Does it make sense to create one collection for users with each document storing username and other info and a separate collection for the posts with each document containing the post and the username of the poster?
In the unlikely event where the number of posts exceeds a million or more, is there an additional cost of querying this kind of massive collection?
Would it make more sense to store each user's posts as a sub-collection under their user document? I believe this would require additional read operations to access each document's sub-collection. Would this be cheaper or more expensive if I end up getting a lot of traffic?
is there an additional cost of querying this kind of massive collection?
The cost and performance of reading from Firestore are purely based on the amount of data (number of documents and their size) you retrieve, and not in any way on the number of documents in the collection.
But what is limited in Firestore is the number of writes you can do to data that is "close to each other". That intentionally vague definition means that it's typically better for write scalability to spread the data over separate subcollections, if the data naturally lends itself to that (such as in your case).
To get a great introduction to Firestore, and to data modeling trade-offs, watch Getting to know Cloud Firestore.
How to structure a Social Network database structure like for example twitter where we can follow a users and get all their tweets in our timeline, i have already checked this Firestore - how to structure a feed and follow system
but the solutions in the post look flawed.
Firestore is different where it requires redundant data to access data efficiently, but suppose i am following 1000 people and if i need to get the posts of all those users by querying data for each 15 users i am following and using limit(10) method then orderBy(timeStamp) there may be unread posts between Queries, because we are getting the post using the last post timeStamp , how to structure the data for a social media app in Firestore
When modeling a use-case on a NoSQL database, you tend to optimize for the features of your application, and for frequent read-operations.
So in a social media application your main feature may be that the user sees the recent posts of everyone they follow. To optimize this operation for frequent reads, you'll want to store the posts that the each user should see in a document for that user. So when compared to Twitter, you'd pretty much have a document containing the twitter feed for each user. Or if there's too much data for a single document, you might want to put that in a collection. I often explain this as modeling the screens of your app in the database.
This is very different from the typical data model in a relational database, so it's normal that it takes time to get used to. For a good introduction, I recommend:
Reading NoSQL data modeling.
Watching Firebase for SQL developers, even though it's for the Realtime Database, it explains how to map common SQL concepts to Firebase's NoSQL model.
Watching Getting to know Cloud Firestore
To develop a social media app like Twitter. The Firestore queries are not enough.
Twitter generates a personalized timeline for every user.
This is where the cloud functions come into the picture.
You need a cloud function that monitors for new posts and copies them in their following user's timelines.
You don't need to copy the entire tweet data. You can just copy the tweet id and other fields which require ordering, like timestamp.
So when I query my timeline, I will get all the tweet ids.
Then I can just load the original tweet when the user is about to scroll.
Because the likes and dislikes should affect the original tweet.
I have a users collection and articles collection. Every user starts with all the articles in articles collection which I store as sub-collection of the user and I only keep the articles which the user has not read. When the user reads an document from article collection, I remove it from user's sub-collection. The problem is when a new user signup, I have to fetch all the documents in articles collection and copy over to user's sub-collection which is unnecessary bandwidth usage. Is there any way to minimise it? Is my database model is good enough?
I am more familiar with firestore than firebase but I can give you some ideas until someone better comes along!
You can avoid copying the entire articles by just keeping a list of article IDs for each user that can be used to look up the article in the main list. This list could either be a sub collection or just an array in the user document.
If you were really concerned about bandwidth you could store a list of all the articles in a top level document (that would have to be updated every time you added or removed an article). This stinks a bit of duplication and makes the model more fragile (you must keep two things in sync) but would allow you just to copy this small list instead.
A different approach, if your articles tend to be read in order, is that you could combine a list of unread articles with another field that indicates the latest read article - anything after this article can be considered unread even if not in the list. A new user would then just have this new field set to 0 to indicate no articles read. This also means you wouldn't need to add new articles to all users as each user would check for any articles newer than this new field when they access your service.
Hopefully this can give you some ideas to try and play around with!
I am developing an iOS app with Firebase Realtime Database. The app will potentially have billions of posts with a number of images and data that needs to be retrieved based on the people a specific user follows (something like Instagram).
I understand that the best practice in Firebase is to structure data as flat as possible which would mean having a "Posts" node with potentially billion of entries, which I would then filter by a kind of 'posted_by' parameter. This begs two questions:
1) Will I be able to retrieve said posts with a query that returns posts by any of the users I follow? (By passing something like an array of the users I follow)
2) Will Firebase be effective enough to loop through potentially billions of posts to find the ones that match my criteria, or is there otherwise a better way to structure data so as to make the app as optimal as possible?
Thanks in advance for the answers.
Billions of entries are no problem.
You should check if Firebase is the most cost efficient solution if you have huge volume of data.
1) Firebase can do that, but you probably don't want the user to wait for all entries (when there are a lot for a single user), but instead request them "page" by "page" and only request more pages on demand when the user scrolls up/down.
2) If you ensure you have an index on the user id, then it doesn't have to go through each one individually. Searching by index is efficient.