I have referred on the below sites for getting ideas on better Firebase schema design for a social network application.
https://firebase.googleblog.com/2015/10/client-side-fan-out-for-data-consistency_73.html
Firebase fan-out structure for news feed
https://highlyscalable.wordpress.com/2012/03/01/nosql-data-modeling-techniques/
https://www.airpair.com/firebase/posts/structuring-your-firebase-data
But I fail to understand the recent developments and features available in Cloud FireStore of Firebase.
I have below requirements to be store.
To Store Image Upload
Comments for the Image
Likes activity
User following activity
I should be able to get them individually or all activities in a single call. How best can I organize the data model?
All the above is present in the below example. But I have question on getting all types of activities to show the master news feed in a single call sorted by time.
https://github.com/firebase/friendlypix/blob/master/design.md
PS: I will be using it with Flutter.
Other models discussed: Firestore - how to structure a feed and follow system
This Udemy course gives a very good idea about how to design a fanned-out social media DB. However, it doesn't implement the "fanoutObject" concept mentioned in this blog. It is updating each duplicate collections independently. So not sure how consistency is guaranteed.
Related
I am working on a personal project to recreate the news feed of Facebook. So what I am trying to do is to recreate the scenario where when the user goes to the news feed, the user gets posts of everyone he follows only. Is there any way to run a query like that using the Firebase real-time database using an of "followings".
I can successfully generate single users posts in the android studio app using snapshot and recycler view.
If you're asking whether you can get posts from multiple userUID values with a single query, that is not possible.
If you're asking whether you can pass a list of postUID values to retrieve, that is also not possible.
In both cases the solution is to execute a separate query/read operation for each of the values, and merge the results in your application code. This is not nearly as slow as you may think, since Firebase pipelines the requests over a single web socket connection - which is quite efficient. For more on this, see Speed up fetching posts for my social network app by using query instead of observing a single event repeatedly
The problem is as follows. We have 2 native apps (ios and android) connected to a Firestore backend. Our current data model stores the information about the users spread across 5 different collections (profile data, user answers, user legal documents, etc). This means, often times, when we need to query data about the user we need to make several queries and manually join them together to get what we need. It’d be much simpler for us to have all the information stored in a single collection. This is a problem we have now, and as our business will continue to evolve, we will have more situations where we need to change the structure of the data model.
Currently, we use the Firestore API to query the user’s data in the frontend to have real-time updates. ATM we don’t use custom endpoints and therefore we don’t have any kind of versioning in place.
Are there any best practices or strategies to perform such a data model migration without forcing the user to upgrade to the latest version of the app?
We can think of some solutions:
Use versioned-endpoints and forget about real-time updates (decouple the FE from the BE)
Migrate the current data to the new data model while keeping the old collections still actives to serve old versions of the app. Additionally, write some cloud function listeners to keep the 2 models in sync (whenever an old version of the app writes in any of the 5 user collections, copy that data to the new data model)
...
…
The problem of an evolving data model and having several FE’s consuming the data is extremely common. The normal best practice would be to have a versioned endpoint that the FE communicate to, to break the direct dependency on the data model. However, it doesn’t seem like Firebase have a set best practice for this. This sounds a bit strange to us, as it is such a common problem to have, and that Firebase solves so many other common challenges out of the box.
What are we missing?
Related questions:
https://www.reddit.com/r/Firebase/comments/dyhzlv/best_practices_of_versioning_with_a_firestore/
Thanks!
I have seen a firebase api in which you can create firebase projects and it occurred to me for example, when a user wants to counter a SaaS, make it easier for me to create a project and connect it to their SaaS but I have the following questions:
If there are supposed to be limits to creating firebase projects in a google account, will there be a time when databases can no longer be created for new users?
If the above is true, how can this be solved?
I have seen that you can ask for more projects, but how many can I have?
I know that with firestore, I can model data and only in a database have all the information, but for example, each user may have special requirements in their system and also provide security and information saving operations that would be impossible if all information is in a single database.
Thanks for the help.
EDIT
"How many projects can you have as a developer?" Yes, that's what I mean, having all the user bases in a single firebase google account. For example, on a web page, the user wants to pay for the application, with firebase admin and google cloud functions, I can automatically create a project and have all the databases in a google account. And what I want to know is if you can have multiple databases. I have seen that you can ask google to give you permission to host several projects but, for example, can you have more than 100 projects or even 1000? (I may never have reached that number, but in that case, I would like to know the limits that can be reached).
Edit 2
This first structure I have all the documents in a "universal" collection and I identify them with an id to know the user who used it.
But the second structure seems to me more organized and clean, in addition to the fact that users at no time need to share information among others. I have read that having nested collections is not good but over the years and the progress that firestore has had, this is no longer or was a problem only that the limitation is that you can only have up to 100 collections anidades but I never think to reach that quantity.
Inside of list, have all products for that user, because inside of document, only can have 1 mb of data and download 1 mb and is not the best option.
in the firestore documentation I see that it does not reveal other problems, it only says that it is difficult to delete the subcollection, but I do not allow users to do that and if I need to delete the subcollections, with the Google Cloud functions i can do it.
So, the second structure is more intuitive for me, but is the best option for that? or actually firestore is not good for this strcucutre?
I have a question about the best way to structure my Firebase database for my iOS app. The basic structure is that there are users and posts. The app will open and the user who is logged in will see all the posts that he/she has uploaded to the database (I may update this down the road so that users can see posts of users they follow). Is it better to have each post have a userId on it? Or is it better to have the an array of posts associated to each user?
If you can shard/partition the data so that you don't need to query a potentially very long list of that, that is always preferable with the realtime database.
So in your case, if you know that you want to show the user a list of their own posts as a starting point, it is a good idea to model that list in your database. You'd typically call this the user's feed or wall, depending on what your social network of choice is.
If you then later want to also show the posts of users they follow, you might want to store those posts in the user's wall. This type of data duplication may seem unnatural at first, but is quite common in NoSQL databases. In fact, this is the exact model that Firebase uses in its class FireFeed example.
To learn more about NoSQL data modeling, see:
NoSQL data modeling.
Firebase for SQL developers.
Getting to know Cloud Firestore, which is about Firebase's newer Firestore database, but has great tips that apply to the Realtime Database too.
I would like to use Firebase Firestore for my next project but I need some help understanding the limitations of what is possible with Firestore queries.
Basically, I would like to perform queries over collections with large amounts of documents but upon doing research I have come across conflicting information.
This video (from the Firebase team): https://youtu.be/W3xIOQu0h1w?t=11m50s states that you can perform a query over a collection with "billions" of documents and maintain the same level of performance compared to a query over a collection with a few documents.
Then, I came across this github issue where the poster states that queries are taking too long and demand alot from the system. A member of the Firebase team answers by stating that performing a query over a collection containing 35k documents is beyond the "performance envelope".
So can someone point me in the right direction concerning Firestore queries and its limitations.
Please let me know if I was not clear in any part of this post.
That GitHub issue you linked to is specifically talking about offline searches. This means the Firestore backend service is not available, so the search is performed on the client. This is a very different situation than when the client is online and the service can perform the query in a massively scalable way. (Client apps are never massively scalable.)