I have the following problem while trying to find the best data structure in firebase cloud firestore (NoSql) for a group travel app.
To put it as simple as I can... We have users. A user creates a new trip where he can include his friends. Those friends would also see the trip they where added to.
In a relational data base, we could represent this with 3 tables: Users - UsersTrips - Trips, where UsersTrips would have foreign key to UserId and TripId.
How can I map this problem to NoSql data base? In particulary, to firestore? I guess this is a many-to-many problem, because a trip could hold many users and a user can participate in many trips.
I had a wrong aproach of having a document "trips" that holds an array of the uids of users, but I find it a little bit strange.
Related
I'm trying to create my first DynamoDB based project and I'm having some trouble figuring out the best practices working with a NoSQL database.
My usecase currently is storing users and teams. I have a table that has a partition key of either USER#{userId} or TEAM{#teamId}. If the PK is TEAM{#teamId} I store records with SK either TEAM#{teamId} for team details, or USER#{userId} for the user's details in the team (acceptedInvite, joinDate etc). I also have a GSI based on the userId/email column that allows me to query all the teams a user has been invted to, or the user's team, depending on the value of acceptedInvite field. Attached screenshots of the table structure at the moment:
The table
The GSI
In my application I have an access pattern of getting a team's team members, given a user id.
Currently, I'm doing two queries in my lambda function:
Get user's team, by querying the GSI on PK = {userId} and fitler acceptedInvite = true
Get the team data by querying the table on PK = {teamId} and SK begins_with USER#
This works fine, but I'm concerned I need to preform two separate DynamoDB calls in my API function.
I'm wondering if there's a better way to represent this access pattern and if multiple dynamoDB calls are actually that bad, since I cannot see another way to do this.
Any kind of feedback is appreciated!
The best way to avoid making two queries like this is to supply the API caller with all the information needed to make a single DynamoDB request. For your case this means supplying the caller with the teamId. You can do this as either as part of a list operation response, or if it is the authenticated user, then as part of their claims in a JWT.
I have an app that helps store owners manage their inventory through a simple API-driven interface.
My app stores all data on Firestore. My simplified database looks like this:
-users
-name
-email
-uid
-products
-atts
...
-ownerId
-someOtherThing
-atts
...
-ownerId
The idea is that only documents with ownerId that matches the current user ID will be accessible to the user. User with ID=5 will only have access to items that match ownerId=5.
Is this a good way of storing this data? I am worried that I will eventually end up with thousands of documents in that collection and querying them by "ownerId" might not be the best way to tackle this. On the other hand, I might end up with hundreds of users too, which probably makes it bad design to introduce several new collections for each of them?
What would be a better approach design-wise?
While "a good way" is subjective and purely dependent on the use-cases of your app, what you're proposing is quite a common way to store data in Firestore.
Your concern about the number of users and other documents is unwarranted, as Firestore guarantees that the performance of returning the (say) products for a specific user depends solely on the number of products returns, not on the total number of products in the database.
So if you have 10 products that you're the ownerId for, then no matter how many other users/products there are, the amount of time it takes to retrieve your 10 products will always be the same.
I'm totally new to Firebase, and I'm trying to get my head round the best db model design for 'relational' data, both 1-1 and 1-many.
We are using the Firestore db (not the realtime db).
Say we have Projects which can contain many Users, and a User can be in multiple Projects
The UI needs to show a list of Users in a Project which shows things like email, firstname, lastname and department.
What is the best way to store the relationship?
An array of User ids in the Project document?
A map of Ids in the Project document?
Ive read the above approaches were recommended, but was that for realtime database? Firestore supports Sub Collections, which sound more appropriate...
A sub collection of Users in the Project document?
A separate collection mapping Project id to User id?
A Reference data type? I've read here https://firebase.google.com/docs/firestore/manage-data/data-types about Reference data type, which sounds like what I want, but I cant find any more on it!
If its just a map or array of Ids, how would you then retrieve the remaining data about the user? Would this have to sit in the application UI?
If its a sub collection of Users documents, is there any way to maintain data integrity? If a user changed their name, would the UI / a cloudFunction then have to update every entry of that users name in the Sub collections?
any help / pointers appreciated...
The approach for modeling many-to-many relationships in Firestore is pretty much the same as it was in Firebase's Realtime Database, which I've answered here: Many to Many relationship in Firebase. The only difference is indeed that you can store the lookup list in a sub-collection of each project/user.
Looking up the linked item is also the same as before, it indeed requires loading them individually from the client. Such a client-side join is not nearly as slow as you may initially expect, so test it before assuming it can't possibly be fast enough.
Ensuring data integrity can be accomplished by performing batched writes or using transactions. These either completely succeed or completely fail.
I have a collection, itemsCollection, which contains a very large amount of small itemDocs. Each itemDoc has a subcollection, statistics. Each itemDoc also has a field "owner" which indicates which user owns the itemDoc.
itemsCollection
itemDoc1
statistics
itemDoc2
statistics
itemDoc3
statistics
itemDoc4
statistics
...
I also have a collection, usersCollection, which contains basic user info.
usersCollection
user1
user2
user3
...
Since each itemDoc belongs to a specific user, it's necessary to display to each user which itemDocs they own. I have been using the query:
db.collection("itemsCollection").where("owner", "==", "user1")
I am wondering if this will scale effectively, i.e. whenever itemsCollection gets to be millions of records? If not, is the best solution to duplicate each itemDoc and its statistics subcollection as a subcollection in the user document, or should I be doing something else?
As Alex Dufter, the product manager from Firebase, explained in one of days at Firebase Dev Summit 2017 that Firestore was inspired in many ways by the feed-back that they had on the Firebase Realtime Database over the years. They faced two types of issues:
Data modelling and querying. Firebase Realtime Database cannot query over multiple properties because it ussaly involves duplication data or cliend-side filtering, which we all already know that is some kind of messy.
Realtime Database does not scale automatically.
With this new product, they say that you can now build an app and grow it to planetary scale without changing a single line of code. Cloud Firestore is also a NoSQL database that was build specifically for mobile and web app development. It's flexible to build all kinds of apps and scalable to grow to any size.
So because the new database was build knowing this iusses, duplication data is not nedeed anymore. So you will not have to worry about using that line of code, even if your data will grow to millions of records, it will scale automatically. But one thing you need to remember, if you will use multiple conditions, don't forget to use the indexes by simply adding them in the Firebase console. Here are two simple examples from the offical documentation:
citiesRef.whereEqualTo("state", "CO").whereEqualTo("name", "Denver");
citiesRef.whereEqualTo("state", "CA").whereLessThan("population", 1000000);
In a Firebase social-media-esque app, here is how I'm storing groups along with its members:
Here, I have a reciprocating relationship in terms of memberships of groups. A group knows all of the users that are a part of it in group-users and vise-versa—a user knows all of the groups it's in via user-groups. The reason I thought to do this is it seems to be quick for a user to access all of its groups, and likewise it seems to be quick for a group to access all of its users. However, having this reciprocating data is costly in terms of space. My other option would be to keep just the user-groups node, and to retrieve all the members of a single group I would need to query that node by group ID. This option seems slower but would take less space in the database. Which option is more advised? Thanks.
The best data model always depends on the use-cases of your app. If you need access to both the groups-for-a-user and the users-for-a-group, the cheapest way to do so is to store both.
See:
Many to Many relationship in Firebase
the answer to this question in a video
this article about NoSQL data modeling
the Firebase video series Firebase for SQL developers