im trying to use firebase to store and retrieve data for my application.. i know that it is recommended to denormalize data and that it may require data replication..
my scenario is as follows:
there are a number of users in the system..
there are a number of posts in the system..
any user should be able to get a list of posts for a particular user..
each posts has a number of users as participants..
i am tempted to use the following structure for this:
users: {
abc: {
name: 'UserA',
profilePicture: 'imageA.png'
},
pqr: {
name: 'UserB',
profilePicture: 'imageB.png'
},
xyz: {
name: 'UserC',
profilePicture: 'imageC.png'
},
...,
...,
...
},
posts: {
def: {
title: 'PostA',
users: {
abc: true,
def: true,
ghi: true,
...,
...,
...
}
},
stu: {
title: 'PostB',
users: {
abc: true,
xyz: true,
...,
...,
...
}
},
...,
...,
...
}
the issue with this is that if i need to show a list of users with each post, i will need to make a query to POST, and then make sequential calls to USER for each user inside that post to get the name/profilePicture data..
if i replicate the user info inside posts as well, the issue becomes that if a user later changes her profilePicture or name, then existing posts will still show the old data..
how can i structure this data better so these cases are efficient?
thanks..
Don't replicate data inside posts. Read Firebase Docs about structuring data
Best practices:
Avoid nesting data
Flatten data structures
if you include data in post you are breaking those 2 rules (and you don't want it).
Multiple calls are not bad.
Related
I have a firebase database like this structure:
-groups
--{group1id}
---groupname: 'group1'
---grouptype: 'sometype'
---groupmembers
----{uid1}:true
----{uid2}:true
--{group2id}
---groupname: 'group2'
---grouptype: 'someothertype'
---groupmembers
----{uid1}:true
----{uid3}:true
----{uid4}:true
Now, I am trying to pull groups of authenticated user. For example for uid1, it should return me group1id and group2id, and for example uid3 it should just return group2id.
I tried to do that with this code:
database().ref('groups/').orderByChild('groupMembers/' + auth().currentUser.uid).equalTo('true').on('value' , function(snapshot) {
console.log('GROUPS SNAPSHOT >> ' + JSON.stringify(snapshot))
})
but this returns null. if I remove "equalTo" and go it returns all childs under 'groups'.
Do you know any solution or better database structure suggestion for this situation ?
Your current structure makes it easy to retrieve the users for a group. It does not however make it easy to retrieve the groups for a user.
To also allow easy reading of the groups for a user, you'll want to add an additional data structure:
userGroups: {
uid1: {
group1id: true,
group2id: true
},
uid2: {
group1id: true,
group2id: true
},
uid3: {
group2id: true
},
uid3: {
group2id: true
}
}
Now of course you'll need to update both /userGroups and /groups when you add a user to (or remove them from) a group. This is quite common when modeling data in NoSQL databases: you may have to modify your data structure for the use-cases that your app supports.
Also see:
Firebase query if child of child contains a value
NoSQL data modeling
Many to Many relationship in Firebase
I have the following situation:
I have a team entity, in each team we have one or more users.
At first I thought about creating an array of IDS inside team. And then download all team and use the javascript to go through these IDS and fetch the corresponding user.
Something like that:
"teams": {
"xxxxxxx": {
"ids": [0: "bKvysPZZCudBKbbjLYV8ZKr1NUo1", 1: XOvysPZZCudBKbbjLYV8ZKr1NUo1]
}
}
But I do not know if it is the best solution. I would like your opinion.
Tks.
I would recommend making a dictionary of IDs where each ID maps with the boolean value of true. I.e.:
"team-users": {
"team1": [
"uid1": true,
"uid2": true,
...
]
}
if you want to get a team that a user is a part of, then use a parallel structure in you database. Add the following node:
"user-teams": {
"uid1": [
"team1": true,
"team2": true,
...
]
}
Reading from this separate node is faster than querying.
The current query you see below is not efficient because I have not setup the proper indexing. I get the suggestion Consider adding ".indexOn": "users/kxSWLGDxpYgNQNFd3Q5WdoC9XFk2" at /conversations in the console in Xcode. I have tried it an it works.
However, I need the user id after users/ to be dynamic. I've added a link to another post below that has tried a similar thing, but I just can't seem to get it. All help would be much appreciated!
Note: The console output user id above does not match the screenshot below, but does not matter to solve the problem I believe. Correct me if I'm wrong. Thanks!
Here is the structure of my DB in Firebase:
{
"conversationsMessagesID" : "-KS3Y9dMLXfs3FE4nlm7",
"date" : "2016-10-19 15:45:32 PDT",
"dateAsDouble" : 4.6601793282986E8,
"displayNames" : [ “Tester 1”, “Tester 2” ],
"hideForUsers" : [ "SjZLsTGckoc7ZsyGV3mmwc022J93" ],
"readByUsers" : [ "mcOK5wVZoZYlFZZICXWYr3H81az2", "SjZLsTGckoc7ZsyGV3mmwc022J93" ],
"users" : {
"SjZLsTGckoc7ZsyGV3mmwc022J93" : true,
"mcOK5wVZoZYlFZZICXWYr3H81az2" : true
}
}
and the Swift query:
FIRDatabase.database().reference().child("conversations")
.queryOrderedByChild("users/\(AppState.sharedInstance.uid!)").queryEqualToValue(true)
Links to other post:
How to write .indexOn for dynamic keys in firebase?
It seems fairly simple to add the requested index:
{
"rules": {
"users": {
".indexOn": ["kxSWLGDxpYgNQNFd3Q5WdoC9XFk2", "SjZLsTGckoc7ZsyGV3mmwc022J93", "mcOK5wVZoZYlFZZICXWYr3H81az2"]
}
}
}
More likely your concern is that it's not feasible to add these indexes manually, since you're generating the user IDs in your code.
Unfortunately there is no API to generate indexes.
Instead you'll need to model your data differently to allow the query that you want to do. In this case, you want to retrieve the conversations for a specific user. So you'll need to store the conversations for each specific user:
conversationsByUser {
"SjZLsTGckoc7ZsyGV3mmwc022J93": {
"-KS3Y9dMLXfs3FE4nlm7": true
},
"mcOK5wVZoZYlFZZICXWYr3H81az2": {
"-KS3Y9dMLXfs3FE4nlm7": true
}
}
It may at first seem inefficient to store this data multiple times, but it is very common when using NoSQL databases. And is really no different than if the database would auto-generate the indexes for you, except that you have to write the code to update the indexes yourself.
On the firebase structure data section, it shows how to structure data with a many-many user-group situation. But, why they have used "referece":true on both the side instead of using a simple array od ids.
Like, it can be used like both the ways:
A user having array of groups
"groups" : [ "groupId1", "groupId2", ... ]
A user having
"groups": {
"groupId1" : true,
"groupId2" : true,
..
}
They have done it a second way. What is the reason for that?
Something was told at the Google I/O 2016 for that in some video. But, I'm unable to recall.
Example from structure your data:
// An index to track Ada's memberships
{
"users": {
"alovelace": {
"name": "Ada Lovelace",
// Index Ada's groups in her profile
"groups": {
// the value here doesn't matter, just that the key exists
"techpioneers": true,
"womentechmakers": true
}
},
...
},
"groups": {
"techpioneers": {
"name": "Historical Tech Pioneers",
"members": {
"alovelace": true,
"ghopper": true,
"eclarke": true
}
},
...
}
}
Firebase recommends against using arrays in its database for most cases. Instead of repeating the reasons here, I'll refer you to this classic blog post on arrays in Firebase.
Let's look at one simple reason you can easily see from your example. Since Firebase arrays in JavaScript are just associative objects with sequential, integer keys, your first sample is stored as:
"groups" : {
0: "groupId1",
1: "groupId2"
]
To detect whether this user is in groupId2, you have to scan all the values in the array. When there's only two values, that may not be too bad. But it quickly gets slower as you have more values. You also won't be able to query or secure this data, since neither Firebase Queries nor its security rules support a contains() operator.
Now look at the alternative data structure:
"groups": {
"groupId1" : true,
"groupId2" : true
}
In this structure you can see whether the user is in groupId2 by checking precisely one location: /groups/groupId2. It that key exists, the user is a member of groupId2. The actual value doesn't really matter in this case, we just use true as a marker value (since Firebase will delete a path if there's no value).
This will also work better with queries and security rules, because you now "just" needs an exists() operator.
For some great insights into this type of modeling, I highly recommend that article on NoSQL data modeling.
I think I've wrapped my head around denormalization as a primary method of optimization when storing data in Firebase as mentioned in question like this one and in this blog post but I'm getting stuck on one small detail.
Assuming I have two things in my domain, users and posts as in the blog article I mentioned, I might have 20,000 users and 20,000 posts. Because I denormalized everything like a good boy, root/users/posts exists as does root/posts. root/users/posts has a set of post keys with a value of true so that I can get all post keys for a user.
users: {
userid: {
name: 'johnny'
posts: {
-Kofijdjdlehh: true,
-Kd9isjwkjfdj: true
}
}
}
posts: {
-Kofijdjdlehh: {
title: 'My hot post',
content: 'this was my content',
postedOn: '3987298737'
},
-Kd9isjwkjfdj: {
title: 'My hot post',
content: 'this was my content',
postedOn: '3987298737'
}
}
Now, I want to list the title of all posts a user has posted. I don't want to load all 20,000 posts in order to get the title. I can only think of the following options:
Query the root/posts path in some way using the subset of keys that are set to true in the root/users/posts path (if this is possible, I haven't figured out how)
Store the title in the root/users/posts so that each entry in that path has the title duplicated looking like this:
posts: {
-Kofijdjdlehh: true
}
becomes
posts: {
-Kofijdjdlehh: {
title: 'This was my content'
}
}
This seems reasonable, but I haven't seen a single example of doing this, so I'm concerned that it's some anti-pattern.
Another way I haven't been able to find
I appreciate any pointers you might have or documentation I might have missed on this use case.
Either are valid solutions. #1 would be more work for whoever is reading the data, while #2 would be more work when data is saved. Also for #2, you'd have to handle updates to post's titles, though this would be pretty easy with the new multi-path updates.
To implement #1, you'd have you essentially do two queries. Here's a really basic solution which only handles adding posts. It listens for posts being added to the user, and then hooks up a listener to each post's title.
var usersPosts = {};
ref.child('users').child(userId).child('posts').on('child_added', function(idSnap) {
var id = idSnap.key();
ref.child('posts').child(id).child('title').on('value', function(titleSnap) {
usersPosts[id] = titleSnap.val();
});
});
For a third solution, you could use firebase-util, which automagically handles the above scenario and more. This code would essentially do the same as the code above, except it comes with the bonus of giving you one ref to handle.
new Firebase.util.NormalizedCollection(
[ref.child('users').child(userId).child("posts"), "posts"],
[ref.child("posts"), "post"]
).select(
{
key: "posts.$value",
alias: "x"
},
{
key: "post.title",
alias: "title"
}
).ref();
Note that the x value will always be true. It's necessary to select that because firebase-util requires you to select at least one field from each path.