I'm creating my first app in Firebase. I have no experience with NoSQL, so working out my data structure is proving to be a challenge. Let's say my app is similar Reddit where users visit the site and read/write posts. I want the app to have a list view where it sorts the post data in several ways, however it is all centered around the date posts where submitted:
Views
Show the latest posts in descending order.
Show the latest posts for a specific tag.
Show the most liked posts in descending order for the last day (24 hours).
I assume the data structure to look this:
{
"posts": {
"post_0": {
"content": "...",
"created_at": 1497112445748,
"likes": 100,
"tags": {
"tag_0": true,
"tag_2": true
}
},
"post_1": {
"content": "...",
"created_at": 1497112549374,
"likes": 30,
"tags": {
"tag_1": true
}
},
"post_2": {
"content": "...",
"created_at": 1497112640376,
"likes": 70,
"tags": {
"tag_1": true,
"tag_2": true
}
},
...
}
}
View 1
This is probably the easiest to resolve. I imagine the script to retrieve the data would be something like this:
const ref = firebase.database().ref("posts");
const query = ref.orderByChild("created_at").limitToLast(50);
query.on("child_added", (snapshot) => {
// Do stuff like add to array for sorting
});
View 2
This is where things get tricky. Since you can only have one orderBy* per query, the only way I can see to pull this off is to have a tags node that duplicates the date and post ID. For example:
{
"tags": {
"tag_2": {
"post_0": {
"created_at": 1497112445748
},
"post_2": {
"created_at": 1497112640376,
}
},
...
}
}
I've read this is the whole concept of denormalization and structuring your data around your views, but isn't there a better way?
View 3
I don't know how to solve this one at all. As the last 1 day is changing every time the view is requested and the likes are fluctuating often, how can I possibly structure my data around this view?
I've read that push keys, which would take place of the post_n key I have in my example, are sequential and can somewhat be relied on as a timestamp. I'm not sure if there's some way to take advantage of that.
I've found a few useful videos by the Firebase team and articles on Medium, but I'm afraid they don't go far enough for me to understand how to accomplish the needs of my app.
Common SQL Queries converted for the Firebase Database
Firebase Data Structures: Pagination
I'm just find this aspect of Firebase really confusing to get my head around to have it return the data I need for my views.
If anybody can provide me with an example of how to accomplish these things, it would be much appreciated! Thanks!
Related
I'm making an app with database structure like this:
{
"Locations": {
"location1": {
"name": "Nice location"
}
},
"User_posts": {
"user1": {
"post1": {
"location_name": "Nice location",
"location_id": "location1",
"description": "Wow!"
},
"post2": {
"location_name": "Nice location",
"location_id": "location1",
"description": "Nice"
}
}
}
If I have to change location1 name, how to change all location_name's that all users posts have? I have to download all the data before and update it or there is other method?
I think that using location id only to get location name for every location when user enters his posts is not a good idea.
By duplicating data you improve your read performance/scalability at the cost of decreased write performance. This is a normal trade-off in NoSQL databases and in highly scaleable systems in general.
If you want to update the location_name of all posts, you will indeed have to query the posts and update each. If you need to do this regularly, consider keeping a separate lookup list for each location to find the posts where it used. Such an inverted index is another common occurrence in NoSQL databases.
I covered strategies for updating the duplicated data in my answer here: How to write denormalized data in Firebase
Coming from a relational/SQL background, this may initially feel uncomfortable, since it goes against the normalization rules we've been taught. To counter that feeling, I recommend reading NoSQL data modeling, watching Firebase for SQL developers and in general just read some more NoSQL data modeling questions.
You can add one more attribute to location1 , say isLocationOf , which will store all the user id or perhaps post id/post names. Like
"Locations": {
"location1": {
"name": "Nice location",
"isLocationOf": {
'post1': true,
'post2': true
}
}
}
Here isLocationOf is an attribute of Locations whose value is an object.Now if locations1's name gets changed then you can retrieve its isLocationOf object , iterate through it , get all posts id/name containing that location.Then use the post ids to update all entries having this address .
Also whenever you add new post , you have to add its post id/name to isLocation object.
im trying to use firebase to store and retrieve data for my application.. i know that it is recommended to denormalize data and that it may require data replication..
my scenario is as follows:
there are a number of users in the system..
there are a number of posts in the system..
any user should be able to get a list of posts for a particular user..
each posts has a number of users as participants..
i am tempted to use the following structure for this:
users: {
abc: {
name: 'UserA',
profilePicture: 'imageA.png'
},
pqr: {
name: 'UserB',
profilePicture: 'imageB.png'
},
xyz: {
name: 'UserC',
profilePicture: 'imageC.png'
},
...,
...,
...
},
posts: {
def: {
title: 'PostA',
users: {
abc: true,
def: true,
ghi: true,
...,
...,
...
}
},
stu: {
title: 'PostB',
users: {
abc: true,
xyz: true,
...,
...,
...
}
},
...,
...,
...
}
the issue with this is that if i need to show a list of users with each post, i will need to make a query to POST, and then make sequential calls to USER for each user inside that post to get the name/profilePicture data..
if i replicate the user info inside posts as well, the issue becomes that if a user later changes her profilePicture or name, then existing posts will still show the old data..
how can i structure this data better so these cases are efficient?
thanks..
Don't replicate data inside posts. Read Firebase Docs about structuring data
Best practices:
Avoid nesting data
Flatten data structures
if you include data in post you are breaking those 2 rules (and you don't want it).
Multiple calls are not bad.
From everything I have read, it doesn't seem possible to query a multilevel value.
My data structure looks like the following:
{
"dinosaurs": {
"bruhathkayosaurus": {
"meta":{
"addedBy":"John",
"addedDate":"02021987"
},
"appeared": -70000000,
"height": 25
},
"lambeosaurus": {
"meta":{
"addedBy":"Peter",
"addedDate":"12041987"
},
"appeared": -76000000,
"height": 2.1
}
}
}
Without knowing the key name of the dinosaurs, is there anyway to query the meta node retrieving only items added by John.
In JS Something like:
var ref = new Firebase('test.firebaseio.com/dinosaurs');
ref.orderByChild("meta/addedBy")
.equalTo('Peter')
.on("child_added", function(snapshot) {
console.log(snapshot);
});
There are hacky solutions but none are scaleable, should I just flatten this data?
Edit:
I need a code review... would this be an acceptable solution?
var ref = new Firebase('test.firebaseio.com/dinosaurs');
ref.orderByChild("meta")
.on('child_added',function(snap1){
snap1.ref().orderByChild("addedBy")
.equalTo("Peter")
.on('child_added', function(snap2) {
console.log(snap2.val());
})
});
Edit Jan 2016: Since this answer, Firebase has Deep Queries so you can query deeper than 1 level.
Queries can only be 1 level deep. There are a number of solutions but flattening your data and linking/referencing is an option.
In the example above you could create another node that links the user names (parent) to the dinosaurs (children) they added. Then John node can be read and immediately know which dinosaurs he added. Then be able to access other relevant data about that dino; date added, appeared,height etc.
users
John
bruhathkayosaurus
Styracosaurus
Lambeosaurus
Spinosaurus
Peter
Lambeosaurus
Seismosaurus
You will probably want to use uid's instead of names but you get the idea.
Also, it's not clear why there is a meta node in the example listed so it could be flattened thusly:
"dinosaurs": {
"bruhathkayosaurus": {
"addedBy":"John"
"addedDate":"02021987"
"appeared": -70000000
"height": 25
},
I've got two items in my Firebase: providers and services, and I'm trying to figure out the best way to structure and build relationships using Firebase's recommended flattened architecture approach.
My data looks something like this:
{
"services" : {
"hip_replacement" : {
"title" : "Hip Replacement"
}
},
"providers" : {
"the_blue_hospital" : {
"title" : "The Blue Hospital"
}
}
}
I would like to link these two items together so that if you were to visit the Hip Replacement page, The Blue Hospital would show up underneath it, if you were to visit The Blue Hospital page, Hip Replacement would show up underneath that. A two-way relationship, essentially.
What would be the best way to structure something like this? I was thinking along the following lines:
{
"services": {
"hip_replacement": {
"title": "Hip Replacement",
"providers": {
"the_blue_hospital": true,
"the_red_hospital": true
}
},
...
},
"providers": {
"the_blue_hospital": {
"title": "The Blue Hospital",
},
"the_red_hospital": {...
},
"the_green_hospital": {...
}
}
}
Is there a better way to achieve this or a more elegant solution? Any help is appreciated.
Thanks in advance!
The problem with joined data in Firebase is that you optimize for certain read or update use cases at the expense of others. In your sample above, creating or deleting a relationship between services and providers requires two separate updates to each "table". There's really nothing wrong with that, but it's not the only way to go.
For a modestly sized data set, you could have a "join table" that maps services to providers, similar to what might be done in the relational DB world. The data might look something like this:
{
"services": {
"hip_replacement": {}
},
"providers": {
"the_blue_hospital": {...},
"the_red_hospital": {...},
"the_green_hospital": {...}
},
"serviceProviders": {
"-JqD5JX0RUDTXsu7Ok3R": {
"provider": "the_blue_hospital",
"service": "hip_replacement"
}
"-JqDoKfyJqPkQlCXDvFM": {
"provider": "the_green_hospital",
"service": "hip_replacement"
}
"-JbE7Ji_JRz2bHgBdMWQ": {
"provider": "the_blue_hospital",
"service": "hip_replacement"
}
}
There are pros and cons of this approach:
Pro
Easy to add mappings in one place
Easy to delete mappings in one place
Flexible options to reformat the data for display, beyond the context of a single provider or service, such as an index.
Con
You have load the whole data set. Firebase doesn't let you filter within a key, clients have to load the whole list, then filter in memory. I suspect this will work fine for hundreds of records, anyways, maybe for low thousands.
You have to do some client work to filter the list for display and merge it with the actual service and provider data. Again, if the data set isn't too big, underscore/lodash groupBy() can make short work of this.
You should consider:
How much updating and deleting will you do?
Is the join information really that simple? Would you need more records (display names, prices, etc.) that make maintenance on the join table more complicated than I suggested?
Given this database structure in Firebase:
{
"users": {
"user1": {
"items": {
"id1": true
}
},
"user2": {
"items": {
"id2": true
}
}
},
"items": {
"id1": {
"name": "foo1",
"user": "user1"
},
"id2": {
"name": "foo2",
"user": "user2"
}
}
}
which is a more efficient way of querying the items belonged to a specific user?
The Firebase docs seem to suggest this:
var itemsRef = new Firebase("https://firebaseio.com/items");
var usersItemsRef = new Firebase("https://firebaseio/users/" + user.uid + "/items");
usersItemsRef.on("child_added", function(data){
itemsRef.child(data.key()).once("value", function(itemData){
//got the item
});
});
but using the .equalTo() query works as well:
var ref = new Firebase("https://firebaseio.com/items");
ref.orderByChild("user").equalTo(user.uid).on("child_added", function(data){
//got the item
});
The latter code seems more concise and doesn't require denormalization of the item keys into the user records but it's unclear to me if it's a less efficient methodology (assuming I create an index on "user").
thanks.
This is rather old one, but when working on the firebase-backed app, I found myself dealing with similar issues quite often.
.equalTo is more time-efficient (especially, if one user owns big number of items). Although n+1 subscriptions does not lead to n+1 networking roundtrips to the cloud, there is some performance penalty for having so many open subscriptions.
Moreover, .equalTo approach does not lead to denormalization of your data.
There is a gotcha however: When you'll want to secure the data, the .equalTo approach may stop working at all.
To allow user to call orderByChild("user").equalTo(user.uid), they must have read privilege to 'items' collection. This read permission is valid for the whole sub-document rooted at /items.
Summary: If user1 is to be prevented from finding out about items of user2, you must use the BYOI (build your own index) approach. That way you can validate that user only reads items that are put to their index.
Finally, disclaimer :) I use firebase only for a short period of time all I got is a few benchmarks and documentation. If I'm mistaken in any way, please correct me.