Structuring Relationships in Firebase - firebase

I've got two items in my Firebase: providers and services, and I'm trying to figure out the best way to structure and build relationships using Firebase's recommended flattened architecture approach.
My data looks something like this:
{
"services" : {
"hip_replacement" : {
"title" : "Hip Replacement"
}
},
"providers" : {
"the_blue_hospital" : {
"title" : "The Blue Hospital"
}
}
}
I would like to link these two items together so that if you were to visit the Hip Replacement page, The Blue Hospital would show up underneath it, if you were to visit The Blue Hospital page, Hip Replacement would show up underneath that. A two-way relationship, essentially.
What would be the best way to structure something like this? I was thinking along the following lines:
{
"services": {
"hip_replacement": {
"title": "Hip Replacement",
"providers": {
"the_blue_hospital": true,
"the_red_hospital": true
}
},
...
},
"providers": {
"the_blue_hospital": {
"title": "The Blue Hospital",
},
"the_red_hospital": {...
},
"the_green_hospital": {...
}
}
}
Is there a better way to achieve this or a more elegant solution? Any help is appreciated.
Thanks in advance!

The problem with joined data in Firebase is that you optimize for certain read or update use cases at the expense of others. In your sample above, creating or deleting a relationship between services and providers requires two separate updates to each "table". There's really nothing wrong with that, but it's not the only way to go.
For a modestly sized data set, you could have a "join table" that maps services to providers, similar to what might be done in the relational DB world. The data might look something like this:
{
"services": {
"hip_replacement": {}
},
"providers": {
"the_blue_hospital": {...},
"the_red_hospital": {...},
"the_green_hospital": {...}
},
"serviceProviders": {
"-JqD5JX0RUDTXsu7Ok3R": {
"provider": "the_blue_hospital",
"service": "hip_replacement"
}
"-JqDoKfyJqPkQlCXDvFM": {
"provider": "the_green_hospital",
"service": "hip_replacement"
}
"-JbE7Ji_JRz2bHgBdMWQ": {
"provider": "the_blue_hospital",
"service": "hip_replacement"
}
}
There are pros and cons of this approach:
Pro
Easy to add mappings in one place
Easy to delete mappings in one place
Flexible options to reformat the data for display, beyond the context of a single provider or service, such as an index.
Con
You have load the whole data set. Firebase doesn't let you filter within a key, clients have to load the whole list, then filter in memory. I suspect this will work fine for hundreds of records, anyways, maybe for low thousands.
You have to do some client work to filter the list for display and merge it with the actual service and provider data. Again, if the data set isn't too big, underscore/lodash groupBy() can make short work of this.
You should consider:
How much updating and deleting will you do?
Is the join information really that simple? Would you need more records (display names, prices, etc.) that make maintenance on the join table more complicated than I suggested?

Related

Updating a string in multiple locations in firebase

I'm making an app with database structure like this:
{
"Locations": {
"location1": {
"name": "Nice location"
}
},
"User_posts": {
"user1": {
"post1": {
"location_name": "Nice location",
"location_id": "location1",
"description": "Wow!"
},
"post2": {
"location_name": "Nice location",
"location_id": "location1",
"description": "Nice"
}
}
}
If I have to change location1 name, how to change all location_name's that all users posts have? I have to download all the data before and update it or there is other method?
I think that using location id only to get location name for every location when user enters his posts is not a good idea.
By duplicating data you improve your read performance/scalability at the cost of decreased write performance. This is a normal trade-off in NoSQL databases and in highly scaleable systems in general.
If you want to update the location_name of all posts, you will indeed have to query the posts and update each. If you need to do this regularly, consider keeping a separate lookup list for each location to find the posts where it used. Such an inverted index is another common occurrence in NoSQL databases.
I covered strategies for updating the duplicated data in my answer here: How to write denormalized data in Firebase
Coming from a relational/SQL background, this may initially feel uncomfortable, since it goes against the normalization rules we've been taught. To counter that feeling, I recommend reading NoSQL data modeling, watching Firebase for SQL developers and in general just read some more NoSQL data modeling questions.
You can add one more attribute to location1 , say isLocationOf , which will store all the user id or perhaps post id/post names. Like
"Locations": {
"location1": {
"name": "Nice location",
"isLocationOf": {
'post1': true,
'post2': true
}
}
}
Here isLocationOf is an attribute of Locations whose value is an object.Now if locations1's name gets changed then you can retrieve its isLocationOf object , iterate through it , get all posts id/name containing that location.Then use the post ids to update all entries having this address .
Also whenever you add new post , you have to add its post id/name to isLocation object.

How to do Complex Querying with Firebase?

I'm creating my first app in Firebase. I have no experience with NoSQL, so working out my data structure is proving to be a challenge. Let's say my app is similar Reddit where users visit the site and read/write posts. I want the app to have a list view where it sorts the post data in several ways, however it is all centered around the date posts where submitted:
Views
Show the latest posts in descending order.
Show the latest posts for a specific tag.
Show the most liked posts in descending order for the last day (24 hours).
I assume the data structure to look this:
{
"posts": {
"post_0": {
"content": "...",
"created_at": 1497112445748,
"likes": 100,
"tags": {
"tag_0": true,
"tag_2": true
}
},
"post_1": {
"content": "...",
"created_at": 1497112549374,
"likes": 30,
"tags": {
"tag_1": true
}
},
"post_2": {
"content": "...",
"created_at": 1497112640376,
"likes": 70,
"tags": {
"tag_1": true,
"tag_2": true
}
},
...
}
}
View 1
This is probably the easiest to resolve. I imagine the script to retrieve the data would be something like this:
const ref = firebase.database().ref("posts");
const query = ref.orderByChild("created_at").limitToLast(50);
query.on("child_added", (snapshot) => {
// Do stuff like add to array for sorting
});
View 2
This is where things get tricky. Since you can only have one orderBy* per query, the only way I can see to pull this off is to have a tags node that duplicates the date and post ID. For example:
{
"tags": {
"tag_2": {
"post_0": {
"created_at": 1497112445748
},
"post_2": {
"created_at": 1497112640376,
}
},
...
}
}
I've read this is the whole concept of denormalization and structuring your data around your views, but isn't there a better way?
View 3
I don't know how to solve this one at all. As the last 1 day is changing every time the view is requested and the likes are fluctuating often, how can I possibly structure my data around this view?
I've read that push keys, which would take place of the post_n key I have in my example, are sequential and can somewhat be relied on as a timestamp. I'm not sure if there's some way to take advantage of that.
I've found a few useful videos by the Firebase team and articles on Medium, but I'm afraid they don't go far enough for me to understand how to accomplish the needs of my app.
Common SQL Queries converted for the Firebase Database
Firebase Data Structures: Pagination
I'm just find this aspect of Firebase really confusing to get my head around to have it return the data I need for my views.
If anybody can provide me with an example of how to accomplish these things, it would be much appreciated! Thanks!

Firebase .indexOn with complex DB structure

The current query you see below is not efficient because I have not setup the proper indexing. I get the suggestion Consider adding ".indexOn": "users/kxSWLGDxpYgNQNFd3Q5WdoC9XFk2" at /conversations in the console in Xcode. I have tried it an it works.
However, I need the user id after users/ to be dynamic. I've added a link to another post below that has tried a similar thing, but I just can't seem to get it. All help would be much appreciated!
Note: The console output user id above does not match the screenshot below, but does not matter to solve the problem I believe. Correct me if I'm wrong. Thanks!
Here is the structure of my DB in Firebase:
{
"conversationsMessagesID" : "-KS3Y9dMLXfs3FE4nlm7",
"date" : "2016-10-19 15:45:32 PDT",
"dateAsDouble" : 4.6601793282986E8,
"displayNames" : [ “Tester 1”, “Tester 2” ],
"hideForUsers" : [ "SjZLsTGckoc7ZsyGV3mmwc022J93" ],
"readByUsers" : [ "mcOK5wVZoZYlFZZICXWYr3H81az2", "SjZLsTGckoc7ZsyGV3mmwc022J93" ],
"users" : {
"SjZLsTGckoc7ZsyGV3mmwc022J93" : true,
"mcOK5wVZoZYlFZZICXWYr3H81az2" : true
}
}
and the Swift query:
FIRDatabase.database().reference().child("conversations")
.queryOrderedByChild("users/\(AppState.sharedInstance.uid!)").queryEqualToValue(true)
Links to other post:
How to write .indexOn for dynamic keys in firebase?
It seems fairly simple to add the requested index:
{
"rules": {
"users": {
".indexOn": ["kxSWLGDxpYgNQNFd3Q5WdoC9XFk2", "SjZLsTGckoc7ZsyGV3mmwc022J93", "mcOK5wVZoZYlFZZICXWYr3H81az2"]
}
}
}
More likely your concern is that it's not feasible to add these indexes manually, since you're generating the user IDs in your code.
Unfortunately there is no API to generate indexes.
Instead you'll need to model your data differently to allow the query that you want to do. In this case, you want to retrieve the conversations for a specific user. So you'll need to store the conversations for each specific user:
conversationsByUser {
"SjZLsTGckoc7ZsyGV3mmwc022J93": {
"-KS3Y9dMLXfs3FE4nlm7": true
},
"mcOK5wVZoZYlFZZICXWYr3H81az2": {
"-KS3Y9dMLXfs3FE4nlm7": true
}
}
It may at first seem inefficient to store this data multiple times, but it is very common when using NoSQL databases. And is really no different than if the database would auto-generate the indexes for you, except that you have to write the code to update the indexes yourself.

Structure a NoSQL database for a chat application (using FireBase)

Coming from years of using relational databases, i am trying to develop a pretty basic chat/messaging app using FireBase
FireBase uses a NoSQL data structure approach using JSON formatted strings.
I did a lot of research in order to understand how to structure the database with performance in mind. I have tried to "denormalize" the structure and ended up with the following:
{
"chats" : {
"1" : {
"10" : {
"conversationId" : "x123332"
},
"17": {
"conversationId" : "x124442"
}
}
},
"conversations" : {
"x123332" : {
"message1" : {
"time" : 12344556,
"text" : "hello, how are you?",
"userId" : 10
},
"message2" : {
"time" : 12344560,
"text" : "Good",
"userId" : 1
}
}
}
}
The numbers 1, 10, 17 are sample user id's.
My question is, can this be structured in a better way? The goal is to scale up as the app users grow and still get the best performance possible.
Using the document-oriented database structure such Firestore, you can store the conversations as below;
{
"chat_rooms":[
{
"cid":100,
"members":[1, 2],
"messages":[
{"from":1, "to":2, "text":"Hey Dude! Bring it"},
{"from":2, "to":1, "text":"Sure man"}
]
},
{
"cid":101,
"members":[3, 4],
"messages":[
{"from":3, "to":4, "text":"I can do that work"},
{"from":4, "to":3, "text":"Then we can proceed"}
]
}
]
}
Few examples of NoSQL queries you could run through this structure.
Get all the conversations of a logged-in user with the user id of 1.
db.chat_rooms.find({ members: 1 })
Get all the documents, messages sent by the user id of 1.
db.chat_rooms.find({ messages: { from: 1 } })
The above database structure is also capable of implementing in RDMS database as table relationships using MySQL or MSSQL. This is also can be implemented for group chat room applications.
This structure is optimized to reduce your database document reading usage which can save your money from paying more for infrastructure.
According to our above example still, you will get 2 document reads since we have 4 messages but if you store all the messages individually and run the query by filtering sender id, you will get 4 database queries which are the kind of massive amount when you have heavy conversation histories in your database.
One case for storing messages could look something like this:
"userMessages":
{ "simplelogin:1":
{ "simplelogin:2":
{ "messageId1":
{ "uid": "simplelogin:1",
"body": "Hello!",
"timestamp": Firebase.ServerValue.TIMESTAMP },
"messageId2": {
"uid": "simplelogin:2",
"body": "Hey!",
"timestamp": Firebase.ServerValue.TIMESTAMP }
}
}
}
Here is a fireslack example this structure came from. This tutorial builds an app like slack using firebase:
https://thinkster.io/angularfire-slack-tutorial
If you want something more specific, more information would be helpful.

OR filter on dashboard in Kibana 4

I want to create a dashboard which shows information about a limited set of request values :
request:("/path1" OR "/path2" OR "/path3")
What I've tried so far:
I can add filters to the dashboard by clicking on a part of a pie chart, but all these filters are applied as AND filters and not OR. This way of working also requires actual data for all possible request values. Which is not always the case in a test environment.
in Discover I created a saved search but I don't know how I can apply this to my Dashboard so it gets part of the dashboard definition.
Is their a way to do this using the Dashboard editor or does it require some json scripting via Settings->Objects->Dashboards ? If so can you point me a good reference to this (escaped) syntax ?
In Kibana 3 you could define filters of type "either". Does this functionality exist in Kibana 4 ?
I'm using Kibana 4.0.2
I am not sure if this is an answer to your actual question, I'll write it anyway as someone may benefit and I found examples on the Kibana filter syntax to be elusive when googling.
I am trying to define a boolean filter instead of a boolean query in my Discover tab, to unclutter the search field and fascilitate further filtering on a limited set of values.
I found this link to the documentation where AND, OR, NOT filter syntax is described. After a bit of experimenting this was what worked for me, example:
I have a field named host containing the name of the server shipping the log entry. There are quite a few servers, each belonging to one of several redundancy groups. To filter only for log entries produced by the servers "SERVER06 OR SERVER07 OR SERVER08" which happen to belong to a distinct redundancy group B-Servers I can make an OR filter like so:
{
"bool": {
"should": [
{
"query": {
"match": {
"host": {
"query": "SERVER06",
"type": "phrase"
}
}
}
},
{
"query": {
"match": {
"host": {
"query": "SERVER07",
"type": "phrase"
}
}
}
},
{
"query": {
"match": {
"host": {
"query": "SERVER08",
"type": "phrase"
}
}
}
}
]
}
}
and save it as a search called B-Servers. Now I get a filtered list, where I can cherry pick a server with a further and more restrictive filter. Before I had all servers and the quick count only listed the five top entries, so I had to pick one and then edit the filter manually if my target wasn't in the list.
This should be useful for other string type fields too. The documentation should have included a couple of more examples I think, to set the context for the placement of the bool statement and not just a demonstration of the principle.
This link was also useful for demonstrating how to do booleans from the search field rather than as a filter.
[EDIT] An update for Kibana 5.2 as I could not get the previous syntax to work. The following did the trick with 5.2, I used this link to figure it out:
{
"query": {
"bool": {
"should": [
{
"match": {
"host": "SERVER06"
}
},
{
"match": {
"host": "SERVER07"
}
},
{
"match": {
"host": "SERVER08"
}
}
],
"minimum_should_match": 1
}
}
}
Kibana 4 is a total rewrite and apparently not all Kibana 3 features are yet implemented. I've found an "enhancement" ticket in the Kibana github:
https://github.com/elastic/kibana/issues/3693
This closes my question for the moment.
Definitely you can add OR filters in your dashboard. As dashboard is created from saved visualizations, In your visualization you can add filter containing OR which will reflect such data.
As per my understanding of your question I am posting my answer (Feel free to correct me):-
Clicking on pie chart under visualize tab (Create a new visualization).
Selecting a search source - From a new search
Go to Split Slices, select aggregation as filters.
Under Query1 enter the filter you want to apply such as request:("/path1" OR "/path2")
Click on add Filter and add Query2 as: request:("/path2" OR "/path3")
Click on Apply to view the changed pie chart as per filters.
Save Visualization by selecting it from toolbar (2nd option beside search bar).
Go to Dashboard & Click on Add Dashboard & select your saved visualization which will reflect your pie chart.
Hope it answers your question :)
The lucene syntax is "OR" (uppercase), so "foo OR bar".
Once your query is corrected, you can save the search in the Discover tab and refer to that saved search in your Visualization.
You can also manually set the query in the visualization if you don't want the overhead of saving it separately.

Resources