The current query you see below is not efficient because I have not setup the proper indexing. I get the suggestion Consider adding ".indexOn": "users/kxSWLGDxpYgNQNFd3Q5WdoC9XFk2" at /conversations in the console in Xcode. I have tried it an it works.
However, I need the user id after users/ to be dynamic. I've added a link to another post below that has tried a similar thing, but I just can't seem to get it. All help would be much appreciated!
Note: The console output user id above does not match the screenshot below, but does not matter to solve the problem I believe. Correct me if I'm wrong. Thanks!
Here is the structure of my DB in Firebase:
{
"conversationsMessagesID" : "-KS3Y9dMLXfs3FE4nlm7",
"date" : "2016-10-19 15:45:32 PDT",
"dateAsDouble" : 4.6601793282986E8,
"displayNames" : [ “Tester 1”, “Tester 2” ],
"hideForUsers" : [ "SjZLsTGckoc7ZsyGV3mmwc022J93" ],
"readByUsers" : [ "mcOK5wVZoZYlFZZICXWYr3H81az2", "SjZLsTGckoc7ZsyGV3mmwc022J93" ],
"users" : {
"SjZLsTGckoc7ZsyGV3mmwc022J93" : true,
"mcOK5wVZoZYlFZZICXWYr3H81az2" : true
}
}
and the Swift query:
FIRDatabase.database().reference().child("conversations")
.queryOrderedByChild("users/\(AppState.sharedInstance.uid!)").queryEqualToValue(true)
Links to other post:
How to write .indexOn for dynamic keys in firebase?
It seems fairly simple to add the requested index:
{
"rules": {
"users": {
".indexOn": ["kxSWLGDxpYgNQNFd3Q5WdoC9XFk2", "SjZLsTGckoc7ZsyGV3mmwc022J93", "mcOK5wVZoZYlFZZICXWYr3H81az2"]
}
}
}
More likely your concern is that it's not feasible to add these indexes manually, since you're generating the user IDs in your code.
Unfortunately there is no API to generate indexes.
Instead you'll need to model your data differently to allow the query that you want to do. In this case, you want to retrieve the conversations for a specific user. So you'll need to store the conversations for each specific user:
conversationsByUser {
"SjZLsTGckoc7ZsyGV3mmwc022J93": {
"-KS3Y9dMLXfs3FE4nlm7": true
},
"mcOK5wVZoZYlFZZICXWYr3H81az2": {
"-KS3Y9dMLXfs3FE4nlm7": true
}
}
It may at first seem inefficient to store this data multiple times, but it is very common when using NoSQL databases. And is really no different than if the database would auto-generate the indexes for you, except that you have to write the code to update the indexes yourself.
Related
I have a firebase database like this structure:
-groups
--{group1id}
---groupname: 'group1'
---grouptype: 'sometype'
---groupmembers
----{uid1}:true
----{uid2}:true
--{group2id}
---groupname: 'group2'
---grouptype: 'someothertype'
---groupmembers
----{uid1}:true
----{uid3}:true
----{uid4}:true
Now, I am trying to pull groups of authenticated user. For example for uid1, it should return me group1id and group2id, and for example uid3 it should just return group2id.
I tried to do that with this code:
database().ref('groups/').orderByChild('groupMembers/' + auth().currentUser.uid).equalTo('true').on('value' , function(snapshot) {
console.log('GROUPS SNAPSHOT >> ' + JSON.stringify(snapshot))
})
but this returns null. if I remove "equalTo" and go it returns all childs under 'groups'.
Do you know any solution or better database structure suggestion for this situation ?
Your current structure makes it easy to retrieve the users for a group. It does not however make it easy to retrieve the groups for a user.
To also allow easy reading of the groups for a user, you'll want to add an additional data structure:
userGroups: {
uid1: {
group1id: true,
group2id: true
},
uid2: {
group1id: true,
group2id: true
},
uid3: {
group2id: true
},
uid3: {
group2id: true
}
}
Now of course you'll need to update both /userGroups and /groups when you add a user to (or remove them from) a group. This is quite common when modeling data in NoSQL databases: you may have to modify your data structure for the use-cases that your app supports.
Also see:
Firebase query if child of child contains a value
NoSQL data modeling
Many to Many relationship in Firebase
I am stuck trying to allow an an array of admins access to their data.
I have a database structure like this:
{
"Respondents": {
"Acme Corp": {
"admins": ["mMK7eTrRL4UgVDh284HntNRETmx1", ""mx1TERNmMK7eTrRL4UgVDh284Hnt"],
"data": {data goes here...}
},
"Another Inc": {
"admins": ["Dh284HmMK7eTrRL4UgVDh284HntN", ""x1TERNmx1TERNmMK7eTrRL4UgVDh"],
"data": {their data goes here...}
}
}
}
And then I tried to set my rules like this
{
"rules": {
"Respondents": {
"$organisation" : {
".read": "root.child('Respondents').child($organisation).child('admins').val().includes(auth.id)",
".read": "root.child('Respondents').child($organisation).child('admins').val().includes(auth.id)"
}
}
}
}
..but that won't parse in the Firebase Database Rules editor
I get "Error saving rules - Line 7: No such method/property 'includes'", but I need something to match the user id with the array of admins.
Any experience or suggestions?
As you've found, there is no includes() operation in Firebase's security rules. This is because Firebase doesn't actually store the data as an array. If you look in the Firebase Database console or read this blog post you will see that Firebase stores it as a regular object:
"admins": {
"0": "mMK7eTrRL4UgVDh284HntNRETmx1",
"1": "mx1TERNmMK7eTrRL4UgVDh284Hnt"
}
And since that is a regular JavaScript object, there is no contains() method on it.
In general creating arrays are an anti-pattern in the Firebase Database. They're often the wrong data structure and when used are regularly the main cause of scalability problems.
In this case: you're not really looking to store a sequence of UIDs. In fact: the order of the UIDs doesn't matter, and each UID can be meaningfully present in the collection at most once. So instead of an array, you're looking to store set of uids.
To implement a set in Firebase, you use this structure:
"admins": {
"mMK7eTrRL4UgVDh284HntNRETmx1": true,
"mx1TERNmMK7eTrRL4UgVDh284Hnt": true
}
The value doesn't matter much. But since you must have a value to store a key, it is idiomatic to use true.
Now you can test whether a key with the relevant UID exists under admins (instead of checking whether it contains a value):
"root.child('Respondents').child($organisation).child('admins').child(auth.uid).exists()",
On the firebase structure data section, it shows how to structure data with a many-many user-group situation. But, why they have used "referece":true on both the side instead of using a simple array od ids.
Like, it can be used like both the ways:
A user having array of groups
"groups" : [ "groupId1", "groupId2", ... ]
A user having
"groups": {
"groupId1" : true,
"groupId2" : true,
..
}
They have done it a second way. What is the reason for that?
Something was told at the Google I/O 2016 for that in some video. But, I'm unable to recall.
Example from structure your data:
// An index to track Ada's memberships
{
"users": {
"alovelace": {
"name": "Ada Lovelace",
// Index Ada's groups in her profile
"groups": {
// the value here doesn't matter, just that the key exists
"techpioneers": true,
"womentechmakers": true
}
},
...
},
"groups": {
"techpioneers": {
"name": "Historical Tech Pioneers",
"members": {
"alovelace": true,
"ghopper": true,
"eclarke": true
}
},
...
}
}
Firebase recommends against using arrays in its database for most cases. Instead of repeating the reasons here, I'll refer you to this classic blog post on arrays in Firebase.
Let's look at one simple reason you can easily see from your example. Since Firebase arrays in JavaScript are just associative objects with sequential, integer keys, your first sample is stored as:
"groups" : {
0: "groupId1",
1: "groupId2"
]
To detect whether this user is in groupId2, you have to scan all the values in the array. When there's only two values, that may not be too bad. But it quickly gets slower as you have more values. You also won't be able to query or secure this data, since neither Firebase Queries nor its security rules support a contains() operator.
Now look at the alternative data structure:
"groups": {
"groupId1" : true,
"groupId2" : true
}
In this structure you can see whether the user is in groupId2 by checking precisely one location: /groups/groupId2. It that key exists, the user is a member of groupId2. The actual value doesn't really matter in this case, we just use true as a marker value (since Firebase will delete a path if there's no value).
This will also work better with queries and security rules, because you now "just" needs an exists() operator.
For some great insights into this type of modeling, I highly recommend that article on NoSQL data modeling.
Coming from years of using relational databases, i am trying to develop a pretty basic chat/messaging app using FireBase
FireBase uses a NoSQL data structure approach using JSON formatted strings.
I did a lot of research in order to understand how to structure the database with performance in mind. I have tried to "denormalize" the structure and ended up with the following:
{
"chats" : {
"1" : {
"10" : {
"conversationId" : "x123332"
},
"17": {
"conversationId" : "x124442"
}
}
},
"conversations" : {
"x123332" : {
"message1" : {
"time" : 12344556,
"text" : "hello, how are you?",
"userId" : 10
},
"message2" : {
"time" : 12344560,
"text" : "Good",
"userId" : 1
}
}
}
}
The numbers 1, 10, 17 are sample user id's.
My question is, can this be structured in a better way? The goal is to scale up as the app users grow and still get the best performance possible.
Using the document-oriented database structure such Firestore, you can store the conversations as below;
{
"chat_rooms":[
{
"cid":100,
"members":[1, 2],
"messages":[
{"from":1, "to":2, "text":"Hey Dude! Bring it"},
{"from":2, "to":1, "text":"Sure man"}
]
},
{
"cid":101,
"members":[3, 4],
"messages":[
{"from":3, "to":4, "text":"I can do that work"},
{"from":4, "to":3, "text":"Then we can proceed"}
]
}
]
}
Few examples of NoSQL queries you could run through this structure.
Get all the conversations of a logged-in user with the user id of 1.
db.chat_rooms.find({ members: 1 })
Get all the documents, messages sent by the user id of 1.
db.chat_rooms.find({ messages: { from: 1 } })
The above database structure is also capable of implementing in RDMS database as table relationships using MySQL or MSSQL. This is also can be implemented for group chat room applications.
This structure is optimized to reduce your database document reading usage which can save your money from paying more for infrastructure.
According to our above example still, you will get 2 document reads since we have 4 messages but if you store all the messages individually and run the query by filtering sender id, you will get 4 database queries which are the kind of massive amount when you have heavy conversation histories in your database.
One case for storing messages could look something like this:
"userMessages":
{ "simplelogin:1":
{ "simplelogin:2":
{ "messageId1":
{ "uid": "simplelogin:1",
"body": "Hello!",
"timestamp": Firebase.ServerValue.TIMESTAMP },
"messageId2": {
"uid": "simplelogin:2",
"body": "Hey!",
"timestamp": Firebase.ServerValue.TIMESTAMP }
}
}
}
Here is a fireslack example this structure came from. This tutorial builds an app like slack using firebase:
https://thinkster.io/angularfire-slack-tutorial
If you want something more specific, more information would be helpful.
Given this database structure in Firebase:
{
"users": {
"user1": {
"items": {
"id1": true
}
},
"user2": {
"items": {
"id2": true
}
}
},
"items": {
"id1": {
"name": "foo1",
"user": "user1"
},
"id2": {
"name": "foo2",
"user": "user2"
}
}
}
which is a more efficient way of querying the items belonged to a specific user?
The Firebase docs seem to suggest this:
var itemsRef = new Firebase("https://firebaseio.com/items");
var usersItemsRef = new Firebase("https://firebaseio/users/" + user.uid + "/items");
usersItemsRef.on("child_added", function(data){
itemsRef.child(data.key()).once("value", function(itemData){
//got the item
});
});
but using the .equalTo() query works as well:
var ref = new Firebase("https://firebaseio.com/items");
ref.orderByChild("user").equalTo(user.uid).on("child_added", function(data){
//got the item
});
The latter code seems more concise and doesn't require denormalization of the item keys into the user records but it's unclear to me if it's a less efficient methodology (assuming I create an index on "user").
thanks.
This is rather old one, but when working on the firebase-backed app, I found myself dealing with similar issues quite often.
.equalTo is more time-efficient (especially, if one user owns big number of items). Although n+1 subscriptions does not lead to n+1 networking roundtrips to the cloud, there is some performance penalty for having so many open subscriptions.
Moreover, .equalTo approach does not lead to denormalization of your data.
There is a gotcha however: When you'll want to secure the data, the .equalTo approach may stop working at all.
To allow user to call orderByChild("user").equalTo(user.uid), they must have read privilege to 'items' collection. This read permission is valid for the whole sub-document rooted at /items.
Summary: If user1 is to be prevented from finding out about items of user2, you must use the BYOI (build your own index) approach. That way you can validate that user only reads items that are put to their index.
Finally, disclaimer :) I use firebase only for a short period of time all I got is a few benchmarks and documentation. If I'm mistaken in any way, please correct me.