Let's take a look at "Instagram-like" app, as an example.
In the feed we got posts, with user avatar and name at the top, photo or video below, and last comments, likes count and post time at the bottom.
Basically, at the client I'm waiting to get from backend something like
{
username: "John",
avatar:"some_link",
photo:"photo_url",
likes:"9",
time:"182937428",
comments:[comments there]
}
but using Firebase, I need to store data in more flat way. so there will be "users", "posts" and "comments" in data JSON.
How am I suppose to aggregate data from those nodes in some kind of single object, which is easy to use at client?
Or should I ask Firebase for posts, than for all users in it, and for all their comments, and do aggregation after all three 'requests' are done?
You should implement "shallow" tree structure, and use references where needed.
That means that for most cases in your app you should use the object as at is, Making sure that it contain the "essential data" (in the example below "the chat title"), and keys for "further" information (in the example, keys to the "members").
from firebase docs (https://firebase.google.com/docs/database/web/structure-data):
bad
{
// This is a poorly nested data architecture, because iterating the children
// of the "chats" node to get a list of conversation titles requires
// potentially downloading hundreds of megabytes of messages
"chats": {
"one": {
"title": "Historical Tech Pioneers",
"messages": {
"m1": { "sender": "ghopper", "message": "Relay malfunction found. Cause: moth." },
"m2": { ... },
// a very long list of messages
}
},
"two": { ... }
}
}
good
{
// Chats contains only meta info about each conversation
// stored under the chats's unique ID
"chats": {
"one": {
"title": "Historical Tech Pioneers",
"lastMessage": "ghopper: Relay malfunction found. Cause: moth.",
"timestamp": 1459361875666
},
"two": { ... },
"three": { ... }
},
// Conversation members are easily accessible
// and stored by chat conversation ID
"members": {
// we'll talk about indices like this below
"one": {
"ghopper": true,
"alovelace": true,
"eclarke": true
},
"two": { ... },
"three": { ... }
},
// Messages are separate from data we may want to iterate quickly
// but still easily paginated and queried, and organized by chat
// conversation ID
"messages": {
"one": {
"m1": {
"name": "eclarke",
"message": "The relay seems to be malfunctioning.",
"timestamp": 1459361875337
},
"m2": { ... },
"m3": { ... }
},
"two": { ... },
"three": { ... }
}
}
Related
Problem
We've been using OPA to do data authorization of our REST HTTP APIs. We secure our APIs as such
allow {
input.method == "GET"
glob.match(input.path, ["/"], "/department/DEPARTMENT_ID/employee/")
some_rule # Where we check that user can list all employee in the particular deparment/DEPARTMENT_ID based on the ACL of department/DEPARTMENT_ID
}
As seen above, each department has its own ACL we authorize against that for any access to it and its child resources (e.g. employee).
We query this policy via OPA's HTTP API, and we push department/DEPARTMENT_ID's ACL to OPA for it to make a decision. See OPA docs.
However, there's been a new requirement where we have to make an API that has to list all employee that the user has access to.
How could one go about doing this given that the authorization can no longer look at just one ACL? (because multiple employee resources will belong in different department, each with their own ACL).
Potential solution
When listing employee, we could send OPA all the ACLs of each of their department (i.e. the parent), and have OPA authorize based on that. This could be highly inefficient, but I'm not sure if there's any better way. The size of this is also bounded if we paginate the employee listing.
I'm not sure I followed entirely, but given that you have data looking something like the below:
{
"departments": {
"department1": {
"permissions": {
"jane": ["read"]
},
"employees": {
"x": {},
"y": {},
"z": {}
}
},
"department2": {
"permissions": {
"jane": ["read"]
},
"employees": {
"a": {},
"b": {},
"c": {}
}
},
"department3": {
"permissions": {
"eve": ["read"]
},
"employees": {
"bill": {},
"bob": {},
"eve": {}
}
}
}
}
And input looking something like this:
{
"user_id": "jane",
"method": "GET",
"department_id": "department1",
"path": "/department/department1/employee"
}
A policy to query for all listable employees for a user might look something like this:
package play
import future.keywords.in
allow {
input.method == "GET"
glob.match(input.path, ["/"], sprintf("/department/%v/employee", [input.department_id]))
can_read
}
# Where we check that user can list all employee in the particular deparment/DEPARTMENT_ID based on the ACL of department/DEPARTMENT_ID
can_read {
"read" in data.departments[input.department_id].permissions[input.user_id]
}
listable_employees[employee] {
some department in data.departments
"read" in department.permissions[input.user_id]
some employee, _ in department.employees
}
The listable_employees in this case would evaluate to:
[
"a",
"b",
"c",
"x",
"y",
"z"
]
Since user jane has read access to department1 and department2, but not department3.
I am checking a if a email is already registered or not
query=googleRef.orderByChild("email").equalTo(newEmail).addValueEventListener(object :ValueEventListener{
override fun onCancelled(p0: DatabaseError) {
println(p0.code)
}
override fun onDataChange(p0: DataSnapshot) {
if(p0.exists())
{
println("Yes user exists")
}
else if(!p0.exists())
{
println("Users dont exists")
}
}
Code from comments:
I had used a push for inserting:
googleRef.child("userID").push().setValue(userId)
googleRef.child("gname").push().setValue(userName)
googleRef.child("email").push().setValue(reEmail)
googleRef.child("photoUrl").push().setValue(userpicUrl)
If you add two users, the way you're adding code is going to result in a structure like this:
"googleRef": {
"userID": {
"-Ldfs32189eqdqA1": "userID1",
"-Ldfs32189eqdqA5": "userID2"
},
"gname": {
"-Ldfs32189eqdqA2": "gname1",
"-Ldfs32189eqdqA6": "gname2"
},
"email": {
"-Ldfs32189eqdqA3": "email1",
"-Ldfs32189eqdqA7": "email2"
},
"photoUrl": {
"-Ldfs32189eqdqA4": "photoUrl1",
"-Ldfs32189eqdqA8": "photoUrl2"
}
}
So you have a separate generated push ID (the keys starting with a -) for each property of each user, which is highly uncommon.
The more idiomatic form of storing user information is either this:
"googleRef": {
"-Ldfs32189eqdqA1": {
"userID": "userID1",
"gname": "gname1",
"email": "email1",
"photoUrl": "photoUrl1"
},
"-Ldfs32189eqdqA5": {
"userID": "userID2",
"gname": "gname2"
"email": "email2"
"photoUrl": "photoUrl2"
},
}
Or (even better) this:
"googleRef": {
"userID1": {
"gname": "gname1",
"email": "email1",
"photoUrl": "photoUrl1"
},
"userID2": {
"gname": "gname2"
"email": "email2"
"photoUrl": "photoUrl2"
},
}
The reasons these last two are more common is that they group the information for each user together, which makes it easier/possible to find information for each user. In both of these cases, you can find users with a specific email address with your query.
The reason the last one is best, is because the information for each user is stored under the user's ID, which is already guaranteed to be unique. This structure makes looking up the user's information by their UID possible without needing a query.
To write a structure like the last example, use:
Map<String, Object> values = new HashMap<>();
values.put("gname", userName)
values.put("email", reEmail)
values.put("photoUrl", userpicUrl)
googleRef.child(userId).setValue(values)
A final note: you can't return whether the node exists or node, since the data is loaded from Firebase asynchronously. To learn more about what that means, and the common workaround (which is to define a callback interface), see getContactsFromFirebase() method return an empty list
We have an application that will store data on Firebase (database) that will then be queried later.
What is the correct format to store the data in.
The example data will be completedGames. They will have data such as:
UserId
TimeToComplete
GameData
Etc...
The query later will then look for all completed games by UserId. We want to ensure the data is collected in the best way possible to query later, rather than refactoring later.
In your case, first off - be sure you have a good reason to use Firebase over Firestore. Once you're confident you should stick with Firebase Realtime Database, look at the below excerpt of documentation. So, you might actually have 2 separate parent nodes, 1 for userId and another for games. Each game node's child is a particular game, which has a child tree of game users (by userId).
Flatten data
structures
If the data is instead split into separate paths, also called
denormalization, it can be efficiently downloaded in separate calls,
as it is needed. Consider this flattened structure:
{
// Chats contains only meta info about each conversation
// stored under the chats's unique ID
"chats": {
"one": {
"title": "Historical Tech Pioneers",
"lastMessage": "ghopper: Relay malfunction found. Cause: moth.",
"timestamp": 1459361875666
},
"two": { ... },
"three": { ... }
},
// Conversation members are easily accessible
// and stored by chat conversation ID
"members": {
// we'll talk about indices like this below
"one": {
"ghopper": true,
"alovelace": true,
"eclarke": true
},
"two": { ... },
"three": { ... }
},
// Messages are separate from data we may want to iterate quickly
// but still easily paginated and queried, and organized by chat
// conversation ID
"messages": {
"one": {
"m1": {
"name": "eclarke",
"message": "The relay seems to be malfunctioning.",
"timestamp": 1459361875337
},
"m2": { ... },
"m3": { ... }
},
"two": { ... },
"three": { ... }
}
}
In the Firebase docs for Structuring Data, they give the follow data structure as an example of mapping users to groups.
{
"users": {
"alovelace": {
"name": "Ada Lovelace",
// Index Ada's groups in her profile
"groups": {
// the value here doesn't matter, just that the key exists
"techpioneers": true,
"womentechmakers": true
}
},
...
},
"groups": {
"techpioneers": {
"name": "Historical Tech Pioneers",
"members": {
"alovelace": true,
"ghopper": true,
"eclarke": true
}
},
...
}
}
With that structure, how would I go about querying only groups with a specific member? So only groups where alovelace is a member, for example.
Would I do this with a rule? If so, what would that rule look like?
OrderByChild works in this case - below the query object
angularfire.database.list('.../groups', {
query: {
orderByChild: 'members/alovelace'+,
startAt: true
}
});
Not sure how the performance is compared to the answer by Frank van Puffelen - might be worse even since it's another list query rather than just a few direct object lookups.
That information is already in the data model. Right under /users/alovelace/groups you have a list of the groups she's a member off.
The reason for recommending this model is that it doesn't even require a query to load the list of groups, it just requires a direct-access read of /users/alovelace/groups.
Is it possible to limit the depth of data returned from Firebase database?
For example, if I want to get some data from a parent object without waiting for all of its children & sub-children, can I specify that I only want x levels of objects?
Firebase does not yet have this capability.
We do intend to add it, but don't have a timetable yet.
There appears to be a shallow option...
{
"message": {
"user": {
"name": "Chris"
},
"body": "Hello!"
}
}
// A request to /message.json?shallow=true
// would return the following:
{
"user": true,
"body": true
}
// A request to /message/body.json?shallow=true
// would simply return:
"Hello!"
From: https://firebase.google.com/docs/database/rest/retrieve-data