How to isolate data by groups of users in firebase - firebase

I’m fairly new to Firebase and its json database. So right now I’m designing the tree hierarchy but when reading the documentation it says that “when you fetch data at a location in your database, you also retrieve all of its child nodes”.
I understand that this means that if you have this
a1
—b1
——c1
——c2
———d1
———d2
———d3
———d4
—b2
——c1
——c2
———etc…
and you want to fetch “c1”, you will download not just “c1” but “c2” and all “d”s. And it makes sense BUT in the example in the documentation it says that to list all “c1”s requires to download the whole tree from “a1”.
Am I correct? And if so, how can I model a tree where I need to isolate data by groups of users and all their internal shared info?

You can address each node in your tree by creating a reference to it. But if you retrieve the value for a node, you will also get each node under it.
So if you attach a listener to /a1/b1:
var root = firebase.database().ref();
root.child('a1/b1').on('value', ...
You will get c1, c2, d1, d2, d3 and d4.
But if you address the c1 directly, you will get only that node:
root.child('a1/b1/c1').on('value', ...

I have some nice examples for you.
You can't access a user's "name" without also downloading all of it's posts in this case:
{
"users": {
"user1": {
"name": "John",
"posts": {
"post1": {...},
"post2": {...}
}
},
"user2": {
"name": "Nate",
"posts": {
"post1": {...},
"post2": {...}
}
},
"user3": {
"name": "Buck",
"posts": {
"post1": {...},
"post2": {...}
}
}
}
}
Denormalize your database for better data querying and fetching:
{
"users": {
"user1": {
"name": "John",
"posts": {
"post2": true,
"post5": true
}
},
"user2": {
"name": "Nate",
"posts": {
"post1": true,
"post6": true
}
},
"user3": {
"name": "Buck",
"posts": {
"post3": true,
"post4": true
}
}
},
"posts":{
"post1":{
...,
"user":"user2"
},
"post2":{
...,
"user":"user1"
},
"post3":{
...,
"user":"user3"
},
"post4":{
...,
"user":"user3"
},
"post5":{
...,
"user":"user1"
},
"post6":{
...,
"user":"user2"
},
}
}

Related

Storing optional attributes in DynamoDB's putItem via step functions

I have defined a state machine in AWS step functions and one of my states is storing an item to DynamoDB
...
"Store item": {
"End": true,
"Type": "Task",
"Resource": "arn:aws:states:::dynamodb:putItem",
"Parameters": {
"Item": {
"foo": {
"S.$": "$.data.foo"
},
"bar": {
"S.$": "$.data.bar"
},
"baz": {
"S.$": "$.data.baz"
},
},
"TableName": "nrp_items"
}
},
...
The problem starts from the fact that baz property is optional, ie not exist in some cases.
On those cases, the putItem task fails:
An error occurred while executing the state 'Store item' (entered at the event id #71). > The JSONPath '$.data.baz' specified for the field 'S.$' could not be found in the input
My backup plan is to use a lambda to perform that type of operation, but can I do it directly using the putItem task in steps function?
I was wondering if:
Is possible to somehow inject via JSONPath my whole $.data item to the "Item" property, something like:
...
"Store item": {
"End": true,
"Type": "Task",
"Resource": "arn:aws:states:::dynamodb:putItem",
"Parameters": {
"Item": "$.data",
"TableName": "nrp_items"
}
},
...
OR
2) Define that the baz property is optional
TL;DR We can deal with optional variables with a "Variable": "$.baz", "IsPresent": true Choice condition to handle no-baz cases.
The Amazon States Language spec does not have optional properties: Step Functions will throw an error if $.baz does not exist in the input. We can avoid undefined paths by inserting a two-branch Choice State, one branch of which handles baz-exists cases, the other no-baz cases. Each branch continues with a Pass State that reworks the data input into dynamo-format Item syntax, using Parameters. The put-item task's "Item.$": "$.data" (as in your #1) contains only foo-bar when baz is not defined, but all three otherwise.
{
"StartAt": "HasBazChoice",
"States": {
"HasBazChoice": {
"Type": "Choice",
"Choices": [
{
"Variable": "$.baz",
"IsPresent": true,
"Next": "MakeHasBazItem"
}
],
"Default": "MakeNoBazItem"
},
"MakeHasBazItem": {
"Type": "Pass",
"Parameters": {
"data": {
"foo": { "S.$": "$.foo"},
"bar": { "S.$": "$.bar"},
"baz": { "S.$": "$.baz"}
}
},
"Next": "PutItemTask"
},
"MakeNoBazItem": {
"Type": "Pass",
"Parameters": {
"data": {
"foo": {"S.$": "$.foo"},
"bar": {"S.$": "$.bar"}
}
},
"Next": "PutItemTask"
},
"PutItemTask": {
...
"Parameters": {
"TableName": "my-table",
"Item.$": "$.data"
}
},
}
}
If you have more than one optional field, your lambda backup plan is the better option - the above workaround would become unwieldy.

How to retrieve firebase documents missing a field using runQuery and the IN operator?

This is my http POST requst body...
{
"structuredQuery": {
"select": {
"fields": [
{
"fieldPath": "name"
},
{
"fieldPath": "taxId"
},
{
"fieldPath": "mailingAddress"
}
]
},
"from": [
{
"collectionId": "orgs"
}
],
"where": {
"fieldFilter": {
"field": {
"fieldPath": "orgId"
},
"op": "IN",
"value": {
"arrayValue": {
"values": [
{
"stringValue": ""
},
{
"nullValue": null
}
]
}
}
}
}
}
It fails to return orgs where the orgId field is completely missing from the document. It correctly includes orgs where the orgId field is present and equal to empty string. This is accessing a Cloud Firestore db.
Due to the way Firestore indexes data, it is not possible to query for documents for which a certain field "is completely missing from the document": the field needs to exist in order for the Firestore index to take it into account. More details on the indexing mechanism in the following official video.
You may store an empty value in this field, as you mention in your question.

Can I apply write rule for anyone for updating a particular item inside the object stored in firebase rt database?

I would like to add firebase database rule to allow anyone to increment the "count" inside the message object. Is it possible to do so? If so, how should I write the rule for updating the count? Thanks!
{
"messages": {
"message0": {
"content": "Hello",
"count": 5
},
"message1": {
"content": "Goodbye",
"count": 10
},
...
}
}
I tried below rule, but it doesn't seem to work:
{
"rules": {
"messages": {
"$message": {
"count": {
".read": true,
".write": true,
}
}
}
}
}

Request probleme with Google Cloud Datastore and Filter

I'm currently doing some tests on google datastore, but I'm having a problem with my queries.
If I believe in the documentation https://cloud.google.com/datastore/docs/concepts/queries we can realize a filter on several columns with the instruction EQUALS.
But when testing, I get an error from the API.
While searching on Datastore's github, I found this reference: https://github.com/GoogleCloudPlatform/google-cloud-dotnet/issues/304 which corresponds to my problem, except that for my case the query to the look good.
Here is the request sent:
{
{
"kind": [{
"name": "talk.message"
}],
"filter": {
"compositeFilter": {
"op": "AND",
"filters": [{
"propertyFilter": {
"property": {
"name": "Conversation"
},
"op": "EQUAL",
"value": {
"stringValue": "2f16c14f6939464ea687d316438ad4cb"
}
}
},
{
"propertyFilter": {
"property": {
"name": "CreatedOn"
},
"op": "LESS_THAN_OR_EQUAL",
"value": {
"timestampValue": "2019-03-15T10:43:31.474166300Z"
}
}
},
{
"propertyFilter": {
"property": {
"name": "CreatedOn"
},
"op": "GREATER_THAN_OR_EQUAL",
"value": {
"timestampValue": "2019-03-14T10:43:31.474175100Z"
}
}
}
]
}
}
}
}
And here is the answer from the API:
{Grpc.Core.RpcException: Status(
StatusCode=FailedPrecondition,
Detail="no matching index found. recommended index is:
- kind: talk.message
properties:
- name: Conversation
- name: CreatedOn"
)
According to the documentation, this should be good... but it's not !
What am I missing ?
Your query includes both an EQUALS (on Conversation) and a non-EQUALS filter (on CreatedOn), therefore you need a composite index to fulfil the query. So your query is valid, but it needs a composite index to be able to run the query.

Firebase complex rules

Let's assume I have the following data structure in my Firebase database:
{
"allProjects": {
"foo": true,
"bar": true,
"baz": true
},
"allUsers": {
"user1": true,
"user2": true,
"user3": true
},
"projects": {
"foo": {
"name": "foo",
"members": {
"user1": true
}
},
"bar": {
"name": "bar",
"members": {
"user2": true
}
},
"baz": {
"name": "baz",
"members": {
"user1": true,
"user3": true
}
}
},
"users": {
"user1": {
"name": "user1"
},
"user2": {
"name": "user2"
},
"user3": {
"name": "user3"
}
}
}
Problem
I'm trying to write a few rules so I can protect my data. Here's a few requirements:
Users should only see projects they are part of (i.e. /project/:id/members/:userId === true)
Users should only see users that belong to the same projects as them.
Here's what I was able to come up with:
{
".read": false,
".write": false,
"allProjects": {
".read": "auth !== null"
},
"allUsers": {
".read": "auth !== null"
},
"projects": {
"$projectId": {
// requirement 1
".read": "data.child('members').hasChild(auth.uid)"
}
},
"users": {
"$userId": {
// requirement 2
".read": "???",
}
}
}
The typical approach would be to keep a list of users that each user can see.
user_friends: {
"user1": {
"user3": true
}
"user3": {
"user1": true
}
}
Then as users get added to/removed from projects, you'll need to keep this derived list up to date. Alternatively you can calculate this derived list in a schedule maintenance operation.
Duplicating data like this is one of the big differences between SQL databases and most NoSQL databases. In Firebase we often store extra data to allow for our use-cases. For a great introduction I recommend reading NoSQL data modeling and watching Firebase for SQL developers.
By the way, I noticed that you've combined metadata and the list of users under /projects/$projectid. This is not recommended. For best results, split them into two top-level lists.
"projects": {
"foo": {
"name": "foo",
},
"bar": {
"name": "bar",
},
"baz": {
"name": "baz",
}
},
"members": {
"foo": {
"user1": true
},
"bar": {
"user2": true
},
"baz": {
"user1": true,
"user3": true
}
},

Resources