Optimizing Firebase data structure for two large paths - firebase

I think I've wrapped my head around denormalization as a primary method of optimization when storing data in Firebase as mentioned in question like this one and in this blog post but I'm getting stuck on one small detail.
Assuming I have two things in my domain, users and posts as in the blog article I mentioned, I might have 20,000 users and 20,000 posts. Because I denormalized everything like a good boy, root/users/posts exists as does root/posts. root/users/posts has a set of post keys with a value of true so that I can get all post keys for a user.
users: {
userid: {
name: 'johnny'
posts: {
-Kofijdjdlehh: true,
-Kd9isjwkjfdj: true
}
}
}
posts: {
-Kofijdjdlehh: {
title: 'My hot post',
content: 'this was my content',
postedOn: '3987298737'
},
-Kd9isjwkjfdj: {
title: 'My hot post',
content: 'this was my content',
postedOn: '3987298737'
}
}
Now, I want to list the title of all posts a user has posted. I don't want to load all 20,000 posts in order to get the title. I can only think of the following options:
Query the root/posts path in some way using the subset of keys that are set to true in the root/users/posts path (if this is possible, I haven't figured out how)
Store the title in the root/users/posts so that each entry in that path has the title duplicated looking like this:
posts: {
-Kofijdjdlehh: true
}
becomes
posts: {
-Kofijdjdlehh: {
title: 'This was my content'
}
}
This seems reasonable, but I haven't seen a single example of doing this, so I'm concerned that it's some anti-pattern.
Another way I haven't been able to find
I appreciate any pointers you might have or documentation I might have missed on this use case.

Either are valid solutions. #1 would be more work for whoever is reading the data, while #2 would be more work when data is saved. Also for #2, you'd have to handle updates to post's titles, though this would be pretty easy with the new multi-path updates.
To implement #1, you'd have you essentially do two queries. Here's a really basic solution which only handles adding posts. It listens for posts being added to the user, and then hooks up a listener to each post's title.
var usersPosts = {};
ref.child('users').child(userId).child('posts').on('child_added', function(idSnap) {
var id = idSnap.key();
ref.child('posts').child(id).child('title').on('value', function(titleSnap) {
usersPosts[id] = titleSnap.val();
});
});
For a third solution, you could use firebase-util, which automagically handles the above scenario and more. This code would essentially do the same as the code above, except it comes with the bonus of giving you one ref to handle.
new Firebase.util.NormalizedCollection(
[ref.child('users').child(userId).child("posts"), "posts"],
[ref.child("posts"), "post"]
).select(
{
key: "posts.$value",
alias: "x"
},
{
key: "post.title",
alias: "title"
}
).ref();
Note that the x value will always be true. It's necessary to select that because firebase-util requires you to select at least one field from each path.

Related

Getting the right permitted fields of conditions

Currently, I'm building an app with with following similar logic:
...
const user = {
isAdmin: true,
company: '5faa6a847b42bf47b8f785a1',
projects: ['5faa6a847b42bf47b8f785a2']
}
function defineAbilityForUser(user) {
return defineAbility((can) => {
if (user.isAdmin) {
can('create', 'ProjectTime', {
company: user.company,
}
);
}
can(
'create',
'ProjectTime',
["company", "project", "user", "start", "end"],
{
company: user.company,
project: {
$in: user.projects
}
}
);
});
}
const userAbility = defineAbilityForUser(user); //
console.log( permittedFieldsOf(userAbility, 'create', 'ProjectTime') );
// console output: ['company', 'project', 'user', 'start', 'end']
Basically an admin should be allowed to create a project time with no field restrictions.
And a none admin user should only be allowed to set the specified fields for projects to which he belongs.
The problem is that I would expect to get [] as output because an admin should be allowed to set all fields for a project time.
The only solution I found was to set all fields on the admin user condition. But this requires a lot of migration work later when new fields are added to the project time model. (also wrapping the second condition in an else-block is not possible in my case)
Is there any other better way to do this? Or maybe, would it be better if the permittedFieldsOf-function would prioritize the condition with no field restrictions?
There is actually no way for casl to know what means all fields in context of your models. It knows almost nothing about their shapes and relies on conditions you provide it to check that objects later. So, it does not have full information.
What you need to do is to pass the 4th argument to override fieldsFrom callback. Check the api docs and reference implementation in #casl/mongoose
In casl v5, that parameter is mandatory. So, this confusion will disappear very soon

Best way to make ReactiveAggregate reactive when data changes on a user

I am currently using ReactiveAggregate to find a subset of Product data, like this:
ReactiveAggregate(this, Products, [
{ $match: {}},
{ $project: {
title: true,
image: true,
variants: {
$filter: {
input: "$variants",
as: "variant",
cond: {
$setIsSubset: [['$$variant.id'], user.variantFollowing]
}
}
}
}}
], { clientCollection: 'aggregateVariants' }
As you can see, a variant is returned if user.variantFollowing matches. When a user 'follows' a product, the ID is added to their object. However, if I understand correctly, this is not triggering ReactiveAggregate to get the new subset when this happens. Only on a full page refresh do I get the correct (latest) data.
Is this the correct way to approach this?
I could store the user's ID as part of the Product object, but the way this would be stored would be nested two places, and I think I would need the Mongo 3.5 updates to then be able to accurately update this. So i'm looking for how to do this in Meteor 1.5+ / Mongo 3.2.12
So, I've been able to get there by adding autorun to the subscription of the aggregate collection, like this:
Template.followedProducts.onCreated(function() {
Meteor.subscribe('products');
this.autorun(() => {
Meteor.subscribe('productsFollowed');
});
... rest of function
For context, productsFollowed is the subscription to retrieve aggregateVariants from the original question.
Thanks to robfallows in this post: https://forums.meteor.com/t/when-and-how-to-use-this-autorun/26075/6

How to update user's feed in Firebase Database when the user follows someone

I am designing an application that each user will a have a timeline (aka feed) based on the posts of his/her followings. In my database, I am keeping the user timeline posts as Shared Key. So each ID in the feedsubtree belong to a post of someone the user following.
"users": {
"userId-1": {
"userName": "Namey McNameface",
"following": {
"followingId-1": true,
"followingId-2": true,
.
.
},
"followers": {
"followerId-1": true,
"followerId-2": true,
.
.
},
"feed": {
"postId-1": true,
"postId-2": true,
.
.
}
},
"userId-2": {},
"userId-3": {},
.
.
}
When somebody posts something, I am also adding the key of that post to the feed of followers of that user.
The problem is that, when somebody starts following someone else, they are not starting to see their older posts. Do you have any idea how to update users timeline immediately after they followed someone?
Note:
This is not a code based problem, its actually a design problem. I am just trying to understand the general idea of user feed in Firebase Database.
Edit:
I am using Java and Spring Framework for my server.
The user feeds are time-ordered. So we need to keep this order while adding and removing references to posts.
Thank you.
If you want to show the historic posts of a user when you start following them, you will need to read those posts at that time. Since your question contains no information on how you store the posts, this is unfortunately as concrete as I can make the answer.
But if we assume that each user also has a list of their own posts, it could be something like (in JavaScript):
function follow(uid) {
var postsRef = ref.child('users').child(uid).child('posts');
var feedRef = ref.child('users').child(auth.uid).child('feed');
postsRef.on('child_added', function(snapshot) {
feedRef.child(snapshot.key).set(true);
});
}

Firebase :: Best way to structure this data

im trying to use firebase to store and retrieve data for my application.. i know that it is recommended to denormalize data and that it may require data replication..
my scenario is as follows:
there are a number of users in the system..
there are a number of posts in the system..
any user should be able to get a list of posts for a particular user..
each posts has a number of users as participants..
i am tempted to use the following structure for this:
users: {
abc: {
name: 'UserA',
profilePicture: 'imageA.png'
},
pqr: {
name: 'UserB',
profilePicture: 'imageB.png'
},
xyz: {
name: 'UserC',
profilePicture: 'imageC.png'
},
...,
...,
...
},
posts: {
def: {
title: 'PostA',
users: {
abc: true,
def: true,
ghi: true,
...,
...,
...
}
},
stu: {
title: 'PostB',
users: {
abc: true,
xyz: true,
...,
...,
...
}
},
...,
...,
...
}
the issue with this is that if i need to show a list of users with each post, i will need to make a query to POST, and then make sequential calls to USER for each user inside that post to get the name/profilePicture data..
if i replicate the user info inside posts as well, the issue becomes that if a user later changes her profilePicture or name, then existing posts will still show the old data..
how can i structure this data better so these cases are efficient?
thanks..
Don't replicate data inside posts. Read Firebase Docs about structuring data
Best practices:
Avoid nesting data
Flatten data structures
if you include data in post you are breaking those 2 rules (and you don't want it).
Multiple calls are not bad.

Meteor - How do you exclude properties in a child collection from being published?

Imagine you have a collection similar to the following...
Tests = [
{
name: 'Some Test',
questions: [
{ question: 'Answer to life, the universe, and everything?', answer: '42' },
{ question: 'What is your favorite color?', answer: 'Blue' },
{ question: 'Airspeed velocity of unladen European Swallow?', answer: '24 mph' }
]
}
];
How do you publish the entire collection except for the answer property?
I understand you can do the following to omit properties from the publish...
Meteor.publish('tests', function() {
return Tests.find({}, {fields: {name:0}});
});
But I'm not sure how to omit a property from an array property.
Thanks!
It can't be done the way you want to do it. Meteor only supports field specifiers that are 1 level deep. You can sometimes get a sub-field specifier to work, but it's not reliable.
You can put your questions into their own collection with a testId field that links them back to the test, relational style. One question per document, and then you'll be able to specify that only the question field gets published.
Meteor.publish ('questions', function(testId) {
return Questions.find({testId: testId}, {fields: {question: 1}})
});
It's not ideal, but pretty painless compared to trying to find a workaround that allows your questions to live in the test document.
There might be a way to do this manually with a more involved publish. There's a similar question here with an answer that gets into it.

Resources