From the blow post below
Firebase client-side fan-out for data consistency
Multi-path updates sound awesome. Does that work the same for Multi-path deletes?
Use case: I add a new post and it is fanned-out to many many followers. I decide to delete the post later on. Does the delete work the same? Do you have an example?
You can delete many posts in a single operation, by setting the value for each key to null.
function deletePostFromFollowers(postId, followers) {
var updates = {};
followers.forEach(function(followerId) {
updates['/users/'+followerId+'/posts/+'postId] = null
});
ref.update(updates);
}
deletePostFromFollowers('-K18713678adads', ['uid1', 'uid2']);
Related
I have a firebase realtime database trigger on a create node. my need is to update a property based on some condition in the create trigger for the same object. The way i am doing currently is below:
exports.on_order_received_validate_doodle_cash_order = functions.database.ref("/orders/{id}")
.onCreate((change, context) => {
console.log("start of on_order_received_deduct_doodle_cash")
const orderId = context.params.id
const order = change.val();
var db = admin.database();
const orderRef = db.ref('orders/')
return orderRef.child(orderId).update({"_verifiedOrder": true})
})
As you can see i am getting order id from context and then querying object again and updating it. My question is do i need to do this circus or can i just update it without querying again?
Generally it looks good. Just some small feedback to make you feel more confident about being on the right track.
Call the parameter snapshot instead of change because the parameter name change only make sense for the onUpdate event trigger.
You do not need to log that you're entering the function. Because entering and leaving the function is automatically logged by Firebase also.
You can remove the order variable that is unused.
You are actually not "querying" the object again. Making a reference to a node in the database doesn't make any network call itself. Not until you subscribe to receiving data. So doing orderRef.child(orderId) is not a query, it's just a database reference.
You can use the snapshot's own reference attribute to shorten your code a bit... effectively throwing away almost all code :-)
So your code code look like this instead. It is doing the exact same thing, just shorter. It was also correct from the beginning.
exports.on_order_received_validate_doodle_cash_order = functions
.database
.ref("/orders/{id}")
.onCreate((snapshot) => {
return snapshot.ref.child("_verifiedOrder").set(true);
});
But as mentioned in my comment above, you are effectively just setting a flag that is confirming that data was saved (or rather: confirming that the function was triggered). You might want to add some logic in there to check whether the order can be placed or not and then set the verified flag to true or false depending on that. Because with the logic of the implementation, all orders will have the value _verifiedOrder set to true, which is a waste of storage in your database.
I have created a simple Pre Trigger in my CosmosDB collection.
function testTrigger() {
var context = getContext();
var request = context.getRequest();
var documentToCreate = request.getBody();
documentToCreate["test"] = "Added by trigger";
request.setBody(documentToCreate);
}
Though it seems, this trigger isn't fired at all.
What's also irritating, the getContext() call is marked with a green squiggly line in the script explorer.
The problem that the trigger doesn't seem to be... well... triggered, is that you aren't specifying it on the triggering operation. Triggers are not automatic. They must be specified for each database operation where you want them executed.
Personally, I think that was a unfortunate design choice and I've decided to not use triggers at all in my own code. Rather, I embed all the business rule constraints in stored procedures. I find this to be a bit cleaner and removes the worry that I'll forget to specify a trigger or (more likely) someone else writing code to the database won't even know that they should. I realize that someone can bypass my stored procedure and violate the business rules by going directly to the documents, but I don't know of any way in DocumentDB to protect against that. If there was a way to make triggers be automatic, that would have protected against that.
When using the REST API, you specify triggers in a header with x-ms-documentdb-pre-trigger-include or x-ms-documentdb-post-trigger-include. In the node.js SDK, you use the RequestOptions preTriggerInclude or preTriggerInclude property. For .NET, you specify it like this:
Document createdItem = await client.CreateDocumentAsync(collection.SelfLink, new Document { Id = "documentdb" },
new RequestOptions
{
PreTriggerInclude = new List<string> { "CapitalizeName" },
});
See comments for discussion on the green squiggly.
I currently have my publications stored in /server/publications.js. I would like to store my client-side subscriptions in a central file too, like in /client/subscriptions.js. Is this a good design decision or are there more cons than pros? Thanks
There are three main places you can subscribe to a Meteor collection.
Globally
In this approach, you create subscriptions in a file somewhere on the client. Like your example, subscriptions.js is what most people name it. As long as it's in the client folder, it's fine.
Meteor.subscribe("posts");
Good candidates for this are collections that you will play around with on all or most of your templates. A friends list similar to Facebook, a feed of some kind.
Router
If you're using something like iron-router or flow-router, you can subscribe to collections based on the URL. I prefer this approach the most. It's flexible, but not too taxing on performance. For example:
// Inside lib/router.js
FlowRouter.route('/blog/:postId', {
subscriptions: function(params) {
this.register('myPost', Meteor.subscribe('blogPost', params.postId));
}
});
Now you can access the data you need using myPost. Very neat, and you can subscribe to as many things as you need.
Template
This is the most flexible of all and the most taxing if you have multiple templates. Honestly, just avoid this approach because it's way too much work. 90% of the time I find myself using Router subscriptions.
Template.posts.onCreated(function () {
// will re-run when the "limit" reactive variables changes
instance.autorun(function () {
// get the limit
var limit = instance.limit.get();
console.log("Asking for "+limit+" posts…")
// subscribe to the posts publication
var subscription = instance.subscribe('posts', limit);
// if subscription is ready, set limit to newLimit
if (subscription.ready()) {
console.log("> Received "+limit+" posts. \n\n")
instance.loaded.set(limit);
} else {
console.log("> Subscription is not ready yet. \n\n");
}
});
});
I suppose you could if you only want global, automatically-activated subscriptions. For some small apps that makes sense. As your code grows in complexity you may want more fine-grained control over your subscriptions. Generally speaking, you have three patters for placing subscriptions in your app (in order of granularity and control):
Globally (as suggested in your question)
In the router - assuming you use iron router
In the template. Also see this post for additional template subscription examples.
While the community has recently fallen in love with template subscriptions, I'm not prepared to make a general recommendation that they are always they way to go. As you move up the subscription hierarchy, you lose some control but also gain reusability.
Let's take a simple example - imagine you have a social app and you have a list of friends. Let's say the majority of routes and templates assume the existence of your fiends' user data. If each template or route made the same subscription for this list, you'd end up with a lot of duplication and, potentially, a lot of unnecessary starting and stopping (this translates to server load and network bandwidth). On the other hand, had you made your friends a global subscription, those performance pitfalls could have been avoided.
Try to use template based subscriptions if possible. In that case, if you have /client/templates/home.html, you would put subscriptions related to that template in /client/templates/home.js for example.
Template.home.onCreated(function() {
this.subscribe('somePublication');
});
In my experience, the better way to manage the subscriptions is using the template subscription pattern pattern
Snippet taken from the reference:
Template.posts.onCreated(function () {
// 1. Initialization
var instance = this;
// initialize the reactive variables
instance.loaded = new ReactiveVar(0);
instance.limit = new ReactiveVar(5);
// 2. Autorun
// will re-run when the "limit" reactive variables changes
instance.autorun(function () {
// get the limit
var limit = instance.limit.get();
console.log("Asking for "+limit+" posts…")
// subscribe to the posts publication
var subscription = instance.subscribe('posts', limit);
// if subscription is ready, set limit to newLimit
if (subscription.ready()) {
console.log("> Received "+limit+" posts. \n\n")
instance.loaded.set(limit);
} else {
console.log("> Subscription is not ready yet. \n\n");
}
});
// 3. Cursor
instance.posts = function() {
return Posts.find({}, {limit: instance.loaded.get()});
}
});
I've read the Firebase docs on Stucturing Data. Data storage is cheap, but the user's time is not. We should optimize for get operations, and write in multiple places.
So then I might store a list node and a list-index node, with some duplicated data between the two, at very least the list name.
I'm using ES6 and promises in my javascript app to handle the async flow, mainly of fetching a ref key from firebase after the first data push.
let addIndexPromise = new Promise( (resolve, reject) => {
let newRef = ref.child('list-index').push(newItem);
resolve( newRef.key()); // ignore reject() for brevity
});
addIndexPromise.then( key => {
ref.child('list').child(key).set(newItem);
});
How do I make sure the data stays in sync in all places, knowing my app runs only on the client?
For sanity check, I set a setTimeout in my promise and shut my browser before it resolved, and indeed my database was no longer consistent, with an extra index saved without a corresponding list.
Any advice?
Great question. I know of three approaches to this, which I'll list below.
I'll take a slightly different example for this, mostly because it allows me to use more concrete terms in the explanation.
Say we have a chat application, where we store two entities: messages and users. In the screen where we show the messages, we also show the name of the user. So to minimize the number of reads, we store the name of the user with each chat message too.
users
so:209103
name: "Frank van Puffelen"
location: "San Francisco, CA"
questionCount: 12
so:3648524
name: "legolandbridge"
location: "London, Prague, Barcelona"
questionCount: 4
messages
-Jabhsay3487
message: "How to write denormalized data in Firebase"
user: so:3648524
username: "legolandbridge"
-Jabhsay3591
message: "Great question."
user: so:209103
username: "Frank van Puffelen"
-Jabhsay3595
message: "I know of three approaches, which I'll list below."
user: so:209103
username: "Frank van Puffelen"
So we store the primary copy of the user's profile in the users node. In the message we store the uid (so:209103 and so:3648524) so that we can look up the user. But we also store the user's name in the messages, so that we don't have to look this up for each user when we want to display a list of messages.
So now what happens when I go to the Profile page on the chat service and change my name from "Frank van Puffelen" to just "puf".
Transactional update
Performing a transactional update is the one that probably pops to mind of most developers initially. We always want the username in messages to match the name in the corresponding profile.
Using multipath writes (added on 20150925)
Since Firebase 2.3 (for JavaScript) and 2.4 (for Android and iOS), you can achieve atomic updates quite easily by using a single multi-path update:
function renameUser(ref, uid, name) {
var updates = {}; // all paths to be updated and their new values
updates['users/'+uid+'/name'] = name;
var query = ref.child('messages').orderByChild('user').equalTo(uid);
query.once('value', function(snapshot) {
snapshot.forEach(function(messageSnapshot) {
updates['messages/'+messageSnapshot.key()+'/username'] = name;
})
ref.update(updates);
});
}
This will send a single update command to Firebase that updates the user's name in their profile and in each message.
Previous atomic approach
So when the user change's the name in their profile:
var ref = new Firebase('https://mychat.firebaseio.com/');
var uid = "so:209103";
var nameInProfileRef = ref.child('users').child(uid).child('name');
nameInProfileRef.transaction(function(currentName) {
return "puf";
}, function(error, committed, snapshot) {
if (error) {
console.log('Transaction failed abnormally!', error);
} else if (!committed) {
console.log('Transaction aborted by our code.');
} else {
console.log('Name updated in profile, now update it in the messages');
var query = ref.child('messages').orderByChild('user').equalTo(uid);
query.on('child_added', function(messageSnapshot) {
messageSnapshot.ref().update({ username: "puf" });
});
}
console.log("Wilma's data: ", snapshot.val());
}, false /* don't apply the change locally */);
Pretty involved and the astute reader will notice that I cheat in the handling of the messages. First cheat is that I never call off for the listener, but I also don't use a transaction.
If we want to securely do this type of operation from the client, we'd need:
security rules that ensure the names in both places match. But the rules need to allow enough flexibility for them to temporarily be different while we're changing the name. So this turns into a pretty painful two-phase commit scheme.
change all username fields for messages by so:209103 to null (some magic value)
change the name of user so:209103 to 'puf'
change the username in every message by so:209103 that is null to puf.
that query requires an and of two conditions, which Firebase queries don't support. So we'll end up with an extra property uid_plus_name (with value so:209103_puf) that we can query on.
client-side code that handles all these transitions transactionally.
This type of approach makes my head hurt. And usually that means that I'm doing something wrong. But even if it's the right approach, with a head that hurts I'm way more likely to make coding mistakes. So I prefer to look for a simpler solution.
Eventual consistency
Update (20150925): Firebase released a feature to allow atomic writes to multiple paths. This works similar to approach below, but with a single command. See the updated section above to read how this works.
The second approach depends on splitting the user action ("I want to change my name to 'puf'") from the implications of that action ("We need to update the name in profile so:209103 and in every message that has user = so:209103).
I'd handle the rename in a script that we run on a server. The main method would be something like this:
function renameUser(ref, uid, name) {
ref.child('users').child(uid).update({ name: name });
var query = ref.child('messages').orderByChild('user').equalTo(uid);
query.once('value', function(snapshot) {
snapshot.forEach(function(messageSnapshot) {
messageSnapshot.update({ username: name });
})
});
}
Once again I take a few shortcuts here, such as using once('value' (which is in general a bad idea for optimal performance with Firebase). But overall the approach is simpler, at the cost of not having all data completely updated at the same time. But eventually the messages will all be updated to match the new value.
Not caring
The third approach is the simplest of all: in many cases you don't really have to update the duplicated data at all. In the example we've used here, you could say that each message recorded the name as I used it at that time. I didn't change my name until just now, so it makes sense that older messages show the name I used at that time. This applies in many cases where the secondary data is transactional in nature. It doesn't apply everywhere of course, but where it applies "not caring" is the simplest approach of all.
Summary
While the above are just broad descriptions of how you could solve this problem and they are definitely not complete, I find that each time I need to fan out duplicate data it comes back to one of these basic approaches.
To add to Franks great reply, I implemented the eventual consistency approach with a set of Firebase Cloud Functions. The functions get triggered whenever a primary value (eg. users name) gets changed, and then propagate the changes to the denormalized fields.
It is not as fast as a transaction, but for many cases it does not need to be.
I have a publication based on server-side user permissions. I want it to be reactive to changes in these permissions.
// SERVER CODE
Meteor.publish("my_publication", function(parent_id) {
//fetch our parent record and lookup this user's permissions
var parent = ParentCollection.findOne({_id: parent_id});
var myPermissionsList = parent.permissionsDict[this.userId];
//use these permissions to make our query
ChildCollection.find({parent_id: parent_id, permissions: {$in: myPermissionsList}})
}
// CLIENT CODE
Tracker.autorun(function () {
Meteor.subscribe('my_publication', Session.get("my_parent_id"));
});
This properly returns all the elements of the "child" collection specified parent, as long as the parent says the user has at least one of the permissions in the child element's list. It does this without the user actually knowing what their permissions are, which is a requirement.
This behaves like one would expect in Meteor:
The subscription does automatically update if any of the returned ChildCollection elements are changed.
The subscription does automatically update if the client changes the "my_parent_id" Session variable, triggering the Tracker.autorun resubscribe.
The subscription does not automatically update if the permissions used to make the query (parent.permissionsDict[this.userId]) are changed.
We're looking for the best (highest performing) way to get an automatic update in the last case.
This article was a helpful, more detailed resource on the topic:
https://www.discovermeteor.com/blog/reactive-joins-in-meteor/
My current understanding is that I need to utilize cursor.observeChanges() to react to changes in my permissions query. However, I am not sure how this fits into the rest of the Meteor publish/subscribe model--where would I call this, and how could the callback instruct Meteor to republish "my_publication"?
I believe https://atmospherejs.com/mrt/reactive-publish addresses this, but I feel like I should try to get a better grasp on core reactivity in meteor before turning to an external package. I also lack an understanding about the performance costs.
Any help would be greatly appreciated!
You can use the reactive-publish package (I am one of authors):
Meteor.publish("my_publication", function(parent_id) {
this.autorun(function (computation) {
//fetch our parent record and lookup this user's permissions
var parent = ParentCollection.findOne({_id: parent_id}, {fields: {permissionsDict: 1}});
var myPermissionsList = parent.permissionsDict[this.userId];
//use these permissions to make our query
return ChildCollection.find({parent_id: parent._id, permissions: {$in: myPermissionsList}});
});
}
It is important that you limit the fields you are interested in the parent document, otherwise autorun would rerun every time any field changes in the document, even if you do not care/use that field.