I have a custom publication on my server (which in some way join 2 collections).
This resulting set of this publication is exactly what I need but for performances issues I would like to avoid sending it entirely to the client.
If I did not care about performances, I would only subscribe to the
publication and do something like
theCollection.find({"my":"filter"})
I am therefore trying to find a way to publish a subset of the custom publication so that the filter would be applied on the custom publication on the server side.
Is there a way to chain or filter publications (server side) ?
For the question we can assume the custom publication to look like this and cannot be modified:
Meteor.publish('customPublication', function() {
var sub = this;
var aCursor = Resources.find({type: 'someFilter'});
Mongo.Collection._publishCursor(aCursor, sub, 'customPublication');
sub.ready();
});
if i understand the question right, you are looking for https://atmospherejs.com/reywood/publish-composite
It let's you "publish a set of related documents from various collections using a reactive join. This makes it easy to publish a whole tree of documents at once. The published collections are reactive and will update when additions/changes/deletions are made."
Ok I came to the following workaround. Instead of working on the publication, I simply added a new collection I update according to the other collections. In order to do so I am using the meteor hooks package
function transformDocument(doc)
{
doc.aField = "aValue"; // do what you want here
return doc;
}
ACollection.after.insert(function(userId, doc)
{
var transformedDocument = transformDocument(doc);
AnotherCollection.insert(transformedDocument);
});
ACollection.after.update(function(userId, doc, fieldNames, modifier, options)
{
var transformedDocument = transformDocument(doc);
delete transformedDocument._id;
AnotherCollection.update(doc._id,{$set:transformedDocument});
});
ACollection.after.remove(function(userId, doc)
{
AnotherCollection.remove(doc._id);
});
Then I have the new collection I can publish subsets the regular way
Benefits:
You can filter whatever you want into this db, no need to worry if the field is virtual or real
Only one operation every time a db changes. This avoid having several publication merging the same data
Cave eats:
This requires one more Collection = more space
The 2 db might not be always synchronised, there is few reasons for this:
The client manually changed the data of "AnotherCollection"
You had documents in "ACollection" before you added "AnotherCollection".
The transform function or source collection schema changed at some point
To fix this:
AnotherCollection.allow({
insert: function () {
return Meteor.isServer;
},
update: function () {
return Meteor.isServer;
},
remove: function () {
return Meteor.isServer;
}
});
And to synchronise at meteor startup (i.e. build the collection from scratch). Do this only once for maintenance or after adding this new collection.
Meteor.startup(function()
{
AnotherCollection.remove({});
var documents = ACollection.find({}).fetch();
_.each(documents, function(doc)
{
var transformedDocument = transformDocument(doc);
AnotherCollection.insert(transformedDocument);
});
});
Related
I'd like if someon could clarify this point about subscriptions in Meteor.js:
as written in the docs the find() function returns a cursor, not the data, that needs to be fetched;
let's say I have a publish function:
Meteor.publish('pages', function() {
return Pages.find()
});
now let's say I need only the page with code: "one"; in a template helper I can fetch that document:
Template.mytemplate.helpers({
data: function() {
return Pages.find({code: "one"});
}
});
The question is: is correct to do this, or if I need only one document would be better to subscribe only that document? (something like:
Meteor.publish('page', function(code) {
return Pages.find({code: code})
});
)
I mean: does it changes something in terms of efficiency/performance?
Yes, it does change a lot. If you subscribe to the entire collection, then the entire collection will be send to the client and kept in sync with the server. So by all means, if you only need one document, and do not need to cache any of the other documents (e.g., for switching to a new document), then only subscribe to the one you need.
Clearly, I am doing something wrong with ReactiveVar because I cannot get it to work as I expect it should.
I am trying to set the value of an ReactiveVar by calling a Meteor.call method which returns the list of usernames. But it does not update when the usernames get changed in another part of the app.
I tried both:
Template.qastatistics.created = function () {
this.trackUsernames = new ReactiveVar(false);
var instance = Template.instance();
Meteor.call('trackUsernames', function (err, data) {
instance.trackUsernames.set(data);
});
};
and:
Template.qastatistics.helpers({
users: function () {
var usernames,
instance = Template.instance();
if (instance.trackUsernames.get() === false) {
Meteor.call('trackUsernames', function (err, data) {
instance.trackUsernames.set(data);
});
}
usernames = instance.trackUsernames.get();
...
But neither updates the list of usernames when these change in the database.
Is this even possible with ReactiveVars or have I completely misunderstood them?
EDIT: The usernames I mention are not from Meteor.users collection, but rather a distinct call from another collection that has usernames in it.
Fist of all I would use the onCreated function instead of defining created. That's a little more extendable and it's the new API. created is just kept around for backwards compatibility.
About your problem. You are right, you seem to have misunderstood what ReactiveVars do. They are a reactive data source. That means that when you call myReactiveVar.get in some Tracker.autorun (aka. reactive computation), the computation will rerun whenever myReactiveVar.set is called.
You got the first part right. Spacebars helpers always run inside their own computation. What you got wrong is thinking that a method call is a reactive action. That means, that you could call trackUsernames and set the trackUsernames ReativeVar again and the value in your template would update itself. But a method is only run once. It doesn't do anything fancy with reactivity.
A method call only transfers data once. When you publish a set of documents (like all users) on the other hand, they will be updated dynamically. Whenever a change happens inside that set of published documents, it will be synced to the client. So in general, it's a better idea to use publications and subscriptions to sync data reactively. If you'd want to use a method for the same thing you'd need to do some kind of polling (so your back in the stone-age again).
The easiest way to implement what you are trying to do is to use Meteor.users.find().fetch(). As it says in the docs fetch registers dependencies for all the documents you are fetching if it's being called from within a reactive computation.
First you'll need to properly set up your publications, so that users can see other users usernames. I'll leave that to you. Then you need to reimplement your helper
Template.qastatistics.helpers({
users: function () {
var usernames = _.pluck(Meteor.users.find().fetch(), 'username');
...
Thanks to suggestions from #kyll, I managed to get what I wanted by publishing the data I need:
server:
cope.publish.usernamesID = Random.id();
Meteor.publish("itemsusernames", function () {
self = this;
var initializing = true;
var handle = Items.find().observeChanges({
added: function (id) {
!initializing && self.changed(
"itemsusernames",
cope.publish.usernamesID,
Items.distinct("p4User"));
},
changed: function (id) {
!initializing && self.changed(
"itemsusernames",
cope.publish.usernamesID,
Items.distinct("p4User"));
},
removed: function (id) {
!initializing && self.changed(
"itemsusernames",
cope.publish.usernamesID,
Items.distinct("p4User"));
}
});
initializing = false;
self.added("itemsusernames", cope.publish.usernamesID, Items.distinct("p4User"));
self.ready();
self.onStop(function () {
handle.stop();
});
});
client:
users: function () {
var usernames = [],
oUsernames = ItemsUsernames.find().fetch();
if (!oUsernames[0]) return [];
usernames = $.map(oUsernames[0], function (value, index) {
if (!isNaN(index)) {
return [value];
}
});
...
And ofcourse: ItemsUsernames = new Mongo.Collection("itemsusernames");
I'd like to create a method that returns the count of a generic collection.
Calling the method would look something like this:
Meteor.call('getCollectionCount', 'COLLECTION_NAME');
And the result would be the collection count.
The server method code would look something like this:
getCollectionCount: function (collectionName) {
return window[collectionName].find().count();
}
This won't work because window isn't defined on the server, but is something similar possible?
Use global instead of window.
Note that this uses the variable name assigned to the collection object, not the name given to the collection. For this to work with Meteor.users you need to assign another variable name.
if (Meteor.isServer) {
users = Meteor.users;
}
if (Meteor.isClient) {
Meteor.call('count', 'users', function (err, res) {
// do something with number of users
});
}
Also probably a good idea to check that global[collectionName] is actually a collection.
I came up with this code which makes the following assumptions :
collections are declared in the global scope as top level objects.
collections are searched by collection name, not the collection variable identifier.
So client code should declare their collections like this :
MyCollection=new Meteor.Collection("my-collection");
And use the function like this :
var clientResult=Meteor.call("getCollectionCount","my-collection",function(error,result){
if(error){
console.log(error);
return;
}
console.log("actual server-side count is : ",result);
});
console.log("published subset count is : ",clientResult);
The method supports execution on the client (this is known as method stub or method simulation) but will only yield the count of the collection subset replicated client-side, to get the real count wait for server-side response using a callback.
/packages/my-package/lib/my-package.js
getCollection=function(collectionName){
if(collectionName=="users"){
return Meteor.users;
}
var globalScope=Meteor.isClient?window:global;
for(var property in globalScope){
var object=globalScope[property];
if(object instanceof Meteor.Collection && object._name==collectionName){
return object;
}
}
throw Meteor.Error(500,"No collection named "+collectionName);
};
Meteor.methods({
getCollectionCount:function(collectionName){
return getCollection(collectionName).find().count();
}
});
As Meteor.users is not declared as a top level variable you have to account for the special case (yes, this is ugly).
Digging into Meteor's collection handling code could provide a better alternative (getting access to a collection handle by collection name).
Final words on this : using a method call to count a collection documents is unfortunately non-reactive, so given the Meteor paradigm this might be of little use.
Most of the time you will want to fetch the number of documents in a collection for pagination purpose (something like a "Load more" button in a posts list for example), and as the rest of the Meteor architecture you'll want this to be reactive.
To count documents in a collection reactively you'll have to setup a slightly more complicated publication as showcased in the "counts-by-room" example in the docs.
http://docs.meteor.com/#meteor_publish
This is something you definitely want to read and understand.
This smart package is actually doing it right :
http://atmospherejs.com/package/publish-counts
It provides a helper function that is publishing the counts of any cursor.
Keep track of the collections on some other property that the server has access too. You could even call it window if you really wanted to.
var wow = new Meteor.Collection("wow");
collections["wow"] = wow;
getCollectionCount: function (collectionName) {
return collections[collectionName].find().count();
}
If you don't want the package users to change how they work with collections in the app then I think you should use MongoInternals to get collections by name from the db. Not tested but here is an example:
//on server
Meteor.methods({
count: function( name ){
var db = MongoInternals.defaultRemoteCollectionDriver().mongo.db;
var collection = db.collection( name );
return collection && collection.count({});
}
});
Another example of MongoInternals use is here. Documentation of the count() function available from the mongo driver is here.
Meteor Collections have a transform ability that allows behavior to be attached to the objects returned from mongo.
We want to have autopublish turned off so the client does not have access to the database collections, but we still want the transform functionality.
We are sending data to the client with a more explicit Meteor.publish/Meteor.subscribe or the RPC mechanism ( Meteor.call()/Meteor.methods() )
How can we have the Meteor client automatically apply a transform like it will when retrieving data directly with the Meteor.Collection methods?
While you can't directly use transforms, there is a way to transform the result of a database query before publishing it. This is what the "publish the current size of a collection" example describes here.
It took me a while to figure out a really simple application of that, so maybe my code will help you, too:
Meteor.publish("publicationsWithHTML", function (data) {
var self = this;
Publications
.find()
.forEach(function(entry) {
addSomeHTML(entry); // this function changes the content of entry
self.added("publications", entry._id, entry);
});
self.ready();
});
On the client you subscribe to this:
Meteor.subscribe("publicationsWithHTML");
But your model still need to create a collection (on both sides) that is called 'publications':
Publications = new Meteor.Collection('publications');
Mind you, this is not a very good example, as it doesn't maintain the reactivity. But I found the count example a bit confusing at first, so maybe you'll find it helpful.
(Meteor 0.7.0.1) - meteor does allow behavior to be attached to the objects returned via the pub/sub.
This is from a pull request I submitted to the meteor project.
Todos = new Meteor.Collection('todos', {
// transform allows behavior to be attached to the objects returned via the pub/sub communication.
transform : function(todo) {
todo.update = function(change) {
Meteor.call('Todos_update', this._id, change);
},
todo.remove = function() {
Meteor.call('Todos_remove', this._id);
}
return todo;
}
});
todosHandle = Meteor.subscribe('todos');
Any objects returned via the 'todos' topic will have the update() and the remove() function - which is exactly what I want: I now attach behavior to the returned data.
Try:
let transformTodo = (fields) => {
fields._pubType = 'todos';
return fields;
};
Meteor.publish('todos', function() {
let subHandle = Todos
.find()
.observeChanges({
added: (id, fields) => {
fields = transformTodo(fields);
this.added('todos', id, fields);
},
changed: (id, fields) => {
fields = transformTodo(fields);
this.changed('todos', id, fields);
},
removed: (id) => {
this.removed('todos', id);
}
});
this.ready();
this.onStop(() => {
subHandle.stop();
});
});
Currently, you can't apply transforms on the server to published collections. See this question for more details. That leaves you with either transforming the data on the client, or using a meteor method. In a method, you can have the server do whatever you want to the data.
In one of my projects, we perform our most expensive query (it joins several collections, denormalizes the documents, and trims unnecessary fields) via a method call. It isn't reactive, but it greatly simplifies our code because all of the transformation happens on the server.
To extend #Christian Fritz answer, with Reactive Solution using peerlibrary:reactive-publish
Meteor.publish("todos", function() {
const self = this;
return this.autorun(function(computation) {
// Loop over each document in collection
todo.find().forEach(function(entry) {
// Add function to transform / modify each document here
self.added("todos", entry._id, entry);
});
});
});
Is it possible yet, to preserve insertion order or set reliable timestamps in Meteor given that MongoDB doesn't guarantee to return items in insertion order if no sort is specified, a document's _id is randomly generated and setting a timestamp manually on insertion would depend upon the client's clock?
I suggest a method.
Meteor.methods({
addItem: function (doc) {
doc.when = new Date;
return Items.insert(doc);
}
});
While the client will run this locally and set when to its own current time, the server's timestamp takes priority and propagates to all subscribed clients, including the original client. You can sort on doc.when.
We'll probably add hooks for setting timestamps automatically as part of document validations and permissions.
If you're willing to use something like these collection hooks (https://gist.github.com/matb33/5258260), along with this fancy Date.unow function (which you can safely sort on even if many documents were inserted with the same timestamp):
if (!Date.unow) {
(function () {
var uniq = 0;
Date.unow = function () {
uniq++;
return Date.now() + (uniq % 5000);
};
})();
}
if (Meteor.isServer) {
// NOTE: this isn't vanilla Meteor, and sometime in the future there may be
// a better way of doing this, but at the time of writing this is it:
Items.before("insert", function (userId, doc) {
doc.created = Date.unow();
});
}