How to 'transform' data returned via a Meteor.publish? - meteor

Meteor Collections have a transform ability that allows behavior to be attached to the objects returned from mongo.
We want to have autopublish turned off so the client does not have access to the database collections, but we still want the transform functionality.
We are sending data to the client with a more explicit Meteor.publish/Meteor.subscribe or the RPC mechanism ( Meteor.call()/Meteor.methods() )
How can we have the Meteor client automatically apply a transform like it will when retrieving data directly with the Meteor.Collection methods?

While you can't directly use transforms, there is a way to transform the result of a database query before publishing it. This is what the "publish the current size of a collection" example describes here.
It took me a while to figure out a really simple application of that, so maybe my code will help you, too:
Meteor.publish("publicationsWithHTML", function (data) {
var self = this;
Publications
.find()
.forEach(function(entry) {
addSomeHTML(entry); // this function changes the content of entry
self.added("publications", entry._id, entry);
});
self.ready();
});
On the client you subscribe to this:
Meteor.subscribe("publicationsWithHTML");
But your model still need to create a collection (on both sides) that is called 'publications':
Publications = new Meteor.Collection('publications');
Mind you, this is not a very good example, as it doesn't maintain the reactivity. But I found the count example a bit confusing at first, so maybe you'll find it helpful.

(Meteor 0.7.0.1) - meteor does allow behavior to be attached to the objects returned via the pub/sub.
This is from a pull request I submitted to the meteor project.
Todos = new Meteor.Collection('todos', {
// transform allows behavior to be attached to the objects returned via the pub/sub communication.
transform : function(todo) {
todo.update = function(change) {
Meteor.call('Todos_update', this._id, change);
},
todo.remove = function() {
Meteor.call('Todos_remove', this._id);
}
return todo;
}
});
todosHandle = Meteor.subscribe('todos');
Any objects returned via the 'todos' topic will have the update() and the remove() function - which is exactly what I want: I now attach behavior to the returned data.

Try:
let transformTodo = (fields) => {
fields._pubType = 'todos';
return fields;
};
Meteor.publish('todos', function() {
let subHandle = Todos
.find()
.observeChanges({
added: (id, fields) => {
fields = transformTodo(fields);
this.added('todos', id, fields);
},
changed: (id, fields) => {
fields = transformTodo(fields);
this.changed('todos', id, fields);
},
removed: (id) => {
this.removed('todos', id);
}
});
this.ready();
this.onStop(() => {
subHandle.stop();
});
});

Currently, you can't apply transforms on the server to published collections. See this question for more details. That leaves you with either transforming the data on the client, or using a meteor method. In a method, you can have the server do whatever you want to the data.
In one of my projects, we perform our most expensive query (it joins several collections, denormalizes the documents, and trims unnecessary fields) via a method call. It isn't reactive, but it greatly simplifies our code because all of the transformation happens on the server.

To extend #Christian Fritz answer, with Reactive Solution using peerlibrary:reactive-publish
Meteor.publish("todos", function() {
const self = this;
return this.autorun(function(computation) {
// Loop over each document in collection
todo.find().forEach(function(entry) {
// Add function to transform / modify each document here
self.added("todos", entry._id, entry);
});
});
});

Related

How to use angular-meteor helpers and Meteor.methods together

I use a bunch of helper methods in my project. Some of them require to load the whole collection into the client due the restriction of the api on client side (the distinct function!). I googled the problem and found Meteor.methods as solution.
Can I use helpers (like this.helpers) into Meteor methods? Or how should I dynamically update my data in the frontend?
Can someone give me an example?
Additional information:
class View2 {
constructor($interval, $scope, $reactive) {
'ngInject';
$reactive(this).attach($scope);
this.helpers({
getOrderNumber(){
this.tempVar = Kafkadata.find().fetch();
this.tempVar2 = _.pluck(this.tempVar, 'orderNumber');
this.tempVar3 = _.uniq(tempVar2, false);
return this.tempVar3;
},
});
}
This is an example for a helpers query. Currently, this code runs client-side. I get ALL orders(tempvar) and then remove ALL data except the ordernumbers(tempvar2). At the end I remove all multiple ordernumbers. ordernumber is not an unique value. Here is an example from one of the collections:
{"orderNumber":"f2a3ed95-fcc3-4da0-9b3f-32cf5ed087f8","value":12480,"booleanValue":false,"intValue":12480,"doubleValue":0,"status":"GOOD","itemName":"MILLING_SPEED","timestamp":1479145734448,"_id":{"_str":"5824f4bc7ff3f0199861f11d"}}
I want to use functions like db.collection.distinct(). But they only work server-side. I think I must use Meteor.methods()to make this thing server-side. But what about this helpers function? How do they work on Meteor.methods()?
EDIT2:
my test:
client-side:
folder:myProject/imports/ui/view1
class View1 {
constructor($interval, $scope, $reactive) {
'ngInject';
$reactive(this).attach($scope);
this.helpers({
// some code
getTestData(){
Meteor.call('allTestData',function(error, result){
if(error){
console.log("error");
}else{
return result;
}
});
}
}); //end of contructor
// this is my testfunction, which is bound to a button!
testFunction(){
Meteor.call('allTestData',function(error, result){
if(error){
alert('Error');
}else{
console.log(result);
}
});
}
on the server-side:
folder:myProject/server/main.js
Meteor.methods({
allTestData:()=>{
var results=Kafkadata.find().count();
console.log(results);
return results;
},
});
and this is my view1.html:
//some code
<md-button ng-click="view1.testFunction()"> It works!</md-button>
<h1>{{view1.getTestData}}</h1>
Why does the button work, but not the helper?
Even though .distinct is supported by Mongo, Meteor does not expose it, even on the server. You just have to use _.uniq as your example shows, but for performance reasons it's better if it runs on the server.
Below is an example of a helper that I use:
aweek: () => {
if (debug)
console.log("Querying weekly appointments for "+this.selectedWeek.format("Do MMMM"));
var weekApts = Appointments.find(
{start: {$gte: new Date(this.getReactively('this.selectedWeek').clone().day(1)),
$lt: new Date(this.getReactively('this.selectedWeek').clone().endOf('week'))},
elderid: Meteor.userId()
}).fetch();
return utils.services.getAWeek(weekApts,utils.data.elderTimeFormat);
},
Note the use of this.getReactively('this.selectedWeek') in the code... basically this tells Meteor to run this helper reactively, so if the value of this.selectedWeek changes, the helper will get re-run. So when I click on a week in the calendar and update the variable, it runs my helper again to get the data.
The utils.services.getAWeek() function does some calculation and formatting on the array of data that makes is easier to display.
If you create a Meteor Method to do processing, I would make it update a collection with its results, and then your helper on the client will update automatically. Best make the technology do the work for you :)

How do I handle values that are dependent on the result of a helper?

So I have an Angular controller with a meteor helper method, as below.
function localeCtrl($scope, $reactive, $stateParams{
$reactive(this).attach($scope);
var self = this;
self.helpers({
locale: function(){ return Locales.findOne($stateParams.id)},
staff: function(){
// Load data from second collection based on current Locale.
// But how?
},
address: function(){
// Take self.location.address and massage it to provide
// google maps link. How?
}
tags: function(){
// Collect all unique instances of a given tag by
// iterating over the available locales.
// E. G. If 10 locales have the 'restaurant' tag, and 5
// more have the 'library' tag, I want an array of
// ['restaurant', 'library'] -- easy enough to do
// by iterating over the locales, but how do I do that
// reactively?
}
});
}
Unfortunately, I need to set additional properties based on the data fetched by locale(). I can't set them up when I initialize the controller because the value in locale() changes as data is fetched from the server. But I need access to the data in locale to, for example, create the google maps address, or fetch associated records. (They aren't imbedded in the locale document for reasons that I'm sure made sense at the time).
Edit:
Additionally, I'm using ground DB to store a local copy of the data for offline access, which makes life even more complicated.
Probably you best bet is to publish your collection using publishComposite which is implemented using the reywood:publish-composite package.
Add the package:
meteor add reywood:publish-composite
Now where you publish the Locales collection you would do something like this:
Meteor.publishComposite('locales', function() {
return {
find() {
//Put whatever you need in the query for locales
const query = {
_userId: this.userId
};
return Locales.find(query);
},
children: [{
find(locale) {
return Staff.find({ localeId: locale._id });
}
}]
};
});
Then in your controller before the helper you add this:
this.subscribe('locales');
Now you should be able to simply call the code like this:
this.helpers({
locale(){
return Locales.findOne(this.$stateParams.id);
}
});
And access it in the template like this:
locale.staff
Give that a try and let me know!

MeteorJS ReactiveVar with database data

Clearly, I am doing something wrong with ReactiveVar because I cannot get it to work as I expect it should.
I am trying to set the value of an ReactiveVar by calling a Meteor.call method which returns the list of usernames. But it does not update when the usernames get changed in another part of the app.
I tried both:
Template.qastatistics.created = function () {
this.trackUsernames = new ReactiveVar(false);
var instance = Template.instance();
Meteor.call('trackUsernames', function (err, data) {
instance.trackUsernames.set(data);
});
};
and:
Template.qastatistics.helpers({
users: function () {
var usernames,
instance = Template.instance();
if (instance.trackUsernames.get() === false) {
Meteor.call('trackUsernames', function (err, data) {
instance.trackUsernames.set(data);
});
}
usernames = instance.trackUsernames.get();
...
But neither updates the list of usernames when these change in the database.
Is this even possible with ReactiveVars or have I completely misunderstood them?
EDIT: The usernames I mention are not from Meteor.users collection, but rather a distinct call from another collection that has usernames in it.
Fist of all I would use the onCreated function instead of defining created. That's a little more extendable and it's the new API. created is just kept around for backwards compatibility.
About your problem. You are right, you seem to have misunderstood what ReactiveVars do. They are a reactive data source. That means that when you call myReactiveVar.get in some Tracker.autorun (aka. reactive computation), the computation will rerun whenever myReactiveVar.set is called.
You got the first part right. Spacebars helpers always run inside their own computation. What you got wrong is thinking that a method call is a reactive action. That means, that you could call trackUsernames and set the trackUsernames ReativeVar again and the value in your template would update itself. But a method is only run once. It doesn't do anything fancy with reactivity.
A method call only transfers data once. When you publish a set of documents (like all users) on the other hand, they will be updated dynamically. Whenever a change happens inside that set of published documents, it will be synced to the client. So in general, it's a better idea to use publications and subscriptions to sync data reactively. If you'd want to use a method for the same thing you'd need to do some kind of polling (so your back in the stone-age again).
The easiest way to implement what you are trying to do is to use Meteor.users.find().fetch(). As it says in the docs fetch registers dependencies for all the documents you are fetching if it's being called from within a reactive computation.
First you'll need to properly set up your publications, so that users can see other users usernames. I'll leave that to you. Then you need to reimplement your helper
Template.qastatistics.helpers({
users: function () {
var usernames = _.pluck(Meteor.users.find().fetch(), 'username');
...
Thanks to suggestions from #kyll, I managed to get what I wanted by publishing the data I need:
server:
cope.publish.usernamesID = Random.id();
Meteor.publish("itemsusernames", function () {
self = this;
var initializing = true;
var handle = Items.find().observeChanges({
added: function (id) {
!initializing && self.changed(
"itemsusernames",
cope.publish.usernamesID,
Items.distinct("p4User"));
},
changed: function (id) {
!initializing && self.changed(
"itemsusernames",
cope.publish.usernamesID,
Items.distinct("p4User"));
},
removed: function (id) {
!initializing && self.changed(
"itemsusernames",
cope.publish.usernamesID,
Items.distinct("p4User"));
}
});
initializing = false;
self.added("itemsusernames", cope.publish.usernamesID, Items.distinct("p4User"));
self.ready();
self.onStop(function () {
handle.stop();
});
});
client:
users: function () {
var usernames = [],
oUsernames = ItemsUsernames.find().fetch();
if (!oUsernames[0]) return [];
usernames = $.map(oUsernames[0], function (value, index) {
if (!isNaN(index)) {
return [value];
}
});
...
And ofcourse: ItemsUsernames = new Mongo.Collection("itemsusernames");

Meteor: Publish a subset of another publication

I have a custom publication on my server (which in some way join 2 collections).
This resulting set of this publication is exactly what I need but for performances issues I would like to avoid sending it entirely to the client.
If I did not care about performances, I would only subscribe to the
publication and do something like
theCollection.find({"my":"filter"})
I am therefore trying to find a way to publish a subset of the custom publication so that the filter would be applied on the custom publication on the server side.
Is there a way to chain or filter publications (server side) ?
For the question we can assume the custom publication to look like this and cannot be modified:
Meteor.publish('customPublication', function() {
var sub = this;
var aCursor = Resources.find({type: 'someFilter'});
Mongo.Collection._publishCursor(aCursor, sub, 'customPublication');
sub.ready();
});
if i understand the question right, you are looking for https://atmospherejs.com/reywood/publish-composite
It let's you "publish a set of related documents from various collections using a reactive join. This makes it easy to publish a whole tree of documents at once. The published collections are reactive and will update when additions/changes/deletions are made."
Ok I came to the following workaround. Instead of working on the publication, I simply added a new collection I update according to the other collections. In order to do so I am using the meteor hooks package
function transformDocument(doc)
{
doc.aField = "aValue"; // do what you want here
return doc;
}
ACollection.after.insert(function(userId, doc)
{
var transformedDocument = transformDocument(doc);
AnotherCollection.insert(transformedDocument);
});
ACollection.after.update(function(userId, doc, fieldNames, modifier, options)
{
var transformedDocument = transformDocument(doc);
delete transformedDocument._id;
AnotherCollection.update(doc._id,{$set:transformedDocument});
});
ACollection.after.remove(function(userId, doc)
{
AnotherCollection.remove(doc._id);
});
Then I have the new collection I can publish subsets the regular way
Benefits:
You can filter whatever you want into this db, no need to worry if the field is virtual or real
Only one operation every time a db changes. This avoid having several publication merging the same data
Cave eats:
This requires one more Collection = more space
The 2 db might not be always synchronised, there is few reasons for this:
The client manually changed the data of "AnotherCollection"
You had documents in "ACollection" before you added "AnotherCollection".
The transform function or source collection schema changed at some point
To fix this:
AnotherCollection.allow({
insert: function () {
return Meteor.isServer;
},
update: function () {
return Meteor.isServer;
},
remove: function () {
return Meteor.isServer;
}
});
And to synchronise at meteor startup (i.e. build the collection from scratch). Do this only once for maintenance or after adding this new collection.
Meteor.startup(function()
{
AnotherCollection.remove({});
var documents = ACollection.find({}).fetch();
_.each(documents, function(doc)
{
var transformedDocument = transformDocument(doc);
AnotherCollection.insert(transformedDocument);
});
});

Publish documents in a collection to a meteor client depending on the existence of a specific document in another collection (publish-with-relations)

I have two collections
Offers (relevant fields: _id)
ShareRelations (relevant fields: receiverId and offerId)
and I'd like to publish only Offers to the logged in user which have been shared to him.
Actually, I'm doing this by using a helper array (visibleOffers) which I fill by looping for each ShareRelations and use this array later on the Offers.find as $in selector.
I wonder if this might be the meteor way to do this, or if I could do with less and/or prettier code?
My actual code to publish the Offers is the following:
Meteor.publish('offersShared', function () {
// check if the user is logged in
if (this.userId) {
// initialize helper array
var visibleOffers = [];
// initialize all shareRelations which the actual user is the receiver
var shareRelations = ShareRelations.find({receiverId: this.userId});
// check if such relations exist
if (shareRelations.count()) {
// loop trough all shareRelations and push the offerId to the array if the value isn't in the array actually
shareRelations.forEach(function (shareRelation) {
if (visibleOffers.indexOf(shareRelation.offerId) === -1) {
visibleOffers.push(shareRelation.offerId);
}
});
}
// return offers which contain the _id in the array visibleOffers
return Offers.find({_id: { $in: visibleOffers } });
} else {
// return no offers if the user is not logged in
return Offers.find(null);
}
});
Furthermore, the actual solution has the downside that if a new share relations is being created, the Offers collection on the client doesn't get updated with the newly visible offer instantly (read: page reload required. But I'm not sure if this is the case because of this publish method or because of some other code an this question is not primary because of this issue).
What you are looking for is a reactive join. You can accomplish this by directly using an observe in the publish function, or by using a library to do it for you. Meteor core is expected to have a join library at some point, but until then I'd recommend using publish-with-relations. Have a look at the docs, but I think the publish function you want looks something like this:
Meteor.publish('offersShared', function() {
return Meteor.publishWithRelations({
handle: this,
collection: ShareRelations,
filter: {receiverId: this.userId},
mappings: [{collection: Offers, key: 'offerId'}]
});
});
This should reactively publish all of the ShareRelations for the user, and all associated Offers. Hopefully publishing both won't be a problem.
PWR is a pretty legit package - several of us use it in production, and Tom Coleman contributes to it. The only thing I'll caution you about is that as of this writing, the current version in atmosphere (v0.1.5) has a bug which will result in a fairly serious memory leak. Until it gets bumped, see my blog post about how to run an updated local copy.
update 2/5/14:
The discover meteor blog has an excellent post on reactive joins which I highly recommend reading.
The way to do this is along the lines of this Question using observeChanges(). Still trying to figure out how to get it all working for my example, see Meteor, One to Many Relationship & add field only to client side collection in Publish?
You can use the reactive-publish package (I am one of authors):
Meteor.publish('offersShared', function () {
// check if the user is logged in
if (this.userId) {
this.autorun(function (computation) {
// initialize helper array
var visibleOffers = [];
// initialize all shareRelations which the actual user is the receiver
var shareRelations = ShareRelations.find({receiverId: this.userId}, {fields: {offerId: 1}});
// loop trough all shareRelations and push the offerId to the array if the value isn't in the array actually
shareRelations.forEach(function (shareRelation) {
if (visibleOffers.indexOf(shareRelation.offerId) === -1) {
visibleOffers.push(shareRelation.offerId);
}
});
// return offers which contain the _id in the array visibleOffers
return Offers.find({_id: { $in: visibleOffers } });
});
} else {
// return no offers if the user is not logged in
return Offers.find(null);
}
});
You can simply wrap your existing non-reactive code into an autorun and it will start to work. Just be careful to be precise which fields you query on because if you query on all fields then autorun will be rerun on any field change of ShareRelations, not just offerId.

Resources