I need to inform clients about changes on server side. In my case I am using different Collections on server and on client (more about it in this question: how would you build pinterest like page with meteor.js).
On the server I am getting new Products from external API. I would like to publish the number of new items to all clients that they could update their local variables needed for layout to work well.
How to do it?
It would be nice if I could publish/subscribe other kinds of data than Meteor.Collection. I found Meteor.deps, but what I understand it works only on client side.
To accomplish what you want you do need another collection - on the client. On the server, in a publish function, build a document from scratch assigning the current count of Products to an attribute. Using observe() and set, modify count when documents are added or removed from Products. Subscribe to the count "record set" on the client.
// Server
Meteor.publish('count', function () {
// Build a document from scratch
var self = this;
var uuid = Meteor.uuid();
var count = Products.find().count();
// Assign initial Products count to document attribute
self.set('count', uuid, {count: count});
// Observe Products for additions and removals
var handle = Products.find().observe({
added: function (doc, idx) {
count++;
self.set('counts', uuid, {count: count});
self.flush();
},
removed: function (doc, idx) {
count--;
self.set('counts', uuid, {count: count});
self.flush();
}
});
self.complete();
self.flush();
self.onStop(function () {
handle.stop();
});
});
// Client
Counts = new Meteor.Collection('count');
Meteor.subscribe('count');
console.log('Count: ' + Counts.findOne().count);
I must say the above solution showed me one way, but still, what if I need to publish to client data that are not connected with observe()? Or with any collection?
In my case I have i.e. 1000 products. To engage visitors I am "refreshig" the collection by updating the timestamp of random number of products, and displaying collection sorted by timestamp. Thank to this visitors have impression that something is happening.
My refresh method returns number of products (it is random). I need to pass that number to all clients. I did it, but using (I think) ugly workaround.
My refresh method sets Session.set('lastRandomNo', random). BTW: I didn't know that Session works on server side. refresh updates Products collection.
Then accoriding to above answer:
Meteor.publish 'refreshedProducts', ->
self = this
uuid = Meteor.uuid()
# create a new collection to pass ProductsMeta data
self.set('products_meta', uuid, { refreshedNo: 0 })
handle = Products.find().observe
changed: (newDocument, atIndex, oldDocument) ->
self.set('products_meta', uuid, { refreshedNo: Session.get('lastRandomNo') })
self.flush()
self.complete()
self.flush()
self.onStop ->
handle.stop()
and on client side:
ProductsMeta = new Meteor.Collection('products_meta')
# subscribe to server 'products_meta' collection that is generated by server
Meteor.subscribe('refreshedProducts')
ProductsMeta.find({}).observe
changed: (newDocument, atIndex, oldDocument) ->
# I have access to refreshedNo by
console.log ProductsMeta.findOne().refreshedNo
What do you think?
Related
We have use publication and subscription with skip and limit in meteor and it fetch all records at client side.
Meteor.publish('jobOrders', function(skip, limit) {
return (jobOrders.find({}, {
skip: skip,
limit: limit
}));
});
Template.templateName.onCreated(function() {
this.subscribe("jobOrders", 0, 2);
});
Template.templateName.helpers({
methodName: function() {
var totalNoOfRecords = jobOrders.find({}).count();
console.og("records are", totalNoOfRecords);
return totalNoOfRecords;
}
});
The reason could be because you have other pub-subs in other templates which are fetching the data from jobOrders collection.
Please comment your subscription in your current template and see whether you are still able to fetch the records from this collection. If you are seeing all the records even after commenting the subscription, it should be because of some other subscription.
To overcome this problem, you have to either do one of the following:
Change the other subscription which is fetching all the records such that the data is available only with in that template by using
Template.instance().subscribe()
instead of using Meteor.subscribe()which will make the data available throughout client side across all templates.
Filter you records on the client side as well.
Template.templateName.helpers({
methodName: function() {
var totalNoOfRecords = jobOrders.find({skip:0, limit: 2}).count(); //Filter on the client side
console.og("records are", totalNoOfRecords);
return totalNoOfRecords;
}
});
Note: Another possibility could be that you did not remove the insecure package which will automatically publish all the records in all the collection.
I have two publish method as below but when I subscribe to one of the publish method in client search page, it is being overwritten with the other one which is meant for index page.
Server
Meteor.publish("task.index", function() {
TaskCollection.find()
}
Meteor.publish("task.index.search", function(state) {
TaskCollection.find({ state: state })
}
Client - search page
Meteor.subscribe("task.index.search", state)
// this will always be overwritten with "task.index" published collection
Client - index page
Meteor.subscribe("task.index")
Does anyone know how to avoid this?
Welcome to SO!
There is a high chance the "override" you see is just the normal Meteor behaviour for Publish/Subscribe mechanism.
Your "task.index" publication sends all your TaskCollection documents to the Client.
Therefore any other publication on that same TaskCollection will send documents that the Client already knows.
Then in your Client, filtering some documents from TaskCollection is independent from your subscription and publication. Just perform your TaskCollection.find({ state: state }) Client side, and you will get the documents you need.
When you publish only a subset of documents of a Collection, it happens that what you publish is exactly already the filtered documents that you want to display on your Client, therefore on your Client you just display all Collection documents you know of. But you have to understand that these are 2 different steps:
Subscription to send some documents to the Client. Several Subscriptions may be set, filling the same Collection on the Client.
Filtering on the Client, based on the documents sent by the (possibly several) Subscription(s).
See also: Publish subscribe doesn't seem to work
If your client index and search pages are different templates, you can subscribe to the documents at respective template level.
Client - search page:
Template.search.created = function () {
const template = this;
template.subscribe('task.index.search', state);
}
Template.search.rendered = function () {
console.log("Client search : " + TaskCollection.find().fetch().length);
}
Client - index page:
Template.index.created = function () {
const template = this;
template.subscribe('task.index');
}
Template.index.rendered = function () {
console.log(""Index : " + TaskCollection.find().fetch().length);
}
But, it is always advisable to filter the documents on the client as well.
I have a custom publication on my server (which in some way join 2 collections).
This resulting set of this publication is exactly what I need but for performances issues I would like to avoid sending it entirely to the client.
If I did not care about performances, I would only subscribe to the
publication and do something like
theCollection.find({"my":"filter"})
I am therefore trying to find a way to publish a subset of the custom publication so that the filter would be applied on the custom publication on the server side.
Is there a way to chain or filter publications (server side) ?
For the question we can assume the custom publication to look like this and cannot be modified:
Meteor.publish('customPublication', function() {
var sub = this;
var aCursor = Resources.find({type: 'someFilter'});
Mongo.Collection._publishCursor(aCursor, sub, 'customPublication');
sub.ready();
});
if i understand the question right, you are looking for https://atmospherejs.com/reywood/publish-composite
It let's you "publish a set of related documents from various collections using a reactive join. This makes it easy to publish a whole tree of documents at once. The published collections are reactive and will update when additions/changes/deletions are made."
Ok I came to the following workaround. Instead of working on the publication, I simply added a new collection I update according to the other collections. In order to do so I am using the meteor hooks package
function transformDocument(doc)
{
doc.aField = "aValue"; // do what you want here
return doc;
}
ACollection.after.insert(function(userId, doc)
{
var transformedDocument = transformDocument(doc);
AnotherCollection.insert(transformedDocument);
});
ACollection.after.update(function(userId, doc, fieldNames, modifier, options)
{
var transformedDocument = transformDocument(doc);
delete transformedDocument._id;
AnotherCollection.update(doc._id,{$set:transformedDocument});
});
ACollection.after.remove(function(userId, doc)
{
AnotherCollection.remove(doc._id);
});
Then I have the new collection I can publish subsets the regular way
Benefits:
You can filter whatever you want into this db, no need to worry if the field is virtual or real
Only one operation every time a db changes. This avoid having several publication merging the same data
Cave eats:
This requires one more Collection = more space
The 2 db might not be always synchronised, there is few reasons for this:
The client manually changed the data of "AnotherCollection"
You had documents in "ACollection" before you added "AnotherCollection".
The transform function or source collection schema changed at some point
To fix this:
AnotherCollection.allow({
insert: function () {
return Meteor.isServer;
},
update: function () {
return Meteor.isServer;
},
remove: function () {
return Meteor.isServer;
}
});
And to synchronise at meteor startup (i.e. build the collection from scratch). Do this only once for maintenance or after adding this new collection.
Meteor.startup(function()
{
AnotherCollection.remove({});
var documents = ACollection.find({}).fetch();
_.each(documents, function(doc)
{
var transformedDocument = transformDocument(doc);
AnotherCollection.insert(transformedDocument);
});
});
I'm working on a simple app where a User can follow other users. Users can star posts. And a user's feed is composed of posts that have been starred by users they follow. Pretty simple actually. However, this all gets complicated in Mongo and Meteor...
There are basically two way of modeling this that I can think of:
A user has a property, following, which is an array of userIds that the user follows. Also, a post has a property, starrers, which is an array of userIds that have starred this post. The good thing about this method is that publications are relatively simple:
Meteor.publish 'feed', (limit) ->
Posts.find({starrers: {$in: Meteor.users.findOne(#userId).following}}, {sort: {date: -1}, limit:limit})
We aren't reactively listening to who the user is following, but thats not too bad for now. The main problem with this approach is that (1) the individual documents will become large and inefficient if 1000000 people star a post. Another problem is that (2) it would be pain to keep track of information like when a user started following another user or when a user starred a post.
The other way of doing this is having two more collections, Stars and Follows. If a user stars a post, then we create a document with properties userId and postId. If a user follows another user, then we create a document with properties userId and followId. This gives us the advantage of smaller document sizes for Users and Posts, but complicated things when it comes to querying, especially because Mongo doesn't handle joins!
Now, I did some research and people seem to agree that the second choice is the right way to go. Now the problem I'm having is efficiently querying and publishing. Based on the Discover Meteor chapter about Advanced Publications, I created a publication that publishes the posts that are starred by user's followers -- sorted, and limited.
# a helper to handle stopping observeChanges
observer = (sub, func) ->
handle = null
sub.onStop ->
handle?.stop?()
() ->
handle?.stop?()
handle = func()
Meteor.publish 'feed', (limit) ->
sub = this
userId = #userId
followIds = null
eventIds = null
publishFollows = observer sub, () ->
followIds = {}
Follows.find({userId:userId}).observeChanges
added: (id, doc) ->
followIds[id] = doc.followId
sub.added('follows', id, doc)
publishStars()
removed: (id) ->
delete followIds[id]
sub.removed('follows', id)
publishStars()
publishStars = observer sub, () ->
eventIds = {}
Stars.find({userId: {$in: _.keys(followIds)}).observeChanges
added: (id, doc) ->
eventIds[id] = null
sub.added('stars', id, doc)
publishEvents()
removed: (id) ->
delete eventIds[id]
sub.removed('stars', id)
publishEvents()
publishEvents = observer sub, () ->
Events.find({_id: {$in: _.keys(eventIds)}}, {sort: {name:1, date:-1}, limit:limit}).observeChanges
added: (id, doc) ->
sub.added('events', id, doc)
changed: (id, fields) ->
sub.changed('events', id, fields)
removed: (id) ->
sub.removed('events', id)
While this works, it seems very limited at scale. Particularly, we have to compile a list of every starred post by every follower. The size of this list will grow very quickly. Then we do a huge $in query against all posts.
Another annoyance is querying for the feed on the client after we subscribe:
Meteor.subscribe("feed", 20)
posts = null
Tracker.autorun ->
followers = _.pluck(Follows.find({userId: Meteor.userId()}).fetch(), "followId")
starredPostIds = _.pluck(Stars.find({userId: {$in: followers}}).fetch(), "postId")
posts = Posts.find({_id: {$in: starredPostIds}}, {sort: {date: -1}, limit: 20}).fetch()
Its like we're doing all this work twice. First we do all the work on the server to publish the feed. Then we need to go through the exact same logic again on the client to get those posts...
My question here is a matter of design over everything. How can I efficiently design this feed based on followers staring posts? What collection / collection schemas should I use? How should I create the appropriate publication? How can I query for the feed on the client?
So it turns out that Mongo and "non-relational" databases simply aren't designed for relational data. Thus, there is no solution here with Mongo. I've ended up using Neo4j, but SQL would work fine as well.
meteor add reywood:publish-composite
Meteor.publishComposite('tweets', function(username) {
return {
find: function() {
// Find the current user's following users
return Relationships.find({ follower: username });
},
children: [{
find: function(relationship) {
// Find tweets from followed users
return Tweets.find({user: relationship.following});
}
}]
}
});
Meteor.publish('ownTweets', function(username) {
return Tweets.find({user: username});
});
I'm making a simple app that informs a client that other clients clicked a button. I'm storing the clicks in a Firebase (db) using:
db.push({msg:data});
All clients get notified of other user's clicks with an on, such as
db.on('child_added',function(snapshot) {
var msg = snapshot.val().msg;
});
However, when the page first loads I want to discard any existing data on the stack. My strategy is to call db.once() before I define the db.on('child_added',...) in order to get the initial number of children, and then use that to discard that number of calls to db.on('child_added',...).
Unfortunately, though, all of the calls to db.on('child_added',...) are happening before I'm able to get the initial count, so it fails.
How can I effectively and simply discard the initial data?
For larger data sets, Firebase now offers (as of 2.0) some query methods that can make this simpler.
If we add a timestamp field on each record, we can construct a query that only looks at new values. Consider this contrived data:
{
"messages": {
"$messageid": {
"sender": "kato",
"message": "hello world"
"created": 123456 // Firebase.ServerValue.TIMESTAMP
}
}
}
We could find messages only after "now" using something like this:
var ref = new Firebase('https://<your instance>.firebaseio.com/messages');
var queryRef = ref.orderBy('created').startAt(Firebase.ServerValue.TIMESTAMP);
queryRef.on('child_added', function(snap) {
console.log(snap.val());
});
If I understand your question correctly, it sounds like you only want data that has been added since the user visited the page. In Firebase, the behavior you describe is by design, as the data is always changing and there isn't a notion of "old" data vs "new" data.
However, if you only want to display data added after the page has loaded, try ignoring all events prior until the complete set of children has loaded at least once. For example:
var ignoreItems = true;
var ref = new Firebase('https://<your-Firebase>.firebaseio.com');
ref.on('child_added', function(snapshot) {
if (!ignoreItems) {
var msg = snapshot.val().msg;
// do something here
}
});
ref.once('value', function(snapshot) {
ignoreItems = false;
});
The alternative to this approach would be to write your new items with a priority as well, where the priority is Firebase.ServerValue.TIMESTAMP (the current server time), and then use a .startAt(...) query using the current timestamp. However, this is more complex than the approach described above.