My situation is as follows:
I have a collection in Mongo which gets updated with new items every few milliseconds, for example log items. I'm displaying these items on the frontend via publish/subscribe in a template, but because of the high volume the list updates so frequently that it's hard to read them. What I would like is to only have the list be updated every (few) seconds. I have tried to use sleep/timeouts on both the client and server side, as indicated here for example, without success so far.
Can I still use publish/subscribe for this or should I switch a polling mechanism with Meteor.setInterval?
Should the time interval part be on the publish or on the subscribe side?
If publish/subscribe is correct for my scenario, how do I only show the updated data every few seconds?
DDP has a rate limiter. it's meant for defeating DDoS attacks, but i suppose it could be repurposed for what you want.
https://blog.meteor.com/rate-limiting-in-meteor-core-762b8ad2412f#.nw6fwlhji
You should be able to use reactive variables and autorun in your Template.name.onCreated to do this :
Template.name.onCreated(function(){
var instance = this;
instance.now = new ReactiveVar( new Date());
instance.autorun(function(){
var test = now.get();
instance.subscribe('yourSubNameHere');
setTimeout(function(){ //will update now and fire the autorun again
instance.now.set(new Date());
},timeoutHere)
});
)};
Although if your collection gets big I'd advise doing this with a limit in your publication maybe?
Related
I'm attempting to use observechanges with iron router but they don't seem to be compatible at all.
Router.route('/gaming', {
waitOn: function() {
return Meteor.subscribe('chat', function() {
window.chatmessagesCache = new ReactiveVar;
chatmessagesCache.set([]);
return chat.find().observeChanges({
added: function(id, doc) {
var tmpArr;
tmpArr = chatmessagesCache.get();
tmpArr.push(doc);
return chatmessagesCache.set(tmpArr);
}
});
});
}
If I leave the route and come back to it, observechanges begins being handled as many times as I've navigated away and returned, for each new record. What's the deal?
If I use subs manager it works as expected, but I don't understand why Meteor.subscribe inside waitOn is so cache/subscription unaware when it ALREADY gets called multiple times per load. Why!? I can't decipher what's causing this behavior at all.
Also, what I'm trying to accomplish is simple. I want to let chat messages that the user's client has received remain on the page even if the chat cursor is no longer publishing them (I'm publishing the last 10 chat messages)
Iron router has reactivity built in, which means when something inside your route function is invalidated, it will repeat the function as well as anything reactive with a Router.current(). These unexpected invalidation runs are a primary reason why folks made the exodus to flow router.
To solve this, you'll want to abstract your code away from the router. You can leave the sub, but I'd suggest you remove the sub's callback from the waitOn and move it into an onRendered callback. If you don't want the history loaded in chunks, you can first do a var collectionCount = chat.find({},{reactive:false}).count() on how many docs are in the collection & then in the added callback you can do something like if (++currentCount === collectionCount) /* add stuff */ to add al the records to the history when it reaches the last record.
On a bigger picture level, consider eliminating the observeChanges & just do an #each over the chat collection in spacebars to show your messages. Fewer cycles, cleaner code.
Iron router just has no management of observations you created yet it manages subscriptions itself, hence the multiple additions.
I figured this out by using a window level variable to check if I'm observing. Even in cases when the subscription is unhooked by iron, if I go back and never re-add the handler, the original observation hook still runs (!). ALSO, if you navigate away and drop the subscription, the handler is no longer called--which is the behavior I want in this scenario (This is all very insane behavior but at least it's now predictable to me )
This is caused by the fact that subscriptions != collections and the API for observations doesn't seem to expose any metadata, unfortunately, so I don't know how the iron router maintainers would account for this. Not to mention you return iron router a subscription, not a collection.
#Matt K if you were correct, this would always be an infinite loop (which admittedly I had a bunch of while trying to solve this) but the posted code is adding too many handlers, not looping indefinitely. Thanks for the post though.
This is what I settled on
Router.route('/gaming',
waitOn: ->
Meteor.subscribe('chat', ->
window.chatmessagesCache = new ReactiveVar(chat.find().fetch().reverse())
if !window.chatListening
window.chatListening = true
after = chat.find().count()
chat.find().observe(
added: _.after(after + 1,(doc) ->
tmpArr = chatmessagesCache.get()
tmpArr.push(doc)
chatmessagesCache.set(tmpArr))
changed : (id, doc) ->
))
I really just wanted to test out a pattern of locally "disconnected" documents. I still may use subs manager because it keeps subscriptions and their handlers alive without rerunning them constantly (which was rerunning the sub handler, which was adding multiple observations)
I'm using subs-manager, but the answer to this may be independent of that lib.
I have a subscription with a single limit argument. Currently, when I call subs.subscribe 'subname', newLimit, another subscription is added.
The old subscriptions are still there. I don't want Meteor to spend time maintaining the old, lower-limit subscriptions. Instead of adding a new subscription, I want to update the argument of the old subscription. What is the best way to do this?
Note that I also don't want to completely tear down eg 'subname', 20 before subscribing to 'subname', 40, because I don't want Meteor to do the extra work of resending the first 20 docs – I want it to just send docs 21 - 40.
You could have a look at your subscription's stop() method. According to the docs:
stop() [cancels] the subscription. This will typically result in the server directing the client to remove the subscription's data from the client's cache.
So the way I see it, you could maybe manage to do:
// globals or whatever
var previousSub, newSub;
// somewhere else
newSub = Meteor.subscribe('subname', newLimit, function () {
if (previousSub)
previousSub.stop();
previousSub = newSub;
});
I have a template in which a user should be able to click on a button to bring up a modal and in the modal choose a handful of items out of a list of about 10,000 items which are displayed there to search or scroll through.
Since this collection is so big, I don't want to keep it around in memory when I don't absolutely need it.
So I would like to subscribe to this collection only when the modal is being viewed and I would like to ensure that I am unsubscribed if the modal is not being viewed.
Is there a way to explicitly unsubscribe from a collection?
There are a couple of ways you can do this:
Use the subscription handle
subscribe returns a handle you can call stop on. For example:
var handle = Meteor.subscribe('stuff');
handle.stop();
Use an autorun
Because an autorun will automatically start and stop subscriptions when its reactive dependencies change, this will work:
Tracker.autorun(function () {
if (Session.get('showingModal'))
Meteor.subscribe('stuff');
});
Side note - it may make more sense to use a method call for searching such a large data set rather than publishing the entire thing to the client. For example you can set a session variable whenever the user's query changes, then use an autorun to update the result set based on the method's return value.
https://docs.meteor.com/#/full/meteor_subscribe
Quoting the docs :
Meteor.subscribe returns a subscription handle, which is an object
with the following methods:
stop() Cancel the subscription. This will typically result in the
server directing the client to remove the subscription's data from the
client's cache.
So basically what you need to do is storing the subscription handle in a variable and call the stop method when you don't need those published documents anymore.
Note that if you're using iron:router (and you probably should), this is taken care of automatically for you on each route change, which is convenient but has the side effect of provoking a lot of sometimes unnecessary calls to Meteor.publish calls which are non trivial for the server and bandwidth... to address this matter you can use meteorhacks:subs-manager but it's another topic anyway.
In an application that allows realtime chat between clients, I aim to integrate functionality that allows to define messages that are delivered at future points in time.
In the following example, I am able to insert messages, which are inserted directly into the template. However, I would like to display only messages that have a time smaller or equal to the current time, but automatically display messages that have future time points as soon as the time is reached. For example, if I insert a message from the console, which should be displayed 30 seconds in the future by calling Meteor.call("createMessage", 30000, "hello in 30 seconds"), the message should be automatically displayed after 30 seconds.
I started restricting the query in the publish function to time: {'$lt': new Date()}. However, I have trouble in making this reactive. I unsuccessfully tried several combinations of Tracker.autorun and cursor.observe.
Can anybody give me a hint how I accomplish the desired reactivity within the following running example?
1) html file
<body>
{{> chat}}
</body>
<template name="chat">
{{#each chatMessages}}
{{time}} - {{message}} <br>
{{/each}}
</template>
2) js file
//server and client
Messages = new Mongo.Collection("messages"); //{time: Sun Nov 02 2014 22:17:32 GMT+0100 (CET), message: "hello"}
//server
if(Meteor.isServer){
Meteor.methods({
'createMessage': function(timeOffset, message){
Messages.insert({
time: new Date(new Date().getTime() + timeOffset),
message: message
});
}
});
Meteor.publish("messages", function(){
return Messages.find({
//time: {'$lt': new Date()}
});
});
}
//client
if (Meteor.isClient) {
Template.chat.helpers({
chatMessages: function(){
return Messages.find({});
}
});
Tracker.autorun(function (){
mySub = Meteor.subscribe('messages');
});
}
If Date() was a reactive datasource, it would work but it's not.
You can create a Timer at server side that will handle it. The best design I see id: pick the next message future date and set a Timer with the time difference and also get the next message time. Of course it's dependes on how your application works.
Read more about Timers in Meteor: https://docs.meteor.com/#/full/timers
Reactivity means that the view reflects the data sources used to make that view, updates when those sources change, and only then (as a rule of thumb).
Therefore, if we want to accomplish what is described using reactivity, we must introduce a reactive change when the message goes live (the outlined model does not have such a change).
Two ways to accomplish this that I can think of:
Add an 'isLive' field to the message, and have the server change it at the right time, using timed callbacks and Meteor.startup (to avoid losing messages in case of a reboot). Somewhat complex, but clean and performant (when properly implemented).
Add a currentDate Session variable and use Meteor.setInterval etc. on the client, to keep it as current as you need (Because Session variables are reactive).
In the second alternative, the user could just change their system clock or use the javascript console to access future messages. Also, reactive events with a set interval seem rather contrived, unless that interval is significant to the problem domain itself (like a calendar app changing the 'today' session variable used to draw the red circle, every midnight).
A simpler (better?) non-reactive solution might be to simply render the future messages as hidden elements, and use javascript timers to show them at the right time. But it all depends on what you are dealing with of course.
I'm having trouble with realizing a reactive publication with a moving date. I have a calendar app with events/meetings. Im displaying these events on a special view page but i only want to display events of today and only with a starting time of currenttime - 30 mins.
The code im having and trying always works fine on initial load, after refreshing and when I add/delete an event from an admin page (collection gets refreshed?). But when I leave the page open the events which have passed just stay on the page. Whenever I add/remove a event via the admin page the publication is updated just fine. I assume this is because the publication isn't refreshing the date in the query or something?
I have tried:
normal publications and just subscribing via either iron-router
before hook or via Deps.autorun
publish all events and filtering on the client side
a publication with observeChanges
keep the vars in a deps.autorun function and passing them via the subscription as parameters
but I just keep getting the same results.
publication with observeChanges:
Meteor.publish('currentEventsObserve', function(calId) {
var self = this;
var nowMin30mins = moment().subtract('minutes',30).valueOf();
var endOfToday = moment(moment().format('DD-MM-YYYY 23:59'), 'DD-MM-YYYY HH:mm').valueOf();
var handle = Events.find({
calId : calId, //CalendarId
eventDate: { $gt: nowMin30mins, $lt: endOfToday }
},
{
sort: { eventDate: 1 },
limit: 5
}).observeChanges({
added: function(id,event){
//console.log("added: ",id,event);
self.added("events", id, event);
},
removed: function (id,event) {
//console.log("removed: ",id,event);
self.removed("events", id);
}
});
self.ready();
self.onStop(function () {
handle.stop();
});
});
As said before: the above works fine on initial load and refreshes, but after leaving the page open for a while the events are staying on there and not being removed from the publication.
Also, whenever I check the collection ones certain events already should have been removed via Events.find().fetch() for example, the events are still there, so it's not the template which isn't updating correctly.
I hope it's clear what I mean. I have read many other questions about this sort of reactivity but I just can't figure it out.
The problem is that nowMin30mins is only computed once when the publication is activated, and won't update afterwards, even though you have an observeChanges on Events. You basically have two constant values (nowMin30mins and endOfToday) bracketing the documents that will be shown and only adding or removing items will cause them to disappear.
I can think of two ways for how you'd want to do this reactively.
You can put all the reactivity on the client. Just send the entire day's events over in a publish, which is very simple, and have the client filter out what's between 30 mins from now and the end of the day using Date.now() in a computation that updates every minute or so, or when new events are added/deleted. It doesn't seem to incur a lot of overhead to store those events anyway and you can offload the computational cost of observes, which can build up if you have a lot of clients
If you want to do it on the server, you'll have to do things a little more carefully than what you have now. Because the live query won't account for the change in time, you'll need to watch all of today's events and then add or remove them from the subscription if they are created, or deleted or go out of range, respectively. To reiterate, you can't do this purely with an observeChanges because the current time is always changing.