I have a situation in which I need to subscribe to the same collection twice. The two publish methods in my server-side code are as follows:
Meteor.publish("selected_full_mycollection", function (important_id_list) {
check(important_id_list, Match.Any); // should do better check
// this will return the full doc, including a very long array it contains
return MyCollection.find({
important_id: {$in: important_id_list}
});
});
Meteor.publish("all_brief_mycollection", function() {
// this will return all documents, but only the id and first item in the array
return MyCollection.find({}, {fields: {
important_id: 1,
very_long_array: {$slice: 1}
}});
});
My problem is that I am not seeing the full documents on the client end after I subscribe to them. I think this is because they are being over-written by the method that publishes only the brief versions.
I don't want to clog up my client memory with long arrays when I don't need them, but I do want them available when I do need them.
The brief version is subscribed to on startup. The full version is subscribed to when the user visits a template that drills down for more insight.
How can I properly manage this situation?
TL/DR - skip to the third paragraph.
I'd speculate that this is because the publish function thinks that the very_long_array field has already been sent to the client, so it doesn't send it again. You'd have to fiddle around a bit to confirm this, but sending different data on the same field is bound to cause some problems.
In terms of subscribing on two collections, you're not supposed to be able to do this as the unique mongo collection name needs to be provided to the client and server-side collections object. In practice, you might be able to do something really hacky by making one client subscription a fake remote subscription via DDP and having it populate a totally separate Javascript object. However, this cannot be the best option.
This situation would be resolved by publishing your summary on something other than the same field. Unfortunately, you can't use transforms when returning cursors from a publish function (which would be the easiest way), but you have two options:
Use the low-level publications API as detailed in this answer.
Use collection hooks to populate another field (like very_long_array_summary) with the first item in the array whenever very_long_array changes and publish just the summary field in the former publication.
A third option might be publishing the long version to a different collection that exists for this purpose on the client only. You might want to check the "Advanced Pub/Sub" Chapter of Discover Meteor (last sub chapter).
Related
I'm doing my meteor app and it has 1 Collection: Students
In Server I made a Publish that receives 3 params: query, limit and skip; to avoid client to subscribe all data and just show the top 10.
I have also 3 Paths:
student/list -> Bring top 10, based on search input and pagination (using find);
student/:id -> Show the student (using findOne)
student/:id/edit -> Edit the student (using findOne)
Each Template subscribe to the Students collection, but every time the user change between this paths, my Template re-render and re-subscribe.
Should I make just one subscribe, and make the find based on this "global" subscription?
I see a lot of people talking about Template level subscription, but I don't know if it is the better choice.
And about making query on server to publish and not send all data, I saw people talking too, to avoid data traffic...
In this case, when I have just 1 Collection, is better making an "global" subscription?
You're following a normal pattern although it's a bit hard to tell without the code. If there many students then you don't really want to publish them all, only what is really necessary for the current route. What you should do is figure out why your pub-sub is slow. Is it the find() on the server? Do you have very large student objects? (In which case you will probably want to limit what fields are returned). Is the search you're running hitting mongo indexes?
Your publication for a list view can have different fields than for a individual document view, for example:
Meteor.publish('studentList',function(){
let fields = { field1: 1, field2: 1 }; // only include two fields
return Students.find({},fields);
});
Meteor.publish('oneStudent',function(_id){
return Students.find(_id); // here all fields will be included
});
I have a template in which a user should be able to click on a button to bring up a modal and in the modal choose a handful of items out of a list of about 10,000 items which are displayed there to search or scroll through.
Since this collection is so big, I don't want to keep it around in memory when I don't absolutely need it.
So I would like to subscribe to this collection only when the modal is being viewed and I would like to ensure that I am unsubscribed if the modal is not being viewed.
Is there a way to explicitly unsubscribe from a collection?
There are a couple of ways you can do this:
Use the subscription handle
subscribe returns a handle you can call stop on. For example:
var handle = Meteor.subscribe('stuff');
handle.stop();
Use an autorun
Because an autorun will automatically start and stop subscriptions when its reactive dependencies change, this will work:
Tracker.autorun(function () {
if (Session.get('showingModal'))
Meteor.subscribe('stuff');
});
Side note - it may make more sense to use a method call for searching such a large data set rather than publishing the entire thing to the client. For example you can set a session variable whenever the user's query changes, then use an autorun to update the result set based on the method's return value.
https://docs.meteor.com/#/full/meteor_subscribe
Quoting the docs :
Meteor.subscribe returns a subscription handle, which is an object
with the following methods:
stop() Cancel the subscription. This will typically result in the
server directing the client to remove the subscription's data from the
client's cache.
So basically what you need to do is storing the subscription handle in a variable and call the stop method when you don't need those published documents anymore.
Note that if you're using iron:router (and you probably should), this is taken care of automatically for you on each route change, which is convenient but has the side effect of provoking a lot of sometimes unnecessary calls to Meteor.publish calls which are non trivial for the server and bandwidth... to address this matter you can use meteorhacks:subs-manager but it's another topic anyway.
In Meteor, I got a collection that the client subscribes to. In some cases, instead of publishing the documents that exists in the collection on the server, I want to send down some bogus data. Now that's fine using the this.added function in the publish.
My problem is that I want to treat the bogus doc as if it were a real document, specifically this gets troublesome when I want to update it. For the real docs I run a RealDocs.update but when doing that on the bogus doc it fails since there is no representation of it on the server (and I'd like to keep it that way).
A collection API that allowed me to pass something like local = true this would be fantastic but I have no idea how difficult that would be to implement and I'm not to fond of modifying the core code.
Right now I'm stuck at either creating a BogusDocs = new Meteor.Collection(null) but that makes populating the Collection more difficult since I have to either hard code fixtures in the client code or use a method to get the data from the server and I have to make sure I call BogusDocs.update instead of RealDocs.update as soon as I'm dealing with bogus data.
Maybe I could actually insert the data on the server and make sure it's removed later, but the data really has nothing to do with the server side collection so I'd rather avoid that.
Any thoughts on how to approach this problem?
After some further investigation (the evented mind site) it turns out that one can modify the local collection without making calls to the server. This is done by running the same methods as you usually would, but on MyCollection._collection instead of just on Collection. MyCollection.update() would thus become MyCollection._collection.update(). So, using a simple wrapper one can pass in the usual arguments to a update call to update the collection as usual (which will try to call the server which in turn will trigger your allow/deny rules) or we can add 'local' as the last argument to only perform the update in the client collection. Something like this should do it.
DocsUpdateWrapper = function() {
var lastIndex = arguments.length -1;
if (arguments[lastIndex] === 'local') {
Docs._collection.update(arguments.slice(0, lastIndex);
} else {
Docs.update(arguments)
}
}
(This could of course be extended to a DocsWrapper that allows for insertion and removals too.)(Didnt try this function yet but it should serve well as an example.)
The biggest benefit of this is imo that we can use the exact same calls to retrieve documents from the local collection, regardless of if they are local or living on the server too. By adding a simple boolean to the doc we can keep track of which documents are only local and which are not (An improved DocsWrapper could check for that bool so we could even omit passing the 'local' argument.) so we know how to update them.
There are some people working on local storage in the browser
https://github.com/awwx/meteor-browser-store
You might be able to adapt some of their ideas to provide "fake" documents.
I would use the transform feature on the collection to make an object that knows what to do with itself (on client). Give it the corruct update method (real/bogus), then call .update rather than a general one.
You can put the code from this.added into the transform process.
You can also set up a local minimongo collection. Insert on callback
#FoundAgents = new Meteor.Collection(null, Agent.transformData )
FoundAgents.remove({})
Meteor.call 'Get_agentsCloseToOffer', me, ping, (err, data) ->
if err
console.log JSON.stringify err,null,2
else
_.each data, (item) ->
FoundAgents.insert item
Maybe this interesting for you as well, I created two examples with native Meteor Local Collections at meteorpad. The first pad shows an example with plain reactive recordset: Sample_Publish_to_Local-Collection. The second will use the collection .observe method to listen to data: Collection.observe().
On my meteor project users can post events and they have to choose (via an autocomplete) in which city it will take place. I have a full list of french cities and it will never be updated.
I want to use a collection and publish-subscribes based on the input of the autocomplete because I don't want the client to download the full database (5MB). Is there a way, for performance, to tell meteor that this collection is "static"? Or does it make no difference?
Could anyone suggest a different approach?
When you "want to tell the server that a collection is static", I am aware of two potential optimizations:
Don't observe the database using a live query because the data will never change
Don't store the results of this query in the merge box because it doesn't need to be tracked and compared with other data (saving memory and CPU)
(1) is something you can do rather easily by constructing your own publish cursor. However, if any client is observing the same query, I believe Meteor will (at least in the future) optimize for that so it's still just one live query for any number of clients. As for (2), I am not aware of any straightforward way to do this because it could potentially mess up the data merging over multiple publications and subscriptions.
To avoid using a live query, you can manually add data to the publish function instead of returning a cursor, which causes the .observe() function to be called to hook up data to the subscription. Here's a simple example:
Meteor.publish(function() {
var sub = this;
var args = {}; // what you're find()ing
Foo.find(args).forEach(function(document) {
sub.added("client_collection_name", document._id, document);
});
sub.ready();
});
This will cause the data to be added to client_collection_name on the client side, which could have the same name as the collection referenced by Foo, or something different. Be aware that you can do many other things with publications (also, see the link above.)
UPDATE: To resolve issues from (2), which can be potentially very problematic depending on the size of the collection, it's necessary to bypass Meteor altogether. See https://stackoverflow.com/a/21835534/586086 for one way to do it. Another way is to just return the collection fetch()ed as a method call, although this doesn't have the benefits of compression.
From Meteor doc :
"Any change to the collection that changes the documents in a cursor will trigger a recomputation. To disable this behavior, pass {reactive: false} as an option to find."
I think this simple option is the best answer
You don't need to publish your whole collection.
1.Show autocomplete options only after user has inputted first 3 letters - this will narrow your search significantly.
2.Provide no more than 5-10 cities as options - this will keep your recordset really small - thus no need to push 5mb of data to each user.
Your publication should look like this:
Meteor.publish('pub-name', function(userInput){
var firstLetters = new RegExp('^' + userInput);
return Cities.find({name:firstLetters},{limit:10,sort:{name:1}});
});
If I change a Session var and trigger a re-subscription via autosubscribe, is there any callback mechanism to wait until the 'latest' data is down from the server? [1]
If you take a look at this gist you'll see some code that logs the content of a collection over time as the subscription changes. A relevant section of the output:
at Subscribed; comments are: first post on #1 - second post on #1
at Flushed; comments are: first post on #1 - second post on #1
at Subscription complete; comments are: first post on #1 - second post on #1 - first post on #2 - second post on #2
So, even after (a) calling .subscribe, (b) calling Meteor.flush (c) being inside the onReady callback for the .subscribe; there is still stale data in the collection, and only in the 3rd case is the 'correct' data in there.
I realise that reactive templates and .observe will eventually receive the correct data and things will 'settle' into the correct state. But is there some way we can tell that we aren't there yet?
For instance, most of the meteor example apps (and my own apps) are prone to jerking around a bit (similar to a FOUC) while data is added + removed from a subscribed collection. If we could tell that a subscription was 'loading' we could do something about this.
[1] Obviously the data on the server is constantly changing, but as you'll see in the gist, I can't (without a timeout) find a point where it's even correct. Thus my use of 'valid' in the question.
A very simple and common use case
Take the madewith app; when you first load it up it appears that there are no apps registered, until after the data comes down the wire and the apps suddenly appear.
The reason for this is that Meteor.subscribe has been called, but the data has not yet come down the wire. But there's no easy way for the template to tell that the data is pending and that it should show a 'loading' template. In madewith they actually do something when the data has loaded, but this is a callback and thus breaks out of the normal meteor way of doing things (i.e. reactive coding).
It would be much nicer (IMO) to be able to write something like:
{{unless apps_loaded}}{{> loading}}{{/unless}}
and
Template.madewith.apps_loaded = function() { return !Apps.isComplete(); }
Interesting question.
The right place to be notified for new subscriptions is the onReady callback. Notice the logging that happens there always includes your new data. Checking at (a) and (b) aren't useful, because there's latency between calling subscribe and when all the data arrives from the server.
The underlying problem is there's no equivalent onRemove callback that runs once the data for a just-stopped subscription has been removed. Moreover, autosubscribe deliberately starts new subs before stopping old ones, to avoid flicker.
What's the real use case? Most of the time such a callback isn't necessary, because templates can also restrict their queries to the data that should be in scope. In your example, the template helper that renders comments might query only for comments with the current post_id in the Session, so there's no harm having extra comments in the database.
Something like this:
Template.post.comments = function () {
return Comments.find({post_id: Session.get('post_id')});
};
That permits strategies more sophisticated that the generic autosubscribe function, like subscribing to comments for the last three posts the user has viewed. Or, subscribing to the first few comments for each post, and then separately subscribing to a post's full set of comments only when that post is selected.
I've come a bit late to the meteor party, so I don't know the history behind when meteor features have been added, but just for completeness it is now possible to do this using template-level subscriptions and 'subscriptionsReady' :
[js]
Template.myTemplate.onCreated(function() {
this.subscribe('myData');
});
[html]
<template name="myTemplate">
<h2>my Template</h2>
{{#if Template.subscriptionsReady}}
// code to loop/display myData
{{else}}
<p>Please wait..</p>
{{/if}}
</template>