I have a meteor project featuring products. They are all in a Products collection.
At the moment, I have only 50 of them. So I subscribe to the entire collection and then filter it client side.
Bu what if my project gets a lots of products. Let's say 10.000. This strategy won't work at all. The application will be slow and will take like forever to load the first datas. What would be the most efficient ?
1 => the classic way : no subscription, I just make meteor method server side and I make calls client side to get the data I want. (this one, I know how to build)
2 => the more meteory way (I feel more into that one) : make a subscription, but filtered and paginated. Meteor.subscribe('products', );
Is it viable on a large scale ? And how do I achieve that ? I'm not sure how to start with the pagination stuff. Do I update my subscription or start a new one ?
I use iron-router. And the first problem I have to tackle is caching the result. In a perfect world, if a user makes a first search, then an other and then goes back to the first one, I'd like the app to keep the first results. When I put my subscription in the waitOn hook, all the datas are lost when I go on another route.
Related
Suppose you have template1 and template2, both subscribing using this.subscribe('samePublication', sameArg) inside their .onCreated() and this.autorun().
What happens when we have something like this:
<template name="template3">
{{>template1}}
{{>template2}}
</template>
Will this.subscribe('samePublication', sameArg) run once for each template, and hit my server and DB twice?
Should I put this.subscribe() inside the .onCreated() on template3?
My own understanding is 'no' and 'no', after having read this: http://docs.meteor.com/#/full/meteor_subscribe
Hoping someone more knowledgeable can comment.
Thanks in advance.
In my experience, the subscriptions load and hit your DB every time you route to the view and render the template.
While the general best practice (probably to promote modularity) is to use template level subscriptions, I have found that if a route is heavily used and frequently changed to and it involves large amounts of docs, then it makes more performance sense to load those subscriptions on client startup and share between views.
Did you know you can pass an argument with a subscription to a publication and use the argument in your publication to filter the results? Using this often makes a big difference on the performance without falling back to global subscriptions.
PS: I am working on parallel subscriptions that do not start until after the loading of 'fast' landing pages, so that when the user switches to a heavily subscribed page, the parallel loading would already have completed, but would not affect the 'first impressions' of the user.
I am trying to make a chat application in meteorJS and i was thinking of making a seperate collection for each groups dynamically when they initiate chat, I want to publish and subscribe that collection for transmitting chat information and all the group user can subscribe it, but i am not being able to create a collection dynamically.
I tried making a function which gets call when the user subscribe the collection.
this.createDb =(name) ->
#ChatDb = new Mongo.Collection(name)
return true
everything is fine, but when i subscribe this collection from client side, ChatDb is unknown, Can any one help me with this, that would be great. :)
ps- i am writing code in angular-meteor framework
Andrew Mao's answer to a similiar question:
In most instances, you probably don't want to create multiple
collections, but instead use one collection and send views of it to
clients depending on their subscription.
You may want to check out the
https://github.com/mizzao/meteor-partitioner package I've built which
is designed especially for this purpose, and includes an example for
how to do this for multiple chat rooms. You can also see
https://github.com/mizzao/CrowdMapper for an implemented example.
I haven't done Meteor for a while now so I couldn't give you a solid answer. But I remember quite clearly that creating collections dynamically is not the recommended way for achieving what you want to do.
Apologies for this long winded question but I just want to walk through my research into trying to solve this problem myself.
My first stackoverflow question... here goes...
My meteor app was plugging along nicely until I wanted to have a standings table. This 30 years of basketball standings article was inspirational. I wanted to tap into that power and flexibility that Mongo provided and keep track of team and player standings I bought this book on Mongo (highly recommended). I have multiple collections and one collection called 'games' that keeps track of games and scores per game. Here is a same document:
{
"leagueId" : "6RtH74QbxGG7xbZXh",
"regionId" : "KDbqfAoKDx2iDDXSS",
"seasonId" : "b5HkGcWFNenpGGvCd",
"gameTime" : "15:00",
"gameDate" : "11/23/2014",
"gameNumber" : 4,
"gameStatus" : "played",
"homeTeam" : "MYBw2RiNwrBhfh9W8",
"homeTeamScore" : 4,
"awayTeam" : "fwx79JJFob5XbaAx6",
"awayTeamScore" : 2,
"gameType" : "regular_season",
"userId" : "4MKaZK84AdZ8j3xr2",
"author" : "league",
"submitted" : "Wed Dec 10 2014 09:51:48 GMT-0800 (PST)",
"_id" : "Gwsu6X6DXXzavdqZQ"
}
I determined the aggregate framework was the way to go but after digging into this I saw that Meteor had some client side limitations with the mongo aggregate framework (stackoverflow.com/questions/11266977/are-group-by-aggregation-queries-possible-in-meteor-yet).
I also thought about [denormalizing my document structure after reading articles like this][4]. And after reading the Denormalization chapter of Discover Meteor (good book!), it seems not denormalizing gives me a lot more flexibility but I can denormalize in Meteor/Mongo with other benefits. The NBA article mentioned earlier is very denormalized and not sure if I can replicate that structure using Meteor/Mongo.
I then jumped in the Aggregate rabbit hole and found some good reading (stackoverflow.com/questions/18520567/average-aggregation-queries-in-meteor/18884223#18884223 - I tried this using my data and I saw the output on the server but when I added the subscription to 'waiton' in my iron router there were no errors and it just hung. So it worked on the server, not the client?).
I then found this article which looked very promising. That led me to the atmosphere package ([meteorhacks:aggregate][8]... and it's dependencies) but the article seemed more directed at experts with mongo as I could not get my data client side.
From all my reading I surmise that I should create a new collection in my 'collections' folder (available on client and server) and then I publish that collection using my aggregate code ($group and $match and cool things like $sum). Then I'm really not sure what to do next. Do I create a server method and call that method from my client side code (tried that and couldn't get it to work)? Am I supposed to subscribe to the collection in iron-router (every time I did, my app just hung with no errors)?
If anyone gets how to use aggregate with meteor I would really appreciate some/any guidance. I think I just need a few more pieces of information before I can generate that standings table with wins, losses and draws.
note: I took a detour and thought MapReduce might be the solution(thebhwgroup.com/blog/2014/09/bi-nodejs-couchdb-mapreduce) but I couldn't get this to work in Meteor either.
You raise a number of topics and there are probably a lot of solutions to get where you want. One key point is that you cannot currently run mongo aggregations on the client side in Meteor. So any approach where you are aggregating game data into standings needs to first aggregate the game data server-side, and only then publish a cursor of the results to the client.
To break your problem down into more manageable simples, I would first focus on assembling an aggregation on the server side that looks the way you want. You will need a package such as meteorhacks:aggregate as you mentioned, or others that accomplish the same purpose.
Then, for starters, you should simply run the aggregation pipeline server-side on Meteor.startup a few times until it looks the way you want. To get a quick look at it, you could save it to a variable and log it in the console (again, server-side). Alternatively, you could save it into a new Collection such as Standings and inspect it using meteor mongo from the console. Part of the issue is that you may be getting ahead of things trying to publish and subscribe to your collections with Iron Router while you are still thinking about how to set up your collections.
This gets to the next point. Once the aggregation document looks the way you want on the server, you'll need to decide whether you want to save the results each time you aggregate a new set of standings, or assemble them on the fly each time you publish it. In the Owens article you linked to, he is aggregating data directly inside of a publish function. This means it is re-run each time it is published or subscribed to, which could be very fast or very slow depending on the size and complexity of the data. It is also important to consider that under this model you are not actually publishing a collection of standings to the client - you are publishing a mutated collection of games. If you expect your standings to acquire their own properties, change over time, or you want to poll them historically, you are best off creating a new Collection such as Standings and inserting a new standing into the collection each time you receive new game results.
Your publish function could then be fairly simple, e.g., Standings.find({}, {limit: 1}) with a query for the most recent standing. You can subscribe in Iron Router's waitOn as you suggested.
To insert new entries into the Standings collection, you could call a server side method like Meteor.Call("createStandings") each time a new game is introduced. This could result in a large document over time, which you could cull or even write over if you don't care about maintaining historical records.
Again, there are a lot of ways you could take this. Hope this helps.
I've been struggling with the following issue in Meteor + iron router:
I have a page (route) that has a subscription to a mongo collection
on that page, I have some logic which relies on a cursor querying the collection, also utilizing an observeChanges handler (namely, I'm running a search on that collection)
the problem in this case is the collection is being preserved in the client throughout route changes, which causes 2 unwanted effects:
1) the collection isn't necessarily needed outside that route, meaning i'm wasting client RAM (the collection, or even a subset of it, is likely to be quite big)
2) whenever i go back to that route, I want to start off with an empty subset for the observeChanges handler to work properly.
Any advice on how to clear the mirrored collection? (using the Collection._collection.remove({}) hack is bad practice, and doesn't even solve the problem)
Thanks!
solved this by storing the subscription handles. used them to unsubscribe the data (i.e. subscription_handle.stop() ) on template.destroyed()
From what I know it seems that Meteor Framework stores part of data on the client. It's clear how to do it for personal todo list - because it's small and you can just copy everything.
But how it works in case of let's say Q&A site similar to this? The collection of questions are huge, you can't possibly copy it to the client. And you need to have filtering by tags and sorting by date and popularity.
How Meteor Framework handles such case? How it partition data? Does it make sense to use Meteor for such use case?
Have a look at the meteor docs, in particular the publish and subscribe section. Here's a short example:
Imagine your database contains one million posts. But your client only needs something like:
the top 10 posts by popularity
the posts your friends made in the last hour
the posts for the group you are in
In other words, some subset of the larger collection. In order to get to that subset, the client starts a subscription. For example: Meteor.subscribe('popularPosts'). Then on the server, there will be a corresponding publish function like: Meteor.publish('popularPosts', function(){...}.
As the client moves around the app (changes routes), different subscriptions may be started and stopped.
The subset of documents are sent to the client and cached in memory in a mongodb-like store called minimongo. The client can then retrieve the documents as needed in order to render the page.