Is it possible yet, to preserve insertion order or set reliable timestamps in Meteor given that MongoDB doesn't guarantee to return items in insertion order if no sort is specified, a document's _id is randomly generated and setting a timestamp manually on insertion would depend upon the client's clock?
I suggest a method.
Meteor.methods({
addItem: function (doc) {
doc.when = new Date;
return Items.insert(doc);
}
});
While the client will run this locally and set when to its own current time, the server's timestamp takes priority and propagates to all subscribed clients, including the original client. You can sort on doc.when.
We'll probably add hooks for setting timestamps automatically as part of document validations and permissions.
If you're willing to use something like these collection hooks (https://gist.github.com/matb33/5258260), along with this fancy Date.unow function (which you can safely sort on even if many documents were inserted with the same timestamp):
if (!Date.unow) {
(function () {
var uniq = 0;
Date.unow = function () {
uniq++;
return Date.now() + (uniq % 5000);
};
})();
}
if (Meteor.isServer) {
// NOTE: this isn't vanilla Meteor, and sometime in the future there may be
// a better way of doing this, but at the time of writing this is it:
Items.before("insert", function (userId, doc) {
doc.created = Date.unow();
});
}
Related
I am trying to make a fairly big meteor app, and I noticed that it has gotten slower over the past few days and I read on the meteor forum that publications can cause slow loading times. After I refresh the page when making a change in the application itself (code change), it usually takes between 1-2 minutes for a single change. Is there anything wrong with my publications? Although, when the page is loaded, and I reload it loads up really fast.
if(Meteor.isServer){
Meteor.publish('notes', function () {
return Notes.find()
});
Meteor.publish('users', function () {
return Meteor.users.find()
});
Meteor.publish("user", function(){
return Meteor.user()
})
Meteor.publish('notes-newest', function () {
return Notes.find({}, {sort: {createdAt: -1}, limit: 10});
});
}
document example:
let noteInfo = { title, subject, description, imageURL, userId, userEmail, createdAt }
let title = this.refs.title.value;
let subject = this.refs.subject.value;
let description = this.refs.description.value;
let allUrls = [this.refs.imageURL.value].concat(this.state.urls);
let imageURL = allUrls.filter(function(entry) { return entry.trim() != ''; });
let userId = Meteor.userId();
let userEmail = Meteor.user().emails[0].address;
let createdAt = Date.parse(new Date());
This is a very broad performance tuning question. You haven't told us how many documents are in your collections or how large these documents are. Some possible issues:
You are over-publishing either by publishing too many documents or because your documents are too big. For example when you do:
Meteor.publish('notes', function () {
return Notes.find()
});
If there are 100,000 Notes documents at 100 bytes each then that's 10 MB that needs to go over the network to the client. This is also the case if there are 1,000 notes documents at 10KB each.
Solution: Limit the number of documents with limit and/or reduce the number of fields transmitted with fields:
Meteor.publish('notes', function () {
return Notes.find({},{ limit: 100, fields: { key1: 1, key2: 1 }});
});
Your collection is missing one or more indexes. When you do:
Meteor.publish('notes-newest', function () {
return Notes.find({}, {sort: {createdAt: -1}, limit: 10});
});
if there are 1M notes documents but there is no index on the createdAt key then this will be terribly slow.
Solution: Adding an index on the createdAt key will make such a publication much faster.
You have an invalid publication.
Meteor.publish("user", function(){
return Meteor.user();
})
Is invalid because Meteor.user() is not a cursor and a publication must return either a cursor or an array of cursors. It is also redundant because Meteor.user() is automatically available on the client, albeit it doesn't include all the keys.
Solution: Remove this unnecessary publication altogether. If you want to publish some of the keys that are not available on the client for the current user you can do so as follows:
Meteor.publish("user", function(){
return Meteor.users.find(this.userId,{fields: {services: 1, emails: 1, profile: 1}})
});
After I refresh the page when making a change, it usually takes between 1-2 minutes for a single change. Is there anything wrong with my publications? Although, when the page is loaded, and I reload it loads up really fast.
What does this mean? Do you mean when you change the code of your website, it takes 1-2 minutes for your changes to appear in a newly loaded page? That's to be expected: it takes time for Meteor to rebuild your application. Watch the terminal for the progress.
Using lots of packages and external code will slow down building. The speed of compilation is unrelated to your end user's experience.
A couple things I do with my meteor projects is that I send specific fields "profile.name" etc. I've also used collection.createIndex({}) which creates an index and inside it and you could put your sort in which is created on server startup and works well with your collection.find.sort. I think limiting your notes would be quite essential as you have a limit on one and not the other.
It happened to me too. At that time I checked with kadira (formerly still opensource but now it is not). Source code at kadira on github, but service is gone kadira.io
The problem is in pubication and subscribtion. When client request subscribtions, he opens up some kind of "connection" - i say that, and it applies to others who call the same publication. imagine how many connections are formed.
where the same data accessed by many people must server side must request data to mongodb, that same to all of number of such connections. This is what makes it slow.
Finally I added redis.io.
meteor npm install --save redis
meteor npm install --save hiredis
On server side :
import { Meteor } from "meteor/meteor";
var redis = require("redis");
var clientRedis = redis.createClient({
host: "YOURREDIS_IP",
port: "YOURREDIS_PORT"
});
clientRedis.setSync = Meteor.wrapAsync(clientRedis.set);
clientRedis.getSync = Meteor.wrapAsync(clientRedis.get);
setRedis_Object = function (keyREDIS, timeRefresh, valueREDIS) {
return clientRedis.setSync(keyREDIS + moment(new Date()).format(timeRefresh), JSON.stringify(valueREDIS));
};
getRedis_Object = function (keyREDIS, timeRefresh) {
return JSON.parse(clientRedis.getSync(keyREDIS + moment(new Date()).format(timeRefresh)));
};
Meteor.methods({
getRedis_YOURCOLLECTIONS: function(timeRefresh) {
var data = getRedis_Object(getRedis_YOURCOLLECTIONS, timeRefresh);
if (data != null) {
return data;
} else {
var data = YOURCOLLECTIONS.find().fetch();
setRedis_Object(key, timeRefresh, data);
return data
}
},
});
On Client Side
Meteor.call("getRedis_YOURCOLLECTIONS", "YYYYMMDD", function (error, result) {
if (error) {
console.log(error);
} else {
Session.set("getRedis_YOURCOLLECTIONS", result);
}
});
After i am add this REDIS consumtion Memory and CPU on server low, and app more Fast. I hope its work for you. Thanks
Clearly, I am doing something wrong with ReactiveVar because I cannot get it to work as I expect it should.
I am trying to set the value of an ReactiveVar by calling a Meteor.call method which returns the list of usernames. But it does not update when the usernames get changed in another part of the app.
I tried both:
Template.qastatistics.created = function () {
this.trackUsernames = new ReactiveVar(false);
var instance = Template.instance();
Meteor.call('trackUsernames', function (err, data) {
instance.trackUsernames.set(data);
});
};
and:
Template.qastatistics.helpers({
users: function () {
var usernames,
instance = Template.instance();
if (instance.trackUsernames.get() === false) {
Meteor.call('trackUsernames', function (err, data) {
instance.trackUsernames.set(data);
});
}
usernames = instance.trackUsernames.get();
...
But neither updates the list of usernames when these change in the database.
Is this even possible with ReactiveVars or have I completely misunderstood them?
EDIT: The usernames I mention are not from Meteor.users collection, but rather a distinct call from another collection that has usernames in it.
Fist of all I would use the onCreated function instead of defining created. That's a little more extendable and it's the new API. created is just kept around for backwards compatibility.
About your problem. You are right, you seem to have misunderstood what ReactiveVars do. They are a reactive data source. That means that when you call myReactiveVar.get in some Tracker.autorun (aka. reactive computation), the computation will rerun whenever myReactiveVar.set is called.
You got the first part right. Spacebars helpers always run inside their own computation. What you got wrong is thinking that a method call is a reactive action. That means, that you could call trackUsernames and set the trackUsernames ReativeVar again and the value in your template would update itself. But a method is only run once. It doesn't do anything fancy with reactivity.
A method call only transfers data once. When you publish a set of documents (like all users) on the other hand, they will be updated dynamically. Whenever a change happens inside that set of published documents, it will be synced to the client. So in general, it's a better idea to use publications and subscriptions to sync data reactively. If you'd want to use a method for the same thing you'd need to do some kind of polling (so your back in the stone-age again).
The easiest way to implement what you are trying to do is to use Meteor.users.find().fetch(). As it says in the docs fetch registers dependencies for all the documents you are fetching if it's being called from within a reactive computation.
First you'll need to properly set up your publications, so that users can see other users usernames. I'll leave that to you. Then you need to reimplement your helper
Template.qastatistics.helpers({
users: function () {
var usernames = _.pluck(Meteor.users.find().fetch(), 'username');
...
Thanks to suggestions from #kyll, I managed to get what I wanted by publishing the data I need:
server:
cope.publish.usernamesID = Random.id();
Meteor.publish("itemsusernames", function () {
self = this;
var initializing = true;
var handle = Items.find().observeChanges({
added: function (id) {
!initializing && self.changed(
"itemsusernames",
cope.publish.usernamesID,
Items.distinct("p4User"));
},
changed: function (id) {
!initializing && self.changed(
"itemsusernames",
cope.publish.usernamesID,
Items.distinct("p4User"));
},
removed: function (id) {
!initializing && self.changed(
"itemsusernames",
cope.publish.usernamesID,
Items.distinct("p4User"));
}
});
initializing = false;
self.added("itemsusernames", cope.publish.usernamesID, Items.distinct("p4User"));
self.ready();
self.onStop(function () {
handle.stop();
});
});
client:
users: function () {
var usernames = [],
oUsernames = ItemsUsernames.find().fetch();
if (!oUsernames[0]) return [];
usernames = $.map(oUsernames[0], function (value, index) {
if (!isNaN(index)) {
return [value];
}
});
...
And ofcourse: ItemsUsernames = new Mongo.Collection("itemsusernames");
I have a custom publication on my server (which in some way join 2 collections).
This resulting set of this publication is exactly what I need but for performances issues I would like to avoid sending it entirely to the client.
If I did not care about performances, I would only subscribe to the
publication and do something like
theCollection.find({"my":"filter"})
I am therefore trying to find a way to publish a subset of the custom publication so that the filter would be applied on the custom publication on the server side.
Is there a way to chain or filter publications (server side) ?
For the question we can assume the custom publication to look like this and cannot be modified:
Meteor.publish('customPublication', function() {
var sub = this;
var aCursor = Resources.find({type: 'someFilter'});
Mongo.Collection._publishCursor(aCursor, sub, 'customPublication');
sub.ready();
});
if i understand the question right, you are looking for https://atmospherejs.com/reywood/publish-composite
It let's you "publish a set of related documents from various collections using a reactive join. This makes it easy to publish a whole tree of documents at once. The published collections are reactive and will update when additions/changes/deletions are made."
Ok I came to the following workaround. Instead of working on the publication, I simply added a new collection I update according to the other collections. In order to do so I am using the meteor hooks package
function transformDocument(doc)
{
doc.aField = "aValue"; // do what you want here
return doc;
}
ACollection.after.insert(function(userId, doc)
{
var transformedDocument = transformDocument(doc);
AnotherCollection.insert(transformedDocument);
});
ACollection.after.update(function(userId, doc, fieldNames, modifier, options)
{
var transformedDocument = transformDocument(doc);
delete transformedDocument._id;
AnotherCollection.update(doc._id,{$set:transformedDocument});
});
ACollection.after.remove(function(userId, doc)
{
AnotherCollection.remove(doc._id);
});
Then I have the new collection I can publish subsets the regular way
Benefits:
You can filter whatever you want into this db, no need to worry if the field is virtual or real
Only one operation every time a db changes. This avoid having several publication merging the same data
Cave eats:
This requires one more Collection = more space
The 2 db might not be always synchronised, there is few reasons for this:
The client manually changed the data of "AnotherCollection"
You had documents in "ACollection" before you added "AnotherCollection".
The transform function or source collection schema changed at some point
To fix this:
AnotherCollection.allow({
insert: function () {
return Meteor.isServer;
},
update: function () {
return Meteor.isServer;
},
remove: function () {
return Meteor.isServer;
}
});
And to synchronise at meteor startup (i.e. build the collection from scratch). Do this only once for maintenance or after adding this new collection.
Meteor.startup(function()
{
AnotherCollection.remove({});
var documents = ACollection.find({}).fetch();
_.each(documents, function(doc)
{
var transformedDocument = transformDocument(doc);
AnotherCollection.insert(transformedDocument);
});
});
Meteor Collections have a transform ability that allows behavior to be attached to the objects returned from mongo.
We want to have autopublish turned off so the client does not have access to the database collections, but we still want the transform functionality.
We are sending data to the client with a more explicit Meteor.publish/Meteor.subscribe or the RPC mechanism ( Meteor.call()/Meteor.methods() )
How can we have the Meteor client automatically apply a transform like it will when retrieving data directly with the Meteor.Collection methods?
While you can't directly use transforms, there is a way to transform the result of a database query before publishing it. This is what the "publish the current size of a collection" example describes here.
It took me a while to figure out a really simple application of that, so maybe my code will help you, too:
Meteor.publish("publicationsWithHTML", function (data) {
var self = this;
Publications
.find()
.forEach(function(entry) {
addSomeHTML(entry); // this function changes the content of entry
self.added("publications", entry._id, entry);
});
self.ready();
});
On the client you subscribe to this:
Meteor.subscribe("publicationsWithHTML");
But your model still need to create a collection (on both sides) that is called 'publications':
Publications = new Meteor.Collection('publications');
Mind you, this is not a very good example, as it doesn't maintain the reactivity. But I found the count example a bit confusing at first, so maybe you'll find it helpful.
(Meteor 0.7.0.1) - meteor does allow behavior to be attached to the objects returned via the pub/sub.
This is from a pull request I submitted to the meteor project.
Todos = new Meteor.Collection('todos', {
// transform allows behavior to be attached to the objects returned via the pub/sub communication.
transform : function(todo) {
todo.update = function(change) {
Meteor.call('Todos_update', this._id, change);
},
todo.remove = function() {
Meteor.call('Todos_remove', this._id);
}
return todo;
}
});
todosHandle = Meteor.subscribe('todos');
Any objects returned via the 'todos' topic will have the update() and the remove() function - which is exactly what I want: I now attach behavior to the returned data.
Try:
let transformTodo = (fields) => {
fields._pubType = 'todos';
return fields;
};
Meteor.publish('todos', function() {
let subHandle = Todos
.find()
.observeChanges({
added: (id, fields) => {
fields = transformTodo(fields);
this.added('todos', id, fields);
},
changed: (id, fields) => {
fields = transformTodo(fields);
this.changed('todos', id, fields);
},
removed: (id) => {
this.removed('todos', id);
}
});
this.ready();
this.onStop(() => {
subHandle.stop();
});
});
Currently, you can't apply transforms on the server to published collections. See this question for more details. That leaves you with either transforming the data on the client, or using a meteor method. In a method, you can have the server do whatever you want to the data.
In one of my projects, we perform our most expensive query (it joins several collections, denormalizes the documents, and trims unnecessary fields) via a method call. It isn't reactive, but it greatly simplifies our code because all of the transformation happens on the server.
To extend #Christian Fritz answer, with Reactive Solution using peerlibrary:reactive-publish
Meteor.publish("todos", function() {
const self = this;
return this.autorun(function(computation) {
// Loop over each document in collection
todo.find().forEach(function(entry) {
// Add function to transform / modify each document here
self.added("todos", entry._id, entry);
});
});
});
I have two collections
Offers (relevant fields: _id)
ShareRelations (relevant fields: receiverId and offerId)
and I'd like to publish only Offers to the logged in user which have been shared to him.
Actually, I'm doing this by using a helper array (visibleOffers) which I fill by looping for each ShareRelations and use this array later on the Offers.find as $in selector.
I wonder if this might be the meteor way to do this, or if I could do with less and/or prettier code?
My actual code to publish the Offers is the following:
Meteor.publish('offersShared', function () {
// check if the user is logged in
if (this.userId) {
// initialize helper array
var visibleOffers = [];
// initialize all shareRelations which the actual user is the receiver
var shareRelations = ShareRelations.find({receiverId: this.userId});
// check if such relations exist
if (shareRelations.count()) {
// loop trough all shareRelations and push the offerId to the array if the value isn't in the array actually
shareRelations.forEach(function (shareRelation) {
if (visibleOffers.indexOf(shareRelation.offerId) === -1) {
visibleOffers.push(shareRelation.offerId);
}
});
}
// return offers which contain the _id in the array visibleOffers
return Offers.find({_id: { $in: visibleOffers } });
} else {
// return no offers if the user is not logged in
return Offers.find(null);
}
});
Furthermore, the actual solution has the downside that if a new share relations is being created, the Offers collection on the client doesn't get updated with the newly visible offer instantly (read: page reload required. But I'm not sure if this is the case because of this publish method or because of some other code an this question is not primary because of this issue).
What you are looking for is a reactive join. You can accomplish this by directly using an observe in the publish function, or by using a library to do it for you. Meteor core is expected to have a join library at some point, but until then I'd recommend using publish-with-relations. Have a look at the docs, but I think the publish function you want looks something like this:
Meteor.publish('offersShared', function() {
return Meteor.publishWithRelations({
handle: this,
collection: ShareRelations,
filter: {receiverId: this.userId},
mappings: [{collection: Offers, key: 'offerId'}]
});
});
This should reactively publish all of the ShareRelations for the user, and all associated Offers. Hopefully publishing both won't be a problem.
PWR is a pretty legit package - several of us use it in production, and Tom Coleman contributes to it. The only thing I'll caution you about is that as of this writing, the current version in atmosphere (v0.1.5) has a bug which will result in a fairly serious memory leak. Until it gets bumped, see my blog post about how to run an updated local copy.
update 2/5/14:
The discover meteor blog has an excellent post on reactive joins which I highly recommend reading.
The way to do this is along the lines of this Question using observeChanges(). Still trying to figure out how to get it all working for my example, see Meteor, One to Many Relationship & add field only to client side collection in Publish?
You can use the reactive-publish package (I am one of authors):
Meteor.publish('offersShared', function () {
// check if the user is logged in
if (this.userId) {
this.autorun(function (computation) {
// initialize helper array
var visibleOffers = [];
// initialize all shareRelations which the actual user is the receiver
var shareRelations = ShareRelations.find({receiverId: this.userId}, {fields: {offerId: 1}});
// loop trough all shareRelations and push the offerId to the array if the value isn't in the array actually
shareRelations.forEach(function (shareRelation) {
if (visibleOffers.indexOf(shareRelation.offerId) === -1) {
visibleOffers.push(shareRelation.offerId);
}
});
// return offers which contain the _id in the array visibleOffers
return Offers.find({_id: { $in: visibleOffers } });
});
} else {
// return no offers if the user is not logged in
return Offers.find(null);
}
});
You can simply wrap your existing non-reactive code into an autorun and it will start to work. Just be careful to be precise which fields you query on because if you query on all fields then autorun will be rerun on any field change of ShareRelations, not just offerId.