I would like to run some client-independent regular tasks in the background of a Meteor app (like scraping some pages). So they should not be inside any client thread, but once they finish, I would like to update all clients with information. What is the best way to achieve this?
Run them on your server side code. If by regular you mean timed tasks every day or something:
You could use a cron job with Tom Coleman's cron package : https://github.com/tmeasday/meteor-cron.
You'll need to install the meteorite package manager first : npm install meteorite -g and then install the cron package in your project dir mrt add cron-tick
Server js
var MyCron = new Cron();
// this job will happen every day (60 seconds * 60 * 24)
MyCron.addJob(60*60*24, function() {
//Scrape your stuff
//Update your collections
});
As soon as you run your update/insert/edit they will be pushed to all clients.
To do this in a way that allows arbitrary external processes update the Meteor clients, use the DDP protocol that's associated with Meteor. Your server processes can write to the DDP channel, and when they do your clients will update. Have a look at this post for an example and a use case, which may be similar to yours:
Using node ddp-client to insert into a meteor collection from Node
The protocol is fairly straight forward, and the post shows an example of a node.js process writing to a Mongo collection that updates the clients in real time.
You could try calling a Meteor.setInterval on the server (perhaps in Meteor.startup). That should work, though it might not be as flexible as the cron solution.
Go to http://atmospherejs.com and search for cron
The best one I found is percolate:synced-cron
Installation
meteor add percolate:synced-cron
Basics
SyncedCron.add({
name: 'Crunch some important numbers for the marketing department',
schedule: function(parser) {
// parser is a later.parse object
return parser.text('every 2 hours');
},
job: function() {
var numbersCrunched = CrushSomeNumbers();
return numbersCrunched;
}
});
SyncedCron.start();
Advanced
See their documentation
Related
Using the WP Crontrol plugin I schedule a process that sends out reminders emails to users. It is working well, but everytime I need to test something using actual data, I am scared that the system will send out reminders that should not have been sent or have already been sent from the live system.
After restoring the backup from the production server, I quickly go to the SMTP plugin I am using and select the option that drops emails sent. That does the job, but there is still a risk that something gets sent before I manage to do that.
So, I am considering my options. One is to wrap the reminder function into a check to see if it is the production server. And only run the function when it is.
I could check using home_url(), and I know it will work because I use this approach for something else.
But I feel there is a better and more correct way, and kindly ask for advice.
I usually use this approach in my projects to separate the code that runs according to the development environment. First I create a constant in the file wp-config.php with the name WP_ENVIRONMENT and assign the value of development to it and then I recognize the execution environment using two helper functions :
function prefix_is_development() {
return defined("WP_ENVIRONMENT") && "development" ===
strtolower(WP_ENVIRONMENT);
}
function prefix_is_production() {
return !defined("WP_ENVIRONMENT") || "production" ===
strtolower(WP_ENVIRONMENT);
}
Continuing the discussion from DDP: how do I wait for a connection?:
Based on the thread above, we an leverage Tracker.autorun to wait and confirm for a successful connection between a client and meteor server.
I would like to do the same on a server : server connection
Basically, I have a meteor server (server1), that will need to “test” and see if another meteor server (server2) is available.
Each time I run DDP.connect(remoteUrl).status() within server1’s meteor method, it always says “connection”. I know it connects in the next second or two, but I’m unable to wait for checking the connection success flag.
How do i do this on the server?
Thanks
The idea of reactivity doesn't exist in this form on the server, so something like the Tracker is not an option. Fortunately though there is the onReconnect callback you can use. You can steal the required logic from my meteor-serversync package:
const connection = DDP.connect(URL);
connection.onReconnect = function() {
console.log("(re)connected");
if (!initialized) {
options.onConnect && options.onConnect();
}
};
I'm writing a Meteor app which allows clients to execute terminal commands on the server at the click of a button.
I know how to do this with a single command:
//server
Meteor.methods({ exec : cmd => { ... } })
//client
Meteor.call('exec', cmd, (err, result) => {
console.log(result)
})
But now I'm trying to implement a more complex protocol and don't quite know what the best way is. I want the client to kick off a series of commands, have the server run them and tell me, step by step, whether they succeeded or failed.
Obviously I could implement this with the above code by writing client-side code that runs exec with the first command, checks the result from the server, runs exec with the next command and so on.
The crux is that in my case the series of commands is always the same, so it would make much more sense to only do one Meteor.call on the client -- the server would know what commands to run. However I would also like to have the results of the individual commands available on the client as they come in -- and this is what I can't do, because Meteor.call only returns once, of course.
What I'm looking for is a sort of stream or iterator through which I can send a number of messages to the client until everything is done. I've seen some outdated packages called meteor-streams and similar that might be able to do something like that, but I'm thinking there must be a smart way in Meteor itself to solve this. Ideas?
A common solution is a Notifications collection. Create the collection with a schema: for: ${userid}, msg: ${msg string}, type: ${err success etc}. Create a Notifications publication, which publishes docs with the users userid.
You can then subscribe to the Notifications collection in some main template page on the client. Use observeChanges to look for changes to the collection and either console.log them, use JavaScript to display them on the page or simply install a package like sAlerts to handle them.
Inside the observe changes callback, a seenNotification method should be called which removes the notification from the db, so it is not shown again.
I'll post code snippets a bit later.
Have a look at this: https://github.com/RocketChat/meteor-streamer
I think it will solve your problem easily.
A very simply scenario: in a local meteor (v 1.2.1) development environment (WebStorm), with autopublish, and insecure enabled, I have a single MongodbDB (v 3.0.4) collection 'Letters'. I wish to respond immediately to any documents being added, removed, or modified in this collection.
For this purpose, I have the following autorun function:
Template.diagram.rendered = function(){
Tracker.autorun(function () {
Letters.find({}).observe({
added: function(document) {
console.log('a new document has been added');
},
changed: function(newDocument) {
console.log('a document has been changed');
},
removed: function(document) {
console.log('a document has been removed');
}
});
})
}
When a new document is added from within the same application, I can see the console messages right away (meteor latency compensation). However, when I connect to the same MongoDB database using an external tool (Robomongo), and add, change, or remove a document within the 'Letters' collection - it takes about 6-10 seconds before this change is detected, and the corresponding console message appears in the browser.
Why does it take this long, instead of being almost instantaneous?
Once I have this question posted on meteor forums, I was pointed to a meteor blog post from 2014, which describes the oplog tailing feature, and the fact, that it is only on by default in the dev instance. This made me realize, that, by using MONGO_URL env variable with my dev application, I was forcing my meteor app to work with the MongoDB instance, which was running on my mac all the time, independently from my meteor development, and, as such, was considered as "production" by my meteor app. Once I have switched the app to work with ad-hock mongo connection / db, the oplog tailing went into effect, and I started to see immediate event propagation to the browser.
Thanks, #dburles from the meteor forums!
We have a problem with our meteor server. When we publish 300 or so items with Meteor.publish/Meteor.subscribe the server increases its memory and eventually becomes unresponsive.
We thought of:
1) monitor the number of reactive subscribtions / memory taken by an active subscription
2) make something like ,,one time publish" - ignore changes in server side collection
Any thoughts on how any of the above can be accomplished ?
Or any other tips to debug /improve meteor app performance ?
Thanks
zorlak's answer is good.
Some other things:
You can do a one-time publish by writing your own custom publisher via the this.set API, based on the code in _publishCursor. You'd do something like:
Meteor.publish("oneTimeQuery", function () {
MyCollection.find().forEach(function (doc) {
sub.added("collectionName", doc._id, doc);
});
sub.ready();
});
This does a query, sends its results down, and then never updates it again.
That said, we hope that Meteor's performance will be such that this is unnecessary!
I'd also like to add an easy way to get stats (like number of observed cursors) from an app to Meteor (exposed as an authenticated subscription) but haven't had the time yet.
As of Meteor 0.5.1, one thing you can do is to remove dependencies on the userId from the publish function. If a publish function does not depend on which user is subscribing, then Meteor will cache the db query so it doesn't get slower as more users subscribe.
See this Meteor blog post: http://meteor.com/blog/2012/11/20/meteor-051-database-scaling