This is my first Meteor application, I'm really excited to try to learn the framework, so I just built an internal website that will manage a bunch of command line processes. Many of these command line processes take 10-20 minutes to execute, so I was hoping I could deliver feedback to the user during execution, such as piping the stdout back to the user as the process executed. Right now I'm doing this:
var require __meteor_bootstrap__.require
var sys = require('sys')
var exec = require('child_process').exec;
Meteor.methods({
foo: function(job_id) {
var select = { _id: job_id };
var execCommand = "dir /s"; // or whatever it is I'm doing
exec(execCommand, function(error, stdout, stderr) {
Fiber (function() {
Jobs.update(select, {$set: { logs: stdout }});
}).run();
})
}
});
This works fine, and when the job completes I see the log, but I was wondering if there was a better way I could do it so that as results are available I can start sending them. Any advise is welcome.
I would append the output line by line using the MongoDB $push operator instead of resetting the content of "logs" every time. That will save you some bandwidth I guess.
But apart from that, exec does not call your function regulary. Take a look at the "node.js execute system command synchronously question for a workaround.
Related
I'm working on an Alexa skill that sometimes takes a while to respond. Sometimes it is running scripts in the background, turning on a TV, connecting a bluetooth device, etc. etc. A successful response can take up to 20+ seconds once all the automation is completed.
On the web, when there is a long-running request, we are used to seeing a progress bar, or at least an animated spinner with a message telling to please wait, or that the processes is underway. I need something similar for Alexa.
I'd like Alexa to respond TWICE to a SINGLE intent, once before the HTTP request is fired, and one once the response has been received. A sample conversation would be:
[User] : Alexa, tell [app name] to switch to theater mode.
[Alexa] : (Immediately) I'm on it! Hang tight.
(...20 seconds later...)
[Alexa] : Done! Theater mode was successfully activated. Enjoy!
I've got some code running on lambda here: http://jsfiddle.net/9gmszmku/embedded/js/
Excerpt:
// ================
// [TODO] RESPONSE HERE: Alexa says: "I'm on it" or "hang on one second..." before getting the response from the http request
// ================
// this request may take many seconds!!!! Ain't nobody got time for staring at Echo for a response!!!
var req = http.request(options, (res) => {
console.log(`STATUS: ${res.statusCode}`);
console.log(`HEADERS: ${JSON.stringify(res.headers)}`);
res.setEncoding('utf8');
var rawData = '';
res.on('data', (chunk) => rawData += chunk);
res.on('end', () => {
try {
var parsedData = JSON.parse(rawData);
console.log(parsedData);
context.succeed(generateResponse(buildSpeechletResponse(parsedData.message, true), {}));
} catch (e) {
context.succeed(generateResponse(buildSpeechletResponse("Error Parsing", true), {}));
}
});
Basically, I want to have Alexa respond upfront without closing the session, and again once the function is complete.
To the best of my knowledge, you can only have one speech output and I don't think you can inject any sort of wait up one sec logic into it. You could work around it by breaking the task up into smaller pieces, chaining them together and having Alexa notify the user at each stage?
Maybe you could play some music with the audioplayer interface while your task is working and/or you can inform the user about the long running task with a speech output.
I have a CasperJS Script duolingo.js and I run the script using a MeteorJS App using a Meteor Method like this:
// define server methods so that the clients will have access to server components
Meteor.methods({
runCasperJS: function() {
// This method call won't return immediately, it will wait for the
// asynchronous code to finish, so we call unblock to allow this client
// to queue other method calls (see Meteor docs)
this.unblock();
// run synchonous system command
var result = process_exec_sync('casperjs duolingo.js --engine=slimerjs --disk-cache=no --proxy=178.166.41.225:80 --proxy-type=HTTP');
// check for error
if (result.error) {
throw new Meteor.Error("exec-fail", "Error running CasperJS: " + result.error.message);
}
// success
return true;
}
});
I need to run the CasperJS Instance using a proxy IP V6. But I don't have any idea, the official documentations of slimerjs and casperjs did not say anything.
Here is the exact part of the code where I use an IP V4 Proxy.
var result = process_exec_sync('casperjs duolingo.js
--engine=slimerjs --disk-cache=no --proxy=178.166.41.225:80
--proxy-type=HTTP');
Thank you for your time and help.
I've created a very simple server using Meteor, to send an email after a timeout. When I use a timeout, the message is successfully sent but an error is thrown: [Error: Can't wait without a fiber].
Here's my code:
if (Meteor.isServer) {
Meteor.startup(function () {
// <DUMMY VALUES: PLEASE CHANGE>
process.env.MAIL_URL = 'smtp://me%40example.com:PASSWORD#smtp.example.com:25';
var to = 'you#example.com'
var from = 'me#example.com'
// </DUMMY>
//
var subject = 'Message'
var message = "Hello Meteor"
var eta_ms = 10000
var timeout = setTimeout(sendMail, eta_ms);
console.log(eta_ms)
function sendMail() {
console.log("Sending...")
try {
Email.send({
to: to,
from: from,
subject: subject,
text: message
})
} catch (error) {
console.log("Email.send error:", error)
}
}
})
}
I understand that I could use Meteor.wrapAsync to create a fiber. But wrapAsync expects there to be a callback to call, and Email.send doesn't use a callback.
What should I do to get rid of the error?
This happens because while your Meteor.startup function runs inside a Fiber (like almost all other Meteor callbacks), the setTimeout you use does not! Due to the nature of setTimeout it will run on the top scope, outside the fiber in which you defined and/or called the function.
To solve, you could use something like Meteor.bindEnvironment:
setTimeout(Meteor.bindEnvironment(sendMail), eta_ms);
And then do so for every single call to setTimeout, which is a painfully hard fact.
Good thing it's not actually true. Simply use Meteor.setTimeout instead of the native one:
Meteor.setTimeout(sendMail, eta_ms);
From the docs:
These functions work just like their native JavaScript equivalents. If you call the native function, you'll get an error stating that Meteor code must always run within a Fiber, and advising to use Meteor.bindEnvironment
Meteor timers just bindEnvironment then delay the call as you wanted.
I'm attempting to run ntwitter streaming API to track tweets about a certain hashtag, populating the Mongo collection Tweets with each tweet.
I've hooked it up server side like so:
t = new nTwitter({
consumer_key: credentials.consumer_key,
consumer_secret: credentials.consumer_secret,
access_token_key: credentials.access_token_key,
access_token_secret: credentials.access_token_secret
});
Meteor.methods({
trackTweets: function () {
this.unblock; // this doesn't seem to work
console.log('... ... trackTweets');
var _this = this;
t.stream(
'statuses/filter',
{ track: ['#love'] },
function(stream) {
stream.on('data', function(tweet) {
// app/packages/mongo-livedata/collection.js:247
// throw e;
// ^
// O yes I love her like money
// Error: Meteor code must always run within a Fiber
console.log(tweet.text);
Tweets.insert(tweet.text); // this call blocks
});
stream.on('error', function(error, code) {
console.log("My error: " + error + ": " + code);
});
}
);
}
});
The line: Tweets.insert(tweet.text) throws the must run inside its own Fiber error – and I've tried putting the this.unblock statement in several different places.
What should I do here?
you dont call the function unblock, you need to replace your
this.unblock;
with this:
this.unblock();
if that doesn't work i would think it has something to do with the way ntwitter is getting the data, you could try to add this
if (Meteor.isClient) return false;
so that the method doesn't run on the client, but only on the server
I believe the code you are running server-side needs to be contained within a Fiber.
Some similar examples can be found in these answers:
Meteor code must always run within a Fiber” when calling Collection.insert on server
Stream stdout to Meteor website
Are console.log/debug/warn/error in node.js asynchrounous? I mean will javascript code execution halt till the stuff is printed on screen or will it print at a later stage?
Also, I am interested in knowing if it is possible for a console.log to NOT display anything if the statement immediately after it crashes node.
Update: Starting with Node 0.6 this post is obsolete, since stdout is synchronous now.
Well let's see what console.log actually does.
First of all it's part of the console module:
exports.log = function() {
process.stdout.write(format.apply(this, arguments) + '\n');
};
So it simply does some formatting and writes to process.stdout, nothing asynchronous so far.
process.stdout is a getter defined on startup which is lazily initialized, I've added some comments to explain things:
.... code here...
process.__defineGetter__('stdout', function() {
if (stdout) return stdout; // only initialize it once
/// many requires here ...
if (binding.isatty(fd)) { // a terminal? great!
stdout = new tty.WriteStream(fd);
} else if (binding.isStdoutBlocking()) { // a file?
stdout = new fs.WriteStream(null, {fd: fd});
} else {
stdout = new net.Stream(fd); // a stream?
// For example: node foo.js > out.txt
stdout.readable = false;
}
return stdout;
});
In case of a TTY and UNIX we end up here, this thing inherits from socket. So all that node bascially does is to push the data on to the socket, then the terminal takes care of the rest.
Let's test it!
var data = '111111111111111111111111111111111111111111111111111';
for(var i = 0, l = 12; i < l; i++) {
data += data; // warning! gets very large, very quick
}
var start = Date.now();
console.log(data);
console.log('wrote %d bytes in %dms', data.length, Date.now() - start);
Result
....a lot of ones....1111111111111111
wrote 208896 bytes in 17ms
real 0m0.969s
user 0m0.068s
sys 0m0.012s
The terminal needs around 1 seconds to print out the sockets content, but node only needs 17 milliseconds to push the data to the terminal.
The same goes for the stream case, and also the file case gets handle asynchronous.
So yes Node.js holds true to its non-blocking promises.
console.warn() and console.error() are blocking. They do not return until the underlying system calls have succeeded.
Yes, it is possible for a program to exit before everything written to stdout has been flushed. process.exit() will terminate node immediately, even if there are still queued writes to stdout. You should use console.warn to avoid this behavior.
My Conclusion , after reading Node.js 10.* docs (Attached below). is that you can use console.log for logging , console.log is synchronous and implemented in low level c .
Although console.log is synchronic, it wont cause a performance issue only if you are not logging huge amount of data.
(The command line example below demonstrate, console.log async and console.error is sync)
Based on Node.js Doc's
The console functions are synchronous when the destination is a terminal or a file (to avoid lost messages in case of premature exit) and asynchronous when it's a pipe (to avoid blocking for long periods of time).
That is, in the following example, stdout is non-blocking while stderr is blocking:
$ node script.js 2> error.log | tee info.log
In daily use, the blocking/non-blocking dichotomy is not something you should worry about unless you > log huge amounts of data.
Hope it helps
Console.log is asynchronous in windows while it is synchronous in linux/mac. To make console.log synchronous in windows write this line at the start of your
code probably in index.js file. Any console.log after this statement will be considered as synchronous by interpreter.
if (process.stdout._handle) process.stdout._handle.setBlocking(true);
You can use this for synchrounous logging:
const fs = require('fs')
fs.writeSync(1, 'Sync logging\n')