Are console.log/debug/warn/error in node.js asynchrounous? I mean will javascript code execution halt till the stuff is printed on screen or will it print at a later stage?
Also, I am interested in knowing if it is possible for a console.log to NOT display anything if the statement immediately after it crashes node.
Update: Starting with Node 0.6 this post is obsolete, since stdout is synchronous now.
Well let's see what console.log actually does.
First of all it's part of the console module:
exports.log = function() {
process.stdout.write(format.apply(this, arguments) + '\n');
};
So it simply does some formatting and writes to process.stdout, nothing asynchronous so far.
process.stdout is a getter defined on startup which is lazily initialized, I've added some comments to explain things:
.... code here...
process.__defineGetter__('stdout', function() {
if (stdout) return stdout; // only initialize it once
/// many requires here ...
if (binding.isatty(fd)) { // a terminal? great!
stdout = new tty.WriteStream(fd);
} else if (binding.isStdoutBlocking()) { // a file?
stdout = new fs.WriteStream(null, {fd: fd});
} else {
stdout = new net.Stream(fd); // a stream?
// For example: node foo.js > out.txt
stdout.readable = false;
}
return stdout;
});
In case of a TTY and UNIX we end up here, this thing inherits from socket. So all that node bascially does is to push the data on to the socket, then the terminal takes care of the rest.
Let's test it!
var data = '111111111111111111111111111111111111111111111111111';
for(var i = 0, l = 12; i < l; i++) {
data += data; // warning! gets very large, very quick
}
var start = Date.now();
console.log(data);
console.log('wrote %d bytes in %dms', data.length, Date.now() - start);
Result
....a lot of ones....1111111111111111
wrote 208896 bytes in 17ms
real 0m0.969s
user 0m0.068s
sys 0m0.012s
The terminal needs around 1 seconds to print out the sockets content, but node only needs 17 milliseconds to push the data to the terminal.
The same goes for the stream case, and also the file case gets handle asynchronous.
So yes Node.js holds true to its non-blocking promises.
console.warn() and console.error() are blocking. They do not return until the underlying system calls have succeeded.
Yes, it is possible for a program to exit before everything written to stdout has been flushed. process.exit() will terminate node immediately, even if there are still queued writes to stdout. You should use console.warn to avoid this behavior.
My Conclusion , after reading Node.js 10.* docs (Attached below). is that you can use console.log for logging , console.log is synchronous and implemented in low level c .
Although console.log is synchronic, it wont cause a performance issue only if you are not logging huge amount of data.
(The command line example below demonstrate, console.log async and console.error is sync)
Based on Node.js Doc's
The console functions are synchronous when the destination is a terminal or a file (to avoid lost messages in case of premature exit) and asynchronous when it's a pipe (to avoid blocking for long periods of time).
That is, in the following example, stdout is non-blocking while stderr is blocking:
$ node script.js 2> error.log | tee info.log
In daily use, the blocking/non-blocking dichotomy is not something you should worry about unless you > log huge amounts of data.
Hope it helps
Console.log is asynchronous in windows while it is synchronous in linux/mac. To make console.log synchronous in windows write this line at the start of your
code probably in index.js file. Any console.log after this statement will be considered as synchronous by interpreter.
if (process.stdout._handle) process.stdout._handle.setBlocking(true);
You can use this for synchrounous logging:
const fs = require('fs')
fs.writeSync(1, 'Sync logging\n')
Related
I've been experimenting with using Tcl as a local server for a desktop application using a browser as the GUI, and using web sockets. Thus far the web sockets are working and I can send a request from the browser to Tcl, retrieve data from a SQLite database, and pass it back to the browser. And it's not difficult to set up a promise-like process in JS to "wait" for Tcl to respond in a non-blocking manner. Also, I've sent a request id on each message to Tcl such that when it responds JS knows which promise to resolve/reject.
My question concerns working in the other direction; that is, initiating a message from Tcl and waiting for the browser to respond and in a non-blocking manner.
In the code below I'm using a coroutine to store the callback procedure and any relevant arguments at the time the request is sent from Tcl to the browser; and then using the id to have the readable event handler call the correct coroutine and pass it the response. Then the coroutine calls the original callback with its arguments and the response received from the browser.
I tried to simulate a delay in sending two messages through a pipe at different times to see if this would work similar to the asynchronous manner of JS. It appears to provide similar results but, of course, is a very simple example.
Would you please tell me if this is the correct approach to take to accomplish this, and tell me of better ones? Thank you for considering my question.
Would you please also tell me why each coro$id doesn't require a separate coSend$id procedure? It doesn't matter if both messages are sent at the same ms, each calls the correct call back with the correct arguments and response. Thanks.
proc coSend {callBack args} {
set response [yield]
$callBack $response $args
}
proc redo {x} {
if {$x < 11} {
chan puts stdout "x: $x => [expr {$x*100}]ms gone by."
incr x
after 100 redo $x
} else {
set ::forever 1
}
}
proc ReadLine {chan} {
if {[catch {chan gets $chan line} len]} {
chan puts stdout "Error in chan $chan"
return
}
set response [string trim [lassign [split $line] id]]
coro$id $response
}
proc SendMessage {callBack args msg delay} {
coroutine coro[set id [GetMsgID]] coSend $callBack $args
after $delay MimicResponse "$id {$msg}"
}
proc MimicResponse {args} {
chan puts $::wchan $args
}
proc GetMsgID {} {
return [incr ::msg_id]
}
proc CallBack_1 {response args} {
chan puts stdout "$args $response"
}
proc CallBack_2 {response args} {
chan puts stdout "$args $response"
}
lassign [chan pipe] rchan wchan
chan configure $rchan -buffering line -blocking 1 -encoding iso8859-1 -translation crlf
chan configure $wchan -buffering line -blocking 1 -encoding iso8859-1 -translation crlf
chan event $rchan readable [list ReadLine $rchan]
set msg_id 0
redo 1
SendMessage CallBack_1 {arg11 arg12} "This is the response to the first message that should return second." 700
SendMessage CallBack_2 {arg21 arg22 arg23} "This is the response to the second message that should return first." 300
set forever 0
vwait forever
# Results are:
# x: 1 => 100ms gone by.
# x: 2 => 200ms gone by.
# x: 3 => 300ms gone by.
# {{arg21 arg22 arg23}} \{This is the response to the second message that should return first.\}
# x: 4 => 400ms gone by.
# x: 5 => 500ms gone by.
# x: 6 => 600ms gone by.
# x: 7 => 700ms gone by.
# {{arg11 arg12}} \{This is the response to the first message that should return second.\}
# x: 8 => 800ms gone by.
# x: 9 => 900ms gone by.
# x: 10 => 1000ms gone by.
`
In Tcl, coroutines are really little stacks; you can yield anywhere in the call hierarchy, and do not need to track what's going on. (In some other languages, this would make them "colorless"; you don't need the async/await ceremony to call them.) The things that need to be aware of how they've been called can use info coroutine to discover if they're running in a coroutine context or not.
The disadvantage of this is that coroutines are comparatively expensive, as they have a (Tcl) stack (but no C stack; Tcl avoids loading that up, and it was work to do that that stack reduction spun out both coroutine and tailcall) so you're best associating one of those per significant ongoing operation. An example would be having one for each connection made to a server when there's a non-trivial protocol present, at least until such time as the effort is transferred to another worker thread.
If you're doing much with I/O and coroutines, the coroutine package in Tcllib is recommended for handling the basics of making apparently synchronous code be actually asynchronous.
I'm working on an Alexa skill that sometimes takes a while to respond. Sometimes it is running scripts in the background, turning on a TV, connecting a bluetooth device, etc. etc. A successful response can take up to 20+ seconds once all the automation is completed.
On the web, when there is a long-running request, we are used to seeing a progress bar, or at least an animated spinner with a message telling to please wait, or that the processes is underway. I need something similar for Alexa.
I'd like Alexa to respond TWICE to a SINGLE intent, once before the HTTP request is fired, and one once the response has been received. A sample conversation would be:
[User] : Alexa, tell [app name] to switch to theater mode.
[Alexa] : (Immediately) I'm on it! Hang tight.
(...20 seconds later...)
[Alexa] : Done! Theater mode was successfully activated. Enjoy!
I've got some code running on lambda here: http://jsfiddle.net/9gmszmku/embedded/js/
Excerpt:
// ================
// [TODO] RESPONSE HERE: Alexa says: "I'm on it" or "hang on one second..." before getting the response from the http request
// ================
// this request may take many seconds!!!! Ain't nobody got time for staring at Echo for a response!!!
var req = http.request(options, (res) => {
console.log(`STATUS: ${res.statusCode}`);
console.log(`HEADERS: ${JSON.stringify(res.headers)}`);
res.setEncoding('utf8');
var rawData = '';
res.on('data', (chunk) => rawData += chunk);
res.on('end', () => {
try {
var parsedData = JSON.parse(rawData);
console.log(parsedData);
context.succeed(generateResponse(buildSpeechletResponse(parsedData.message, true), {}));
} catch (e) {
context.succeed(generateResponse(buildSpeechletResponse("Error Parsing", true), {}));
}
});
Basically, I want to have Alexa respond upfront without closing the session, and again once the function is complete.
To the best of my knowledge, you can only have one speech output and I don't think you can inject any sort of wait up one sec logic into it. You could work around it by breaking the task up into smaller pieces, chaining them together and having Alexa notify the user at each stage?
Maybe you could play some music with the audioplayer interface while your task is working and/or you can inform the user about the long running task with a speech output.
I would like to use LuaSocket's HTTP module to download a large file while displaying progress in the console and later on in a GUI. The UI must never block, not even when the server is unresponsive during the transfer. Additionally, creating a worker thread to handle the download is not an option.
Here's what I got so far:
local io = io
local ltn12 = require("ltn12")
local http = require("socket.http")
local fileurl = "http://www.example.com/big_file.zip"
local fileout_path = "big_file.zip"
local file_size = 0
local file_down = 0
-- counter filter used in ltn12
function counter(chunk)
if chunk == nil then
return nil
elseif chunk == "" then
return ""
else
file_down = file_down + #chunk
ui_update(file_size, file_down) -- update ui, run main ui loop etc.
return chunk -- return unmodified chunk
end
end
-- first request
-- determine file size
local r, c, h = http.request {
method = "HEAD",
url = fileurl
}
file_size = h["content-length"]
-- second request
-- download file
r, c, h = http.request {
method = "GET",
url = fileurl,
-- set our chain, count first then write to file
sink = ltn12.sink.chain(
counter,
ltn12.sink.file(io.open(fileout_path, "w"))
)
}
There are a few problems with the above, ignoring error checking and hard-coding:
It requires 2 HTTP requests when it is possible with only 1 (a normal GET request also sends content-length)
If the server is unresponsive, then the UI will also be unresponsive, as the filter only gets called when there is data to process.
How could I do this making sure the UI never blocks?
There is an example on non-preemptive multithreading in Programming in Lua that uses non-blocking luasocket calls and coroutines to do a multiple parallel downloads. It should be possible to apply the same logic to your process to avoid blocking. I can only add that you should consider calling this logic from IDLE event in your GUI (if there is such a thing) to avoid getting "attempt to yield across metamethod/c-call boundary" errors.
I am invoking multiple async calls of thrift from my code. I would like to wait
for all of them to complete before going on with my next stage.
for (...) {
TNonblockingTransport transport = new TNonblockingSocket(host, port);
TAsyncClientManager clientManager = new TAsyncClientManager();
TProtocolFactory protocolFactory = new TBinaryProtocol.Factory();
AsyncClient c = new AsyncClient(protocolFactory, clientManager, transport);
c.function(params, callback);
}
// I would like to wait for all the calls to be complete here.
I can have a countdown in the callback like wait/notify and get this done. But does the thrift system allow a way for me to wait on my async function call, preferably with a timeout ?
I didnt see any in the TAsyncClientManager or in the AsyncClient. Please help.
Given that it was not possible to do this, I used the sync api client and managed the launch and wait using executors and launchAll. I am leaving this as my answer for people to have an alternative.
This is my first Meteor application, I'm really excited to try to learn the framework, so I just built an internal website that will manage a bunch of command line processes. Many of these command line processes take 10-20 minutes to execute, so I was hoping I could deliver feedback to the user during execution, such as piping the stdout back to the user as the process executed. Right now I'm doing this:
var require __meteor_bootstrap__.require
var sys = require('sys')
var exec = require('child_process').exec;
Meteor.methods({
foo: function(job_id) {
var select = { _id: job_id };
var execCommand = "dir /s"; // or whatever it is I'm doing
exec(execCommand, function(error, stdout, stderr) {
Fiber (function() {
Jobs.update(select, {$set: { logs: stdout }});
}).run();
})
}
});
This works fine, and when the job completes I see the log, but I was wondering if there was a better way I could do it so that as results are available I can start sending them. Any advise is welcome.
I would append the output line by line using the MongoDB $push operator instead of resetting the content of "logs" every time. That will save you some bandwidth I guess.
But apart from that, exec does not call your function regulary. Take a look at the "node.js execute system command synchronously question for a workaround.