What is the correct pattern to consume a highland stream using an async function? It seems the .each method does not work with an node-style async function wrapped using wrapCallback.
I would like to do something like the following. Note that I would like to catch errors generated in the .each function as well and trigger a .done function when all async consumption is done.
function asyncConsume(item,cb) {
// perform async operation based on data stream here
return cb(null,item);
}
var wrappedFunction = _.wrapCallback(asyncConsume);
highlandStream.each(wrappedFunction).done('All successfully done!')
.stopOnError(function(err) {
console.log('This error handler catches errors in .each() as well.');
});
Thank you.
As far as I know, you can only use an async function with Highland map function (after the function has been wrapCallbacked). The map does not actually call the function, so you need a Highland series or parallel call to follow and actually handle the calls.
I think you want flatMap here because you are transforming your items into a new stream:
highlandStream
.flatMap(function(item) {
return wrappedFunction(item);
}).each(someFn).done('All successfully done!')
.stopOnError(function(err) {
console.log('This error handler catches errors in .each() as well.');
});
Related
I'm still struggeling with the async/await pattern so I'm here to ask you some precisions.
I saw this page explaining the async/await pattern pretty well. I'm posting here the example that bother me :
import 'dart:async';
Future<String> firstAsync() async {
await Future<String>.delayed(const Duration(seconds: 2));
return "First!";
}
Future<String> secondAsync() async {
await Future<String>.delayed(const Duration(seconds: 2));
return "Second!";
}
Future<String> thirdAsync() async {
await Future<String>.delayed(const Duration(seconds: 2));
return "Third!";
}
void main() async {
var f = await firstAsync();
print(f);
var s = await secondAsync();
print(s);
var t = await thirdAsync();
print(t);
print('done');
}
In this example, each async method is called one after another, so the execution time for the main function is 6 seconds (3 x 2 seconds). However, I don't understand what's the point of asynchronous function if they are executed one after another.
Are async functions not supposed to execute in the background ? Is it not the point of multiple async functions to fastens the process with parrallel execution ?
I think I'm missing something about asynchronous functions and async/await pattern in flutter so if you could explain me that, it would be very appreciated.
Best
Waiting on multiple Futures to complete using Future.wait()
If the order of execution of the functions is not important, you can use Future.wait().
The functions get triggered in quick succession; when all of them complete with a value, Future.wait() returns a new Future. This Future completes with a list containing the values produced by each function.
Future
.wait([firstAsync(), secondAsync(), thirdAsyncC()])
.then((List responses) => chooseBestResponse(responses))
.catchError((e) => handleError(e));
or with async/await
try {
List responses = await Future.wait([firstAsync(), secondAsync(), thirdAsyncC()]);
} catch (e) {
handleError(e)
}
If any of the invoked functions completes with an error, the Future returned by Future.wait() also completes with an error. Use catchError() to handle the error.
Resource:https://v1-dartlang-org.firebaseapp.com/tutorials/language/futures#waiting-on-multiple-futures-to-complete-using-futurewait
The example is designed to show how you can wait for a long-running process without actually blocking the thread. In practice, if you have several of those that you want to run in parallel (for example: independent network calls), you could optimize things.
Calling await stops the execution of the method until the future completes, so the call to secondAsync will not happen until firstAsync finishes, and so on. If you do this instead:
void main() async {
var f = firstAsync();
var s = secondAsync();
var t = thirdAsync();
print(await f);
print(await s);
print(await t);
print('done');
}
then all three futures are started right away, and then you wait for them to finish in a specific order.
It is worth highlighting that now f, s, and t have type Future<String>. You can experiment with different durations for each future, or changing the order of the statements.
If anyone new in this problem use the async . Dart has a function called FutureGroup. You can use it to run futures in parallel.
Sample:
final futureGroup = FutureGroup();//instantiate it
void runAllFutures() {
/// add all the futures , this is not the best way u can create an extension method to add all at the same time
futureGroup.add(hello());
futureGroup.add(checkLocalAuth());
futureGroup.add(hello1());
futureGroup.add(hello2());
futureGroup.add(hello3());
// call the `.close` of the group to fire all the futures,
// once u call `.close` this group cant be used again
futureGroup.close();
// await for future group to finish (all futures inside it to finish)
await futureGroup.future;
}
This futureGroup has some useful methods which can help you ie. .future etc.. check the documentation to get more info.
Here's a sample usage Example One using await/async and Example Two using Future.then.
you can always use them in a single future
final results = await Future.wait([
firstAsync();
secondAsync();
thirdAsync();
]);
results will be an array of you return type. in this case array of strings.
cheers.
Try this resolve.
final List<Future<dynamic>> featureList = <Future<dynamic>>[];
for (final Partner partner in partnerList) {
featureList.add(repository.fetchAvatar(partner.uid));
}
await Future.wait<dynamic>(featureList);
If want parallel execution you should switch to multi thread concept called Isolates
mix this with async/await concepts . You can also check this website for more
https://buildflutter.com/flutter-threading-isolates-future-async-and-await/
Using async / await like that is useful when you need a resource before executing the next task.
In your example you don't do really useful things, but imagine you call firstAsync, that gives you a stored authorization token in your phone, then you call secondAsync giving this token get asynchronously and execute an HTTP request and then checking the result of this request.
In this case you don't block the UI thread (user can interact with your app) and other tasks (get token, HTTP request...) are done in background.
i think you miss understood how flutter works first flutter is not multi threaded.....!
second if it isn't multi threaded how can it executes parallel tasks, which doesnt happen....! here is some links that will help you understand more https://webdev.dartlang.org/articles/performance/event-loop
https://www.dartlang.org/tutorials/language/futures
flutter doesn't put futures on another thread but what happens that they are added to a queue the links that i added are for event loop and how future works. hope you get it , feel free to ask me :)
I have an array of JSON objects. I want to iterate through the array, and for each object check some parameters, and if certain conditions are true, make a request to create a record using the Air-table module.
Because of the rate limit, I want to throttle how quickly i make those requests, essentially calling the functions in series, and also implementing a delay in between them.
Promises have always confused the crap out of me. How would Iterate over an array to achieve this with promises. Can I do this with Async Await? I know that some promise libraries like Bluebird have both a delay and a concurrent function, but I am pretty confused about how to use them correctly.
If you want to iterate through array synchronously you can use for ... of construction:
async function doSomethingWithArray(array, delayTime) {
for (const item of array) {
await someMethod(item);
await delay(delayTime);
}
}
the simplest delay function is:
function delay(delayTime) {
return new Promise(resolve => setTimeout(resolve, delayTime));
}
I am having trouble figuring out how to use wrapAsync properly using Meteor. I am currently working with node-apac and the Amazon Product Advertising API.
If am trying to run the following code, how do I run it asynchronously:
opHelper.execute('ItemSearch', {
'SearchIndex': 'Books',
'Keywords': 'harry potter',
'ResponseGroup': 'ItemAttributes, Offers'
}, function(err, results) {
console.log(results);
});
I have tried to watch several videos, but am having trouble
Meteor.wrapAsync takes an asynchronous method like opHelper.execute and makes it synchronous. It can do this so long as the last param taken by the method returns a callback where the first param is the error and second the result. This is just like your method!
You make a new method that is synchronous:
var opExecuteSynchronous = Meteor.wrapAsync(opHelper.Execute, opHelper);
The first param opHelper.Execute is the method you want to make asynchronous, and the second param is the context of the method (opHelper).
You can use this synchronously now:
var results = opExecuteSynchronous('ItemSearch', {
'SearchIndex': 'Books',
'Keywords': 'harry potter',
'ResponseGroup': 'ItemAttributes, Offers'
})
This will throw an error if err is called instead of results in the callback.
I've used caolan's async module which is very good, however tracking errors and the varying way of passing data through for control flow causes development to sometimes be very difficult.
I would like to know if there are any better options, or what is currently being used in production environments.
Thanks for reading.
I use async as well. To help tracking errors it's recommended you name your functions, instead of having loads of anonymous functions:
async.series([
function doSomething() {...},
function doSomethingElse() {...},
function finish() {...}
]);
This way you'll get more helpful information in stack traces.
...however tracking errors and the varying way of passing data through for control flow causes development to sometimes be very difficult.
I've recently created a simple abstraction named "wait.for" to call async functions in sync mode (based on Fibers): https://github.com/luciotato/waitfor
Using wait.for, you can use 'try/catch' while still calling async functions, and you keep function scope (no closures needed). Example:
function inAFiber(param){
try{
var data= wait.for(fs.readFile,'someFile'); //async function
var result = wait.for(doSomethingElse,data,param); //another async function
otherFunction(result);
}
catch(e) {
//here you catch if some of the "waited.for"
// async functions returned "err" in callback
// or if otherFunction throws
};
see the examples at https://github.com/luciotato/waitfor
Sometimes it is hard to put all the functions in an array. When you have an array of objects and want to do something for each object, I use something like the example below.
read more in: http://coppieters.blogspot.be/2013/03/iterator-for-async-nodejs-operations.html
var list = [1, 2, 3, 4, 5];
var sum = 0;
Application.each(list, function forEachNumber(done) {
sum += this;
// next statement most often called as callback in an async operation
// file, network or database stuff
done(); // pass an error if something went wrong and automatically end here
}, function whenDone(err) {
if (err)
console.log("error: " + err);
else
console.log("sum = " + sum);
});
I name the functions, because it is easier to debug (and easier to read)
Based on Dart official page When using Async/Wait:
When the app sees the word async it execute the function normally synchronously, until it sees await or return
Note that an async function starts executing right away
(synchronously). The function suspends execution and returns an
uncompleted future when it reaches the first occurrence of any of the
following:
The function’s first await expression (after the function gets the
uncompleted future from that expression).
Any return statement in the function.
The end of the function body.
and when it sees any of them it returns an uncompleted Future and stops executing the async function until it execute all of the other functions, and when all of the other functions are executed, the app goes back to the async function and executes what's inside of it.
Here's a photo from the Dart official page explaining it in more details:
But when I was testing that I tried to add a print statement before returning the future result as you can see in the below code, but the result wasn't as stated in the site, as it's saying that the app stops executing once it sees the word awaitbut the statement: "Async - Hi Called 1st" was printed as you can see before the other functions were executed.
import 'dart:async';
Future<void> print1stAsync() async {
var test = await callAsync();
print(test);
}
main() {
print1stAsync();
print2nd();
print3rd();
print4th();
}
print2nd() {
print('Called 2nd');
}
print3rd() {
print("Called 3rd");
}
print4th() {
print('Called 4th');
}
Future<String> callAsync() {
print("Async - Hi Called 1st");
return Future(() => "Async - Called 1st ");
}
Output:
Async - Hi Called 1st
Called 2nd
Called 3rd
Called 4th
Async - Called 1st
So why is this happening? have I miss understood something?
The app doesn't stop executing, only the execution of the code after await is delayed until the returned Future completes.
You also need to await the call to the async function print1stAsync(), otherwise the execution of main continues after the call to callAsync(); (after the call, not after the Future it returns completes)
main() async {
await print1stAsync();
print2nd();
print3rd();
print4th();
}
Add async to a function also means that this function returns a Future.
There is no way to go back from async to sync. Async is contagious.
await callAsync(); means code below that line within the same function (like print(test); in your example) will be delayed.
It doesn't say anything about code in callAsync() or code that calls print1stAsync();