My actual program is more sophisticated than this, but I've tried to simplify things.
So let's say I'm reading a file with a list of URLs. I want to download the HTML from each URL and process it. The processing may be a bit complex, so I'd like it to be done on a separate thread.
The basic problem is how to tell when all the processing is done. For example, if the user tries to close the program before all URLs are processed, I want to give him a message and not exit the program. Alternatively, I want to terminate the program (perhaps with a MsgBox("Done") message) as soon as all the URLs are processed.
I'd like my code to look something as follows (assuming I've got an outer loop reading the URLs and calling this routine)...
List<Task> TaskList = new List<Task>();
async void ProcessSingleUrl(string url) {
var web = new HttpClient();
var WebPageContents = await web.GetStringAsync(url);
Task t = Task.Run(() => ProcessWebPage(WebPageContents);
TaskList.Add(t);
}
The above code should run very quickly (Async methods run pretty well instantly) and return to the caller almost immediately.
But at that point, I may well have no entries whatsoever in TaskList, since a task isn't defined until the GetStringAsync is completed, and none (or maybe just a few) may have finished by then. So
Task.WaitAll(TaskList.ToArray());
doesn't work the way I need it to.
If absolutely necessary, I could first read in all the URLs and know how many Tasks to expect, but I'm hoping for a more elegant solution.
I suppose I could increment a counter just before the await, but that feels a bit kludgy.
I assume I'm structuring things incorrectly, but I'm not sure how to reorganize things.
Note: I'm not wedded to Task.Run. Good ol' QueueWorkItem is a possibility, but I think it has pretty well the same problems.
I assume I'm structuring things incorrectly, but I'm not sure how to reorganize things.
I think that is true. Here is a possible solution: Store the whole computation as a Task in your list, not just the second part:
async Task ProcessSingleUrlInner(string url) {
var web = new HttpClient();
var WebPageContents = await web.GetStringAsync(url);
Task t = Task.Run(() => ProcessWebPage(WebPageContents);
await t;
}
void ProcessSingleUrl(string url) {
var t = ProcessSingleUrlInner(url);
TaskList.Add(t);
}
Waiting on all tasks of this list will guarantee that everything is done. Probably, you need to adapt this idea to your exact needs.
I'm assuming that you're getting the list of urls as an IEnumerable<string> or some such.
You can use LINQ to convert each url into a Task, and then await them all to complete:
async Task ProcessUrls(IEnumerable<string> urls)
{
var tasks = urls.Select(async url =>
{
var web = new HttpClient();
var WebPageContents = await web.GetStringAsync(url);
await Task.Run(() => ProcessWebPage(WebPageContents);
});
await Task.WhenAll(tasks);
}
Note that if you use this solution and there are multiple different urls that have errors, then Task.WhenAll will only report one of those errors.
Related
I have some slow part of my code that does a http request then waits for the answer and calls an adf pipeline so it can update a database parameter at the end. Since this took 2 minutes and my client app seemed so slow for my clients, I wrapped this call inside Task.Run() and set my logic inside. This works good but the last part, update a database parameter, doesn't get executed at all.
Task.Run(async () =>
{
await client.ExecuteAsync(request, CancellationToken.None);
await RunADFPipeline(pipeline, _config);
await _projectRepository.UpdateValue(id, "status", "unlock");
});
I found this example online and I applied it in my code. The first two awaits seem to execute correctly but the third one doesn't execute at all. If I set it outside of Task.Run(), it executes. The third part literally just calls a stored procedure
return await _context.Database.ExecuteSqlRawAsync("EXEC core.UpdateValue #ProjectId, #Name, #Value", parameters);
I also tried by adding ContinueWith() at the end of the Task.Run() but still no luck.
Task.Run(async () =>
{
await client.ExecuteAsync(request, CancellationToken.None);
await _adfHelper.RunADFPipeline(pipeline, _config);
}).ContinueWith((result) => _projectRepository.UpdateValue(id, "status", "unlock"));
All this code is inside a controller. Any thoughts and ideas on how can I achieve my goal and update this parameter after Task.Run() finishes or the functions inside finish?
I'm using Navision to call ReadAsMultipartAsync().Result in a C# library, but this blocks Navision. If I change the library with await and sync, I can't manage a Task result from Navision, so I would like to call the "ReadAsMultipartAsync" function but in a synchronous way.
If I do something like this: streamCont.ReadAsMultipartAsync(provider).Wait(TimeSpan.FromSeconds(20)); works, but I spent 20 seconds on this case.
There are any way of wait the task of ReadAsMultipartAsync without block Navision?. Thanks
I have found a way to do it that works. It's funny but this blocks Nav:
Task<MultipartMemoryStreamProvider> task = streamContent.ReadAsMultipartAsync();
task.Wait();
And these two work fine:
Task thread1 = Task.Factory.StartNew(() => result = streamContent.ReadAsMultipartAsync().Result);
Task.WaitAll(thread1);
Task tarea1 = new Task(() => result = streamContent.ReadAsMultipartAsync().Result);
tarea1.Start();
tarea1.Wait();
I don't know what the difference is between the first option and the others, or if it can give an error in some cases, but for now it seems to work.
I'm still struggeling with the async/await pattern so I'm here to ask you some precisions.
I saw this page explaining the async/await pattern pretty well. I'm posting here the example that bother me :
import 'dart:async';
Future<String> firstAsync() async {
await Future<String>.delayed(const Duration(seconds: 2));
return "First!";
}
Future<String> secondAsync() async {
await Future<String>.delayed(const Duration(seconds: 2));
return "Second!";
}
Future<String> thirdAsync() async {
await Future<String>.delayed(const Duration(seconds: 2));
return "Third!";
}
void main() async {
var f = await firstAsync();
print(f);
var s = await secondAsync();
print(s);
var t = await thirdAsync();
print(t);
print('done');
}
In this example, each async method is called one after another, so the execution time for the main function is 6 seconds (3 x 2 seconds). However, I don't understand what's the point of asynchronous function if they are executed one after another.
Are async functions not supposed to execute in the background ? Is it not the point of multiple async functions to fastens the process with parrallel execution ?
I think I'm missing something about asynchronous functions and async/await pattern in flutter so if you could explain me that, it would be very appreciated.
Best
Waiting on multiple Futures to complete using Future.wait()
If the order of execution of the functions is not important, you can use Future.wait().
The functions get triggered in quick succession; when all of them complete with a value, Future.wait() returns a new Future. This Future completes with a list containing the values produced by each function.
Future
.wait([firstAsync(), secondAsync(), thirdAsyncC()])
.then((List responses) => chooseBestResponse(responses))
.catchError((e) => handleError(e));
or with async/await
try {
List responses = await Future.wait([firstAsync(), secondAsync(), thirdAsyncC()]);
} catch (e) {
handleError(e)
}
If any of the invoked functions completes with an error, the Future returned by Future.wait() also completes with an error. Use catchError() to handle the error.
Resource:https://v1-dartlang-org.firebaseapp.com/tutorials/language/futures#waiting-on-multiple-futures-to-complete-using-futurewait
The example is designed to show how you can wait for a long-running process without actually blocking the thread. In practice, if you have several of those that you want to run in parallel (for example: independent network calls), you could optimize things.
Calling await stops the execution of the method until the future completes, so the call to secondAsync will not happen until firstAsync finishes, and so on. If you do this instead:
void main() async {
var f = firstAsync();
var s = secondAsync();
var t = thirdAsync();
print(await f);
print(await s);
print(await t);
print('done');
}
then all three futures are started right away, and then you wait for them to finish in a specific order.
It is worth highlighting that now f, s, and t have type Future<String>. You can experiment with different durations for each future, or changing the order of the statements.
If anyone new in this problem use the async . Dart has a function called FutureGroup. You can use it to run futures in parallel.
Sample:
final futureGroup = FutureGroup();//instantiate it
void runAllFutures() {
/// add all the futures , this is not the best way u can create an extension method to add all at the same time
futureGroup.add(hello());
futureGroup.add(checkLocalAuth());
futureGroup.add(hello1());
futureGroup.add(hello2());
futureGroup.add(hello3());
// call the `.close` of the group to fire all the futures,
// once u call `.close` this group cant be used again
futureGroup.close();
// await for future group to finish (all futures inside it to finish)
await futureGroup.future;
}
This futureGroup has some useful methods which can help you ie. .future etc.. check the documentation to get more info.
Here's a sample usage Example One using await/async and Example Two using Future.then.
you can always use them in a single future
final results = await Future.wait([
firstAsync();
secondAsync();
thirdAsync();
]);
results will be an array of you return type. in this case array of strings.
cheers.
Try this resolve.
final List<Future<dynamic>> featureList = <Future<dynamic>>[];
for (final Partner partner in partnerList) {
featureList.add(repository.fetchAvatar(partner.uid));
}
await Future.wait<dynamic>(featureList);
If want parallel execution you should switch to multi thread concept called Isolates
mix this with async/await concepts . You can also check this website for more
https://buildflutter.com/flutter-threading-isolates-future-async-and-await/
Using async / await like that is useful when you need a resource before executing the next task.
In your example you don't do really useful things, but imagine you call firstAsync, that gives you a stored authorization token in your phone, then you call secondAsync giving this token get asynchronously and execute an HTTP request and then checking the result of this request.
In this case you don't block the UI thread (user can interact with your app) and other tasks (get token, HTTP request...) are done in background.
i think you miss understood how flutter works first flutter is not multi threaded.....!
second if it isn't multi threaded how can it executes parallel tasks, which doesnt happen....! here is some links that will help you understand more https://webdev.dartlang.org/articles/performance/event-loop
https://www.dartlang.org/tutorials/language/futures
flutter doesn't put futures on another thread but what happens that they are added to a queue the links that i added are for event loop and how future works. hope you get it , feel free to ask me :)
Sorry for the beginers question, I read a lot of post here and on the web and there is something fondemental I can't understand.
As I understood, the usage of async actions in WebAPI is mainly for scalability reasons, so any incoming request will be diverted to a worker instead to a thread and by that, more requests could be served.
My project consists on several huge actions that read/insert/update from DB by EF6 many times. the action looks like that:
public async Task<HttpResponseMessage> Get(int id)
{
PolicyModel response = await policyRepository.GetPolicyAsync(id);
return Request.CreateResponse<PolicyModel>(HttpStatusCode.OK,response);
}
and GetPolicyAsync(int id) looks like that:
public async Task<PolicyModel> GetPolicyAsync(int id)
{
PolicyModel response = new PolicyModel();
User currentUser = await Repositories.User.GetUserDetailsAsync(id);
if(currentUser.IsNew)
{
IEnumerable<Delivery> deliveries = await Repositories.Delivery.GetAvailableDeliveries();
if(deliveries == null || deliveries.Count() == 0
{
throw new Exception("no deliveries available");
}
response.Deliveries = deliveries;
Ienumerable<Line> lines = await Repositores.Delivery.GetActiveLinesAsync();
lines.AsParallel().ForAll(line => {
await Repositories.Delivery.AssignLineAsync(line,currentUser);
}
...
return response;
}
I didn't write the entire code but it goes quite a bit and it is also broken into several methods but that is the spirit of it
now the question i have is: is it a good practice to use so many awaiters in one method? I saw that it is more difficult to debug, is it hard to maintain the thread context and for the sake of worker assignment, shouldn't I just use Task.Factory.StartNew() or maybe call a simple Task.Delay() so that the request will immidiatelly be diverted to a worker?
I know that it is not a good practice (async all the way) so maybe just one async method at the end/begining of the GetpolicyAsync(int id) method
EDIT:
as I understood the mechanics of the async methods in .net, for every async method, the compiler is looking for a free thread and let it deal withthe method, the thread is looking for a free worker and assign the method to it and then report back to the compiler that it is free. so if we have 10 threads and for every thread there are 10 workers, the program can deals with 100 concurrent async methods.
so back to web developement, the IIS assign x thread to each app pool, 10 for instance. that means that the async WebAPI method can handle 100 requests but if there is another async method inside, the amount of requests that can be dealth with are 50 and so on, am I right?
and as I understood, I must call an async method in order to make the WebAPI method truely async and now, since it is a bad practice to use Task.Factory.StartNew(), I must at least use Task.Delay()
what I really want to gain is the scalability of the async WebAPI methods and the context awareness of synced methods
in all the examples I've seen so far, they only show a very simple code but in real life, methods are far more complex
Thanks
There's nothing wrong with having many awaits in a single method. If the method becomes too complicated for you you can split it up into several methods:
public async Task<PolicyModel> GetPolicyAsync(int id)
{
PolicyModel response = new PolicyModel();
User currentUser = await Repositories.User.GetUserDetailsAsync(id);
if(currentUser.IsNew)
{
await HandleDeliveriesAsync(await Repositories.Delivery.GetAvailableDeliveries());
}
...
return response;
}
public async Task HandleDeliveriesAsync(IEnumerable<Delivery> deliveries)
{
if(deliveries == null || deliveries.Count() == 0
{
throw new Exception("no deliveries available");
}
response.Deliveries = deliveries;
Ienumerable<Line> lines = await Repositores.Delivery.GetActiveLinesAsync();
lines.AsParallel().ForAll(line => {
await Repositories.Delivery.AssignLineAsync(line,currentUser);
}
Don't use Task.Factory.StartNew or Task.Delay as it just offloads the same work to a different thread. It doesn't add any value in most cases and actually harms in ASP.Net.
After much search, I came across this artical:
https://msdn.microsoft.com/en-us/magazine/dn802603.aspx
it explains what #i3arnon said in the comment. there are no workers at all, the threads are doing all the work.
In a nutshell, the thread that handles a web request reach to an opporation that is designed to be done asyncronically on the driver stack, it creates a request and pass it to the driver. The driver mark it as pending and reports "done" to the thread which goes back to the thread pool to get another assignment. The actual job is not being done by threads but by the driver that borrowing cpu time from all threads and leave them free to attend to their businesses. When the opporation is done, the driver notifies it and an available thread comes to continue...
so, from that I learn that I should look into each and every async method and check that it actually does an opporation that uses the drivers asyncronically.
Point to think: the scallability of your website is dependent on the amount of opporations that are truly being done asyncronically, if you don't have any of them, then your website won't scale.
moderators, I'm a real newbe, if I got it all wrong, please correct me
Thanks!
As far as I know, when runtime comes across the statement below it wraps the rest of the function as a callback to the method which is invoked asynchronously (someCall() in this example). In this case anotherCall() will be executed as a callback to someCall():
await someCall();
await anotherCall();
I wonder if it is possible to make runtime perform like this: call someCall() in async fashion and return immediately to the calling thread, then invoke anotherCall() similarly (without waiting someCall to complete). Because I need these two methods to run asynchronously and suppose these calls are just fire and forget calls.
Is it possible to implement this scenario using just async and await (not using old begin/end mechanism)?
The async/await includes a few operators to help with parallel composition, such as WhenAll and WhenAny.
var taskA = someCall(); // Note: no await
var taskB = anotherCall(); // Note: no await
// Wait for both tasks to complete.
await Task.WhenAll(taskA, taskB);
// Retrieve the results.
var resultA = taskA.Result;
var resultB = taskB.Result;
The simplest way is probably to do this:
var taskA = someCall();
var taskB = someOtherCall();
await taskA;
await taskB;
This is especially nice if you want the result values:
var result = await taskA + await taskB;
so you don't need to do taskA.Result.
TaskEx.WhenAll might be faster than two awaits after each other. i don't know since I haven't done performance investigation on that, but unless you see a problem I think the two consecutive awaits reads better, especially if you ewant the result values.
The Async CTP is no longer needed provided you're using .NET 4.5. Note that the async functionality is implemented by the compiler so .NET 4 apps can use it but VS2012 is required to compile it.
TaskEx is not needed anymore. The CTP couldn't modify the existing framework so it used extensions to accomplish things that the language would handle in 5.0. Just use Task directly.
So herewith, I have re-written the code(answered by Stephen Cleary) by replacing TaskEx with Task.
var taskA = someCall(); // Note: no await
var taskB = anotherCall(); // Note: no await
// Wait for both tasks to complete.
await Task.WhenAll(taskA, taskB);
// Retrieve the results.
var resultA = taskA.Result;
var resultB = taskB.Result;