Here's a sample code from Jersey docs for Asynchronous Services:
#Path("/resource")
public class AsyncResource {
#GET
public void asyncGet(#Suspended final AsyncResponse asyncResponse) {
new Thread(new Runnable() {
#Override
public void run() {
String result = veryExpensiveOperation();
asyncResponse.resume(result);
}
private String veryExpensiveOperation() {
// ... very expensive operation
}
}).start();
}
}
Considering that container is already in charge of releasing the connection handling thread back to the pool and handing the request processing to a worker thread, I'm wondering why we still need to spawn a new thread programmatically? Shouldn't this be just a matter of container configuration (setting number of worker threads)?
There is no need to spwan a new thread like this. If you are in an JavaEE environment you can simply place an #Asynchronous annotation to your method:
#GET
#Asynchronous
public void asyncGet(#Suspended final AsyncResponse asyncResponse) {
String result = veryExpensiveOperation();
asyncResponse.resume(result);
}
I you need extended thread pooling you can also have a look at ManagedExecutorService.
Shouldn't this be just a matter of container configuration (setting number of worker threads)?
Actually you can configure the number of container worker threads, but you are still bound by them. If you have 10 container threads then you can process 10 requests at the same time. If you have 100 threads, then you can have 100 simultaneous requests
Async web requests on the other hand decouple the request/response objects (and jax-rs uses servlets underneath in case of servlet-container deployment) from the worker thread. Thus you can have 10 threads, but 1000 requests for instance (keep reading).
I'm wondering why we still need to spawn a new thread programmatically?
It's meaningless to decouple the request from the thread if you are still going to process it in the same thread, because in that case there is absolutely no difference between async/sync requests.
Also you don need to spawn a new thread. Actually that's a horrible solution. Thread creation is expensive. Instead you should submit the runnables to a thread pool.
So what good are async requests then ? They are good for operations that take a lot of time to complete. Do the following experiment.
Configure tomcat (or your server of choice) to use a pool of 10 threads.
Make your resource to just sleep for 10 seconds.
Request it as many times as you can.
Make your resource asynchronous and instead of waiting in it, submit the runnables which just wait for 10seconds to a thread pool -> how many requests can you make now ?
Related
Service layer
public async Task<string> getAllAsync()
{
return await WCFMethodAsync();
}
Presentation layer
public class Customer : Controller
{
public async Task<string> Index()
{
var r = await Task.Run(getAllAsync());
return r;
}
}
I will describe what I think is happening. I would like to know if I am wrong.
the request takes a thread from ThreadPool
await returns an incomplete Task
Task.Run queues a Task to run getAllAsync
3.1. that will take a thread from the ThreadPool.
when the getAllAsync method finishes the taken thread returns to the group of threads
when the service method call end .net is notified and a thread is taken from the group of threads o terminate the request.
Where is the incomplete homework returned to? Since the method was executed inside Task.Run.
I would like to know the flow of the whole process.
Since a thread was initially taken from the thread group and task.run takes a work thread, the question is when the task ends the job (wcf) and a thread is taken to finish the request. How many threads were used to process the request?
Your getAllAsync method is a I/O bound operation, because you are making a network call to reach your WCF service instance.
That means the getAllAsync will be executed by the network driver. It is an asynchronous non-blocking promise task. The calling thread is not blocked until the asynchronous I/O operation finishes. When it finishes the ThreadPool will be notified about it.
On the other hand the Task.Run is design for CPU bound operation.
The Task.Run returns a delegate task and it is used to queue a work on the ThreadPool. That work will be processed by one of the CPU core.
So if you put an asynchronous I/O work (kick-off) call on a dedicated Thread then that is just wasting valuable resources.
I have a .NET Core Web API application. The app contains models for db access and a way to send emails to users. My end goal is to call a method nightly (to email users that their registration expired and to mark it expired in the database).
So, in short, I can build an endpoint and call it manually every night. Or build a windows service to call the endpoint. Or build a windows service to do the work. But I want to keep the logic in one application.
My ideal solution would be to have a timer running inside my app and calling a method in a service every 24 hours. Of course, that's not possible, so I am looking at Hangfire. The official documentation seems to indicate that there is a lot of overhead.
Hangfire keeps background jobs and other information that relates to the processing inside a persistent storage. Persistence helps background jobs to survive on application restarts, server reboots, etc.
Do I need this if I just want to call a method?
Background jobs are processed by Hangfire Server. It is implemented as a set of dedicated (not thread pool’s) background threads that fetch jobs from a storage and process them. Server is also responsible to keep the storage clean and remove old data automatically.
Do I even need jobs?
Is there a way to JUST call a method without all this overhead with Hangfire?
tl;dr: Are there options to opt out of the dashboard, database connectivity, etc and just have Hangfire work as a timer?
My ideal solution would be to have a timer running inside my app and calling a method in a service every 24 hours. Of course, that's not possible...
It's very possible, actually, using IHostedService. You should take some time to read the full documentation, but simply, for your scenario, you'd just need something like:
internal class NightlyEmailHostedService : IHostedService, IDisposable
{
private Timer _timer;
public Task StartAsync(CancellationToken cancellationToken)
{
_timer = new Timer(DoWork, null, TimeSpan.Zero,
TimeSpan.FromHours(24));
return Task.CompletedTask;
}
private void DoWork(object state)
{
// send email
}
public Task StopAsync(CancellationToken cancellationToken)
{
_timer?.Change(Timeout.Infinite, 0);
return Task.CompletedTask;
}
public void Dispose()
{
_timer?.Dispose();
}
}
Then, in Startup.cs just add:
services.AddHostedService<NightlyEmailHostedService>();
Now, that's an extremely naive approach. It basically just kicks off a timer that will run once every 24 hours, but depending on when your app started, it may not always be at night. In reality, you'd likely want to have the timer run every minute or so, and check against a particular time you actually want the email to go out. There's an interesting implementation of handling cron-style times via an IHostedService you might want to reference.
The long and short is that it's very possible to do this all in your app, without requiring anything additional like Hangfire. However, you have to a do a bit more work than you would have to using something like Hangfire, of course.
I know the Task Parallel Library(TPL) makes use of the Thread Pool to run tasks.
I've also read that IIS makes use of a "Thread Pool" to process incoming Http requests.
My question is: Are these two referring to the same Thread Pool??
Say I have this controller method:
[HttpGet]
public async void GetAsyncTest()
{
await Task.Run(() => {//Do some heavy CPU work here});
}
Will the task be processed using a thread from the same thread pool as the request thread?
The short answer is yes.
This is why it does not make sense starting a task for a CPU-heavy operation: it will use a thread from the same pool as other incoming requests so your application will not be able to serve more requests as before. And that beats the purpose of making things async in a web app: it is about releasing threads for other requests to use while an external IO-heavy operation (db request or http request to another service) is executed.
I know that is a common question, but I've read a kiloton of articles and feel confused. And now I think that it would be better not to read them at all )).
So, how ASP.NET works (only about threads):
http request is served by thread from the thread pool.
while request is processing this thread is busy, because request is processing inside exactly this thread.
when request processing finishes, the thread returns to thread pool, server sends a response.
Is this described behaviour right ?
What really happens when I start new task inside ASP.NET MVC controller?
public ActionResult Index()
{
var task1 = Task.Factory.StartNew(() => DoSomeHeavyWork());
return View();
}
private static async Task DoSomeHeavyWork()
{
await Task.Delay(5000);
}
controller action starts to execute inside thread that processes current request - T1.
the thread pool allocates another one thread (T2) for task1.
task1 starts "immediately" inside T2.
the View result returns "immediately".
ASP.NET do some work, server sends a response, T1 returns to the thread pool, T2 is still alive.
after some time when the DoSomeHeavyWork will be finished, the T2 thread will be returned to the thread pool.
Is it correct ?
Now lets look to async action
public async Task<ActionResult> Index()
{
await DoSomeHeavyWork();
return View();
}
, I understand the difference with previous code sample, but not the process, in this example the behaviour is the following:
action starts to execute inside thread that processes current request - T1.
DoSomeHeavyWork "immediately" returns a task, let's call it "task1" too.
T1 returns to the thread pool.
after the DoSomeHeavyWork finishes, Index action continue to execute.
after Index action execution the server will send a response.
Please explain what happening between points 2 and 5, the questions are:
is the DoSomeHeavyWork processed inside task1 or where (where it is "awaited") ? I think this a key question.
which thread will continue to process the request after await - any new one from the thread pool, right ?
request produces thread allocating from the thread pool, but response will not be sent until the DoSomeHeavyWorkAsync finished and it doesn't matter in which thread this method executes. In other words, according to single request and single concrete task (DoSomeHeavyWork) there is no benefits of using async. Is it correct ?
if the previous statement is correct, then I don't understand how the async can improve performance for multiple requests with the same single task. I'll try to explain. Let's assume that the thread pool has 50 threads available to handle requests. Single request should be processed at least by one single thread from the thread pool, if the request starts another threads, then all of them will be taken from the thread pool, e.g. request takes one thread to process himself, starts 5 different tasks in parallel and wait all of them, thread pool will have 50 - 1 - 5 = 44 free threads to handle incoming requests - so this is a parallelism, we can improve performance for single request, but we reduce number of requests which can be processed. So according request processing in ASP.NET I suppose that only task that somehow starts IO completion thread can achieve a goal of async (TAP). But how IO completion thread calls back thread pool thread in this case ?
Is this described behaviour right ?
Yes.
Is it correct ?
Yes.
is the DoSomeHeavyWork processed inside task1 or where (where it is
"awaited") ? I think this a key question.
From the current code, DoSomeHeavyWork will asynchronously wait for Task.Delay to complete. Yes, this will happen on the same thread allocated by the thread-pool, it won't spin any new threads. But there isn't a guarantee that it will be the same thread, though.
which thread will continue to process the request after await?
Because we're talking about ASP.NET, that will be an arbitrary thread-pool thread, with the HttpContext marshaled onto it. If this was WinForms or WPF app, you'd be hitting the UI thread again right after the await, given that you don't use ConfigureAwait(false).
request produces thread allocating from the thread pool, but response
will not be sent until the DoSomeHeavyWorkAsync finished and it
doesn't matter in which thread this method executes. In other words,
according to single request and single concrete task (DoSomeHeavyWork)
there is no benefits of using async. Is it correct ?
In this particular case, you won't see the benefits of async. async shines when you have concurrent requests hitting the server, and alot of them are doing IO bound work. When using async while hitting the database, for example, you gain freeing the thread-pool thread for the amount of time the query executes, allowing the same thread to process more requests in the meanwhile.
But how IO completion thread calls back thread pool thread in this
case ?
You have to separate parallelism and concurrency. If you need computation power for doing CPU bound work in parallel, async isn't the tool that will make it happen. On the other hand, if you have lots of concurrent IO bound operations, like hitting a database for CRUD operations, you can benefit from the usage of async by freeing the thread while to IO operation is executing. That's the major key point for async.
The thread-pool has dedicated pool of IO completion threads, as well as worker threads, which you can view by invoking ThreadPool.GetAvailableThreads. When you use IO bound operations, the thread that retrieves the callbacks is usually an IO completion thread, and not a worker thread. They are both have different pools.
Since the very begining of writing ASP.NET applications when I wanted to add a threading there are 3 simple ways I can accomplish threading within my ASP.NET application :
Using the System.Threading.ThreadPool.
Using a custom delegate and calling its BeginInvoke method.
Using custom threads with the aid of System.Threading.Thread class.
The first two methods offer a quick way to fire off worker threads for your application. But unfortunately, they hurt the overall performance of your application since they consume threads from the same pool used by ASP.NET to handle HTTP requests.
Then I wanted to use a new Task or async/await to write IHttpAsyncHandler. One example you can find is what Drew Marsh explains here : https://stackoverflow.com/a/6389323/261950
My guess is that using Task or async/await still consume the thread from the ASP.NET thread pool and I don't want for the obvious reason.
Could you please tell me if I can use Task (async/await) on the background thread like with System.Threading.Thread class and not from thread pool ?
Thanks in advance for your help.
Thomas
This situation is where Task, async, and await really shine. Here's the same example, refactored to take full advantage of async (it also uses some helper classes from my AsyncEx library to clean up the mapping code):
// First, a base class that takes care of the Task -> IAsyncResult mapping.
// In .NET 4.5, you would use HttpTaskAsyncHandler instead.
public abstract class HttpAsyncHandlerBase : IHttpAsyncHandler
{
public abstract Task ProcessRequestAsync(HttpContext context);
IAsyncResult IHttpAsyncHandler.BeginProcessRequest(HttpContext context, AsyncCallback cb, object extraData)
{
var task = ProcessRequestAsync(context);
return Nito.AsyncEx.AsyncFactory.ToBegin(task, cb, extraData);
}
void EndProcessRequest(IAsyncResult result)
{
Nito.AsyncEx.AsyncFactory.ToEnd(result);
}
void ProcessRequest(HttpContext context)
{
EndProcessRequest(BeginProcessRequest(context, null, null));
}
public virtual bool IsReusable
{
get { return true; }
}
}
// Now, our (async) Task implementation
public class MyAsyncHandler : HttpAsyncHandlerBase
{
public override async Task ProcessRequestAsync(HttpContext context)
{
using (var webClient = new WebClient())
{
var data = await webClient.DownloadDataTaskAsync("http://my resource");
context.Response.ContentType = "text/xml";
context.Response.OutputStream.Write(data, 0, data.Length);
}
}
}
(As noted in the code, .NET 4.5 has a HttpTaskAsyncHandler which is similar to our HttpAsyncHandlerBase above).
The really cool thing about async is that it doesn't take any threads while doing the background operation:
An ASP.NET request thread kicks off the request, and it starts downloading using the WebClient.
While the download is going, the await actually returns out of the async method, leaving the request thread. That request thread is returned back to the thread pool - leaving 0 (zero) threads servicing this request.
When the download completes, the async method is resumed on a request thread. That request thread is briefly used just to write the actual response.
This is the optimal threading solution (since a request thread is required to write the response).
The original example also uses threads optimally - as far as the threading goes, it's the same as the async-based code. But IMO the async code is easier to read.
If you want to know more about async, I have an intro post on my blog.
I've been looking for information through internet for a couple of days. Let me sum up what I found until now :
ASP.NET ThreadPool facts
As Andres said: When async/await will not consume an additional ThreadPool thread ? Only in the case you are using BCL Async methods. that uses an IOCP thread to execute the IO bound operation.
Andres continues with ...If you are trying to async execute some sync code or your own library code, that code will probably use an additional ThreadPool thread unless you explicitely use the IOCP ThreadPool or your own ThreadPool.
But as far as I know you can't chose whetever you want to use a IOCP thread, and making correct implementation of the threadPool is not worth the effort. I doubt someone does a better one that already exists.
ASP.NET uses threads from a common language runtime (CLR) thread pool to process requests. As long as there are threads available in the thread pool, ASP.NET has no trouble dispatching incoming requests.
Async delegates use the threads from ThreadPool.
When you should start thinking about implementing asynchronous execution ?
When your application performs relatively lengthy I/O operations (database queries, Web service calls, and other I/O operations)
If you want to do I/O work, then you should be using an I/O thread (I/O Completion Port) and specifically you should be using the async callbacks supported by whatever library class you're using. Theirs names start with the words Begin and End.
If requests are computationally cheap to process, then parallelism is probably an unnecessary overhead.
If the incoming request rate is high, then adding more parallelism will likely yield few benefits and could actually decrease performance, since the incoming rate of work may be high enough to keep the CPUs busy.
Should I create new Threads ?
Avoid creating new threads like you would avoid the plague.
If you are actually queuing enough work items to prevent ASP.NET from processing further requests, then you should be starving the thread pool! If you are running literally hundreds of CPU-intensive operations at the same time, what good would it do to have another worker thread to serve an ASP.NET request, when the machine is already overloaded.
And the TPL ?
TPL can adapt to use available resources within a process. If the server is already loaded, the TPL can use as little as one worker and make forward progress. If the server is mostly free, they can grow to use as many workers as the ThreadPool can spare.
Tasks use threadpool threads to execute.
References
http://msdn.microsoft.com/en-us/magazine/cc163463.aspx
http://blogs.msdn.com/b/pfxteam/archive/2010/02/08/9960003.aspx
https://stackoverflow.com/a/2642789/261950
Saying that "0 (zero) threads will be servicing this request" is not accurate entirely.
I think you mean "from the ASP.NET ThreadPool", and in the general case that will be correct.
When async/await will not consume an additional ThreadPool thread?
Only in the case you are using BCL Async methods (like the ones provided by WebClient async extensions) that uses an IOCP thread to execute the IO bound operation.
If you are trying to async execute some sync code or your own library code, that code will probably use an additional ThreadPool thread unless you explicitely use the IOCP ThreadPool or your own ThreadPool.
Thanks,
Andrés.
The Parallel Extensions team has a blog post on using TPL with ASP.NET that explains how TPL and PLINQ use the ASP.NET ThreadPool. The post even has a decision chart to help you pick the right approach.
In short, PLINQ uses one worker thread per core from the threadpool for the entire execution of the query, which can lead to problems if you have high traffic.
The Task and Parallel methods on the other hand will adapt to the process's resources and can use as little as one thread for processing.
As far as the Async CTP is concerned, there is little conceptual difference between the async/await construct and using Tasks directly. The compiler uses some magic to convert awaits to Tasks and Continuations behind the scenes. The big difference is that your code is much MUCH cleaner and easier to debug.
Another thing to consider is that async/await and TPL (Task) are not the same thing.
Please read this excellent post http://blogs.msdn.com/b/ericlippert/archive/2010/11/04/asynchrony-in-c-5-0-part-four-it-s-not-magic.aspx to understand why async/await doesn't mean "using a background thread".
Going back to our topic here, in your particular case where you want to perform some expensive calculations inside an AsyncHandler you have three choices:
1) leave the code inside the Asynchandler so the expensive calculation will use the current thread from the ThreadPool.
2) run the expensive calculation code in another ThreadPool thread by using Task.Run or a Delegate
3) Run the expensive calculation code in another thread from your custom thread pool (or IOCP threadPool).
The second case MIGHT be enough for you depending on how long your "calculation" process run and how much load you have. The safe option is #3 but a lot more expensive in coding/testing. I also recommend always using .NET 4 for production systems using async design because there are some hard limits in .NET 3.5.
There's a nice implementation of HttpTaskAsyncHandler for the .NET 4.0 in the SignalR project. You may want to ckeck it out: http://bit.ly/Jfy2s9