I was reading the documentation of Microsoft specifically the Async programming article and I didn't understand this section while he is explaining the work of the server's threads when using Async code.
because it(The server) uses async and await, each of its threads is freed up when the I/O-bound work starts, rather than when it finishes.
Could anyone help what does it mean by the threads r freed up when the I/O starts??
Here is the article : https://learn.microsoft.com/en-us/dotnet/standard/async-in-depth
When ASP.NET gets an HTTP request, it takes a thread from the thread pool and uses that to execute the handler for that request (e.g., a specific controller action).
For synchronous actions, the thread stays assigned to that HTTP request until the action completes. For asynchronous actions, the await in the action method may cause the thread to return an incomplete task to the ASP.NET runtime. In this case, ASP.NET will free up the thread to handle other requests while the I/O is in flight.
Further reading about the difference between synchronous and asynchronous request handling and how asynchronous work doesn't require a thread at all times.
When your application makes a call to an external resource like Database or HttpClient thread, that initiated connection needs to wait.
Until it gets a response, it waits idly.
In the asynchronous approach, the thread gets released as soon as the app makes an external call.
Here is an article about how it happens:
https://medium.com/#karol.rossa/asynchronous-programming-73b4f1988cc6
And performance comparison between async and sync apporach
https://medium.com/#karol.rossa/asynchronous-performance-1be01a71925d
Here's an analogy for you: have you ever ordered at a restaurant with a large group and had someone not be ready to order when the waiter came to them? Did they bring in a different waiter to wait for him or did the waiter just come back to him after he took other people's orders?
The fact that the waiter is allowed to come back to him later means that he's freed up immediately after calling on him rather than having to wait around until he's ready.
Asynchronous I/O works the same way. When you do a web service call, for example, the slowest part (from the perspective of the client at least) is waiting for the result to come back: most of the delay is introduced by the network (and the other server), during which time the client thread would otherwise have nothing to do but wait. Async allows the client to do other things in the background.
Related
Do I need to do anything to make all requests asynchronous or are they automatically handled that way?
I ran some tests and it appears that each request comes in on its own thread, but I figure better to ask as I might have tested wrong.
Update: (I have a bad habit of not explaining fully - sorry) Here's my concern. A client browser makes a REST request to my server of http://data.domain/com/employee_database/?query=state:Colorado. That comes in to the appropriate method in the controller. That method queries the database and returns an object which is then turned into a JSON structure and returned to the calling app.
Now let's say 10,000 clients all make a similar query to the same server. So I have 10,000 requests coming in at once. Will my controller method be called simultaneously in 10,000 distinct threads? Or must the first request return before the second request is called?
I'm not asking about the code in my handler method having asynchronous components. For my case the request becomes a single SQL query so the code has nothing that can be handled asynchronously. And until I get the requested data, I can't return from the method.
No REST is not async by default. the request are handled synchronously. However, your web server (IIS) has a number of max threads setting which can work at the same time, and it maintains a queue of the request received. So, the request goes in the queue and if a thread is available it gets executed else, the request waits in the IIS queue till a thread is available
I think you should be using async IO/operations such as database calls in your case. Yes in Web Api, every request has its own thread, but threads can run out if there are many consecutive requests. Also threads use memory so if your api gets hit by too many request it may put pressure on your system.
The benefit of using async over sync is that you use your system resources wisely. Instead of blocking the thread while it is waiting for the database call to complete in sync implementation, the async will free the thread to handle more requests or assign it what ever process needs a thread. Once IO (database) call completes, another thread will take it from there and continue with the implementation. Async will also make your api run faster if your IO operations take longer to complete.
To be honest, your question is not very clear. If you are making an HTTP GET using HttpClient, say the GetAsync method, request is fired and you can do whatever you want in your thread until the time you get the response back. So, this request is asynchronous. If you are asking about the server side, which handles this request (assuming it is ASP.NET Web API), then asynchronous or not is up to how you implemented your web API. If your action method, does three things, say 1, 2, and 3 one after the other synchronously in blocking mode, the same thread is going to the service the request. On the other hand, say #2 above is a call to a web service and it is an HTTP call. Now, if you use HttpClient and you make an asynchronous call, you can get into a situation where one request is serviced by more than one thread. For that to happen, you should have made the HTTP call from your action method asynchronously and used async keyword. In that case, when you call await inside the action method, your action method execution returns and the thread servicing your request is free to service some other request and ultimately when the response is available, the same or some other thread will continue from where it was left off previously. Long boring answer, perhaps but difficult to explain just through words by typing, I guess. Hope you get some clarity.
UPDATE:
Your action method will execute in parallel in 10,000 threads (ideally). Why I'm saying ideally is because a CLR thread pool having 10,000 threads is not typical and probably impractical as well. There are physical limits as well as limits imposed by the framework as well but I guess the answer to your question is that the requests will be serviced in parallel. The correct term here will be 'parallel' but not 'async'.
Whether it is sync or async is your choice. You choose by the way to write your action. If you return a Task, and also use async IO under the hood, it is async. In other cases it is synchronous.
Don't feel tempted to slap async on your action and use Task.Run. That is async-over-sync (a known anti-pattern). It must be truly async all the way down to the OS kernel.
No framework can make sync IO automatically async, so it cannot happen under the hood. Async IO is callback-based which is a severe change in programming model.
This does not answer what you should do of course. That would be a new question.
I have a variation of the benefits-of-async/await-on-ASP.NET from this question.
My understanding is that asynchrony is not the same thing as parallelism. So, on a web server, I wonder about how much benefit async/await brings to ASP.NET pages.
Isn't IIS+ASP.NET already really good at allocating threads for requests, and if onen page is busy waiting for a resource the server will just switch to processing another request that has work to do?
There are a limited number of threads in the pool for ASP.NET to use - does async use them any more effectively?
As Mr. Skeet pointed out in answering the question above, we're not talking about blocking a UI thread. We're already multi-threaded and the web response can't be completed until all the request's tasks are done, async or not, right?
I guess what it boils down to is this:
Is there any benefit to an async read of a resource (say a file or DB request) in an ASP.NET page vs. blocking on it?
if one page is busy waiting for a resource the server will just switch to processing another request that has work to do?
I don't think so. I would be very surprised if this were the case. It's theoretically possible, but very complex.
There are a limited number of threads in the pool for ASP.NET to use - does async use them any more effectively?
Yes, because when you await something, the thread for that request is immediately returned to the pool.
We're already multi-threaded and the web response can't be completed until all the request's tasks are done, async or not, right?
That is correct. async in a server scenario is all about removing pressure on the thread pool.
Is there any benefit to an async read of a resource (say a file or DB request) in an ASP.NET page vs. blocking on it?
Absolutely!
If you block on a file/service call/db request, then that thread is used for the duration of that operation. If you await a file/service call/db request, then that thread is immediately returned to the thread pool.
One (really cool!) result of this is that you can have a request in progress, and while it's (a)waiting some operation, there are no threads servicing that request! Zero-threaded concurrency, if you will.
When the operation completes, the method resumes after the await - on a (possibly different) thread from the thread pool.
In conclusion: async scales better than threads, so there is definitely a benefit on the server side.
More info: my own intro to async post and this awesome video.
This comment by Stephen Cleary says this:
AspNetSynchronizationContext is the strangest implementation. It treats Post as synchronous rather than asynchronous and uses a lock to execute its delegates one at a time.
Similarly, the article that he wrote on synchronization contexts and linked to in that comment suggests:
Conceptually, the context of AspNetSynchronizationContext is complex. During the lifetime of an asynchronous page, the context starts with just one thread from the ASP.NET thread pool. After the asynchronous requests have started, the context doesn’t include any threads. As the asynchronous requests complete, the thread pool threads executing their completion routines enter the context. These may be the same threads that initiated the requests but more likely would be whatever threads happen to be free at the time the operations complete.
If multiple operations complete at once for the same application, AspNetSynchronizationContext will ensure that they execute one at a time. They may execute on any thread, but that thread will have the identity and culture of the original page.
Digging in reflector seems to validate this as it takes a lock on the HttpApplication while invoking any callback.
Locking the app object seems like scary stuff. So my first question: Does that mean that today, all asynchronous completions for the entire app execute one at a time, even ones that originated from separate requests on separate threads with separate HttpContexts? Wouldn't this be a huge bottleneck for any apps that make 100% use of async pages (or async controllers in MVC)? If not, why not? What am I missing?
Also, in .NET 4.5, it looks like there's a new AspNetSynchronizationContext, and the old one is renamed LegacyAspNetSynchronizationContext and only used if the new app setting UseTaskFriendlySynchronizationContext is not set. So question #2: Does the new implementation change this behavior? Otherwise, I imagine with the new async/await support marshaling completions through the synchronization context, this kind of bottleneck would be noticed much more frequently going forward.
The answer to this forum post (linked from SO answer here) suggests that something fundamentally changed here, but I want to be clear on what that is and what behaviors have improved, since we have a .NET 4 MVC 3 app which is pretty much 100% async action methods making web service calls.
Let me answer your first question. In your assumption you didn't consider the fact that separate ASP.NET requests are processed by different HttpApplication objects. HttpApplication objects are stored in pool. Once you request a page, an application object is retrieved from pool and belongs to the request till its completion. So, my answer to your question:
all asynchronous completions for the entire app execute one at a time, even ones that originated from separate requests on separate threads with separate HttpContexts
is: No, they don't
Separate requests are processed by separate HttpApplication objects, locked HttpApplication will affect only single request.
Synchronization context is a powerful thing that helps developers to synchronize access to shared (in scope of request) resources. That is why all callbacks are executed under lock. Synchronization context is a heart of event-based synchronization pattern.
I was working on a project and there is bulk e-mail sending part of it and when user clicks on the button, he/she gets the "Thanks the e-mails have been sent" immediately as a Response and the same method is firing an async thread as well.
ThreadPool.QueueUserWorkItem(SendEMail, _message);
This thread is queued when user clicks on the send button but since this is default Background Thread I was expecting when the page response ended, this thread would be terminated but it didn't happen because the current thread which fires this Thread was also a Background Thread and a Worker Thread, so which means there is an unfinished foreground thread(s) (Could be MainThread or Worker Threads) still alive but I don't know when they finish because their finish time will effect my background worker threads; When the last foreground thread ends, it causes the process to be terminated, so do background threads.
Should I be afraid of this or Asp.NET can handle it automatically and I am kinda confused because I've read a lot of things about it and now everything is mixed up.
Could you please clarify things a little bit ?
Thanks in advance.
Using the ThreadPool for long-running tasks will negatively influence the performance of your application (because ASP.NET also uses the ThreadPool to handle incoming requests).
Creating threads manually for each task can also become a problem if too much threads are created.
One technique I've used in the past, is to create a custom ThreadPool when starting the application, and enqueuing tasks on that ThreadPool. A simple custom ThreadPool could consist of a single Thread and a Queue of tasks.
When you call QueueUserWorkItem a new thread will be drawn from the thread pool if available and execute the callback function on this thread. If no threads are available, the function will block until a thread is freed. Once it finishes the execution of the method the thread will be suspended and returned to the pool for further reuse. Your only concern should be the fact that threads from the pool are also used to service requests, so if you perform lengthy tasks on them you could jeopardize the whole system request servicing capabilities because there's a limited number of threads in the pool and if all of them are busy performing lengthy operations future HTTP requests will be queued. As an alternative you could consider creating threads manually: new Thread(state => { }).Start();
You can use MSMQ for creating your email queue and have a single thread process the queue.
The following post might be useful as it fits your problem domain. Although it does not use MSMQ but is a good post on processing scheduled tasks using ASP.net.
Simulate a Windows Service using ASP.NET to run scheduled jobs
Is it possible to use BackGroundWorker thread in ASP.NET 2.0 for the following scenario, so that the user at the browser's end does not have to wait for long time?
Scenario
The browser requests a page, say SendEmails.aspx
SendEmails.aspx page creates a BackgroundWorker thread, and supplies the thread with enough context to create and send emails.
The browser receives the response from the ComposeAndSendEmails.aspx, saying that emails are being sent.
Meanwhile, the background thread is engaged in a process of creating and sending emails which could take some considerable time to complete.
My main concern is about keeping the BackgroundWorker thread running, trying to send, say 50 emails while the ASP.NET workerprocess threadpool thread is long gone.
If you don't want to use the AJAX libraries, or the e-mail processing is REALLY long and would timeout a standard AJAX request, you can use an AsynchronousPostBack method that was the "old hack" in the .net 1.1 days.
Essentially what you do is have your submit button begin the e-mail processing in an asynchronous state, while the user is taken to an intermediate page. The benefit to this is that you can have your intermediate page refresh as much as needed, without worrying about hitting the standard timeouts.
When your background process is complete, it will put a little "done" flag in the database/application variable/whatever. When your intermediate page does a refresh of itself, it detects this flag and automatically redirects the user to the "done" page.
Again, AJAX makes all of this moot, but if for some reason you have a very intensive or timely process that has to be done over the web, this solution will work for you. I found a nice tutorial on it here and there are plenty more out there.
I had to use a process like this when we were working on a "web check-in" type application that was interfacing with a third party application and their import API was hideously slow.
EDIT: GAH! Curse you Guzlar and your god-like typing abilities 8^D.
You shouldn't do any threading from ASP.NET pages. Any thread that is long running is in danger of being killed when the worker process recycles. You can't predict when this will happen. Any long-running processes need to be handled by a windows service. You can kick off these processes by dropping a message in MSMQ, for example.
ThreadPool.QueueUserWorkItem(delegateThatSendsEmails)
or on System.Net.Mail.SmtpServer use the SendAsync method.
You want to put the email sending code on another thread, because then it will return the the user immediately, and will just process, no matter how long it takes.
It is possible. Once you start a new thread asynchronously from page, page request will proceed and send the page back to the user. The async thread will continue to run on the server but will no longer have access to the session.
If you have to show task progress, consider some Ajax techniques.
What you need to use for this scenario is Asynchronous Pages, a feature that was added in ASP.NET 2.0
Asynchronous pages offer a neat
solution to the problems caused by
I/O-bound requests. Page processing
begins on a thread-pool thread, but
that thread is returned to the thread
pool once an asynchronous I/O
operation begins in response to a
signal from ASP.NET. When the
operation completes, ASP.NET grabs
another thread from the thread pool
and finishes processing the request.
Scalability increases because
thread-pool threads are used more
efficiently. Threads that would
otherwise be stuck waiting for I/O to
complete can now be used to service
other requests. The direct
beneficiaries are requests that don't
perform lengthy I/O operations and can
therefore get in and out of the
pipeline quickly. Long waits to get
into the pipeline have a
disproportionately negative impact on
the performance of such requests.
http://msdn.microsoft.com/en-us/magazine/cc163725.aspx
If you want using multitheading in your ASP page, you might using simple threading model like this:
{
System.Threading.Thread _thread = new Thread(new ThreadStart(Activity_DoWork));
_thred.Start();
}
Activity_DoWork()
{
/*Do some things...
}
This method is correct working with ASP pages. The ASP page with BackgroundWorker will not start while BackgroundWorker will finish.
5 years later, but problems the same… If you want to perform fire-and-forget operations from your application and forget about all difficulties related to background job processing in ASP.NET applications, you can use http://hangfire.io.
It does not loose your jobs on recycling process, because it uses persistent storage to keep information about background jobs.
It automatically retries your background jobs that were aborted or failed due to transient exception (SMTP Server connectivity errors).
It allows you to easily debug background jobs through the integrated web interface.
It is very easy to install/configure/use HangFire.
There is also tutorial Sending Mail in Background with ASP.NET MVC for using HangFire with Postal.