I'm somewhat vague about when to use asynchronous operations in ASP.NET. I understand, whenever I make a call to external web services such as calling Twitter APIs and what not, I should be using asynchronous operations so the CLR threads could be freed and service other requests, make sense.
I read an excellent blog once mentioned that if your operation is using CPU efficiently, then you shouldn't do asynchronous operation because it has a context switch penalty, however, if you are doing a long operation and waits a lot, then it's worth to do the context switch.
What about a page that use ajax call to a local web service which in turn makes a database operations (takes around 3 seconds), returns JSON and then the page itself using JQuery, renders it for another second for a total of 4 seconds?
What about a traditional webform, from page_load makes the same database operation call (3 seconds) and then take another 3 seconds to render? For instance, a big forum post with 1000 comments?
My general impression is that shouldn't IIS be designed such that EVERY operation is asynchronous by default in the background such that all operation is non-blocking without the context switch penalty? Is that the idea of node.js? and If you do have static pages, no wait operations, only then should you specifically write a synchronous page? Basically the reverse of what is happening now?
Thanks a lot.
The only benefit of asynchronous requests is that it frees up worker threads. So you only notice it if you run out of worker threads. By default, there are 100 threads per CPU. So you won't notice the performance improvement unless you have more than 100 requests per 4 seconds per CPU.
The drawback of asynchronous requests is that it makes your code harder to understand and maintain.
Related
I understand the benefits of async on the frontend (some kind of UI). With async the UI does not block and the user can "click" on other things while waiting for another operation to execute.
But what is the benefit of async programming on the backend?
The main benefit is that there could be various slow operations on the backend, which can prevent other requests from using the cpu at the same time. These operations could be: 1. Database operations 2. File operations 3. Remote calls to other services (servers), etc. You don't want to block the cpu while these operations are in progress.
First of all, there a benefit of handling more than one request at a time. Frameworks like ASP.net or django create new (or reuse existing) threads for each requests.
If you mean async operations from the thread of particular request, that's more complicated. In most cases, it does not help at all, because of the overhead of spawning new thread. But, we have things like Schedulers in C# for example, which help a lot. When correctly used, they free up a lot of CPU time normally wasted on waiting.
For example, you send a picture to a server. Your request is handled in new thread. This thread can do everything on it's own: upack the picture and save it to disk, then update the database.
Or, you can write to disk AND update the database at the same time. The thread that is done first is our focus here. When used without scheduler, it starts spinning a loop, checking if the other thread is done, which takes CPU time. If you use scheduler, it frees that thread. When the other task is done, it uses propably yet another precreated thred to finish handling of your request.
That scenario does make it seem like it's not worth the fuss, but it is easy to imagine more coplicated tasks, that can be done in the same time instead of sequentailly. On top of that, schedulers are rather smart and will make it so the total time needed will be lowest, and CPU usage moderate.
I am working on a .net client which consumes a REST api (JSON). The client is a web application with high traffic and considerable user interaction
When writing the wrappers around the external REST api i am trying to decide
If i should make all calls made for the API async? This will be all the way from UI to the API as explained here http://blog.stephencleary.com/2012/07/dont-block-on-async-code.html. This will help me in achieving the desired performance but i will have to figure out a way to handle UI when Tasks are waiting to be completed
Or Is it an overkill? And do i just use sync/sequential code? I could still get (some) performance by Horizontally scaling the application?
I am keen to understand what is the preferred way of calling an external REST api from a client (if there is any) and how is the UI handled for those case where people do use async?
So you have about 10 requests per second in a busy period. This by itself does not require asynchronous IO. Assuming 1sec per request that's 10 threads. 10 threads are nothing.
There is one special case, though: What if the backend service you are calling sometimes takes a long time to respond (a bug, overload, index rebuild, ...)? If it takes 30 seconds to respond or timeout that means that 300 requests are in flight. This is too much for the default thread-pool settings. This will effectively shut down the entire app by pool exhaustion until the requests are cleared.
You can do two things:
Use async IO for all high-volume actions.
Reduce timeouts and have a load-breaker for the number of in-flight requests. Example:
.
SemaphoreSlim sem = new SS(50); //max 50 in-flight
//...
if (!sem.WaitOne(TimeSpan.Zero))
throw new Exception("Load limit exceeded");
Both are equally safe and well-performing. Do not make the mistake to think that async IO causes your IOs to become faster.
The semaphore solution requires less architectural changes but it requires permission to drop requests. These requests would not have completed anyway with high likelihood.
Benefits of reducing threadcount
It is fashionable these days to attempt to improve the scalability of web-servers by reducing the number of threads handling requests, in an effort to prevent unnecessary context switching from having tens or hundreds of threads. This is particularly important when waiting on some kind of application event for a 'long polling' (aka COMET) request - it simply is rude to consume all the resources of an entire thread and do nothing with it.
Node.ns as an extreme example
Node.ns is a rather extreme example of reducing thread-count, requiring handlers to give up the single thread handling all HTTP requests when waiting on any event (even I/O from a local disk!). In this way, a CPU core is kept busy doing useful computational work, instead of 'wasting' instructions pausing and resuming many different threads.
But API support is needed
However, arranging for resumption of the handler once the waited-for event has completed typically requires explicit support in the handler API. For example, there is a part of the Java Servlet Specification which deals specifically with 'asynchronous' responses.
Wherefore, ASP.NET?
So, does ASP.NET have a similar API for asynchronous responses (aka. continuations, aka. (a bit loosely) 'reactor pattern')? Is there authoritative documentation which directly addresses this matter and its interaction with IIS? What nomenclature does ASP.NET / IIS use to refer to these properties?
I am coming to IIS/.NET shortly, and don't want to write any code that's unnecessarily 'thread-per-request' if I can help it. I'll be using C# by preference.
With regards to asynchronous ASP.NET web pages article on MSDN.
The advantages are obvious with long-running pages or high server load. So, given projects where you think demand may be high somewhere down the track, when usage grows, is there any reason NOT to implement async ASP.NET in every web application as a standard? Are there any disadvantages to the approach?
Secondary question: are there any real-world studies/examples of where the advantages start to appear, in different web app situations? Or is it just a matter of suck it and see?
From your own link:
Only I/O-bound operations are good candidates for becoming async action methods on an asynchronous controller class. An I/O-bound operation is an operation that doesn’t depend on the local CPU for completion. When an I/O-bound operation is active, the CPU just waits for data to be processed (that is, downloaded) from external storage (a database or a remote service). I/O-bound operations are in contrast to CPU-bound operations, where the completion of a task depends on the activity of the CPU.
Async pages are not free, they do come at a price. They are generally good when your page is making an external call to a service or performing some long-running, non-CPU bound, operation. Otherwise, you are likely to thrash the CPU, leaving you with a worse situation that you had before going async.
The idea is to use async when you would be eating up a thread from your application's thread pool doing non-CPU intensive work (waiting for a response from a long-running service). That way, your application can continue processing requests and doesn't start queuing new ones, slowly draining the responsiveness from your app.
Here is another link with information when/when not to use async pages.
Edit
As for what is considered "long running," you're faced with the crummy answer of "It depends." To figure this out, you would need to profile your application, see how many of your "long running" requests cause subsequent requests to be queued, instead of processed, by IIS. The decision comes down to being in a situation in which paying the costly toll of context switching is less than the return you're going to get for doing so. If your bottleneck is a certain page or service that causes incoming requests to be held off, it is probably a good idea to start thinking about async work. But, you might also be doing too much work in the request and it could be a "code smell" that you need to refactor your code.
In the end, It depends.
Here is an exerpt from MSDN.
In general, use asynchronous pipelines when the following conditions are true:
The operations are network-bound or I/O-bound instead of CPU-bound.
Testing shows that the blocking operations are a bottleneck in site performance and that IIS can service more requests by using asynchronous action methods for these blocking calls.
Parallelism is more important than simplicity of code.
You want to provide a mechanism that lets users cancel a long-running request.
While the link is about MVC, the idea holds true for other flavors of ASP.NET, too.
What kind of multi-threading issues do you have to be careful for in asp.net?
It's risky to spawn threads from the code-behind of an ASP.NET page, because the worker process will get recycled occasionally and your thread will die.
If you need to kick off long-running processes as a result of user actions on web pages, your best bet is to drop a message off in MSMQ and have a separate background service monitoring the queue. The service could take as long as it wants to accomplish the task, and the web page would be finished with its work almost immediately. You could accomplish the same thing with an asynch call to a web method, but don't rely on getting the response when the web method is finished working. From code-behind, it needs to be a quick fire-and-forget.
One thing to watch out for at things that expire (I think httpContext does), if you are using it for operations that are "fire and forget" remember that all of a sudden if the asp.net cleanup code runs before your operation is done, you won't be able to access certain information.
If this is for a web service, you should definitely consider thread pooling. Too many threads will bring your application to a grinding halt because they will eventually start competing for CPU time.
Is this for file or network IO? If so, you should also consider using asynchronous IO. It can be a bit more of a pain to program, but you don't have to worry about spawning off too many threads at once.
Programmatic Caching is one area which immediately comes to my mind. It is a great feature which needs to be used carefully. Since it is shared across requests, you have to put locks around it before updating it.
Another place I would check is any code accessing filesystem like writing to log files. If one request has a read-write lock on a file, other concurrent requests will error out if not handled properly.
Isn't there a Limit of 25 Total Threads in the IIS Configuration? At least in IIS 6 i believe. If you exceed that limit, interesting things (read: loooooooong response times) may happen.
Depending on what you need, as far as multi threading is concerned, have you thought of spawning requests from the client. It's safe to spawn requests using AJAX, and then act on the results in a callback. Or use a service as a backgrounding mechanism, which runs every X minutes and processes in the background that way.