I am looking to to make my web application scalable and was going through articles in which it was said that the queue length should be increased by a factor of say 1.5 * the number of users.
Settings:
IIS-> Application Pool Default Settings -> Queue Length
IIS ->ASP.NET-> Queue Length
Question:
What is the difference between the queue length properties?
I am aware that one says it handles HTTP.sys requests and the other concurrent asp.net requests. However, how will it help an application scale i.e. will changing only one suffice or both are performing some different role.
The application pool can handle multiple websites, so the queue is for every site inside the pool. The ASP.NET queue is restricted for that website.
Answer 1: Appilcation Pool Default Settings -> Queue Length:
Optional uint attribute.
Indicates to HTTP.sys how many requests to queue for an application
pool before rejecting future requests.
When the value set for this property is exceeded, IIS rejects
subsequent requests with a 503 error. If the loadBalancerCapabilities
setting is true, the connection is closed instead of rejecting
requests with a 503.
https://learn.microsoft.com/en-us/iis/configuration/system.applicationhost/applicationpools/applicationpooldefaults/
Answer 2: IIS -> ASP.NET -> Queue Length:
Its the same, but related to the website selected.
Related
When you set the httpRuntime executionTimeout, does that cut off a response that has already been partially sent to the client?
I've noticed in the IIS request queue that occasionally there are some requests that run for a lot longer than our executionTimeout setting and the state is SendResponse in the IIS Web Core module. Does the executionTimeout include the time that it takes to send the response to the client?
The only documentation that I could find is on the following page in this snip, but it is from 2003 for older versions of IIS:
https://msdn.microsoft.com/en-us/library/ms972959.aspx
Request Execution Time. The number of milliseconds taken to execute the last request. In version 1.0 of the Framework, the execution time begins when the worker process receives the request, and stops when the ASP.NET ISAPI sends HSE_REQ_DONE_WITH_SESSION to IIS. For IIS version 5, this includes the time taken to write the response to the client, but for IIS version 6, the response buffers are sent asynchronously, and so the time taken to write the response to the client is not included.
The IIS version is 7.5 on Windows Server 2008 R2 running an ASP.NET 4.5 web application and debug is set to false.
I apologize for not addressing your question directly, but there is something that might help with your problem:
There's a minBytesPerSecond setting for setting a lower bound on transmission speed in the <webLimits> section. I suspect it was originally added as a defense against SlowLoris, but you can bump the value up to kick out users whose pipe can't support your site. The default value is 240, which is pretty low.
Our front-end MVC3 web application is using AsyncController, because each of our instances is servicing many hundreds of long-running, IO bound processes.
Since Azure will terminate "inactive" http sessions after some pre-determined interval (which seems to vary depending up on what website you read), how can we keep the connections alive?
Our clients MUST stay connected, and our processes will run from 30 seconds to 5 minutes or more. How can we keep the client connected/alive? I initially thought of having a timeout on the Async method, and just hitting the Response object with a few bytes of output, sort of like chunking the response, and then going back and waiting some more. However, I don't think this will work, since MVC3 is handling the hookup of an IIS thread back to the asynchronous response, which will have already rendered a view at that time.
How can we run a really long process on an AsyncController, but have the client not be disconnected by the Azure Load Balancer? Sending an immediate response to the caller, and asking that caller to poll or check another resource URL is not acceptable.
Azure load balancer idle time-out is 4 minutes. Can you try to configure TCP keep-alive on the client side for less than 4 minutes, that should keep the connection alive?
On the other hand, it's pretty expensive to keep a connection open per client for a long time. This will limit the number of clients you can handle per server. Also, I think IIS may still decide to close a connection regardless of keep-alives if it thinks it need the connection to serve other requests.
Question applies to IIS 6.0 and .Net 3.5
ASP.Net maintains a request queue for each AppDomain. Requests are moved to this queue if the CLR threads are busy executing.
Assumption 1:
My assumption is this request queue size is derived from IIS 6.0 metabase property called "AspRequestQueueMax" which has a default value of 3000.
Is this assumption correct?
The number of worker threads & IO threads that are available to execute a request and to accept incoming requests/execute callbacks can be set at Worker Process Level by using "processModel" element of IIS 6.0 metabase.
Assumption 2:
There is another property called "AspProcessorThreadMax" which is applied at each Virtual Directory level and hence (assumption) at each AppDomain level.
Now if my assumption (Assumption 2) holds good then:
1. Irrespective of number of max/min worker threads configured at a Worker process level, does each domain restrict the number of CLR threads to the value configured for AspProcessorThreadMax?
Or
Is it that this property only applies to ASP requests and not ASP.Net?
The AspXXX properties are ASP specific. The size of the various request queues used by ASP.NET is configured through settings in machine.config.
Specifically, the global request queue limit is configured by the <processModel requestQueueLimit> property in machine.config. The application request queue limit is configured in Machine.config by the <httpRuntime appRequestQueueLimit> property.
The following article by Thomas Marquardt describes how IIS handles ASP.Net requests, the max/min CLR worker threads/Managed IO threads that can be configured to run, the various requests queues involved and their default sizes.
Now as per the article, the following occurs in IIS 6.0:
ASP.NET picks up the requests from a IIS IO Thread and posts "HSE_STATUS_PENDING" to IIS IO Thread
The requests is handed over to a CLR Worker thread
If the requests are of high latency and all the threads are occupied (the thread count approaches httpRuntime.minFreeThreads), then the requests are posted to the Application level request queue (this queue is per AppDomain)
Also ASP.NET checks the number of concurrently executing requests. The article states that "if the number of concurrently executing requests is too high" it queues the incoming requests to a ASP.NET global request queue (this is per Worker Process) (Please check Update 2)
I want to know what is the "threshold value" at which point ASP.NET considers that the number of requests currently executing it too high and then starts queuing the requests to the global ASP.NET request queue?
I think this threshold will depend upon the configuration of max number of worker threads, but there might be some formula based on which ASP.NET will determine that the number of concurrently executing requests is too high and starts queuing the requests to the ASP.NET global request queue. What might this formula? or is this setting configurable?
Update
I read through the article again and in the comments sections I found this:
1) On IIS 6 and in IIS 7 classic mode, each application (AppDomain)
has a queue that it uses to maintain the availability of worker
threads. The number of requests in this queue increases if the number
of available worker threads falls below the limit specified by
httpRuntime minFreeThreads. When the limit specified by
httpRuntime appRequestQueueLimit is exceeded, the request is
rejected with a 503 status code and the client receives an
HttpException with the message "Server too busy." There is also an
ASP.NET performance counter, "Requests In Application Queue", that
indicates how many requests are in the queue. Yes, the CLR thread
pool is the one exposed by the .NET ThreadPool class.
2) The requestQueueLimit is poorly named. It actually limits the
maximum number of requests that can be serviced by ASP.NET
concurrently. This includes both requests that are queued and
requests that are executing. If the "Requests Current" performance
counter exceeds requestQueueLimit, new incoming requests will be
rejected with a 503 status code.
So essentially requestQueueLimit limits the number of requests that are queued (I am assuming it will sum the number of requests queued in Application queues plus the global ASP.Net request queue plus the number of requests currently executing) and are executing. All though this does not answer the original question, it does provide information about when we might receive a 503 server busy error because of high number of concurrent requests/high latency requests.
(Check update 2)
Update 2
There was a mistake in my part in the understanding. I had mixed up the descriptions for IIS 6 and IIS 7.
Essentially when ASP.NET is hosted on IIS 7.5 and 7.0 in integrated mode, the application-level queues are no longer present, ASP.NET maintains a global request queue.
So IIS 7/7.5 will start to queue requests to the global request queue if the number of executing requests is deemed high. The question applies more to IIS 7/7.5 rather than 6.
As far IIS 6.0 is concerned, there is no global ASP.NET request queue, but the following is true:
1. ASP.NET picks up the requests from a IIS IO Thread and posts "HSE_STATUS_PENDING" to IIS IO Thread
2. The requests is handed over to a CLR Worker thread
3. If the requests are of high latency and all the threads are occupied (the thread count approaches httpRuntime.minFreeThreads), then the requests are posted to the Application level request queue (this queue is per AppDomain)
4. Also ASP.NET checks the number of requests queued and currently executing before accepting a new request. If this number is greater than value specified by processModel.requestQueueLimit then incoming requests will be rejected with 503 server busy error.
This article might help to understand the settings a little better.
minFreeThreads: This setting is used by the worker process to queue all the incoming requests if the number of available threads in
the thread pool falls below the value for this setting. This setting
effectively limits the number of requests that can run concurrently to
maxWorkerThreads minFreeThreads. Set minFreeThreads to 88 * # of
CPUs. This limits the number of concurrent requests to 12 (assuming
maxWorkerThreads is 100).
Edit:
In this SO post, Thomas provides more detail and examples of request handling in the integrated pipeline. Be sure to read the comments on the answer for additional explanations.
A native callback (in webengine.dll) picks up request on CLR worker
thread, we compare maxConcurrentRequestsPerCPU * CPUCount to total
active requests. If we've exceeded limit, request is inserted in
global queue (native code). Otherwise, it will be executed. If it was
queued, it will be dequeued when one of the active requests completes.
Firstly there's a great overview of the IIS7 HTTP request lifecycle and various settings that affect performance here:
ASP.NET Thread Usage on IIS 7.0 and 6.0
Very specifically though, in dotNet 4 the defaults for maxConcurrentRequestsPerCPU and requestsQueueLimit are set to 5000. E.g. equivalent to: (in aspnet.config):
<system.web>
<applicationPool
maxConcurrentRequestsPerCPU="5000"
maxConcurrentThreadsPerCPU="0"
requestQueueLimit="5000" /> (** see note below)
</system.web>
Seems to me that on a multi-CPU/core server the requestQueueLimit here will always be invoked well berfore the 'perCPU' limit. Thus, if a max of 5000 requests per CPU is what you actually want then I would expect that the requestQueueLimit needs to be increased to 5000 * CPUCount or just disabled altogether.
Is my interpretation correct? If so can I disable requestQueueLimit? (set it to zero?). The documentation on this setting doesn't appear to address this question (so maybe I'm missing something or misreading?)
** side note from the above article: The requestQueueLimit is poorly named. It actually limits the maximum number of requests that can be serviced by ASP.NET concurrently. This includes both requests that are queued and requests that are executing. If the "Requests Current" performance counter exceeds requestQueueLimit, new incoming requests will be rejected with a 503 status code)
***Is my interpretation correct?
Yes, if you want to execute more than 5000 requests concurrently, you'll need to increase the requestQueueLimit. The requestQueueLimit restricts the total number of requests in the system. Due to its legacy, it is actually the total number of requests in the system, and not the number of requests in some queue. It's goal is to prevent the server from toppling over due to lack of physical memory, virtual memory, etc. When the limit is reached, incoming requests will receive a quick 503 "Server Too Busy" response. By the way, the current number of requests in the system is exposed by the "ASP.NET\Requests Current" performance counter.
***can I disable requestQueueLimit? (set it to zero?)
You can effectively disable it by setting it to a large value, like 50000. You must set the value in the aspnet.config fileI doubt your server can handle 50000 concurrent requests, but if so, then double that. Setting it to zero will not disable it...oddly, it means no more than one request can execute concurrently.
By the way, it looks like there is a bug in v4. For integrated mode, it only successfully reads the value of requestQueueLimit if it is configured in the aspnet.config file as described on MSDN. For some reason, v4 was not reading it from machine.config when I experimented with it a little bit ago.
You might want to check this application's source code. IIS tuner is an open source application which optimizes IIS settings for better performance. On the other hand this page could be useful for your questions.