We have a http module that measures how long server requests spend in individual states. In one of our servers we see requests take a large amount of time in the AuthenticateRequest state - seconds, sometimes even minutes.
This is Windows 2008, IIS 7, Integrated pipeline, .NET 3.5 ASP.NET application
We do not use any authentication. "Anonymous authentication" is the only one enabled.
Any suggestion where to look at would be very appreciated.
Related
We have a ASP.NET/WCF app hosted in Window Server 2012 (IIS 7). We used the basicHttpBinding. This ASP.NET/WCF application exposes two methods; one is to receive messages and the other is to download a text file (1MB) onto the server.
On another server, we have ASP.NET hosted in Window Server 2012 (IIS 7) which is the client that consumes the exposed method mentioned earlier. This client application sends a message and uploads a text file at a high frequency. This communication between this ASP.NET/WCF application and client application works fine for a few hours until we get the following error at the ASP.NET/WCF side.
Application pool 'XXXXXXXXXX' is being automatically disabled due to a series of failures in the process(es) serving that application pool.
So, could you please shine some light regarding this issue that we are facing?
This is due to something called "Rapid Fail Protection." When your underlying application crashes a certain number of times in a certain time period, the application pool is automatically disabled.
The default settings are 5 crashes in 5 minutes, but you can configure this yourself. See this link for details.
We've implemented a "background service" in our Asp.Net web app that receives messages from MSMQ at random intervals without requiring an HTTP request to start up the application. We auto start this web app using a serviceAutoStartProvider.
Everyhing works when IIS initially starts up, the server is rebooted and so on, we receive messages just fine. BUT if we just stop the site in IIS (not touching the application or app pool), the application stops receiving MSMQ messages. And when we start the web site again, the serviceAutoStartProvider is not called again, so our app does not start listening to MSMQ messages again!
If we issue a HTTP request against the web app after the IIS site has been stopped and started again, it starts listening to MSMQ messages again.
Shouldn't our "background service" web app continue to listen to MSMQ messages even if the IIS site is stopped? It won't get any requests, but I think it should continue to run.
What exactly happens in an Asp.Net application/app pool when the IIS site is stopped? Any events fired that we can hook up to? The app pool claims to be "started" in IIS manager, but code is not running in it.
Why isn't our serviceAutoStartProvider called when the site is started again? I believe it is "by design", since the application isn't really stopped. But the applications isn't running, either, has to be waken up by an actual HTTP request.
When the IIS Web App shuts down (eg. due to no new HTTP(S) requests for the timeout time) the .NET app domain (within the app pool worker process) completely closes and unloads. This includes all background threads, including those used by the .NET thread pool.
A Web App can be configured with a longer (or no) timeout, then background worker threads could continue to process work.
But better would be to either run such workers in a specialist service process managed completely separately.
Or, even better, use IIS application hosting with WCF to create the MSMQ listener. I understand in this case the integration of Windows Process Activation Services with IIS would restart the Web App if a new message arrived after it had been shutdown.
I would host the MSMQ listenner in a windows service. Why couple it to IIS?
UPDATE
Actually what I mean is why couple the MSMQ and ASPNET in the same app pool?
You can now use "Application Initialization" feature of IIS8 (and IIS7.5), more information including version availability and usage documentation can be found at:
http://www.iis.net/learn/get-started/whats-new-in-iis-8/iis-80-application-initialization
This replaces "Application Warm-Up Module" which is no longer supported, and provides us with proper control over component/service initialization in an "always running" scenario.
I am in a bit of confusion here. In IIS 5.0 all the ASP.NET applications run inside the same worker process(aspnet_wp.exe). The various applications are isolated by appdomain. I believe the fundamental functionality of an appdomain is to provide application isolation within a process. Now on to IIS 6.0. I have read about application pools in IIS 6. A lot of sites say that in IIS 5.0 when there is a problem with an application it affects the other applications. But isnt that safeguarded by appdomains? I am being told that application pools in IIS 6.0 prevent that mishaps. I am a bit confused here as to what exactly is advantage that application pools provide over IIS 5.0. I have also been told that in IIS 6 worker process isolation mode, configuring different application pools(and thereby different worker processes) is a perfomant thing to do. But spawning new processes increases the memory utilization and how can it be claimed to be a performant way to do things? Also how does the HTTP.SYS know to route the request to the concerned application poool?
Don't get application pools and AppDomains confused. An AppDomain is a isolation boundary within a process. They are designed to isolate and provide security around managed code that is executing.
Application pools are a feature of IIS that allow a pool to handle requests for multiple sites. They handle this by running multiple AppDomain instances in the worker processes (aspnet_wp.exe, or w3wp.exe). Although technically you can run multiple AppDomain instances per site, generally it is a single AppDomain per IIS Application.
If you have multiple sites in a single application pool (and this sharing the worker processes), if one of those sites starts consuming a lot of resources, this can directly effect the other sites running in the same application pool.
By grouping (and segregating) sites in logical groups (application pools), you can better manage how sites behave.
HTTP.SYS is a kernel mode driver stack that is part of the Windows subsystems. It provides management of the HTTP protocol in kernel mode (versions prior to IIS6 used winsock in user mode). With HTTP.SYS running at the kernel level, should a worker process fail, the request can be queued, and then forwarded to a new worker process which will be spun up.
When you create a site in IIS6, it registers the site with HTTP.SYS which can then route requests coming in to the appropriate worker process.
The Appdomain is about sharing data
http://msdn.microsoft.com/en-us/library/system.appdomain.aspx
and in IIS 6 you can append several applications to an application pool. If you have 2 applications in the same application pool and one of them keeps crashing, it affects the other and IIS will decide to shutdown the whole application pool because of the instability.
Therefore, if you have critical applications, it is recommended to keep them in their own application pool. If you have multiple apps in IIS, you route the requests with the hostheaders (or extra external IIP addresses)
extra resource that recommends an application pool for each production website: http://forums.iis.net/t/1151476.aspx
a nice resource which explains the difference between 5.0 and 6.0 with isolation:
http://www.tech-faq.com/understanding-iis-5-and-iis-6.html
Our (reasonably busy) intranet web site has recently been sometimes taking 2-3 minutes to log in users on the production server, and we haven't been able to find why.
The web site is a "top-level" web site (mapped to an IP address) containing an additional 10 virtual directories. The web site contains both classic ASP and ASP.NET pages, and is running .NET 4.0 (version 4.0.30319.235; the latest, I believe) and the entire site is HTTPS (SSL). It uses ASP.NET Forms authentication with an LDAP provider (the corporation's LDAP in the same domain). It's a very straightforward authentication, with an LDAP connection string and provider configuration in the root web.config, and an "asp:login" control, using the defined LDAP provider, on the login page. The web site is configured with wildcard mapping in IIS to allow the classic ASP pages to be authenticated with the same ASP.NET login process. Sessions are the default "Inproc", not using SQLServer or a state service.
Intermittently, starting about 3 weeks ago, it has been taking 2-3 minutes for users to log in to the web site. There are some pages in the site that don't require authentication and those still perform fine, and after loggin in, authenticated pages also work fine without any delays. It's only the login process itself that's slow.
The exact same website configuration and code running on the development server and workstations never encounters the same delays in authentication.
The login code has not changed in well over a year, and the site has been running .NET 4.0 for probably a year; the servers were updated to the latest .NET version in October-2010. The wildcard mapping was also set up well over a year ago. The login slowness only started about 3 weeks ago. The web server department is not aware of any changes that were made around that time to the web servers and/or the network.
While the long delay occurs on the login page, I don't think the delay is actually in the authentication process; it seems to be in some sort of setup before that. I have added the setting of a Session variable with the current date/time to the beginning (in the LogginIn event) and end of the login code, and there is usually less than a second between these times; however, the clock time between clicking the "Login" button on the login page and the setting of the first of these Session variables is a couple of minutes (consistently around 2 minutes and 14 seconds). I have tried this with page buffering enabled and disabled with no difference in the times.
The same configuration and code is set up in these various environments:
(OK) Development server: Windows Server 2003 SP2 64-bit (32-bit IIS) in SSL mode
(OK) Development server: Windows Server 2008 R2 64-bit (32-bit IIS) in SSL mode
(OK) dev. workstations: Windows 7 64-bit (32-bit IIS), NOT running SSL
(Sometimes slow) Staging web server: Windows Server 2003 SP2 32-bit in SSL mode
(Sometimes slow) Production web cluster: Windows Server 2003 SP2 32-bit in SSL mode
That is, in the development servers and workstaions, I cannot seem to cause the same slowness no matter what I try - resetting IIS, recycling the app pool, updating web.config, etc.
The development servers are using self-signed certificates for SSL; the staging and production servers are using "official" (Verisign) certificates. The production web servers are also used for other web sites (and the other web sites are all still at .NET 2.0), but none of those other sites are using LDAP authentication. All machines (production and development) are up-to-date with Windows Updates.
I can reproduce the problem on the staging or production servers by forcing an application restart (such as by updating web.config) or asking for an IIS Reset, or (on the staging server) waiting for more than 20 minutes with no activity (I think, for the application pool default lifetime to end).
We have checked the following:
The "machineKey" section in every web.config in the entire site is identical
The application pool for the site is set to .NET 4.0, and no other sites are using the same application pool
The application pool identity is "NETWORK SERVICE"
We have tried the following in production with no change in the occasional login delay:
Adding connectionProtection="None" to the LPAP provider configuration
Adding an "applicationName" to the LDAP provider configuration
Adding a port number (the SSL/secure port number) to the LDAP connection string
We have looked through Windows event logs and the IIS logs on the staging and production servers, without finding an obvious connection to the issue. The only possibly-related error that does sometimes occur is the following (that's been logged sometimes about a second before the login succeeds, not at the time that the "Login" button was clicked), however, this may also be because a previously-logged in session has timed out:
Event code: 4006
Event message: Membership credential verification failed.
Application information:
Application domain: /LM/W3SVC/ [...]
Trust level: Full
Application Virtual Path: /
Process information:
Process name: w3wp.exe
Account name: NT AUTHORITY\NETWORK SERVICE
Can anyone suggest any other troubleshooting ideas or potential configuration issues that should be checked or changed?
You can take a memory dump when the problem is happening, and then you use windbg do the analysis.
How to take memory dump:
http://support.microsoft.com/kb/286350
This blog has lots information about how to analysis memory dump
http://blogs.msdn.com/b/tess/
I have a ASP.NET 3.5 web application that uses long polling for chat. It long polls an asynchronous handler for new messages. Each request can last a maximum of 15 seconds before being closed and reconnected.
The application runs fine using Visual Studio Built-in web server but once deployed to the IIS (i.e. IIS 7.5 ) the long polling requests start getting aborted after i login around 10 users.
Please help.