We have a ASP.NET/WCF app hosted in Window Server 2012 (IIS 7). We used the basicHttpBinding. This ASP.NET/WCF application exposes two methods; one is to receive messages and the other is to download a text file (1MB) onto the server.
On another server, we have ASP.NET hosted in Window Server 2012 (IIS 7) which is the client that consumes the exposed method mentioned earlier. This client application sends a message and uploads a text file at a high frequency. This communication between this ASP.NET/WCF application and client application works fine for a few hours until we get the following error at the ASP.NET/WCF side.
Application pool 'XXXXXXXXXX' is being automatically disabled due to a series of failures in the process(es) serving that application pool.
So, could you please shine some light regarding this issue that we are facing?
This is due to something called "Rapid Fail Protection." When your underlying application crashes a certain number of times in a certain time period, the application pool is automatically disabled.
The default settings are 5 crashes in 5 minutes, but you can configure this yourself. See this link for details.
Related
Platform: Windows Server 2016
Website: Default website using https
Custom app pools - 4 - one used as the default for our application[
Recycle settings: default
We have an issue with our web servers, where occasionally the existing worker process stops processing requests. But for whatever reason, IIS does not spin up a new worker process. The net effect is that the web server is dead in the water.
Is it possible that the existing process is not terminated until a new child process successfully announces itself?
This problem only occurs in the app pool that our application creates, and uses as the default app pool. (The other worker threads probably don't do enough to run into this situation)
The attached screenshot taken from our monitoring shows the server doing nothing, until we manually intervened and recyled IIS.
I have a .Net application hosted in IIS 10 running on Windows Service 2019. Sometimes the website stops responding (usually when running E2E tests). Even after restarting Website/Application Pool/IIS/Machine it doesn't work.
Looking in Event Viewer I see errors like these:
Forms authentication failed for the request. Reason: The ticket supplied has expired
Failed to stop a listening channel for protocol 'http' at allotted time from worker process serving application pool
A process serving application pool exceeded time limits during shut down
In HTTPERR files I see a lot of messages containing Connection_Abandoned_By_ReqQueue and Connection_Dropped
In inetpub log files I can't see any relevant, just the url requests.
To add more information, we have signalr installed and sometimes errors appear in the events, errors with messages like:
The user identity cannot change during an active SignalR connection.
Any idea what might be causing this?
Have a web application running on several servers. Two of the servers are having issues where the application pool becomes disabled. Message in the Event viewer System Log: Application pool 'xxxxx' is being automatically disabled due to a series of failures in the process(es) serving that application pool.
Just prior to this message are several other 'Warning' Messages: A process serving application pool 'xxxxx' suffered a fatal communication error with the Windows Process Activation Service. The process id was '1072'. The data field contains the error number. Or A process serving application pool 'xxxxx' terminated unexpectedly. The process id was '3644'. The process exit code was '0x0'.
Running IIS 7. The servers that are failing are running 2008R2 with Service Pack1 and the other are running 2008R2 (no Service Pack).
In the HTTP log right before I see the AppOffline message, there are the several Connection_Abandoned_By_ReqQueue and Client_Reset messages.
I have read and reread many posting about changing the Rapid-Fail Protection settings from the default of failure interval (minutes) 5 and Maximum failures 5 as I can see that after five failures in five minutes the AppPool is stopped. However, doing this just changes the number of issues during a particular time before the AppPool will be stopped and not really addressing the root cause of the problem.
What is the correct method for determine why the application is failing?
Could the difference in service pack on the server be a culprit?
Thanks.
We've implemented a "background service" in our Asp.Net web app that receives messages from MSMQ at random intervals without requiring an HTTP request to start up the application. We auto start this web app using a serviceAutoStartProvider.
Everyhing works when IIS initially starts up, the server is rebooted and so on, we receive messages just fine. BUT if we just stop the site in IIS (not touching the application or app pool), the application stops receiving MSMQ messages. And when we start the web site again, the serviceAutoStartProvider is not called again, so our app does not start listening to MSMQ messages again!
If we issue a HTTP request against the web app after the IIS site has been stopped and started again, it starts listening to MSMQ messages again.
Shouldn't our "background service" web app continue to listen to MSMQ messages even if the IIS site is stopped? It won't get any requests, but I think it should continue to run.
What exactly happens in an Asp.Net application/app pool when the IIS site is stopped? Any events fired that we can hook up to? The app pool claims to be "started" in IIS manager, but code is not running in it.
Why isn't our serviceAutoStartProvider called when the site is started again? I believe it is "by design", since the application isn't really stopped. But the applications isn't running, either, has to be waken up by an actual HTTP request.
When the IIS Web App shuts down (eg. due to no new HTTP(S) requests for the timeout time) the .NET app domain (within the app pool worker process) completely closes and unloads. This includes all background threads, including those used by the .NET thread pool.
A Web App can be configured with a longer (or no) timeout, then background worker threads could continue to process work.
But better would be to either run such workers in a specialist service process managed completely separately.
Or, even better, use IIS application hosting with WCF to create the MSMQ listener. I understand in this case the integration of Windows Process Activation Services with IIS would restart the Web App if a new message arrived after it had been shutdown.
I would host the MSMQ listenner in a windows service. Why couple it to IIS?
UPDATE
Actually what I mean is why couple the MSMQ and ASPNET in the same app pool?
You can now use "Application Initialization" feature of IIS8 (and IIS7.5), more information including version availability and usage documentation can be found at:
http://www.iis.net/learn/get-started/whats-new-in-iis-8/iis-80-application-initialization
This replaces "Application Warm-Up Module" which is no longer supported, and provides us with proper control over component/service initialization in an "always running" scenario.
Our (reasonably busy) intranet web site has recently been sometimes taking 2-3 minutes to log in users on the production server, and we haven't been able to find why.
The web site is a "top-level" web site (mapped to an IP address) containing an additional 10 virtual directories. The web site contains both classic ASP and ASP.NET pages, and is running .NET 4.0 (version 4.0.30319.235; the latest, I believe) and the entire site is HTTPS (SSL). It uses ASP.NET Forms authentication with an LDAP provider (the corporation's LDAP in the same domain). It's a very straightforward authentication, with an LDAP connection string and provider configuration in the root web.config, and an "asp:login" control, using the defined LDAP provider, on the login page. The web site is configured with wildcard mapping in IIS to allow the classic ASP pages to be authenticated with the same ASP.NET login process. Sessions are the default "Inproc", not using SQLServer or a state service.
Intermittently, starting about 3 weeks ago, it has been taking 2-3 minutes for users to log in to the web site. There are some pages in the site that don't require authentication and those still perform fine, and after loggin in, authenticated pages also work fine without any delays. It's only the login process itself that's slow.
The exact same website configuration and code running on the development server and workstations never encounters the same delays in authentication.
The login code has not changed in well over a year, and the site has been running .NET 4.0 for probably a year; the servers were updated to the latest .NET version in October-2010. The wildcard mapping was also set up well over a year ago. The login slowness only started about 3 weeks ago. The web server department is not aware of any changes that were made around that time to the web servers and/or the network.
While the long delay occurs on the login page, I don't think the delay is actually in the authentication process; it seems to be in some sort of setup before that. I have added the setting of a Session variable with the current date/time to the beginning (in the LogginIn event) and end of the login code, and there is usually less than a second between these times; however, the clock time between clicking the "Login" button on the login page and the setting of the first of these Session variables is a couple of minutes (consistently around 2 minutes and 14 seconds). I have tried this with page buffering enabled and disabled with no difference in the times.
The same configuration and code is set up in these various environments:
(OK) Development server: Windows Server 2003 SP2 64-bit (32-bit IIS) in SSL mode
(OK) Development server: Windows Server 2008 R2 64-bit (32-bit IIS) in SSL mode
(OK) dev. workstations: Windows 7 64-bit (32-bit IIS), NOT running SSL
(Sometimes slow) Staging web server: Windows Server 2003 SP2 32-bit in SSL mode
(Sometimes slow) Production web cluster: Windows Server 2003 SP2 32-bit in SSL mode
That is, in the development servers and workstaions, I cannot seem to cause the same slowness no matter what I try - resetting IIS, recycling the app pool, updating web.config, etc.
The development servers are using self-signed certificates for SSL; the staging and production servers are using "official" (Verisign) certificates. The production web servers are also used for other web sites (and the other web sites are all still at .NET 2.0), but none of those other sites are using LDAP authentication. All machines (production and development) are up-to-date with Windows Updates.
I can reproduce the problem on the staging or production servers by forcing an application restart (such as by updating web.config) or asking for an IIS Reset, or (on the staging server) waiting for more than 20 minutes with no activity (I think, for the application pool default lifetime to end).
We have checked the following:
The "machineKey" section in every web.config in the entire site is identical
The application pool for the site is set to .NET 4.0, and no other sites are using the same application pool
The application pool identity is "NETWORK SERVICE"
We have tried the following in production with no change in the occasional login delay:
Adding connectionProtection="None" to the LPAP provider configuration
Adding an "applicationName" to the LDAP provider configuration
Adding a port number (the SSL/secure port number) to the LDAP connection string
We have looked through Windows event logs and the IIS logs on the staging and production servers, without finding an obvious connection to the issue. The only possibly-related error that does sometimes occur is the following (that's been logged sometimes about a second before the login succeeds, not at the time that the "Login" button was clicked), however, this may also be because a previously-logged in session has timed out:
Event code: 4006
Event message: Membership credential verification failed.
Application information:
Application domain: /LM/W3SVC/ [...]
Trust level: Full
Application Virtual Path: /
Process information:
Process name: w3wp.exe
Account name: NT AUTHORITY\NETWORK SERVICE
Can anyone suggest any other troubleshooting ideas or potential configuration issues that should be checked or changed?
You can take a memory dump when the problem is happening, and then you use windbg do the analysis.
How to take memory dump:
http://support.microsoft.com/kb/286350
This blog has lots information about how to analysis memory dump
http://blogs.msdn.com/b/tess/