I have a .Net application hosted in IIS 10 running on Windows Service 2019. Sometimes the website stops responding (usually when running E2E tests). Even after restarting Website/Application Pool/IIS/Machine it doesn't work.
Looking in Event Viewer I see errors like these:
Forms authentication failed for the request. Reason: The ticket supplied has expired
Failed to stop a listening channel for protocol 'http' at allotted time from worker process serving application pool
A process serving application pool exceeded time limits during shut down
In HTTPERR files I see a lot of messages containing Connection_Abandoned_By_ReqQueue and Connection_Dropped
In inetpub log files I can't see any relevant, just the url requests.
To add more information, we have signalr installed and sometimes errors appear in the events, errors with messages like:
The user identity cannot change during an active SignalR connection.
Any idea what might be causing this?
Related
Have a web application running on several servers. Two of the servers are having issues where the application pool becomes disabled. Message in the Event viewer System Log: Application pool 'xxxxx' is being automatically disabled due to a series of failures in the process(es) serving that application pool.
Just prior to this message are several other 'Warning' Messages: A process serving application pool 'xxxxx' suffered a fatal communication error with the Windows Process Activation Service. The process id was '1072'. The data field contains the error number. Or A process serving application pool 'xxxxx' terminated unexpectedly. The process id was '3644'. The process exit code was '0x0'.
Running IIS 7. The servers that are failing are running 2008R2 with Service Pack1 and the other are running 2008R2 (no Service Pack).
In the HTTP log right before I see the AppOffline message, there are the several Connection_Abandoned_By_ReqQueue and Client_Reset messages.
I have read and reread many posting about changing the Rapid-Fail Protection settings from the default of failure interval (minutes) 5 and Maximum failures 5 as I can see that after five failures in five minutes the AppPool is stopped. However, doing this just changes the number of issues during a particular time before the AppPool will be stopped and not really addressing the root cause of the problem.
What is the correct method for determine why the application is failing?
Could the difference in service pack on the server be a culprit?
Thanks.
We have a ASP.NET/WCF app hosted in Window Server 2012 (IIS 7). We used the basicHttpBinding. This ASP.NET/WCF application exposes two methods; one is to receive messages and the other is to download a text file (1MB) onto the server.
On another server, we have ASP.NET hosted in Window Server 2012 (IIS 7) which is the client that consumes the exposed method mentioned earlier. This client application sends a message and uploads a text file at a high frequency. This communication between this ASP.NET/WCF application and client application works fine for a few hours until we get the following error at the ASP.NET/WCF side.
Application pool 'XXXXXXXXXX' is being automatically disabled due to a series of failures in the process(es) serving that application pool.
So, could you please shine some light regarding this issue that we are facing?
This is due to something called "Rapid Fail Protection." When your underlying application crashes a certain number of times in a certain time period, the application pool is automatically disabled.
The default settings are 5 crashes in 5 minutes, but you can configure this yourself. See this link for details.
I have an asp.net application running on IIS 7.5 , Windows Server 2008 R2 – using an application pool in classic mode, framework version 4.
Sometimes I am running into the following problem:
The application can work for a few days, but all of a sudden I receive an http error 503 – server unavailable.
When I look at the application pool it seems to be running (I see it started), but it actually FREEZES – every request to it is responded with 503.
At the worker processes list (in the IIS manager) I see a lot of requests unhandled.
It's important to mention that other asp.net application running under other application pools are working just fine which means the IIS is working fine and the problem is only in this specific application pool.
When I researched the http error logs I saw the following error logs in the Windows\System32\LogFiles\HTTPERR folder:
In regular settings when everything works fine I noticed records of
"Timer_ConnectionIdle" (a normal thing from what I have read).
At a certain times I notice an appearance of "Client_Reset" records.
15 minutes after the "Client_Reset" errors started->records of "QueueFull appear".
In order to work with the application I am doing an iisreset (I guess a recycle for the pool will be enough also).
I will be happy to receive any help or suggestions.
EDIT:
It's important to mention that nothing related gets written to the IIS logs, or the System and Application logs. This error occurs before.
Without more information about your problem, most quick fix will be to Configure Recycling Settings for an Application Pool. Since your problem is about request queues, you can choose option After reaching a number of requests.
When I encounter HTTP 500 errors, I find that enabling IIS Tracing is very helpful. Below is a link that describes the process of enabling tracing and then reviewing the trace. The first section of the site describes how to install IIS, so you probably want to skip to the section labeled "Enable Failed-Request Tracing."
Troubleshooting Failed Requests Using Tracing in IIS 7
Edit:
Since you're getting a QueueFull error, you may want to monitor the request queues. The easiest way to do this is using Perfmon. On the server with IIS, open Performance Monitor and add the appropriate counters under "HTTP Service Request Queue." In your case, "Current Request Queue" for the ailing Application Pool would likely be of value.
I'm struggling to setup the environment in IIS8, I searched a lot but couldn't find a right solution.
I checked the error logs, but no idea.
C:\Windows\System32\LogFiles\HTTPERR
2013-10-09 09:28:39 192.168.43.205 60172 192.168.43.205 80 HTTP/1.1
GET / 503 2 AppOffline qa.hti.local
2013-10-09 09:28:39 192.168.43.205 60192 192.168.43.205 80 HTTP/1.1
GET /favicon.ico 503 2 AppOffline qa.hti.local
Then in Event Viewer:
WARNINGS:
A listener channel for protocol 'http' in worker process '11188'
serving application pool 'qa.hti.local' reported a listener channel
failure. The data field contains the error number.
A listener channel for protocol 'http' in worker process '7492'
serving application pool 'qa.hti.local' reported a listener channel
failure. The data field contains the error number.
A listener channel for protocol 'http' in worker process '9088'
serving application pool 'qa.hti.local' reported a listener channel
failure. The data field contains the error number.
A listener channel for protocol 'http' in worker process '9964'
serving application pool 'qa.hti.local' reported a listener channel
failure. The data field contains the error number.
A listener channel for protocol 'http' in worker process '7716'
serving application pool 'qa.hti.local' reported a listener channel
failure. The data field contains the error number.
I don't understand what the warning means.
ERROR: Application pool 'qa.hti.local' is being automatically disabled due to a series of failures in the process(es) serving that
application pool.
Note: I learned that consecutive 5 failures leads to APP Pool crash, and this can increased. I also tried increasing this but no success.
Please share your thoughts.
I came across this question as I was experiencing a similar issue and searching for a solution.
My problem specifically had to do with our IIS shared configuration. We had enabled a feature in IIS on one of the servers (Http Redirect) that was not installed on any of the others so the server 'features' were out of sync with all the servers.
I was able to resolve the issue by uninstalling the new addition on the first server so it was back to matching the others. An IIS reset later and the AppPools were no longer going down and all was back to normal.
So if you are using IIS Shared Configuration and the IIS is creating 'Service Unavailable' errors and the AppPools are going down, this can be a symptom of the system configuration being out of synch which is corrupting the shared configuration. Hopefully this post will help someone find the solution faster than I was able to.
I had a similar problem, and it was due to another error (2282 IIS-W3SVC-WP) that the pool stopped itself. My problem was that de module owssvr.dll could not be loaded due to a configuration problem.
The solution for me was to set the setting "Enable 32-bit applications" from True to false.
I was deploying a solution on Sharepoint 2013 on a Windows Server 2012 with Visual studio 2013.
I know this was supposedly a very specific problem, but I want to help all those with the same problem.
Propably you do not have the permissions to read the directory.
The directory (and the files) need to have read-access for either windows-group "USERS" or windows-grou "IIS_IUSRS" and also for your apppoolidentity.
This occurred for me too on Windows 10 1803 after an update.
Earlier in the event log there were errors to do with missing DLLS, specifically iiswsock.dll and compstat.dll.
After turning on the following Windows features (under IIS > WWWServices > Performance Features and AppDev Features):
Dynamic and Static Content Compression, and
WebSocket Protocol Windows features
those errors disappeared after an IIS restart.
503 2
Is that 2 the substatus code? If so, you might want to make sure your site is not being hit with excessively (5000+) # of requests.
http://support.microsoft.com/kb/943891
The data field contains the error number.
what's the error number?
This also can be caused by you're software vendor not realizing that they installed the 32bit version of the application pool apps on your 2008 R2 server. After a little troubleshooting my server because they wanted me to reinstall IIS i checked the windows app logs and emailed them the x86 architecture error for their app.
This can occur if a piece of IIS is missing (e.g. one of the many IIS modules has not been installed on the new server).
The thing to do is to carefully compare the IIS config of the source and target, and add the missing pieces to the target.
In a recent IIS8.5->8.5 migration, I had this issue. Went through the whole stack. The last piece was that the Web Cert auth piece of IIS was missing.
To install:
powershell
import-module servermanager
add-windowsfeature Web-Cert-Auth
I reinstalled server and copied applicationHost.config to new server, but not installed corresponding modules, and got this error. I fixed that by check modules in IIS manager and ensure modules are installed.
Update: In Windows Logs-> Application, there're some info about which module is missing.
In case this helps anyone else: We run multiple https sites on an IIS 10 server. One of the steps in configuring a new site is to give the new application pool permission to the system certificate. In the registry, under HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SystemCertificates\MY, grant the application pool on the local machine full control. That fixed this issue for us.
I had the same problem and solved it by adding the domain account the application pool was using as an identity to the local group of Administrators on a web server. Perhaps it would also do the trick to grant access to the application directory for the application pool identity account, as #Lisa-Berlin stated above.
We've implemented a "background service" in our Asp.Net web app that receives messages from MSMQ at random intervals without requiring an HTTP request to start up the application. We auto start this web app using a serviceAutoStartProvider.
Everyhing works when IIS initially starts up, the server is rebooted and so on, we receive messages just fine. BUT if we just stop the site in IIS (not touching the application or app pool), the application stops receiving MSMQ messages. And when we start the web site again, the serviceAutoStartProvider is not called again, so our app does not start listening to MSMQ messages again!
If we issue a HTTP request against the web app after the IIS site has been stopped and started again, it starts listening to MSMQ messages again.
Shouldn't our "background service" web app continue to listen to MSMQ messages even if the IIS site is stopped? It won't get any requests, but I think it should continue to run.
What exactly happens in an Asp.Net application/app pool when the IIS site is stopped? Any events fired that we can hook up to? The app pool claims to be "started" in IIS manager, but code is not running in it.
Why isn't our serviceAutoStartProvider called when the site is started again? I believe it is "by design", since the application isn't really stopped. But the applications isn't running, either, has to be waken up by an actual HTTP request.
When the IIS Web App shuts down (eg. due to no new HTTP(S) requests for the timeout time) the .NET app domain (within the app pool worker process) completely closes and unloads. This includes all background threads, including those used by the .NET thread pool.
A Web App can be configured with a longer (or no) timeout, then background worker threads could continue to process work.
But better would be to either run such workers in a specialist service process managed completely separately.
Or, even better, use IIS application hosting with WCF to create the MSMQ listener. I understand in this case the integration of Windows Process Activation Services with IIS would restart the Web App if a new message arrived after it had been shutdown.
I would host the MSMQ listenner in a windows service. Why couple it to IIS?
UPDATE
Actually what I mean is why couple the MSMQ and ASPNET in the same app pool?
You can now use "Application Initialization" feature of IIS8 (and IIS7.5), more information including version availability and usage documentation can be found at:
http://www.iis.net/learn/get-started/whats-new-in-iis-8/iis-80-application-initialization
This replaces "Application Warm-Up Module" which is no longer supported, and provides us with proper control over component/service initialization in an "always running" scenario.