IIS - All application pools stopped on a specific time - iis-10

I'm facing a weird behaviour from IIS where all the app pools configured in IIS has been stopped state exact time daily. No logs on event viewer or IIS.
Setup:
We have 2 servers having OS "Windows server 2019" and IIS 10.0. we hosted a service handlers on both the servers and it was working fine til last month. this service handler has separate app pool that has been created.
Issue:
For last couple of weeks, we are facing an issue where all the app pools (default pool, handlers pool, classic 4.0) on the IIS stopped unexpectedly exactly at 3pm on server1 and 4pm on server2 on a daily basis.
Already tried:
1. Site running in "app pool identity" and not custom. no password change required.
2. No trace of event log in event viewer or iis
3. Enabled Operational logging and no log while stopped.
4. Made sure all the required dependencies installed on Windows features for IIS.
Any clues?

Related

IIS restarts suspended aplications after configuration change

I have encountered a (mis?)behavior of IIS/ASP.NET related to app pool suspension (Idle Worker Process Page-Out), and I can't find documentation about it. When an ASP.NET app (.NET Framework 4.8) in IIS (10.0) is suspended, and then a change is made to IIS configuration, the next time the app is accessed, it restarts (new AppDomain in same w3wp process).
It doesn't matter What is changed in IIS (appcmd add apppool, appcmd add app, etc.). Even touching applicationhost.config (changing LastWriteTime without content change) is enought to fire a restart. To be clear, the restart only happens if the app is SUSPENDED when IIS configuration is changed, and not if the app is live.
We have a lot of apps in the same IIS server, with frequent app deployments, and each time a deployment is made, all the suspended apps are affected (next user must wait the restart time). This renders useless the suspension mechanism. ¿Is this restart behavior by design or it's a bug? ¿Is there a way to prevent it?

IIS application pool crashes several times a day

I have an application that has a application pool that shuts down a couple of times a day and I can't figure out why. the website is hosted on a VM with SQL Server R2. the version of IIS is 7 and the website has a dedicated application pool using .NET framework v4.0.30319.
At the application level the website is using a target framework of 4.5 and seems to run just fine for a 24-48 hour period and then for some reason the application pool just stops. We do have other .NET 4.5 applications running on this machine and their application pools do not have this problem. The only thing slightly different about this application is that we initiate a separate thread to send a order confirmation email when an order is completed. We average about a couple of hundred orders a day for this website.
According to the event log the error reported prior to the shutdown is
"A process serving application pool 'RetailSupport' suffered a fatal communication error with the Windows Process Activation Service"
Please help
I would check, if your code could hit under some circumstances an endless loop or a recursive
call of methods. This is often the reason for this error.
What always helps me in such cases is to add debug logs (e.g. log4net) in your case especially before the email is sent, status info about resources, etc. They can be more informative then the IIS logs.
If this does not help, check http://support.microsoft.com/kb/919789/en how to use the debug Diagnostic tool, to get more details for the error.
Look also for IIS: Web Application hangs periodically needs system reboot it could help you.
Also: what recycle strategy on IIS do you have?

View state errors in development but not production

I have two websites which are more or less identical, both running on the same server, but under different app pools. The app pools are both configured to run with "NETWORK SERVICE" credentials.
On one website - call this the development copy, I receive the infamous "validation of view state MAC failed" error. This happens when a postback occurs after the page is idle for 20 minutes. I've been informed this is because I don't specify the machine key in the web.config or the machine config file. Manually recycling the app pool reproduces the error as expected. So my question is not about how to stop the issue, but rather I want to know why the error happens in dev but not production. The only difference between the setup of the two sites is the authentication method.
In the "basic settings" section of the IIS config, the dev version of the site is set to connect using a service account, while the production version is using "application user" (pass through authentication). Under authentication settings, both pages are configured with impersonation and windows authentication.
This is the only difference I can see. My knowledge of how IIS handles authentication at the site and app pool level is fairly limited. Is there a specific reason why one setup should result in the machine key being refreshed but the other does not?
After installing .NET 3.5 SP1 you may get Validation of viewstate MAC failed exceptions when doing post backs on ASP.NET pages.

IIS7 Application Pool Crash

I am running IIS 7.0 on Windows server 2008 R2 and created 1 Application Pool per site and I have 1 site which is causing the application pool to stop. These are the following errors in the Event Viewer:
This shows several times:
A process serving application pool 'Pilot.Mobile.SyncV2' terminated unexpectedly. The process id was '3620'. The process exit code was '0x80131506'.
Then I get this:
Application pool 'Pilot.Mobile.SyncV2' is being automatically disabled due to a series of failures in the process(es) serving that application pool.
Two Things- How can I findout why this is crashing e.g. tools or scritps, also has anyone get any scripts I can run on a schedule task that will check the application pools and if 1 is down it will restart it and email me.
Here's a list of "Useful tools and add-ons for IIS troubleshooting and IIS7".
Tess Ferrandez (ASP.NET Escalation Engineer at Microsoft) has a good blog on debugging.
I don't have any scripts to give you. We use Nagios/OpsView to monitor websites/servers and notify us.

Application Pools not starting after iisreset

Before I start, I know using iisreset is considered bad practice, but this shouldn't happen anyway..
What we have:
Several machines with IIS6 on Windows Server 2003 R2 (both 64 and 32 bits)
Several WCF webservices (.NET runtime 2.0) deployed in several applications, each with it's own application pool, each application pool running under an other windows account.
What happens:
All appPools are started, all services operational
IISReset is executed (or the machine is rebooted)
IIS comes back up, but not all application pools start properly. Sometimes they all come back up, sometimes one or more pools won't start. They can be started manually however.
Is this "normal" iis behavior and should I just avoid using iisreset, or are we doing something wrong in our .NET code?
The application pools should restart on an iisreset, but they do run outside of iis (in COM+) for reliability. This mean they may may not come back if the application is misbehaving, but IIS and the other apps will(should) come back. So yes, This is "normal".
P.S.
I would also like to "OUT" myself as a proud user of iisreset. Bad practice? Bah! ;D
IIS does not immediately start ASP.NET worker processes (w3wp.exe) until the first request comes in. When you say "not started", does it mean you attempt to access some WCF web services (after iisreset), and you get a Service Unavailable error because the appPool cannot be started? Do you see any IIS W3SVC related entries in the Event logs?
If there are, they may be able to clue you in why they cannot start; post them up here.
Reason:
IIS does not immediately start ASP.NET worker processes (w3wp.exe) until the first request comes in. When it says "not started" it mean you attempt to access some WCF web services (after iisreset) failed due to object was holding some space in memory, and you get a Service Unavailable error because the appPool cannot be started.
Workaround:
Create Batch file with following commands & schedule it.
net stop 23svc
net stop msftpsvc
net stop smtpsvc
net stop PleskControlPanel
net stop HTTPFilter
iisreset /restart
net start w3svc
net start msftpsvc
net start smtpsvc
net start PleskControlPanel
net Start HTTPFilter
Had similiar issue - after IIS has been restarted, DefaultAppPool was stopped.
In application event logs found an error:
Windows cannot log you on because your profile cannot be loaded. Check that you are connected to the network, or that your network is functioning correctly. If this problem persists, contact your network administrator.
DETAIL - Access is denied.
Fixed by setting in DefaultAppPool Advanced Settings option Load User Profile to False.
Hope it could be usefull.
In a prior support role, I managed several IIS servers running all kinds of .NET mess. When an AppPool failed to start, it was usually a bad login credential.
Re-setting the identity (auth credentials) via advanced settings of the Application pool worked for me.
Earlier I changed my active directory password and as the app pool was already running, it worked fine until I performed IISRESET.

Resources