I've a a customer using IIS and an application, developed by us, in ASP.NET 1.1.
Monday, for 4 times in a row the following error has appeared:
A process serving application pool 'xxxx' suffered a fatal communication error with the World Wide Web Publishing Service. The process id was 'yyyy'. The data field contains the error number.
Any idea about how to diagnose this? The only link I've found talks about installing low level debug tools but before proceeding in this kind of low level analysis I would know if someone has a better idea or a suitable alternative.
The problem (from what I can see) is something in the customer environment, because the same application it's installed at other customer sites on at least 20 or 30 different servers and the problem do not happen.
Had the same issue on IIS 7, had few reports all worked except one report that was very long and it never worked on IIS7 (it was fine on low spec server).
on IIS7 in application pool's Advance Setting I set "Enable 32-Bit Application" to true and all worked nicely
I'm getting the same error, here are some more details:
running: Windows Server 2003, IIS 6.0 / ASP 3.0,
2.13 GHz, 1 GB ram
My web site is in Beta, so I hardly have any visitors to the site.
According to the event viewer I'm getting this warning 3 times every 3 minutes,
then it stops for a few hours.
Then sometimes I get the error:
A process serving application pool 'DefaultAppPool' terminated unexpectedly. The process id was '3900'. The process exit code was '0x800703e9'.
which follow by:
Application pool 'DefaultAppPool' is being automatically disabled due to a series of failures in the process(es) serving that application pool.
Which then causes a 'Service Unavailable' message when browsing to the website.
after reading too many post about this issue, I did the following steps:
I read that it might be registry access right, so I installed a monitor and trace all the W3SVC Access Denied errors and grant permition
I read that the 0x800703e9 error means stack over flow which causes a w3wp.exe crash and I should install a Debug tool and try to get a memory dump.
I did that but I didn't get any dump, so I installed a new debugging tool, but didn't got a crash yet.
My web site is doing some data mining which keeps the server busy.
Conclusion:
I don't know what is going on there... but I do know that my server machine is way to slow on resource, so I'm going to upgrade and reinstall it, I'm certain that it will solve the problem...
The problem happens all the time, even when my .net code is idle, therefore it is a problem in the server, and not in my code.
I think that the first warnning “A process serving application pool ..." happens every some time, and every now and then it causes the Application Pool to restart, therefore a attaching a debugger doesn't help - the process keeps restarting and the debugger is not effective any longer...
I think that the 0x800703e9 error (which causes the Service Unavailable) maybe happens when the app pool restart, I guess that it needs a lot of resource and since my machine is too slow it gets the 0x800703e9 ... as said before this is a stack over flow, but I think that it is caused by low resource and not by an endless recursion.
I think that the 'registry access right' which is claimed by Microsoft to be the issue, is nonsense, but I didn't got 'Service Unavailable' since so it might helped (thought I still get the warnning “A process serving application pool ..." ).
Hope this help someone...
I'm sure you already know this, but the App Pool contains 1.1 applications only right? I don't remember the error you get when the pool dies by trying to mix frameworks (something like Server Unavailable), but it's more common then I thought in the wild so I would double check.
While not likely the case, it's somewhere to start.
Edit: This KB article also had the error message you described relating to registry permissions, what version of IIS is the client running?
I had this same problem when a website was deployed to a client's web server. This Microsoft support article says:
"This issue may occur if the NT AUTHORITY\NETWORK SERVICE account does not have the permissions to the required registry keys."
And the resolution is: "set the permissions to the required registry keys, and then restart IIS 6.0."
The linked article has the steps for doing this.
One more popular reason (as in my case) - one of windows logs is full.
Related
I have a Windows Server 2008 R2 box serving a couple .NET websites from IIS 7.5 using SQL Server Express 2008 R2.
The sites have been running for a number of years without issue. In the last couple of weeks they have become unreliable and will randomly fail with the error "Cannot open database 'foo' requested by the login. The login failed". They both fail at the same time and once in the broken state they stay broken for all requests. It's failing roughly once a day.
The credentials and permissions are absolutely correct and have worked for years, work in the management studio and work again when I bounce the server.
I have found that stopping/starting each website in IIS does not resolve the issue, but stopping/starting IIS does. The leads me to think that something is getting corrupt or badly cached in memory.
Debugging is a pain as the websites need to be fixed ASAP but it's impossible to trace the issue once it's been fixed.
Anybody seen this before or got any idea where I could look for clues next time it goes down?
I have an application that has a application pool that shuts down a couple of times a day and I can't figure out why. the website is hosted on a VM with SQL Server R2. the version of IIS is 7 and the website has a dedicated application pool using .NET framework v4.0.30319.
At the application level the website is using a target framework of 4.5 and seems to run just fine for a 24-48 hour period and then for some reason the application pool just stops. We do have other .NET 4.5 applications running on this machine and their application pools do not have this problem. The only thing slightly different about this application is that we initiate a separate thread to send a order confirmation email when an order is completed. We average about a couple of hundred orders a day for this website.
According to the event log the error reported prior to the shutdown is
"A process serving application pool 'RetailSupport' suffered a fatal communication error with the Windows Process Activation Service"
Please help
I would check, if your code could hit under some circumstances an endless loop or a recursive
call of methods. This is often the reason for this error.
What always helps me in such cases is to add debug logs (e.g. log4net) in your case especially before the email is sent, status info about resources, etc. They can be more informative then the IIS logs.
If this does not help, check http://support.microsoft.com/kb/919789/en how to use the debug Diagnostic tool, to get more details for the error.
Look also for IIS: Web Application hangs periodically needs system reboot it could help you.
Also: what recycle strategy on IIS do you have?
Back in October we upgraded one of our sites (on IIS 6) from .net 3.5 to .net 4.0. Since then we have been getting the following warning and error pretty much everyday:
Warning:
ISAPI
'c:\windows\microsoft.net\framework\v4.0.30319\aspnet_isapi.dll'
reported itself as unhealthy for the
following reason: 'Deadlock detected'.
Error:
It is not possible to run two
different versions of ASP.NET in the
same IIS process. Please use the IIS
Administration Tool to reconfigure
your server to run the application in
a separate process.
We have checked all applications and ensured that they are running on the correct versions of ASP.NET and also in their own separate app pools.
We have looked at the IIS logs at the time the errors are recorded but this doesn't give any clues as to the issue. Is there a tool available that could give us more information as as to which application is causing the errors and what was happening at the time?
The discussion here might be helpfull for you as well:
You can configure the application pool to orphan the worker process rather than kill it when this condition happens - you can then use adplus to collect a hang dump of the worker process which can be analyzed to figure out what in the process is deadlocked.
(...)
It means that the asp.net thrads have not returned from whatever activity they were doing for a very long time and asp.net has run out of threads - this would normally indicate deadlock. This kb also talks about how to generate dumps for analysis for this condition.
I had the exact same error, and it was due to a .NET 2.0 custom error page that my .NET 4.0 app was referring to.
I had set up a custom error page that was developed in .NET 2.0 and it was applied at the top level of the IIS web site.
The .NET 4.0 application was using a unique MVC URL routing scheme, and for some reason just that application kept locking up and eventually those deadlock errors would come up.
I got rid of the custom error page setting which pointed to the .NET 2.0 web page and switched it back to the default "Not Found" static error. I restarted the application pool and the web page started working without issue.
Hope this helps someone else out there.
http://adamantinewolverine.blogspot.kr/2012/10/running-iis6-in-32-bit-mode-on-64bit.html
Backstory
Last month our development team created a new asp.net 3.5 application to place out on our production website. Once we had the work completed, we requested from the group that manages are server to copy the app out to our production site, and configure the virtual directory as a new application.
On 12/27/2010, two public 'Gineau Pigs' were selected to use the app, and it worked great.
On 12/30/2010, We received notification by internal staff, that when that staff member tried to access the application (this was the Business Process Owner) they recieved the 'Server Application Unavailable' message.
When I called the group that does our server support, I was told that it probably failed, because I didn't close the connections in my code. However, the same group went in and then created a separate app pool for this Extension Request application. It has had no issues since.
I did a little googling, since I do not like being blamed for things. I found that the 'Server Application Unavailable' message will also appear when you have multiple applications using different frameworks and you do not put them in different application pools.
Technical Details - Tree of our website structure
Main Website <-- ASP Classic
+-Virtual Directory(ExtensionRequest) <-- ASP 3.5
From our server support group:
'Reviewed server logs and website setup in IIS. Had to reset the application pool as it was not working properly. This corrected the website and it is now back online. We went ahead and created a application pool for the extension web so it is isolated from the main site pool. In the past we have seen other application do this when there is a connection being left open and the pool fills up. Would recommend reviewing site code to make sure no connections are being left open.'
The Real Question:
What really caused the failure? Isn't the connection being left open issue an ASP Classic issue? Wouldn't the ExtensionRequest application have to be used (more than twice) in the first place to have the connections left open? Is it more likely the failure is caused by them not bothering to setup the new Application in it's own App Pool in the first place?
Sorry for the long windedness
You'd really need to obtain and review the server's Application & System event and HTTPERR logs for the period the server was reporting these errors.
Without these it'd be hard speculate what was the root cause of the problem.
Update:
OP incorrectly tagged his question so this next section no longer applies. However I'll leave in place because I think the information is useful for those encountering these issues and perhaps thinking about migrating to IIS7.x.
You are correct that running two different .NET Framework's in the same application pool can cause these errors but that's something you'd tend to see on Windows 2003/IIS6, not Windows 2008/IIS7.
IIS7 uses a slightly different approach to specifying which .NET Framework version is loaded and it's determined by the Application pool's managedRunTimeVersion property. When requests are processed by IIS/ASP.NET the site's Handler Mapping's use a preCondition attribute to determine when to load the requisite handler (which is kind of like a script mapping in previous versions of IIS).
This mechanism prevents the incorrect runtime version being loaded into the application pool's worker process.
So if an application pool is configured to run .NET Framework version v4.0 only that version will load, even if your application is built against v2.0.
There's a great article on how this works here:
Achtung! IIS7 Preconditions
The section on Handlers about half way through explains why the dangers of accidentally loading the wrong .NET version into a pool are mitigated by the preCondition feature.
A Server Application Unavailable error usually means something catastrophic has happened (like loading the wrong ASP.NET version's ISAPI filter into an already running worker process).
Not closing SQL connections is unlikely to cause this type of serious error. You'd more than likely be seeing a yellow screen of death runtime errors if that were the case. Running out of SQL connections usually doesn't bend ASP.NET so out of shape that the whole service tops itself.
My prime suspect would be a permissions problem where the application pool identity was unable to correctly access the application folders. But it's just a hunch.
Again, what you need to do is get the Application & System event logs and the HTTPERR logs (they reside in %systemroot%\System32\LogFiles\HTTPERR. That will contain clues and facts about what went wrong.
Update 2:
On Windows 2003/IIS6, if you have two applications running different ASP.NET versions that reside in the same pool you will get this error. In my experience (I work for a web hoster) it is the primary cause of this infamous error page:
There's also a tell-tale event logged to the Application Event log:
Event Type: Error
Event Source: ASP.NET 2.0.50727.0
Event Category: None
Event ID: 1062
Date: 12/01/2011
Time: 12:31:43
User: N/A
Computer: KK-DEBUG
Description:
It is not possible to run two different versions of ASP.NET in the same
IIS process. Please use the IIS Administration Tool to reconfigure your
server to run the application in a separate process.
Whilst your root application may not be written in ASP.NET it's likely that something has triggered loading of a different version of the framework into your site's application pool.
there's a rogue web.config in the root...this will trigger ASP.NET to load
there's a wildcard mapping to ASP.NET 1.1 in the site script maps (less likely, but possible)
I'm inclined to think that your new application most certainly ended up in a pool where other sites or applications were running a different framework version. The only way to really find out is to obtain the Application event logs and look for the event shown above.
It's hard to tell; there could be many causes (too many resources used, calling outside of .NET caused something to crash, etc). I would look in the Event log and see if you can find something there.
If you're running different versions of .NET you definitely want separate pools. If you have the option, I would recommend separate pools for each application (even if in the same .NET version).
As far as "closing the connection" (I assume you mean the connection to the database). If you're creating "low level" connections (i.e. SqlConnection, SqlCommand) then make sure you're wrapping them in a "using" statement, otherwise your connection pool can fill up. In my experience though, you should receive regular .NET errors in this case. If you're using an ORM this shouldn't be an issue.
Edit:
If you can't find anything useful in the Event Log, you could try this: http://learn.iis.net/page.aspx/266/troubleshooting-failed-requests-using-tracing-in-iis-7/
Before I start, I know using iisreset is considered bad practice, but this shouldn't happen anyway..
What we have:
Several machines with IIS6 on Windows Server 2003 R2 (both 64 and 32 bits)
Several WCF webservices (.NET runtime 2.0) deployed in several applications, each with it's own application pool, each application pool running under an other windows account.
What happens:
All appPools are started, all services operational
IISReset is executed (or the machine is rebooted)
IIS comes back up, but not all application pools start properly. Sometimes they all come back up, sometimes one or more pools won't start. They can be started manually however.
Is this "normal" iis behavior and should I just avoid using iisreset, or are we doing something wrong in our .NET code?
The application pools should restart on an iisreset, but they do run outside of iis (in COM+) for reliability. This mean they may may not come back if the application is misbehaving, but IIS and the other apps will(should) come back. So yes, This is "normal".
P.S.
I would also like to "OUT" myself as a proud user of iisreset. Bad practice? Bah! ;D
IIS does not immediately start ASP.NET worker processes (w3wp.exe) until the first request comes in. When you say "not started", does it mean you attempt to access some WCF web services (after iisreset), and you get a Service Unavailable error because the appPool cannot be started? Do you see any IIS W3SVC related entries in the Event logs?
If there are, they may be able to clue you in why they cannot start; post them up here.
Reason:
IIS does not immediately start ASP.NET worker processes (w3wp.exe) until the first request comes in. When it says "not started" it mean you attempt to access some WCF web services (after iisreset) failed due to object was holding some space in memory, and you get a Service Unavailable error because the appPool cannot be started.
Workaround:
Create Batch file with following commands & schedule it.
net stop 23svc
net stop msftpsvc
net stop smtpsvc
net stop PleskControlPanel
net stop HTTPFilter
iisreset /restart
net start w3svc
net start msftpsvc
net start smtpsvc
net start PleskControlPanel
net Start HTTPFilter
Had similiar issue - after IIS has been restarted, DefaultAppPool was stopped.
In application event logs found an error:
Windows cannot log you on because your profile cannot be loaded. Check that you are connected to the network, or that your network is functioning correctly. If this problem persists, contact your network administrator.
DETAIL - Access is denied.
Fixed by setting in DefaultAppPool Advanced Settings option Load User Profile to False.
Hope it could be usefull.
In a prior support role, I managed several IIS servers running all kinds of .NET mess. When an AppPool failed to start, it was usually a bad login credential.
Re-setting the identity (auth credentials) via advanced settings of the Application pool worked for me.
Earlier I changed my active directory password and as the app pool was already running, it worked fine until I performed IISRESET.