I have a website which has been up and running absolutely fine for about 8 months now. It's running .net 4 intergrated mode.
Recently i've started to get some "random" w3wp.exe crashes, and after 5 of them, IIS rapid fail protection kicks in and I have to manually login to the server and start the application pool again.
Here's what the event viewer says for the Error:
Faulting application name: w3wp.exe, version: 7.5.7601.17514, time stamp: 0x4ce7afa2
Faulting module name: nlssorting.dll, version: 4.0.30319.296, time stamp: 0x504835c7
Exception code: 0xc00000fd
Fault offset: 0x000000000000191f
Faulting process id: 0x1998
Faulting application start time: 0x01ce6e6b9b80c949
Faulting application path: c:\windows\system32\inetsrv\w3wp.exe
Faulting module path: C:\Windows\Microsoft.NET\Framework64\v4.0.30319\nlssorting.dll
Report Id: d9cf3164-da5e-11e2-8cc5-f46d0440f6d5
Straight after the crashes, i get an "Information" log in the event viewer which at the bottom gives me the location of a .wer file.
This is what the .wer files contains:
Version=1
EventType=APPCRASH
EventTime=130162847687759734
ReportType=2
Consent=1
ReportIdentifier=d7c5e520-da5e-11e2-8cc5-f46d0440f6d5
IntegratorReportIdentifier=d7c5e51f-da5e-11e2-8cc5-f46d0440f6d5
Response.type=4
Sig[0].Name=Application Name
Sig[0].Value=w3wp.exe
Sig[1].Name=Application Version
Sig[1].Value=7.5.7601.17514
Sig[2].Name=Application Timestamp
Sig[2].Value=4ce7afa2
Sig[3].Name=Fault Module Name
Sig[3].Value=nlssorting.dll
Sig[4].Name=Fault Module Version
Sig[4].Value=4.0.30319.296
Sig[5].Name=Fault Module Timestamp
Sig[5].Value=504835c7
Sig[6].Name=Exception Code
Sig[6].Value=c00000fd
Sig[7].Name=Exception Offset
Sig[7].Value=000000000000197d
DynamicSig[1].Name=OS Version
DynamicSig[1].Value=6.1.7601.2.1.0.1296.17
DynamicSig[2].Name=Locale ID
DynamicSig[2].Value=2057
DynamicSig[22].Name=Additional Information 1
DynamicSig[22].Value=6141
DynamicSig[23].Name=Additional Information 2
DynamicSig[23].Value=61419d6dee6cf74b8ac2b00b4c3b3373
DynamicSig[24].Name=Additional Information 3
DynamicSig[24].Value=c19b
DynamicSig[25].Name=Additional Information 4
DynamicSig[25].Value=c19b8acf029a3088171b1f5f3dd9dc77
UI[2]=c:\windows\system32\inetsrv\w3wp.exe
UI[5]=Check online for a solution (recommended)
UI[6]=Check for a solution later (recommended)
UI[7]=Close
UI[8]=IIS Worker Process stopped working and was closed
UI[9]=A problem caused the application to stop working correctly. Windows will notify you if a solution is available.
UI[10]=&Close
LoadedModule[0]=c:\windows\system32\inetsrv\w3wp.exe
LoadedModule[1]=C:\Windows\SYSTEM32\ntdll.dll
LoadedModule[2]=C:\Windows\system32\kernel32.dll
LoadedModule[3]=C:\Windows\system32\KERNELBASE.dll
LoadedModule[4]=C:\Windows\system32\ADVAPI32.dll
LoadedModule[5]=C:\Windows\system32\msvcrt.dll
LoadedModule[6]=C:\Windows\SYSTEM32\sechost.dll
LoadedModule[7]=C:\Windows\system32\RPCRT4.dll
LoadedModule[8]=C:\Windows\system32\pcwum.DLL
LoadedModule[9]=C:\Windows\system32\USER32.dll
LoadedModule[10]=C:\Windows\system32\GDI32.dll
LoadedModule[11]=C:\Windows\system32\LPK.dll
LoadedModule[12]=C:\Windows\system32\USP10.dll
LoadedModule[13]=C:\Windows\system32\ole32.dll
LoadedModule[14]=c:\windows\system32\inetsrv\IISUTIL.dll
LoadedModule[15]=C:\Windows\system32\IMM32.DLL
LoadedModule[16]=C:\Windows\system32\MSCTF.dll
LoadedModule[17]=C:\Windows\system32\CRYPTBASE.dll
LoadedModule[18]=C:\Windows\system32\ntmarta.dll
LoadedModule[19]=C:\Windows\system32\WLDAP32.dll
LoadedModule[20]=c:\windows\system32\inetsrv\w3wphost.dll
LoadedModule[21]=C:\Windows\system32\OLEAUT32.dll
LoadedModule[22]=c:\windows\system32\inetsrv\nativerd.dll
LoadedModule[23]=C:\Windows\system32\CRYPT32.dll
LoadedModule[24]=C:\Windows\system32\MSASN1.dll
LoadedModule[25]=C:\Windows\system32\XmlLite.dll
LoadedModule[26]=C:\Windows\system32\ktmw32.dll
LoadedModule[27]=c:\windows\system32\inetsrv\IISRES.DLL
LoadedModule[28]=C:\Windows\system32\CRYPTSP.dll
LoadedModule[29]=C:\Windows\system32\rsaenh.dll
LoadedModule[30]=C:\Windows\system32\mscoree.dll
LoadedModule[31]=C:\Windows\system32\CLBCatQ.DLL
LoadedModule[32]=C:\Windows\system32\mlang.dll
LoadedModule[33]=C:\Windows\Microsoft.NET\Framework64\v4.0.30319\webengine4.dll
LoadedModule[34]=C:\Windows\system32\MSVCR100_CLR0400.dll
LoadedModule[35]=C:\Windows\system32\USERENV.dll
LoadedModule[36]=C:\Windows\system32\profapi.dll
LoadedModule[37]=C:\Windows\system32\PSAPI.DLL
LoadedModule[38]=C:\Windows\Microsoft.NET\Framework64\v4.0.30319\mscoreei.dll
LoadedModule[39]=C:\Windows\system32\SHLWAPI.dll
LoadedModule[40]=C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll
LoadedModule[41]=C:\Windows\system32\inetsrv\iiscore.dll
LoadedModule[42]=c:\windows\system32\inetsrv\W3TP.dll
LoadedModule[43]=c:\windows\system32\inetsrv\w3dt.dll
LoadedModule[44]=C:\Windows\system32\HTTPAPI.dll
LoadedModule[45]=C:\Windows\system32\slc.dll
LoadedModule[46]=C:\Windows\system32\WS2_32.dll
LoadedModule[47]=C:\Windows\system32\NSI.dll
LoadedModule[48]=C:\Windows\system32\Normaliz.dll
LoadedModule[49]=C:\Windows\system32\faultrep.dll
LoadedModule[50]=C:\Windows\system32\Secur32.dll
LoadedModule[51]=C:\Windows\system32\SSPICLI.DLL
LoadedModule[52]=C:\Windows\system32\NLAapi.dll
LoadedModule[53]=C:\Windows\system32\napinsp.dll
LoadedModule[54]=C:\Windows\System32\mswsock.dll
LoadedModule[55]=C:\Windows\system32\DNSAPI.dll
LoadedModule[56]=C:\Windows\System32\winrnr.dll
LoadedModule[57]=C:\Windows\System32\wshtcpip.dll
LoadedModule[58]=C:\Windows\System32\wship6.dll
LoadedModule[59]=C:\Windows\system32\IPHLPAPI.DLL
LoadedModule[60]=C:\Windows\system32\WINNSI.DLL
LoadedModule[61]=C:\Windows\system32\rasadhlp.dll
LoadedModule[62]=C:\Windows\System32\fwpuclnt.dll
LoadedModule[63]=C:\Windows\System32\inetsrv\cachuri.dll
LoadedModule[64]=C:\Windows\System32\inetsrv\cachfile.dll
LoadedModule[65]=C:\Windows\System32\inetsrv\cachtokn.dll
LoadedModule[66]=C:\Windows\System32\inetsrv\cachhttp.dll
LoadedModule[67]=C:\Windows\System32\inetsrv\compdyn.dll
LoadedModule[68]=C:\Windows\System32\inetsrv\compstat.dll
LoadedModule[69]=C:\Windows\System32\inetsrv\defdoc.dll
LoadedModule[70]=C:\Windows\System32\inetsrv\protsup.dll
LoadedModule[71]=C:\Windows\System32\inetsrv\redirect.dll
LoadedModule[72]=C:\Windows\System32\inetsrv\static.dll
LoadedModule[73]=C:\Windows\System32\inetsrv\authanon.dll
LoadedModule[74]=C:\Windows\System32\inetsrv\authbas.dll
LoadedModule[75]=C:\Windows\System32\inetsrv\authsspi.dll
LoadedModule[76]=C:\Windows\system32\NETAPI32.dll
LoadedModule[77]=C:\Windows\system32\netutils.dll
LoadedModule[78]=C:\Windows\system32\srvcli.dll
LoadedModule[79]=C:\Windows\system32\wkscli.dll
LoadedModule[80]=C:\Windows\System32\inetsrv\iprestr.dll
LoadedModule[81]=C:\Windows\System32\inetsrv\modrqflt.dll
LoadedModule[82]=C:\Windows\System32\inetsrv\logcust.dll
LoadedModule[83]=C:\Windows\System32\inetsrv\custerr.dll
LoadedModule[84]=C:\Windows\System32\inetsrv\loghttp.dll
LoadedModule[85]=C:\Windows\System32\inetsrv\isapi.dll
LoadedModule[86]=C:\Windows\System32\inetsrv\filter.dll
LoadedModule[87]=C:\Windows\System32\inetsrv\validcfg.dll
LoadedModule[88]=c:\Windows\Microsoft.NET\Framework64\v4.0.30319\aspnet_filter.dll
LoadedModule[89]=C:\Windows\system32\inetsrv\wbhst_pm.dll
LoadedModule[90]=C:\Windows\Microsoft.NET\Framework64\v4.0.30319\webengine.dll
LoadedModule[91]=C:\Windows\assembly\NativeImages_v4.0.30319_64\mscorlib\4f52500ab48877b85e71430f4f46670f\mscorlib.ni.dll
LoadedModule[92]=C:\Windows\Microsoft.NET\Framework64\v4.0.30319\nlssorting.dll
LoadedModule[93]=C:\Windows\assembly\NativeImages_v4.0.30319_64\System\a91f32875cb3ba779f1b3ceff1690251\System.ni.dll
LoadedModule[94]=C:\Windows\assembly\NativeImages_v4.0.30319_64\System.Core\0a8d99339ffe6b25debb8f8201c27664\System.Core.ni.dll
LoadedModule[95]=C:\Windows\assembly\NativeImages_v4.0.30319_64\System.Web\5b905bd7b71f9fd6bea2d05cc1ae85f8\System.Web.ni.dll
LoadedModule[96]=C:\Windows\system32\sxs.dll
LoadedModule[97]=C:\Windows\system32\RpcRtRemote.dll
LoadedModule[98]=C:\Windows\assembly\NativeImages_v4.0.30319_64\System.Configuration\fa65f89fd682c459fc5e7bcbd0418317\System.Configuration.ni.dll
LoadedModule[99]=C:\Windows\assembly\NativeImages_v4.0.30319_64\System.Xml\f4afb233f160b8e55aad4660e45b374c\System.Xml.ni.dll
LoadedModule[100]=C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clrjit.dll
LoadedModule[101]=C:\Windows\assembly\NativeImages_v4.0.30319_64\Microsoft.Build.Uti#\14e16d61fae3cd1d9a1fa79b789f8438\Microsoft.Build.Utilities.v4.0.ni.dll
LoadedModule[102]=C:\Windows\assembly\NativeImages_v4.0.30319_64\System.Runtime.Cach#\8fdbe304abab0631b8a4310b35f3d93a\System.Runtime.Caching.ni.dll
LoadedModule[103]=C:\Windows\system32\shfolder.dll
LoadedModule[104]=C:\Windows\system32\SHELL32.dll
LoadedModule[105]=C:\Windows\assembly\NativeImages_v4.0.30319_64\Microsoft.JScript\85204dde340780329b569b025e249c23\Microsoft.JScript.ni.dll
LoadedModule[106]=C:\Windows\system32\version.dll
LoadedModule[107]=C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\root\587f6661\a99d8ff8\App_Code.cgixlnxh.dll
LoadedModule[108]=C:\Windows\assembly\NativeImages_v4.0.30319_64\System.Data.Linq\feaa494ad67542d2060b31b9eeb6458b\System.Data.Linq.ni.dll
LoadedModule[109]=C:\Windows\assembly\NativeImages_v4.0.30319_64\System.Data\b928128fca867546a858a1a39240d85c\System.Data.ni.dll
LoadedModule[110]=C:\Windows\Microsoft.Net\assembly\GAC_64\System.Data\v4.0_4.0.0.0__b77a5c561934e089\System.Data.dll
LoadedModule[111]=C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\root\587f6661\a99d8ff8\assembly\dl3\595a888a\f26c0653_7f81cd01\HtmlAgilityPack.dll
LoadedModule[112]=C:\Windows\assembly\NativeImages_v4.0.30319_64\System.Drawing\5ae853f556290da9399b15b3619f7e15\System.Drawing.ni.dll
LoadedModule[113]=C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\root\587f6661\a99d8ff8\assembly\dl3\85ba5013\f0c8f388_706bce01\TweetSharp.dll
LoadedModule[114]=C:\Windows\assembly\NativeImages_v4.0.30319_64\System.Web.Extensio#\0180a2d993d2a9699cf07f7163524fff\System.Web.Extensions.ni.dll
LoadedModule[115]=C:\Windows\assembly\NativeImages_v4.0.30319_64\System.Transactions\7b2099a1386e38ff198a51939304ce6e\System.Transactions.ni.dll
LoadedModule[116]=C:\Windows\Microsoft.Net\assembly\GAC_64\System.Transactions\v4.0_4.0.0.0__b77a5c561934e089\System.Transactions.dll
LoadedModule[117]=C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\root\587f6661\a99d8ff8\App_global.asax.yxdky-qn.dll
LoadedModule[118]=C:\Windows\assembly\NativeImages_v4.0.30319_64\System.ServiceModel#\7a5a5ff4a0b3bb4ba4bcc13166918e36\System.ServiceModel.Activation.ni.dll
LoadedModule[119]=C:\Windows\system32\bcrypt.dll
LoadedModule[120]=C:\Windows\assembly\NativeImages_v4.0.30319_64\System.Runtime.Dura#\799274e49455d0fe4ca563f42143bef2\System.Runtime.DurableInstancing.ni.dll
LoadedModule[121]=C:\Windows\assembly\NativeImages_v4.0.30319_64\System.Numerics\a66416296451fe6d2d8a5506ca41b23d\System.Numerics.ni.dll
LoadedModule[122]=C:\Windows\assembly\NativeImages_v4.0.30319_64\System.ServiceModel\15834d73d2846fc01ed54488ccfff5c8\System.ServiceModel.ni.dll
LoadedModule[123]=C:\Windows\assembly\NativeImages_v4.0.30319_64\SMDiagnostics\31f93b6be386908ff2727bcd825de0ca\SMDiagnostics.ni.dll
LoadedModule[124]=C:\Windows\assembly\NativeImages_v4.0.30319_64\System.Xaml.Hosting\cf8401f4952deb5303e0d7fd459ce530\System.Xaml.Hosting.ni.dll
LoadedModule[125]=C:\Windows\system32\inetsrv\gzip.dll
LoadedModule[126]=C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\root\587f6661\a99d8ff8\assembly\dl3\3d63b311\fe7c9b8a_706bce01\Hammock.ClientProfile.dll
LoadedModule[127]=C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\root\587f6661\a99d8ff8\assembly\dl3\6a128bd2\c184e08a_706bce01\Newtonsoft.Json.dll
LoadedModule[128]=C:\Windows\system32\rasapi32.dll
LoadedModule[129]=C:\Windows\system32\rasman.dll
LoadedModule[130]=C:\Windows\system32\rtutils.dll
LoadedModule[131]=C:\Windows\system32\winhttp.dll
LoadedModule[132]=C:\Windows\system32\webio.dll
LoadedModule[133]=C:\Windows\system32\credssp.dll
LoadedModule[134]=C:\Windows\system32\dhcpcsvc6.DLL
LoadedModule[135]=C:\Windows\system32\dhcpcsvc.DLL
LoadedModule[136]=C:\Windows\system32\security.dll
LoadedModule[137]=C:\Windows\system32\schannel.DLL
LoadedModule[138]=C:\Windows\system32\ncrypt.dll
LoadedModule[139]=C:\Windows\system32\bcryptprimitives.dll
LoadedModule[140]=C:\Windows\system32\GPAPI.dll
FriendlyEventName=Stopped working
ConsentKey=APPCRASH
AppName=IIS Worker Process
AppPath=c:\windows\system32\inetsrv\w3wp.exe
That nlssorting.dll seems to crop up a lot but I can't seem to find anything online related. The only thing I can find which matches my error is here, but that doesn't really help me.
I'm completely stumped as to where to go from here to fix this. Here's what I've tried:
Loading up IIS log files and trying every request from about 30 minutes before a crash, and none of the pages cause any errors.
Searching my code for any recursion which might cause a stackoverflow, but there isn't any
trawling online for ANYTHING that might help
Has anyone else ever had any problems with nlssorting.dll ? Can i get some more information from the .wer file that might help me pin point where this is happening?
Thanks in advance for any help!
UPDATE
I was using a 3rd party DLL, which was causing a stack overflow exception (0xc00000fd)
After more investigation, it was only happening after a certain chain of events happened - hence the "random" in the title. Removing the DLL fixed the problem.
We had the same problem with one of our sites. Using SVN we tracked it down to a method that was scaning for images within a folder.
I modified the code as follows:
Checking array length of scan results to be > 0 instead of == 1
Adding CultureInfo.InvariantCulture to all Int32.ToString() calls
After this we no longer experienced the error. The exact reason is still unknown.
I believe that none of the above points should make a difference in our environment. I believe that the problem could have been, people modifying image files and folders while the image scanning method was called.
I hope this helps somebody.
For anyone who's curious, this is a PITA to debug. Here are three reasons rumored for this to happen:
(1) Stack overflows, as in the original post.
(2) Too much CPU / memory usage, which becomes obvious and rapid fail protection closes the process.
(3) Unable to respond to pings / requests due to application hogging resources, but in a way that rapid fail protection deems appropriate to end the process, not explicitly because of either (1) or (2).
Our solution was to add manual log tracing in the production environment until we eventually found recursion which was leading the application to be stopped by reliability services (for inability to respond to pings, or process randomly crashing) rather than throwing an in-application exception.
I had the issue where w3p would throw an unhandled error as soon I spun up the site\api URI from the web browser, then it would crash.
I was able to pinpoint what part of my code was causing it in my case it was in the Owin Startup class and I was reading some configuration records from a database but prior to that it gets the connection string from a configuration file outside of the web app directory.
I checked the ownership of the folder it showed my account but apparently the subfolders were not owned by me so I set ownership to me again, clicked OK to allow permissions to traverse the objects and voila bye bye w3p error and the API loaded.
So in my case it was an access denied error on the folder \ file that contained the connection string.
Related
We've one IIS application server on which we have deployed ASP.NET application.
Now the issue is Server is hanging very frequently and every time we are recycling or restarting application pool. Could you please help me to fix it.
What I've tried:
1) Looked into IIS log and found that there are few requests throwing 500 error some time. All the time except few instances same requests are working fine and getting 200 status.
2) Few resources are taking longer time like
a) "Report.aspx" - State- SendResponse - Time Elapsed- 2-3 minutes
I checked the query and found that sp is returning data in 00:00:02.
b) "NewReport.aspx" - State- ExecuteRequestHandler- Time Elapsed- 1 minutes
I checked the query and found that sp is returning data too quickly.
3) Default recycle time is 29 hrs, and each application is using individual pool.
4) In code application connection has been opened and not closed at very few places.
5) We are using connection pooling, having 2 different connection string with Max Pool=500 and min Pool=50 each.
6) Most of the time there is no error in Event Viewer, some time have received TCP/IP error.
What I am looking for:
1) Do I need to make any changes in IIS to track down or please help if any changes required in IIS?
2) Is there any setting in SQL Server which can help me to find out or fix it?
3) Please help if anything else I've to do to fix this issue from your previous experiences.
I think you should capture a hang dump for W3WP.exe with Procdump or debug diagnostic tool first. We need to check all managed stack trace via WINDBG, sos.dll and mex extension. So that we can figure out worker thread is pending on which method and which module. Then might tell us about how to fix this hang issue. If you could find a way to always reproduce this problem. The problem would become much easier.
This link is about how to capture hang dump.
https://blogs.msdn.microsoft.com/asiatech/2014/01/09/debug-diagnostic-2-0-generate-a-manual-hang-dump-for-all-processes-owned-by-iis/
If you don't know how to analyze dump file manually, debug diagnostic analysis tool could help you achieve this.
ASP.NET WebAPI2 application on .NET 4.6.2, hosted on IIS on Windows Server 2016. From time to time, there is a lot (hundreds) of requests stuck for hours (despite the fact I have request timeout 60s set) with no CPU usage. So, I took the memory dump of w3wp process, along with sos.dll, clr.dll and mscordacwks.dll and all my project's dlls and pdbs from bin directory from server and used WinDbg as described in many blogs and tutorials. But, in all of them, they are able to directly see CLR stack by calling ~*e !clrstack. I can see CLR stacktrace for some Redis and ApplicationInsights workers, but for all other managed threads I can see only:
OS Thread Id: 0x1124 (3)
Child SP IP Call Site
GetFrameContext failed: 1
0000000000000000 0000000000000000
!dumpstack for any of these gives just this:
0:181> !dumpstack
OS Thread Id: 0x1754 (181)
Current frame: ntdll!NtWaitForSingleObject+0x14
Child-SP RetAddr Caller, Callee
000000b942c7f6a0 00007fff33d63acf KERNELBASE!WaitForSingleObjectEx+0x8f, calling ntdll!NtWaitForSingleObject
000000b942c7f740 00007fff253377a6 clr!CLRSemaphore::Wait+0x8a, calling kernel32!WaitForSingleObjectEx
000000b942c7f7b0 00007fff25335331 clr!GCCoop::GCCoop+0xe, calling clr!GetThread
000000b942c7f800 00007fff25337916 clr!ThreadpoolMgr::UnfairSemaphore::Wait+0xf1, calling clr!CLRSemaphore::Wait
000000b942c7f840 00007fff253378b1 clr!ThreadpoolMgr::WorkerThreadStart+0x2d1, calling clr!ThreadpoolMgr::UnfairSemaphore::Wait
000000b942c7f8e0 00007fff253d952f clr!Thread::intermediateThreadProc+0x86
000000b942c7f9e0 00007fff253d950f clr!Thread::intermediateThreadProc+0x66, calling clr!_chkstk
000000b942c7fa20 00007fff37568364 kernel32!BaseThreadInitThunk+0x14, calling ntdll!LdrpDispatchUserCallTarget
000000b942c7fa50 00007fff3773e821 ntdll!RtlUserThreadStart+0x21, calling ntdll!LdrpDispatchUserCallTarget
So I have no idea, where to look for bug in my code.
(here is the full result:
https://gist.github.com/rouen-sk/eff11844557521de367fa9182cb94a82
and here is the results of !threads:
https://gist.github.com/rouen-sk/b61cba97a4d8300c08d6a8808c4bff6e)
What can I do? Google search for GetFrameContext failed gives nothing helpful.
As mentioned, this is not trivial, however you can find a case study of similar problem here: https://blogs.msdn.microsoft.com/rodneyviana/2015/03/27/the-case-of-the-non-responsive-mvc-web-application/
In a nutshell:
Download NetExt. It is the zip file here:
https://github.com/rodneyviana/netext/tree/master/Binaries
Open your dump and load NetExt
Run !windex to index the heap
Run !whttp -order -running to see a list of running requests
If the requests contains thread number you can go to the thread to see what is happening
If the requests contains --- instead of thread number, they are waiting a thread and this is a sign that some throttling is happening
If it is a WCF service, run !wservice to see the services
Run !wruntime to see runtime information
Run !wapppool to see Application Pool information
Run !wdae to list all errors
... And so it goes. When you do this again and again you will be able to spot issues easily
i have created one web application in asp.net[.net 4 framework and visual studio 2010].Now i am running with IIS 5.1 version.When i running my application, i am getting the following error
Server Application Unavailable
The web application you are attempting to access on this web server is currently unavailable. Please hit the "Refresh" button in your web browser to retry your request.
Administrator Note: An error message detailing the cause of this specific request failure can be found in the application event log of the web server. Please review this log entry to discover what caused this error to occur.
I am also checked with EventViewer... It says
Event Type: Error
Event Source: ASP.NET 4.0.30319.0
Event Category: None
Event ID: 1001
Date: 8/10/12
Time: 9:55:47 AM
User: N/A
Computer: SARAVANAN-
Description:
aspnet_wp.exe (PID: 1724) was recycled because memory consumption exceeded the 1221 MB (60 percent of available RAM).
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
I dont know how to resolve this issue? Is there any way to increase the memory consumption limit of RAM for my IIS.
Please guide me to get out of this issue...
Bottom line - you should debug the code to seee why it takes this much memory.
But to answer your actual question, yes, there is a way to increase this limit.
In the machine.config file there is a section named 'processModel' and by default it looks like this:
<processModel autoConfig="true"/>
If you change it to add a 'memoryLimit' value this will modify the percentage value used to determine when to recycle the process. For example:
<processModel autoConfig="true" memoryLimit="90"/>
This would allow the process to consume 90% of available RAM before the process was recycled. The default is '60'. You can of course also set a lower value.
I'm not quite sure what's going on, but several issues are occurring on our website supported by Tridion 2011 SP1 that I can only think is being caused by the broker db
1, Loading the website results in the error with the Ambient Data Module:
[NullReferenceException: Object reference not set to an instance of an object.]
Tridion.ContentDelivery.AmbientData.HttpModule.OnRequestStart(Object sender, EventArgs e) +292
System.Web.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() +79
System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) +269
2, Audience manager profile synchronisation between the broker and subscription management databases is failing, receiving the following errors from the logs:
Error occurred while fetching a synchronization batch from the presentation system with url: [URL] Profilesync.aspx. Message class java.sql.SQLException No Data Access Object for AudienceManagerProfile java.sql.SQLException: No Data Access Object for AudienceManagerProfile
3, Publishing the website publication repeatedly fails at the transport stage, returning:
Transport failed: Could not transport tcm_0-10689-66560.Content.zip using HTTPS
I know these sound like several issues happening at once, but my site was running fine up until another web publication was added into blueprinting. Since then we've been getting these errors, and like to think it's all related to an issue with the broker db.
Anyone come across something like this?
UPDATE: Should also add the fact that dynamic linking has stopped working as well, which strengthens my belief there is an issue either connecting to or with the broker database
That's full of loaded questions, oh.. boy.. let's take one by one.
Did you check your license file is valid and did not expire recently.
I would rule the license file out first. If that's not the case then see below.
First, Adding publication blueprinting publication does not have any impact on CDA site. So, I would totally rule this out.
Second, Your site up and running until publication added.. This can't be happening. There should be some configurations changed/added on your web site. Sometimes, the configurations updated but the APPPool did not recycled so you don't see the impact. At later time, when the AppPool restarted/recycled you will notice the errors and might have the impression that the site is breaking all of sudden.
Did you double check your configs and dlls?
Are you able to connect to your database directly with broker user
id/password?
Any firewall changes happened recently, try connecting to DB from
CDA Server
For #3, I had this same problem... is it possible that adding the new publication caused the package size to increase from less than 30mb to more than 30mb? If so, check IIS Request Filtering >> Edit Feature Settings and look at Maximum allowed content length. This defaults to 30MB and for us increasing this size to something larger than our package size solved the Transport Failed error (since IIS would just reject our transports due to the size being greater than 30mb).
About issue #1: Could it be you have the Tridion.ContentDelivery.AmbientData.HttpModule configured in the Web.config, but are lacking the java installation? Do you have a cd_ambient_conf.xml in your config folder? And a cd_ambient.jar in the lib?
About issue # 2: you seem to be missing the audience manager DAOs (Data Access Objects). Do you have the following in your cd_storage_conf.xml?
<StorageBindings>
<Bundle src="AudienceManagerDAOBundle.xml"/>
</StorageBindings>
About issue # 3: do you have more information in the cd_transport.log or windows event viewer?
I agree with Ram that the added publication couldn't have caused this. It looks like your CD installation was changed somehow. Did you do an upgrade or something?
Log Name: System
Source: Microsoft-Windows-WAS
Date: 13.06.2012 08:34:45
Event ID: 5002
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: WIN-QU022VN756J
Description:
Application pool 'DefaultAppPool' is being automatically disabled due to a series of failures in the process(es) serving that application pool.
Please provide near solution for this problem. I will appreciate the suggestion
IIS comes with a feature called Rapid-Fail Protection. By default an App Pool is configured to become disabled if a worker in the pool crashes 5 times within a 5 minute span.
The thresholds of this setting are able to be modified by right clicking on the App Pool going to Advanced Settings and modifying the 'Rapid-Fail Protection' properties. However, this event indicates that there is likely a serious issue in your application that needs to be resolved.
I got tensed about this issue and finally i got solved.
Please make sure that the following option is correct in Advance Settings in Application pool.
Application Pools>Default Application Pool > Right click and select Advance Settings.
And then check whether the option "Enable 32 bit applications" has been set as "False". It should be False
I got solved by did the above change.
I Got this error when i installed URL Rewrite Module, install correct URL Rewrite Module to fix this.
To verify this error cause of URL Rewrite Module check events logs .
windows logs>> System>> Click on recent Error, In Error details
it will show "Application pool 'DefaultAppPool(or
YourApplicationpool)' is being automatically disabled due to a
series of failures in the process(es) serving that application
pool."
Windows logs>> Application>> click on recent error, In error
details it will show "The Module DLL
'C:\Windows\system32\inetsrv\rewrite.dll' failed to load. the data
is the error."
I hope this will help You.
It's a registry permission issue. This should help:
http://support.microsoft.com/kb/885654
You might also need to add read/write permissions for NT AUTHORITY\NETWORK SERVICE (or NT AUTHORITY\SYSTEM or the account used by your application pool) to c:\Windows\TEMP.
See http://social.technet.microsoft.com/Forums/en-US/winservergen/thread/176e9b6d-7b1b-4667-b55e-330bec133f37/