I have this error in ASP.NET application , NET 4.7.1.
The request queue limit of the session is exceeded.
Full:
System.Web.HttpException (0x80004005): The request queue limit of the session is exceeded.
at System.Web.SessionState.SessionStateModule.QueueRef()
at System.Web.SessionState.SessionStateModule.PollLockedSession()
at System.Web.SessionState.SessionStateModule.GetSessionStateItem()
at System.Web.SessionState.SessionStateModule.BeginAcquireState(Object source, EventArgs e, AsyncCallback cb, Object extraData)
any suggestions ?
The default behavior has changed in .NET 4.7. Retargeting guide suggests:
To restore the old behavior, you can add the following setting to your web.config file to opt-out of the new behavior.
<appSettings>
<add key="aspnet:RequestQueueLimitPerSession" value="2147483647"/>
</appSettings>
Clarification of changed behavior:
In the .NET Framework 4.6.2 and earlier, ASP.NET executes requests
with the same Sessionid sequentially and ASP.NET always issues the
Sessionid through cookies by default. If a page takes a long time to
respond, it will significantly degrade server performance just by
pressing F5 on the browser. In the fix, we added a counter to track
the queued requests and terminate the requests when they exceed a
specified limit. The default value is 50. If the limit is reached, a
warning will be logged in the event log, and an HTTP 500 response may
be recorded in the IIS log.
Also addressed here: https://knowledgebase.progress.com/articles/Article/The-request-queue-limit-of-the-session-is-exceeded-in-sitefinity-11-2
Some time this error is generated by to many redirects on server side, after investigation I detect that in the fact user is redirected to same action by ActionsFilter after I fixed this error has not occurred, I think if you investigate IIS logs you are more likely to find the same problem.
PS. For this case setting RequestQueueLimitPerSession will not solve the problem.
TO REPRODUCE: Open IE 11 open the specified path and press F5 for 60 sec. It will generate a lot of requests to this path and if we'll take a look to iis then we will find some requests with win-32 status = 64
Quite analyzing of IIS logs will give you a lot of information about nature of this requests/user agent/all accessed paths/request status/...
I was getting the same error in my MVC application (.NET version 4.7.2) on days with unusually high activity. I fixed it by adding the necessary table indexes in the application's database. In my case, the solution was not to adjust the "aspnet:RequestQueueLimitPerSession" setting but to address the underlying problem regarding database performance that caused the session requests to exceed the default limit.
Related
I have an awkward issue with IIS 10.0 on Windows Server 2016 and ASP.Net 4.5.2 and MVC 5.2.7.
At times, certain requests do not receive a response and run for minutes, maybe 10 or so, before ending in a lost connection (PR_CONNECT_RESET_ERROR in Firefox on Windows, NSURLDomainError in Firefox on iOS). These are mostly POST requests. When this issue occurs, other GET requests will receive a swift response and a correct result. Normally, POST-request do no take long to be processed, typically less than 3 seconds.
Recycling the associated worker process will make the issue go away, for hours or days.
When today inspected the web server when the issue was going on, I saw little CPU usage, less than 10%, memory 56%, the worker process a modest 615 MB. I saw neither logging in the W3C log of these requests, nor in my custom application logs.
I added the Web-Request-Monitor conform How do I see currently executing web request on IIS 8, but in doing so, the the worker process probably got recycled, as the issue is not currently occurring.
There are a reverse proxy and an access manager between the internet and my web server. I suppose they can have something to do with this issue, but it certainly is related to IIS, as recycling helps.
All of this is happening on a acceptation web server running a newer version of my application. I am not aware of any big changes to the application's architecture that could be involved. Also, there will be very little traffic from other clients, if none at all.
What could be next steps to investigate this issue further?
Update
This issue was definitely caused by log4net. However, it was not related to the log4net.Internal.Debug setting. It was caused by two application domains accessing the same log file. This occasionally resulted in concurrency issues with accessing the log file. It appeared that log4net could not properly handle this and got stuck while writing to the log file.
This log file was configured with the RollingFileAppender option. Since we also used AdoNetAppender, we decided to remove file logging all together.
Original
I have found a probable cause. I'll report the steps I took to investigate the issue.
I activated the Worker Processes feature in IIS.
When, after a couple of days of waiting, the issue started again, I found long running requests. They all had State ExecuteRequestHandler and Module Name ManagedPipelineHandler. They had Time Elapsed of hundreds of seconds.
I also activated the Failed Requests Tracing with a rule for long running requests with a Time Taken of 1 minute.
After a couple of days, I started to receive failed request reports. The failed request all have a GENERAL_SET_RESPONSE_HEADER event as their last event.
I added additional debug logging events for each requests. When debugging in my development environment, at one point, I started to see the hanging behaviour there, on one of the new logging statements(!). The application uses log4net.
I captured a stack trace:
log4net.dll!log4net.Appender.AppenderSkeleton.DoAppend(log4net.Core.LoggingEvent loggingEvent) log4net.dll!log4net.Util.AppenderAttachedImpl.AppendLoopOnAppenders(log4net.Core.LoggingEvent loggingEvent) log4net.dll!log4net.Repository.Hierarchy.Logger.CallAppenders(log4net.Core.LoggingEvent loggingEvent) log4net.dll!log4net.Repository.Hierarchy.Logger.Log(System.Type callerStackBoundaryDeclaringType, log4net.Core.Level level, object message, System.Exception exception) log4net.dll!log4net.Core.LogImpl.DebugFormat(string format, object arg0)
The DoAppend method uses lock(this), which may very well cause hangs.
I also found out that the config setting log4net.Internal.Debug was set to true, which I do not want under normal circumstances and this may be related. I did not attempt to understand the log4net code, but I remember that logging initially did not work, in the acceptance environment, so the setting may very well have been set to true then, causing the issue to start.
Another indication that this is happening with log4net is that when the issue last occurred, I realized that logging of level standard, only occurs in some POST requests. I found a POST-request that does not log and requests to it where handled normally, while the other POST-requests still hung.
For now, I have set log4net.Internal.Debug to false and will wait to see what happens.
IIS recycle fix this issue doesn't mean that this is an IIS issue because all asp.net application run in .net runtime unless it is proved that the request is hang in IIS module.
So you may need to wait this issue happen again, then create a Failed request tracing rule for time-taken. Then it will tell us this issue is happening on IIS pipeline module or .net runtime.
If all request hang in .net runtime. Then you may have to capture a hang dump and do a deep analysis via WINDGB and mex extension. It will tell us what's happening there.
(Cross posting from Server Fault where I wasn't getting any traction):
I have been trying to chase down the root cause of a condition where ASP .NET session state remains locked after a web request has been terminated due to an unexpected error. We use the SQL Server session state provider for session because we have several servers in a web farm. This issue first presented itself in the form of many requests getting stuck on the 'AcquireRequestState' event of their lifecycle for no apparent reason. I was able to finding corresponding entries for these requests in the session state database in SQL server that were all locked (column Locked = 1). I was also able to correlate these requests to entries in the IIS log with HTTP status codes of 500 (with a sub status of 0). These findings lead me to believe that, in some cases, a request was erroring out but was NOT releasing its lock on session state like it should.
I enabled Failed Request Tracing in IIS for the website in question for status code 500 with all available providers selected each with the 'Verbose' setting for verbosity. I've since gathered several failed traces that have caused permanently locked ASP .NET sessions. They all share the same characteristics:
They are all 'POST' requests where the browser is posting data to be processed/saved.
They all have events indicating that the 'Session' module was invoked during the REQUEST_ACQUIRE_STATE event. At this point the request would have marked the row in the session state database as being "locked". This is normal and expected.
They all have GENERAL_READ_ENTITY_START, GENERAL_READ_ENTITY_END, and GENERAL_REQUEST_ENTITY entries that appear to be reading in the data that was posted to the server as part of the request. This appears to be a buffered operation as these events get repeated over and over with each one reading in some subset of the posted data.
At some point during the 'read entity' related events an error occurs. Some have the error code "Incorrect function. (0x80070001)" and others have "The I/O operation has been aborted because of either a thread exit or an application request. (0x800703e3)".
Once the error has been encountered, they all jump directly to the END_REQUEST events.
The issue here is that, under normal circumstances, there should be a RELEASE_REQUEST_STATE event that will allow the Session module to release the lock it has on the session. This event is being skipped in this scenario. Just to be sure, I enabled failed request tracing for the '200' status code as well and generated several traces of successful requests that do have the RELEASE_REQUEST_STATE event being handled by the Session module.
A co-worker pointed out that you can also cause a request to skip directly to the 'END_REQUEST' event by calling HttpContext.Current.ApplicationInstance.CompleteRequest(). I tested this out and saw that using this method during a post request creates a trace very similar to the ones I've been capturing when this issue has been happening, but session does still get cleaned up properly. This lead to me to running SQL Profiler on the SQL Server database where the session is stored to trace all calls to stored procedures. When we skip directly to END_REQUEST due to calling CompleteRequest(), a call is made to update the session state (and release the lock) as expected. When we skip to END_REQUEST as a result of an error during GENERAL_REQUEST_ENTITY, the call to update or release the lock on session state is never made.
My theory at this point is that some kind of network issue is causing the 'Incorrect function' and 'I/O operation has been aborted because of either a thread exit or an application request' errors, but I don't understand why this seems to be causing the request handling to skip over the releasing the lock on session state. If the request went through REQUEST_ACQUIRE_STATE it seems like it should also release the lock at some point toward the end of the request as well. I'm loathe to say that this is a bug in IIS or ASP .NET, but it certainly appears that way to me at this point.
Are there any known conditions under which errors will lead to a session state lock not being released?
As it turns out, this was related to this question: ManagedPipelineHandler for an AJAX POST crashes if an IE9 user navigates away from a page while that call was in progress
The workaround specified in the accepted answer on that question does work, but Microsoft has also since released a hotfix (not yet publicly available as of this writing) that patches the session handling logic to avoid the issue all together.
From time to time, some requests on my website starts to hang on the RequestAcquireState state of the Session module. When that spiral begins all requests timeout and we need to restart the IIS on the affected server.
I investigated it a lot and the only conclusion I got is that somehow a deadlock is happening while the application tries to access user data stored in Session.
The only option I can think of to fix this issue is to either reduce or stop using Sessions in my application. This is definetely part of the plan, but it will take a while before we can complete that.
We run 6 machines with IIS 7.5, out of proc StateServer and server affinity on in our Load Balance.
Any hints on how to workaround this issue or fix it at all without having to remove Sessions entirely?
Lock mechanism exist on both provider and session module (IIS Session Module). You can develop custom session module, but you still need provider without locking or You can develop custom provider without locking but you still need IIS session module and it is not so simple to implement at that level.
The Solution is UnlockedStateProvider [aka Unlocked]
Follow the white rabbit :P (Check the demo project, it explains everything.)
The answer is Hotfix Rollup 2828841 for .NET Framework 4.5 , here all the explanation:
http://forums.asp.net/t/1888889.aspx/2/10?Question+regarding+a+possible+bug+within+NET+4+5
and here the download link
It works for me on IIS 7.5 Windows Server 2008 rs x64 , asp.net web forms application with lot of ajax request.
I just found out today that if you have a long running request (or in my case, an infinite loop), then all subsequent requests will be locked, because by default ASP.NET locks on session.
So if you have users with requests in RequestAcquireState, then check if there is a request in ExecuteRequestHandler that is locking the session, and thus preventing other requests from starting.
There is a discussion here on how to prevent locking on session.
(Basically, create most of your pages as Session-Read-Only, and modify session as rarely as you can.)
Is it possible those users have another long running request and the requests you see piling up are actually secondary requests? By default, ASP.NET will lock Session until a request is complete. If a second request comes in before the first one is complete, it will have to wait. If you are using MVC, you can change this behavior by adding an attribute to your controller.
[SessionState(SessionStateBehavior.ReadOnly)]
This makes Session read-only, removing the locking behavior allowing subsequent requests to be processed.
I have an ASP.net application where Users aren't able to successfully complete certain actions, for reasons, I'm assuming, can only be related to losing their session (which is where I maintain their current user information, and how determine whether they are logged in)
I'm at a loss as to why they would lose their session, so my first question is:
What (in general) would cause a user to lose their session in ASP.net?
and since I don't know when a user loses their session and can't reproduce it myself:
How can I track when I user loses their session
Below is my sessionState config for reference
<sessionState
mode="InProc"
cookieless="false"
cookieName="My.Site.Com"
timeout="480"/>
A number of things can cause session state to mysteriously disappear.
Your sessionState timeout has expired
You update your web.config or other file type that causes your AppDomain to recycle
Your AppPool in IIS recycles
You update your site with a lot of files, and ASP.NET proactively destroys your AppDomain to recompile and preserve memory.
-
If you are using IIS 7 or 7.5, here are a few things to look for:
By default, IIS sets AppPools to turn themselves off after a period of inactivity.
By default, IIS sets AppPools to recycle every 1740 minutes (obviously depending on your root configuration, but that's the default)
In IIS, check out the "Advanced Settings" of your AppPool. In there is a property called "Idle Time-out". Set that to zero or to a higher number than the default (20).
In IIS, check the "Recycling" settings of your AppPool. Here you can enable or disable your AppPool from recycling. The 2nd page of the wizard is a way to log to the Event Log each type of AppPool shut down.
If you are using IIS 6, the same settings apply (for the most part but with different ways of getting to them), however getting them to log the recycles is more of a pain. Here is a link to a way to get IIS 6 to log AppPool recycle events:
http://web.archive.org/web/20100803114054/http://surrealization.com/sample-code/getnotifiedwhenapppoolrecycles/
-
If you are updating files on your web app, you should expect all session to be lost. That's just the nature of the beast. However, you might not expect it to happen multiple times. If you update 15 or more files (aspx, dll, etc), there is a likelyhood that you will have multiple restarts over a period of time as these pages are recompiled by users accessing the site. See these two links:
http://support.microsoft.com/kb/319947
http://msdn.microsoft.com/en-us/library/system.web.configuration.compilationsection.numrecompilesbeforeapprestart.aspx
Setting the numCompilesBeforeAppRestart to a higher number (or manually bouncing your AppPool) will eliminate this issue.
-
You can always handle Application_SessionStart and Application_SessionEnd to be notified when a session is created or ended. The HttpSessionState class also has an IsNewSession property you can check on any page request to determine if a new session is created for the active user.
-
Finally, if it's possible in your circumstance, I have used the SQL Server session mode with good success. It's not recommended if you are storing a large amount of data in it (every request loads and saves the full amount of data from SQL Server) and it can be a pain if you are putting custom objects in it (as they have to be serializable), but it has helped me in a shared hosting scenario where I couldn't configure my AppPool to not recycle couple hours. In my case, I stored limited information and it had no adverse performance effect. Add to this the fact that an existing user will reuse their SessionID by default and my users never noticed the fact that their in-memory Session was dropped by an AppPool recycle because all their state was stored in SQL Server.
I was having a situation in ASP.NET 4.0 where my session would be reset on every page request (and my SESSION_START code would run on each page request). This didn't happen to every user for every session, but it usually happened, and when it did, it would happen on each page request.
My web.config sessionState tag had the same setting as the one mentioned above.
cookieless="false"
When I changed it to the following...
cookieless="UseCookies"
... the problem seemed to go away. Apparently true|false were old choices from ASP.NET 1. Starting in ASP.Net 2.0, the enumerated choices started being available. I guess these options were deprecated. The "false" value has never presented a problem in the past - I've only noticed in on ASP.NET 4.0. I don't know if something has changed in 4.0 that no longer supports it correctly.
Also, I just found this out not long ago. Since the problem was intermittent before, I suppose I could still encounter it, but so far it's working with this new setting.
In my case setting AppPool->AdvancedSettings->Maximum Worker Proccesses to 1 helped.
Your session is lost becoz....
JUST MAKE SURE THERE ARE NO RUNTIME ERRORS, ANY FATAL EXCEPTION WOULD
KILL THE SESSION!
In Microsoft stack, Visual Studio - put Ctrl + Alt + E - All Exceptions ON, then run the code in Debugging mode. Any Fatal ones are THE reason for session loss..
You could add some logging to the Global.asax in Session_Start and Application_Start to track what's going on with the user's Session and the Application as a whole.
Also, watch out of you're running in Web Farm mode (multiple IIS threads defined in the application pool) or load balancing because the user can end up hitting a different server that does not have the same memory. If this is the case, you can switch the Session mode to SQL Server.
I was only losing the session which was not a string or integer but a datarow.
Putting the data in a serializable object and saving that into the session worked for me.
Had a problem on IIS 8 when retrieving Content via Ajax. The issue was that MaximumWorkerProcesses was set to 2 and Javascript opened 17 concurrent requests. That was more than the AppPool could handle and a new pool (without auth-data) was opened.
Solution was to Change MaximumWorkerProcesses to 0 in IIS -> Server -> Application Pools -> [myPool] -> Advanced Settings -> Process Model -> MaximumWorkerProcesses.
Dont know is it related to your problem or not BUT Windows 2008 Server R2 or SP2 has changed its IIS settings, which leads to issue in session persistence. By default, it manages separate session variable for HTTP and HTTPS. When variables are set in HTTPS, these will be available only on HTTPS pages whenever switched.
To solve the issue, there is IIS setting. In IIS Manager, open up the ASP properties, expand Session Properties, and change
New ID On Secure Connection to False.
I had same problem by losing sessions. every time , every page reload, my sessions clear and by new reload any page, my sessions returned by valid value...
i fixed it by change MaximumWorkerProcesses from 0 to 1 in iis
I was struggling with this issue for 14 days.
Here's what helped me:
Check your recycling options in App Pool > Advanced settings. Turn off all of the options so it won't recycle on its own.
Check your web.config file for the executionTimeout property under httpRuntime and increase its value.
Check your web.config file for the timeout property under sessionState and increase its value (I set it to 300 minutes).
Go to the server's event log and check the Application log for unhandled exceptions that may cause the worker process to crash. Fix them in your code or use try and catch to eliminate this crash.
Try changing the value of your maximum worker process from 0 to 1 or the other way around, this may also solve this issue.
In my case, session state was loosing due to Load Balancer. Session was storing in one server and Load balancer was redirecting next call to another server where session state was missing.
Here is my current question:
I'm guessing that my problem (described below) is being caused by ASP.NET worker processes being recycled, per the answers below—I'm using InProc sessions storage and don't see much chance of moving away, due to the restriction for other types of storage that all session objects be serializable. However, I can't figure out what would make the worker process be recycled as often as I'm seeing it—there wasn't any changing of the files in the app directory as far as I know, and the options in IIS seem to imply that the process would only be recycled every 1,740 minutes—which is much less frequent than the actual session loss. So, my question is now, what different cases can cause an ASP.NET worker process to be recycled?
Here is my original question:
I have a difficult-to-reproduce problem that occurs in my ASP.NET web application. The application has one main .aspx page that is loaded and initializes a number of session variables. This page uses the ASP.NET Ajax Sys.Net.WebRequest class to repeatedly access another .aspx page, which uses the session variables to make database queries and update the main page (the main page is never re-requested).
Occasionally, after a period of time using the page, causing successful HTTP requests where the session created in the main page properly carries over to the subpage, one of the requests seems to cause a new ASP.NET session to be created—all the session variables are lost (causing an exception to be thrown in my code), and a new session id is reported in the dynamically requested page. That means that suddenly, the main page is disconnected from the server—as far as the server is concerned, the user is no longer logged in.
I'm nearly positive it's not a session timeout—the timeout time is set to something ridiculous, the amount of time it takes to get this to happen is variable but is never long enough to cause the session to time out, and the constant Sys.Net.WebRequests should refresh the session timer.
So, what else could be happening that would cause the HTTP requests to lose contact with the ASP.NET session? I unfortunately haven't been sniffing network traffic when this has happened to me, or I would've checked if the ASP.NET session cookie has stuck around or not.
One solution would be to use a StateServer, rather than InProc session management.
Lots of things can cause the session state to be lost:
Editing Web.Config
IIS resetting
etc.
If the session state is important to your app then use either SQL state management, or the State Server which ships with ASP. NET.
Cheers,
RB.
We had problems of Session when we did migrating the AnkerEx application to the
new server. The new server had Microsoft Windows Server 2008 as operation system
and Microsoft Internet Information Services 7. Also in the server were installed
.NET Framework of versions 1.0.3705, 1.1.4322, 2.0.50727, 3.0 and 3.5.
For solving of this problem i have done enabling health monitoring for
application's Lifetime related events in ASP.NET 2.0. I had added to the web.config:
...
...
<system.web>
...
...
<healthMonitoring>
<rules>
<add name="Application Events"
eventName="Application Lifetime Events"
provider="EventLogProvider"
profile="Default"
minInterval="00:01:00" />
</rules>
</healthMonitoring>
...
...
It is help to us to check the AppDomain recycles. We can see it at our Event Viewer.
The link to more details is http://blogs.msdn.com/rahulso/archive/2006/04/13/575715.aspx
After I have done adding to web.config, the Event Viewer showed me that my
application is restarting every time when i do click to almost any link in my
application.
From the article of http://blogs.msdn.com/toddca/archive/2005/12/01/499144.aspx i
found out that ASP.NET has the new behavior - if we will do deleting, for example
a sub-directory of the application's root directory, then ASP.NET 2.0 will do the
restarting AppDomain.
The problem was in that that I had in the web.config the instruction:
...
<compilation debug="true" tempDirectory="c:\AnkerEx\Temporary ASP.NET files">
...
I.e. the ASP.NET did compiling of aspx pages in folder of my application root.
I think he created folders, may be and did removing some of them also. I removed
tempDirectory instruction and the application began work stable.
The worker process is probably cycling.
http://www.lattimore.id.au/2006/06/03/iis-dropping-sessions/
It could be caused by an unhandled exception in a background thread. It can cause your ASP.NET worker process to terminate. A new process is started very quickly so you don't actually notice it but all your sessions are lost.
Here is an article that explains it much better than I can: ASP.NET 2.0 Unhandled Exception Issues
quote:
An unhandled exception in a running ASP.NET 2.0 application will usually terminate the W3WP.exe process, and leave you with a very cryptic EventLog entry something like this:
"EventType clr20r3, P1 w3wp.exe, P2 6.0.3790.1830, P3 42435be1, P4 app_web_ncsnb2-n, P5 0.0.0.0, P6 440a4082, P7 5, P8 1, P9 system.nullreferenceexception, P10 NIL."
Here is a Microsoft KB article that explains the same issue: KB911816 Unhandled exceptions cause ASP.NET-based applications to unexpectedly quit in the .NET Framework 2.0
My guess would be memory consumption - but, set up IIS to log recycles and you'll know for sure.