Yesterday, my customer played with the IIS settings and he changed the number of Working Process to 2 which made my Web Application ran very weird, the Session State sometimes was lost, sometimes was recovered and took me a day to find out what happened. So, the question is Multiple Working Process is useful in what situation?
It can be useful for scaling a web app vertically. Especially poorly designed ones that do too much work in the web processes or one where processes crash frequently so you always have a hot one. It isn't an option that should be exercised lightly as you have found out, but it is good to have when you need it.
The reason your user sessions started to fail was because you are using the default in-process session state module. This is fairly easy to fix as well -- just run the session state out of process using either the session state service or a database. Note that some behavior of the session state changes when you do this as well, so you will need to test carefully that you don't break something else.
Related
in an old ASP.NET Web Forms application, developed in Visual Studio 2010,
suddenly does not run anymore, and in the log file appears this message:
Session state has created a session id,
but cannot save it because the response was already flushed by the application.
No new deployment has been made, and no code modifications take place.
Until now I didn't find any solution to this.
What I have to check?
I state that the source code is no longer available, and therefore it would be very difficult to change the code and proceed with a new deployment.
Thanks in advance.
Luis
This would suggest that someone might be hitting the site and jumping directly to some URL (and thus code) that say does some response redirect to another page or some such.
Remember, when code behind runs, and say re-directs to another page, in most cases the running code for the current page is terminated, and that is normal behaviors.
However, the idea that you going to debug code and debug a web site when you don't have the code to debug? Gee, I don't see how that's going to work at all. As noted, if this just started, then it sounds like incoming requests are to pages that don't expect to be hit "first", but some pages that expect to be ONLY called from other pages in the site when some session() and imporant values are setup BEFORE such pages are to be hit.
It also not clear if the site is using sql based sessions, or just in-memory sessions. In memory can (and is) faster, but it also not particually relaible. Now, if you deployed to a new web server or new hosting, then often session errrors can now start to appear, and this is due to the MASSIVE HUGE LARGE DIFFERENT of using cloud based hosting vs that of older hosting soluions that run on a single server.
Clould computing is real utility computing, and thus when you host a web site on such systems, then in-memory session() cannot be used anymore, since multiple servers can and will be used to "dish out" web pages. Since more then one server might be used, then obvisouly in-memory sesson() can't work, since a few web pages might be served out by one server, and then a few more pages might be served out by another server. And using shared memory for a session is limited to ONE server, since multiplel servers don't and can't transfer their memory to other servers.
So, this suggests that you want to be sure that sql server based sessions are being used here - and for any kind of server farm, or any kind of system that does load balances between more then one server, then of course you HAVE to use sql server based sessions, since in memory can't work in that kind of environment.
The error could also be due to excessive server loads - often the session database is "locked" for a short period of time, and can thus often be a bottleneck. So, for say years you might not see a issue, but then as load and use of the web site increases, then this can become noticed where as in the past it was not. I suppose the database used for storing sessions could be checked, or looked at, since as you note, you don't have the ability to test + debug the code. I would REALLY but REALLY work towards solving and fixing this lack of source code for the web site, since without that, you have really no means to manage, maintain, and fix issues for that web site.
But, abrupt terminations of web pages? As noted, this could be a error triggered in code, and thus the page never finished what it was supposed to do. And as noted, perhaps a page that expects some session() values but does not have them as explained above could be the problem. It not clear if your errors also shows what URL this was occurring for.
While nothing seems to have changed - something obviously did.
Ultimate, you need to get that source code, or deal with the people + vendor that supplies the code for that site. If you don't have a vendor, and you don't have source code, you quite much attempting to work on a car that you cant even open the hood to check what's going on under that hood.
so, one suggestion here? Someone is hitting a page that expected some value(s) in session to exist. Often the simple solution is to shove ANY simple or dummy value into session so session REALLY does get created, and then when the page attempts to save the session(), there is one to save!!!
In other words, this error often occurs when session is attempted to be saved, but no sesison exists. For such pages, as noted, a simple tiny small code change of doing this session("zoozoo") = "my useless text" will fix this error. So, it sounds like session is being lost.
As noted, a error on a web page can also trigger a app-pool re-start. If app-pool re-starts, then session is lost (in memory session). Now, with session being lost, then any page that decides to terminate early AND ALSO having used session() will trigger this error.
So, this sounds like app-pool is being re-started and session is being lost. (you can google why app-pool restarts and for the many reasons). However, critical to this issue would be are you using sql based sessions, or in-memory (server) sessions? So, this sounds like some code is triggering a error, and with a error triggered, app-pool re-starts. And with app-pool being restarted, then in-memory session is blown away. And now, without ANY session at all, then attempts to save the session trigger the exact error message you see. (and this is why shoving a dummy value into the session allows and can fix some pages - since you can't save a "nothing" session, and if you do, then you get that exact error message.
but, as noted, you can't make these simple changes to code anyway, right?
But, first on this issue - are you using memory based sessions or not? And that feature can be setup and configured in IIS, and without changes to the code base. So, one quick fix might be to turn on sql server based sessions. It will cost web site performance (10%), but the increased reliability is more then worth the performance hit.
Another area to look at? Are AJAX calls being made to a page, and again without any previous session having been created? So, once again, we down to a change in end user behaviors, and possible those hitting a page first before having logged in, or done other things - and again one would see this error crop up.
At work we have a fairly good process for zero-downtime deployments of a busy ASP.NET MVC web site. There's two server groups so we can move traffic over, we write db SP's to be backwards compatible and roll them out in advance. Generally, it works well.
However, one issue has just tripped me up. The last deployment included a lot of code namespace changes.
When the build was deployed it broke existing user sessions because the objects in their session state no longer existed on the new codebase e.g. their session contained App_Code.UserDetails and the site now has a class of X.Y.Z.UserDetails
I understand the problem and why it happened, but what I can't work out is if there's anything that can be done apart from a full site shutdown to ensure there are no active users on the site. Or maybe restarting the state service so everyone gets booted off and has to log in again. Any ideas folks?
After thinking this over I've arrived at two possible solutions:
a) Always deploy web site updates with downtime. Keep the site down long enough for all sessions to expire. This is a safe option technically but unacceptable commercially
b) Simulate the problem and look at what can be done address the issue in the exception handler. Although not an ideal end-user experience it may be better to catch SerializationExceptions on session state, and flush the user session. They will have to authenticate again but it's not as bad as a broken session.
A decision I made upfront is that it's going to be difficult to pick up on live session dependencies during development, there are just too many other things to consider.
I will be investigating (b) as it sounds quite feasible.
I was wondering if anyone had experience they could share using the Response.IsClientConnected property as a performance optimization for asp.net websites.
The reason I ask is that I am a bit skeptical on how effective it would be in real life scenarios. I understand the concept of checking the value before performing a large task but I just can't see how useful this would be as clients could disconnect at any point time.
I think the main usage would be for optimizing the delivery of long processes. For example, if you had to generate a huge report or something, you might run the report in a separate thread and then periodically check to see if the user is still connnected. If not, you could kill this long running process so that it is not running needlessly since the user is no longer expecting a response.
This helps to prevent users from starting long processes and then making more requests over and over because they might think it is slow or something. If you were not doing this type of checking, you could tax your server due to all the requests even though all but one is valid. This scenario could be handled by allowing only one user to run one long running task, but it would also help in a multi-user environment as well to make sure you are only spending time serving up requests where the user is still connected and waiting for the response.
Note: I have never actually used this before, this is just based on my very basic understanding of what I have read.
I have used this extensively in my applications and it can give you a huge saving on resources.
Try this: create a page that needs -some- time to complete and try refresh it many many times before it complete. You will see that requests are queued to be executed. Imagine a user that has a slow connection and refreshes his page many many times thinking this will fetch the page (a very common issue from what a site can die out of resources when all users are connected and for some reason it becomes slow).
Now, change it and at the start of each page load, (or sooner at page init) check if HttpContext.Current.Response.IsClientConnected and in the case that he is not connetced throw a threadabord exception. You will see, your site will respond much sooner.
Actually I check if client is connected before any heavy action on the page so as to prevent needless executions. In production environments, I have seen that especially in cases where the system becomes slow, this validation will help much.
How do you work-around the fact that sessions are dropped every time you deploy certain code files to an ASP.NET website? Sometimes we need to deploy a crucial fix in the middle of the day but don't want to boot off all our users for it.
By default Sessions are stored InProc. You should choose an out-of-process option.
Maybe just a StateServer is enough in your scenario
One way would be to have a load-balanced server set-up. You could direct all traffic to server A, patch Server B and then repeat the other way around.
Alternatively, as #Curtisk states, better to get to the stage where you don't need to do "hot patches" through rigourous testing and then proceed to planned outages advertised in advance.
Hope this helps.
The reason why this happens is that deploying the new code causes the application pool to get recycled. You then lose everything you have in memory.
The way to get around this is then not to save anything in memory.
How difficult this is depends on your architecture.
One solution could be to save your session information in SQL Server, using the out of process state. Note do not use an in memory state server, as if the app pool is recycled you will lose this.
We have an ASP.Net 2.0 web application running in a web farm which is using the ASP.Net State service to store sessions.
We have been having problmes with the service intermittently and have changed a few things such as the machineKey in the machine.config.
My actual question is around the monitoring of the state service service. We have all 4 available performance counters running on the server that hosts the service and as yet we have not seen a single session time out. We have also seen the number of active sessions slowly rise over a period of time, but never become less.
Does the state service recognise when sessions time out? Is there something we should be doing manually?
Edit: We have given up on the state service and gone with SQL server sessions.
To answer the questions below, it seems that sessions go up forever until the service falls over and it is very doubtful that any oen threads are linked to the state server. This is a fairly basic web app at the end of the day.
It seems from the reading I am doing that anumber of other people have experienced similar things, but there seems to be a general lack of common sense and knowledge in any responses flying about.
MS seem to have almost no documentation on this topic.
In ASP.Net session time outs can be configured in web.config and machine.config. The default time out assuming nothing has changed will be 20 mins. The machine.config file can be set to not allow overriding, which means that any changes specified in web.config files will not override these settings.
Have you ensured that the appropriate settings are in place in both machine and web config files?
The state service should drop each session after 20 mins of inactivity assuming the default settings.
At what point are your inactive sessions dropped? I assume they are not exponentially increasing, unless your are restarting the service in order to clear them they must be being dropped at some point.
Do you have something that might be hitting the session and keeping it alive without you knowing? Is there are thread being spawned somehwere that is doing work inteh background and holding on to your session? As far as my expeireince goes the timeout is set int he web config file and it just doesn't it's magic from there.
In my experience we've found out that native state server or even using SQL Server for sessions is a very scary scenario as both have issues.
I think you can explore other products for this to achive the absolute best.
A free option would be Velocity but it is still not released.
And another comprehensive but proven product will be (Very expensive actually) NCache
Take a look and see which looks best for you.
About SQL Server, you server will die very soon if you have enough number of hits coming in (I belive you have some hits already which yielded you to do Web Farm or you do it just for the sake of redundancy)
I am sure this will get modded down, but I have to say it.
If you are having issues with the state server, then there is likely an error somewhere in your web application. Charles' comment above seems like good places to start checking, but somewhere there is a life cycle issue.
Go back over the code and check your assumptions. Take a new computer, visit your website (create a session) and let it sit for an hour. If your session is still alive, then something is wrong. Create a new web application that just has a single page reporting the the age of the current session and try the same thing. You should find that after an hour (default is 20 minutes) the session is no longer valid. Now you have a system that is working as expect and one that is not, both using the same session server, so you can rule that out as the problem, now start going through code/configuration and see where you could be keeping it alive (or preventing the time-out).
Here, by the way, is a 'valid' session config. If you don't have your looking something like this, you have likely found your issue:
<sessionState
mode="StateServer"
stateConnectionString="tcpip=10.1.1.1:55455"
cookieless="false"
timeout="20" />
Also make sure you are not overriding your web.config with your machine.config to have a longer timeout.