How do you work-around the fact that sessions are dropped every time you deploy certain code files to an ASP.NET website? Sometimes we need to deploy a crucial fix in the middle of the day but don't want to boot off all our users for it.
By default Sessions are stored InProc. You should choose an out-of-process option.
Maybe just a StateServer is enough in your scenario
One way would be to have a load-balanced server set-up. You could direct all traffic to server A, patch Server B and then repeat the other way around.
Alternatively, as #Curtisk states, better to get to the stage where you don't need to do "hot patches" through rigourous testing and then proceed to planned outages advertised in advance.
Hope this helps.
The reason why this happens is that deploying the new code causes the application pool to get recycled. You then lose everything you have in memory.
The way to get around this is then not to save anything in memory.
How difficult this is depends on your architecture.
One solution could be to save your session information in SQL Server, using the out of process state. Note do not use an in memory state server, as if the app pool is recycled you will lose this.
Related
Since we have moved to azure, we have numerous session lost issues only on production.
We have InProc, cookie based, sticky session, large timeout, no high traffic and no high memory/process usage.
We use HAProxy as loadbalancer.
I have done basic research and none of the following seems to be the cause:
session timeout
application pool settings/recycling
memory size and usage thresholds
no eaten exceptions
there is no changes to file system to cause a restart
I'm particularly more suspicious about how loadbalancer/ssl and application work together and if http headers are fine, but I don't know any tools to really monitor that.
I'm assigned to find a solution at the same time I have no privilege to access the machines.
Logs(Log4Net) are all stored in database but doesn't help to give a clear understanding of what is going on the system and cannot follow a user session using them.
I'm allowed to find the problem by adding required logs to code or to develop some kind of monitoring module or to use profiling/debugging tools.
Only once a month there will be a production deployment so I'm trying to use the opportunity as best as possible.
Question:
Is there any useful monitoring/profiling tool that can give me a clear view of what is happening in the system by aggregating information I may need? for example following a user/session between requests from time of login until session drop plus information about headers and other system application parameters.
if there is not such a tool out there, please give me your ideas to write one?
This is a common issue in load balanced environment. As mentioned in this answer for a similar question,
InProc mode, which stores session state in memory on the Web server. Which means that session data is maintained inside your web server on a given VM and is not shared outside of the VM. So when you have multiple server for load balancing, the session state isn't shared with each other. To solve this, you must store your session state external to the web server.
Use Redis, or SQL Database, or something else.
Yesterday, my customer played with the IIS settings and he changed the number of Working Process to 2 which made my Web Application ran very weird, the Session State sometimes was lost, sometimes was recovered and took me a day to find out what happened. So, the question is Multiple Working Process is useful in what situation?
It can be useful for scaling a web app vertically. Especially poorly designed ones that do too much work in the web processes or one where processes crash frequently so you always have a hot one. It isn't an option that should be exercised lightly as you have found out, but it is good to have when you need it.
The reason your user sessions started to fail was because you are using the default in-process session state module. This is fairly easy to fix as well -- just run the session state out of process using either the session state service or a database. Note that some behavior of the session state changes when you do this as well, so you will need to test carefully that you don't break something else.
At work we have a fairly good process for zero-downtime deployments of a busy ASP.NET MVC web site. There's two server groups so we can move traffic over, we write db SP's to be backwards compatible and roll them out in advance. Generally, it works well.
However, one issue has just tripped me up. The last deployment included a lot of code namespace changes.
When the build was deployed it broke existing user sessions because the objects in their session state no longer existed on the new codebase e.g. their session contained App_Code.UserDetails and the site now has a class of X.Y.Z.UserDetails
I understand the problem and why it happened, but what I can't work out is if there's anything that can be done apart from a full site shutdown to ensure there are no active users on the site. Or maybe restarting the state service so everyone gets booted off and has to log in again. Any ideas folks?
After thinking this over I've arrived at two possible solutions:
a) Always deploy web site updates with downtime. Keep the site down long enough for all sessions to expire. This is a safe option technically but unacceptable commercially
b) Simulate the problem and look at what can be done address the issue in the exception handler. Although not an ideal end-user experience it may be better to catch SerializationExceptions on session state, and flush the user session. They will have to authenticate again but it's not as bad as a broken session.
A decision I made upfront is that it's going to be difficult to pick up on live session dependencies during development, there are just too many other things to consider.
I will be investigating (b) as it sounds quite feasible.
This may be a dumb question, and based on the fact that googling has failed me I'm betting the answer is "no", but I thought I'd ask in case someone else has figured it out.
We're finally putting our website on a server farm, which means we can't use InProc session management. We're using SQLServer mode instead, but we had a situation where our SQL Cluster crashed. During this time, none of our newer web apps were able to load because of an inability to connect to the session database.
So here's the question: Is it possible to automatically fall back to a different session management (StateServer for example) or dynamically change the connectionstring so that we can use a backup Sql Server?
For now, our plan is to use DNS and if the main SQL Cluster fails, simply switch the DNS to a backup, but that's a manual task, and takes some time. We were hoping to have some sort of automatic failover.
I am afraid that there is no way. Also switching the session state mode would also make your application crash because users won't be able to find the data that was stored in their sessions. So an advice I can give you is the following: use a dedicated SQL server for the sessions, don't use the same server as the one serving your application data. And if you can, progressively start to update your application so that it uses less and less sessions, store very small amounts of data until you completely get rid of it. Make it stateless. Then your application will become very scalable.
My shared hosting provider set up IIS recycle app pool every 3 minutes for idle.
So my session factory often recreates (at application startup). As I have about 70-100 entities it takes about 2-5 seconds to construct factory. So cold start of my application is rather long. I haven't access to IIS setting.
You can offset a lot of the cost of setting up your factory by generating your proxies at build-time instead of runtime. This article explains the steps how.
Being realistic, the simplest change is to ask that the app-pool isn't recycled so frequently (since this is an expensive operation for your application). I'm sure they've set the timeout very low as a "performance" setting, but really this is generating work and slowing things down.
You might not have access to the IIS settings directly, but this shouldn't stop you from contacting your supplier's technical support and getting it resolved.
If you are in a full trust environment (doubtful, but provider may be willing to work with you on this), you can try serializing your configuration so it doesn't need to be rebuilt each time. Merging all your entity mappings into a single XML doc can help also (just do this as build step so its not a nightmare to work with mappings).
More info here: http://nhibernate.info/blog/2009/03/13/an-improvement-on-sessionfactory-initialization.html
Have you tried to stop your site from being idle in the first place? I use uptime robot that is FREE and pings your site every 5 minutes. The benefit of this service is that it only requests the headers of the page you set up as a monitor and therefore does not affect logging such as Google Analytics.
However said you will need to test this to see when your app does indeed recycle to see if uptime robot works with your shared hosting provider. The best way is to log every time the session factory is re-built.
not much you can do. app pool recycle shuts down your app...
I guess you could try to fool the recycler by having the application do something every 2:45.