Before anyone says it, Yes we have validation & machine key explicitly specified in Web.config
Yet we are still getting this error from time to time. I've spent an awful long time trying to track down this problem and have come up completely empty.
We do have load balancing using TMG but we've reduced the farm down to 1 server for testing, and these errors still occur, so to my mind that rules out load balancing.
Because both ViewState integrity checks are occurring as well as Authentication ticket failures, this to me, points squarely at a situation where the Validation and Machine keys are being regenerated periodically, But how?! is the explicit key declaration in Web.config being ignored for some reason?
Once upon a time our setup worked perfectly, it was only when the servers were upgraded from Server 2008 to Server 2008R2 that this started happening, everything else in terms of the web apps is the same as the old setup.
What I'm thinking now is, if there's any way to get those keys at run-time, so we can see if they're changing somehow?!
Any other ideas welcome!
Related
Since we have moved to azure, we have numerous session lost issues only on production.
We have InProc, cookie based, sticky session, large timeout, no high traffic and no high memory/process usage.
We use HAProxy as loadbalancer.
I have done basic research and none of the following seems to be the cause:
session timeout
application pool settings/recycling
memory size and usage thresholds
no eaten exceptions
there is no changes to file system to cause a restart
I'm particularly more suspicious about how loadbalancer/ssl and application work together and if http headers are fine, but I don't know any tools to really monitor that.
I'm assigned to find a solution at the same time I have no privilege to access the machines.
Logs(Log4Net) are all stored in database but doesn't help to give a clear understanding of what is going on the system and cannot follow a user session using them.
I'm allowed to find the problem by adding required logs to code or to develop some kind of monitoring module or to use profiling/debugging tools.
Only once a month there will be a production deployment so I'm trying to use the opportunity as best as possible.
Question:
Is there any useful monitoring/profiling tool that can give me a clear view of what is happening in the system by aggregating information I may need? for example following a user/session between requests from time of login until session drop plus information about headers and other system application parameters.
if there is not such a tool out there, please give me your ideas to write one?
This is a common issue in load balanced environment. As mentioned in this answer for a similar question,
InProc mode, which stores session state in memory on the Web server. Which means that session data is maintained inside your web server on a given VM and is not shared outside of the VM. So when you have multiple server for load balancing, the session state isn't shared with each other. To solve this, you must store your session state external to the web server.
Use Redis, or SQL Database, or something else.
Validation of viewstate MAC failed. If this application is hosted by a Web Farm or cluster, ensure that configuration specifies the same validationKey and validation algorithm. AutoGenerate cannot be used in a cluster.
Is it possible that this error occures if the session expires ?
The MAC is validated on a per request basis unless you turn it off. I don't immediately see why a session ending could cause this error, but I'm not going to say it's absolutely impossible - you can check this for yourself by following the stack track and using Reflector.
This would be more likely to happen if multiple physical machines serve different requests in the same session, as opposed to a session expiring.
I got this error before and I was able to fix it by creating a static read only machine key. That way the key would never change after the first time it was set, so it would always validate. In my particular circumstance MAC validation was unimportant to me, but depending on your security concerns, you may want to further research the implications of this approach.
I'm having a problem with a web app I'm managing. Users starting receiving the following error occasionally:
Validation of viewstate MAC failed. If
this application is hosted by a Web
Farm or cluster, ensure that
configuration specifies
the same validationKey and validation
algorithm. AutoGenerate cannot be used
in a cluster.
The problem is that it's not a cluster - it's a single Windows 2003 server. After digging around, it appears that adding a machineKey section and some extra attributes to the Pages directive in my web.config resolves this error:
<machineKey validationKey='MACHINE KEY SNIPPED'
decryptionKey='DECRYPTION KEY SNIPPED'
validation='SHA1'/>
<pages validateRequest="true" enableEventValidation="false">
After changing these two things in my web.config, the error goes away, but now I have a new problem - Instead of an error that my viewstate is invalid, the app just "Forgets" who my user is, and sends them back to the login page. Now, the users are browsing through the application, and then they're unexpectedly sent to the login page, even after they've already been logged in for a few minutes. While I can't force this to happen, it usually happens within visiting 10-12 different pages, so pretty frequently.
I'd love a resolution to this - does anybody know what else might be causing the viewstate error on a single server, or what I can do to ensure that it's validated properly?
It sounds as though the worker process is recycling itself (assuming you're storing session state in-process). Picking a fixed key means that the viewstate is still valid when the process comes back up, but you've lost the session state. You could try storing the session state in a database, but I'd be more concerned to fix the underlying problem. Does your application suddenly allocate vast amounts of memory, or anything like that? Is there anything suspicious in the event log?
It turned out that this began happening when I added additional worked processes to the app pool that was running our application. Because the session state was being stored InProc (and not in a state service or a SQL Server), it was losing track of who the user was when it switched them between working processes. For now, dropping the number of processes on our server back to one corrected the problem, since raising it didn't seem to have any improvement in the first place.
We have an ASP.Net 2.0 web application running in a web farm which is using the ASP.Net State service to store sessions.
We have been having problmes with the service intermittently and have changed a few things such as the machineKey in the machine.config.
My actual question is around the monitoring of the state service service. We have all 4 available performance counters running on the server that hosts the service and as yet we have not seen a single session time out. We have also seen the number of active sessions slowly rise over a period of time, but never become less.
Does the state service recognise when sessions time out? Is there something we should be doing manually?
Edit: We have given up on the state service and gone with SQL server sessions.
To answer the questions below, it seems that sessions go up forever until the service falls over and it is very doubtful that any oen threads are linked to the state server. This is a fairly basic web app at the end of the day.
It seems from the reading I am doing that anumber of other people have experienced similar things, but there seems to be a general lack of common sense and knowledge in any responses flying about.
MS seem to have almost no documentation on this topic.
In ASP.Net session time outs can be configured in web.config and machine.config. The default time out assuming nothing has changed will be 20 mins. The machine.config file can be set to not allow overriding, which means that any changes specified in web.config files will not override these settings.
Have you ensured that the appropriate settings are in place in both machine and web config files?
The state service should drop each session after 20 mins of inactivity assuming the default settings.
At what point are your inactive sessions dropped? I assume they are not exponentially increasing, unless your are restarting the service in order to clear them they must be being dropped at some point.
Do you have something that might be hitting the session and keeping it alive without you knowing? Is there are thread being spawned somehwere that is doing work inteh background and holding on to your session? As far as my expeireince goes the timeout is set int he web config file and it just doesn't it's magic from there.
In my experience we've found out that native state server or even using SQL Server for sessions is a very scary scenario as both have issues.
I think you can explore other products for this to achive the absolute best.
A free option would be Velocity but it is still not released.
And another comprehensive but proven product will be (Very expensive actually) NCache
Take a look and see which looks best for you.
About SQL Server, you server will die very soon if you have enough number of hits coming in (I belive you have some hits already which yielded you to do Web Farm or you do it just for the sake of redundancy)
I am sure this will get modded down, but I have to say it.
If you are having issues with the state server, then there is likely an error somewhere in your web application. Charles' comment above seems like good places to start checking, but somewhere there is a life cycle issue.
Go back over the code and check your assumptions. Take a new computer, visit your website (create a session) and let it sit for an hour. If your session is still alive, then something is wrong. Create a new web application that just has a single page reporting the the age of the current session and try the same thing. You should find that after an hour (default is 20 minutes) the session is no longer valid. Now you have a system that is working as expect and one that is not, both using the same session server, so you can rule that out as the problem, now start going through code/configuration and see where you could be keeping it alive (or preventing the time-out).
Here, by the way, is a 'valid' session config. If you don't have your looking something like this, you have likely found your issue:
<sessionState
mode="StateServer"
stateConnectionString="tcpip=10.1.1.1:55455"
cookieless="false"
timeout="20" />
Also make sure you are not overriding your web.config with your machine.config to have a longer timeout.
I am seeing a very strange problem on one of my production boxes. We have an application hosted in IIS 6 on a single machine with an apache web server in front of it. My application is using ASP.NET Membership for authentication and relies on session state. I am seeing a problem right now where some server requests will hit a null exception when trying to access any session variables, but if the request is retried, the page hits no exceptions and behaves properly.
I believe this has something to do with the Session ID cookie either getting corrupted or lost on the request, but I have no idea what could cause that. The reason I believe this is because it seems like ASP.NET is not seeing the cookie and creating a new session, which would explain why the variables are null. When it gets the exception, it cannot write the new SessionID cookie back to the client, so the client still maintains the original SessionID. Then when the next request is sent, the original SessionID cookie is used, which now ASP.NET finds and is able to retrieve the session state. This is pure speculation, but seems to fit the symptoms.
Also this site is using no other cookies other then the ones required by ASP.NET Membership and ASP.NET Session, so I am well under the cookie limit for IE. The site has been functioning properly for about 8 months and this issue just popped up recently. I have tried IIS resets and actually rebooting the machine, but nothing has seemed to help the problem.
Updates:
Here are some clarifications that were asked for.
1.)Our Apache server is the only thing exposed to the internet. All requests occur over HTTPS to this box. The Apache box then forwards all request past over HTTP to our application server. This is being done for security reasons. We have looked to see if Apache might have been the issue, but there seems to be no error in the Apache logs.
2.)The null exception is occurring when trying to access an object stored in session that the application expects to be there as opposed to the exception happening with the actual session object itself.
We have found the root of the issue. It looks like the IIS meta-base got corrupted on our app server. The best way to fix this issue is to do a clean install of IIS, but because of business constraints, this is not an option for us. So another solution is to actually just create a new App Pool for the application to run under. According to some people with more IIS expertise than myself, this will fix the problem in the short term, but it is very likely that the same thing will happen to this App Pool. So we are required to create new App Pools if this starts occurring again.