In our production environment (only) an excessive number of sessions seem to be getting created from an ASP.NET web application. The eye-catching symptom was that the ASPStateTempSessions table was generating ~25K records per hour (when google analytics indicates less than 500 unique users on this site per hour). This is resulting in a high number of waiting tasks which is then causing slowdowns and issues across other databases and therefore impeding site performance. The vast majority of the sessions don't appear to have any significant amount of data in them.
Any thoughts on what could be causing the phantom sessions? I was originally thinking that image requests and the like were somehow causing new sessions, but that doesn't seem to be sufficient to explain such a high multiplier. Is that even reasonable? Should I explore that avenue further? Why would that not have the same symptoms in my development environment?
Thanks!
Environment Details (I can provide more details, I'm just not sure what else is relevant):
IIS 7
SQL Server 2008
Session mode is SQLServer:
<sessionState mode="SQLServer" sqlConnectionString="[Connection String]" allowCustomSqlDatabase="true" cookieless="false" timeout="120" cookieName="XYZ_SessionId" />
I agree that the likely culprit is flat files running the SessionStateModule, especially if you are using MVC instead of webforms. The reason is that in order to support extensionless URL routing, MVC adds the following tag into your web.config:
<modules runAllManagedModulesForAllRequests="true">
...
The attribute does pretty much what its name implies, which will cause you to experience overhead if you host images off of the same website in IIS. If you don't use extensionless URLs, you can consider turning off that attribute, or you can try moving images to a CDN like akamai or AWS cloudfront.
Alternatively you could look for a SessionStateModule alternative, there are some that exist that might provider alternate behavior. You could even roll your own by inheriting System.Web.SessionState.SessionStateStoreProviderBase, there is an MSDN article on doing that: custom session provider tutorial. If you do that last one, I recently found ResetItemTimeout to be the most commonly run item on every request by the default SessionState module.
Lastly, the reason I think that the expansion you are experiencing is being caused by this is the fact that SessionState module is synchronous by default. This means a browser requesting both imageA.jpg and imageB.jpg will receive imageA then imageB only after imageA has released the Session lock. This is much slower than the default behavior of a webserver, which is to serve up both on separate threads.
Another way to troubleshoot, if this does not solve it, is to look at the current requests going through IIS7. To do that, go to the top level server name in the left hand side of IIS manager and click on Worker Processes. It should list your website process, double-click that and it will show all current requests. You should see a ton of image files in there.
Related
We are trying to provide fix for vulnerability issues from our website. We are facing Cross Site Scripting threat in our website(Vulnerability Testing Result). We put four lines of code in web.config. Now i need a clarification that how much time it will take to reflect the changes in IIS that were made in web.config and also is there any time criteria for this.
It will do not take a longer time to affect the web.conifg changes in iis. it is instant. you just need to refresh the site in iis to reflect the changes you made in config file.
I'm a beginner in ASP.NET 2.0.Probably this could sound too basic and stupid issue for someone expert in the ASP.NET.But this is giving me sleepless nights.
Basically i have developed a simple multilingual website with a master page and content pages which fills inside the content place holder portions of the master page. The application works great when it is configured to run on the ASP.NET Development Server 2.0. But once i publish it to run on the IIS web server it will no longer function. :( I could see from the trace that none of the session variables i use are stored and redirected to the relevant content pages.
Although the contents are displayed, the session variable values by which i take some decisions on the redirected pages are lost and i run into exceptions.
Please guide me where am i going wrong and exact procedure for Publishing an application.
Ex: my home page has URL which runs something like
http://localhost/Onlineupdate/Home.aspx?vers=1.1&lang=fr-FR
Based on the above URL, i strip and save the vers and the lang variables in a Session variable. However these are lost when hosted on IIS.
There are a dozen or so things that could cause the session data to be lost:
IIS restarting
The app pool restarting
due to a change to the web.config
due to a change to anything in the \bin directory
memory limit reached, or a bug causing the app pool to reset.
several other possible causes
Your host is actually a web farm, and you're using in-process memory, which will cause issues when one server fails over to the other, unless you're using SQL Server session state mode.
Since we don't have enough information to answer exactly what's happening in your specific situation, I'd ask you to start by reading up, starting here: http://msdn.microsoft.com/en-us/library/ms178581.aspx
Edit: I did find this blog article, which may be helpful: http://blogs.msdn.com/b/amenon/archive/2007/08/21/troubleshooting-session-loss.aspx
The following is not necessarily part of the answer, but added to try to be helpful.
If it's feasible, from my own personal experience, we've had success in eliminating our lost session issues by using the SqlServer Session State mode. Since we implemented this, our session issues have all but disappeared.
also i found the main problem that you should initialize session before use
like:
session[“id”]=””;
and after that it well work fine
In order to prevent this to happen first in the web.config set restartOnExternalChanges to false.
Now in web.config changes must be propagated manually(this means that the dev is now responsible to build a mechanism for config change propagation).
Hint: You can use file watcher for this that will listen for the web.config (or any config you use in you web. app) for changes and wrap it as a watchable configuration so you can reload the configs when they are changed.
Hope this helps
I faced the same issue in my ASP.NET MVC website .
and i have resolve it by next steps :
open IIS Manager
go to the application pools
right click on the application pool which related to your website
click on "Advanced Settings"
set "Idle Time-out (minutes)" to be "20"
set "Maximum Worker Processes" to be "1"
Click Ok to close the window
these steps has resolved my issue.
On a site I'm working with, we've got two classes of changes they can ask for. On one hand, they've got stuff that I'd have to rebuild and redeploy. They count these as "downtime" changes, because we display a nice little splash screen and we test the site thoroughly when we come back up.
On the other hand, they ask us to do a number of text changes, turning features on and off, etc., which we've isolated to the web.config. We offer to do these either inside or outside of deployment windows - we just edit the file, check that the change is right, and go back to work.
But one of the smart guys on the client side pointed out that editing web.config recycles the app pool, and that's downtime right there. I'd never noticed, but I suppose that's right - while the app pool is unavailable, the app is "down".
But for how long? I'm not asking you to sort through the client's level of comfort with downtime intervals, but is this a common perspective? Or should we just not worry that web.config editing is accompanied by a second or two of application downtime?
All said so far is correct.
However there is a way to avoid this downtime, as long as your values you are pulling are not cached.
You can port part of your .config file to another file, that won't recylce the app pool.
It would look something like this in the web.config file:
<appSettings file="moresettings.config"></appSettings>
Then your outside file would look like this:
<?xml version="1.0" encoding="utf-8" ?>
<appSettings>
<add key="SOMEKEY" value="MYVALUE"/>
</appSettings>
IIS recycles the app pool by itself normally, and if those recycles don't cause you concern, this once shouldn't either.
The user shouldn't receive any sort of "service unavailable" errors, afaik.
If you're concerned at all with downtime, and this happens a lot, I would consider moving these settings to the database.
That said, the downtime in your case will be minimal. The app pool is recycled when you SAVE the web.config file, and we're talking milliseconds.
As said, IIS is indeed recycling the App Pool. This is not as bad as doing a full iisreset though - users shouldn't get the "Service unavailable." error as the Web Server is still online and serving requsts - it just has to wait for the AppPool to restart, which means that the response time for the users accessing in this moment are very high. This may be a problem of course if you have a public website and are turning visitors away.
The other side effects of AppPool recycling are the same as an iisreset: It flushes an InProc Session Cache if I'm not mistaken, and it executes the Application_Start event.
So even though it's relatively harmless, I would still treat it as downtime.
I am trying to get a grasp on how to handle updates to a live, functioning ASP.NET (2.0 or greater) Application while there are users on the site.
For example, suppose SO is an ASP.NET Web Application project. The project code compiles down to the single .DLL in the BIN folder. Now, there are constantly users on SO, so what would happen to users' actions/sessions if you would use the Visual Studio .NET "Publish" feature (or just FTP everything again manually) while they are using the site?
Would creating an ASP.NET Web Site, instead, alleviate any problems that may or may not exist with the scenario above? I am beginning to develop a web site as a user-driven Web Application, and I want to make sure that my inexperience with this would not potentially annoy the [potentially] many users that I [want to] have 24/7.
EDIT: Sorry, I should have put this in a more exact context. Assume that this site is being hosted by a web hosting service with monthly fees. I won't be managing the server itself, just what the web host allows as a user of their services.
I create two Web sites in IIS. One is the production Web site, and the other is a static Web site with an HttpHandler that sends all requests to a single static "We're updating" HTML page served with an HTTP 503 Service Unavailable. Typically the update Web site is turned off. When it's time to update, we stop the production Web site, start the update Web site, and now we can fiddle with the production Web site all we want without worrying about DLLs being locked or worker processes needing to be spun down.
I started doing this because
App_Offline.htm really does not work well in Web Gardens, which we use.
App_Offline.htm serves its page as 404, which is bad if you're down for a meaningful period of time.
We can start the upgraded production Web site with modified settings (only listening on localhost), where we can do a last-minute acceptance/verification that everything is working before we flip the switch, turning off the update Web site and re-enabling the production Web site.
Things this does not solve include
Any maintenance that requires a restart of the server--you still have downtime where no page is served.
Any maintenance that diddles with the .NET runtime, like upgrading to the latest service pack.
Other approaches I've seen include
Having two servers. Send all load balancing requests to one server, upgrade the other one; then rinse and repeat. Most of us don't have this luxury.
Creating multiple bin directories, like bin-1.0.0.0 and bin-1.1.0.0 and telling ASP.NET which bin directory to use in the web.config file. (One advantage of this is that reverting to a previous binary is just editing a config file. A disadvantage is that it's harder to revert resources that don't end up in your binaries, like templates and images and such.) I don't remember how this actually worked--I think the application did some late assembly loading in its Global.asax based on its own web.config section (since you touched the web.config, the app had restarted, so it was okay).
If you find a better way, let me know!
Changing to the asp.net web site model won't have any effect, as the recycle will also happen, some of changes that trigger it for sure: web.config, global.asax, app_code.
After the recycle, user will still be logged in because asp.net will just validate the syntax. That is given you use a fixed machine key, otherwise it will change on each recycle. This is something you want to do anyway as other stuff can break if the key change across requests i.e. viewstate validation, embedded resources (decryption of the url fails).
If you can put the session out of process, like in sql server, you will avoid loosing the session. If you can't, your code will have to consider that. There are plenty of scenarios where you can avoid using session, and others were you can wrap it and re-retrieve the info if the session was cleaned. This should leave you with a handful specific cases that you know can give trouble to the users, so for those you do some of the suggestions others have already made.
One solution could be to deploy your application into a load balanced environment (web farm).
When deploying a new version you would use the load balancer to redirect requests to the server you are not deploying to.
App_offline.htm is great solution for this I think.
in SO we see application currently unavailable page when a deployment begins.
I am not sure how SO handles it.. But we usually put a holding page. So what ever the user has done (adding question or answering questions) does not get updated. As soon as he updates something he will see a holding page asking him to try after sometime.
And if I am the user I usually press the back button to make sure what I entered is saved in the browser history so that I can post later.
Some site use use are in clustered environment so I take one server offline and inform the load balancer that she will not be available and once I make sure that the new version is working fine I make it live.. I do the same thing for the next server.
Do we have any other option?
It is not a technical solution, but set up a scheduled maintenance window. You can annoucement in advance giving your user base fair warning that there is a possiblity that the application will not be available during that time frame.
Following recent hardware problems, I attempted to switch a couple of our websites to use new, individual application pools. A test run on our staging server worked fine, and has had no visible negative consequences.
Unfortunately, trying the same operation on our live machine left one of our key applications struggling - my best guess is with some kind of mismatch in Session state. I could log in fine, but a few clicks later would be presented with a screen that was part login screen, but with all menus visible. This indicates to me that part of the system thinks the session had been lost (redirect to login page), but IIS itself had not lost the session (hence the menus showing on the master page).
I tried recycling all the Application Pools (new and old), and each website using IIS Manager. I also tried a single-space change to the web.config file, and a full release of the dll's. Still, I could intermittently use the system for a few clicks, do some useful stuff, then maybe find myself at a login screen again or similar. We have some logging and on some occasions I could see that the session was being timed-out after a couple of seconds, substantially less than the settings on the App-pool (default 20mins).
As soon as I switched the web site's app-pool back to the default, everything was ok again.
What have I missed? Any suggestions gratefully received!
EDIT:
Just thought... on the staging environment I did name the App-pool differently from the website name (e.g. Xxxx_Dev, Xxx_Test etc) but on live I just called it the same name as the website. Could this cause an issue?
do your various applications all use Forms Authentication? Have you specified unique path attributes in each form tag in the web.config under the Authentication tag?
OK. I think I've found the problem.
I was actually using an Application Pool that had been set up by someone else - of the expected name - but they had set it up with the Properties, Performance tab | Web Garden option to use 4 worker processes. I have now changed that to 1.
As the session state was being stored 'In Process' (the default), each time the connection hit a new thread it also essentially lost any stored session variables, as I now understand things.
Its early days, but a simple switch to the newly altered Application Pool (no restarts or web.config saves necessary thus far) and everything appears to be behaving normally.