I have an ASP.net (c#) application, that has a portion of code that modifies a globally accessible resource (like a web.config file). When modifying the resource, naturally, to prevent race conditions only one user is allowed at a time so I need to lock the code using a monitor:
lock(Global.globallyAccessibleStaticObject)
{
//..modify resource
//..save resource
}
I was satisfied with this locking approach but then I thought, what if this isn't enough? should i use a mutex instead? I know a mutex is useful for inter-process locking (across many processes and applications), and thus slower, but given the nature of a deployed asp.net page (multiple requests at once across multiple app domains), is this necessary?
The answer it seems, would depend on how asp pages are handled on the server side. I have done research regarding the http pipeline, app domain, thread pooling etc. but i remain confused as to whether it is necessary to implore inter-process locking for my synchronization, or is intra-process locking sufficient for a web app???
Note: I don't want to get caught up in the specific task because I want this question to remain general, as it can be relevant in many (mult-threading) scenarios. Furthermore, I know there are more ways to accomplish these tasks (async handlers/pages, web services, etc) that I don't care about right now.
If your application only runs in one AppPool, then it is running in one physical w3wp.exe process, so the monitors/lock should be sufficient for guarding the shared resource. With that strategy, you only need to protect across threads running in the same process.
We encountered a situation in work where we had an IIS application configured to run in a single AppDomain but lock was not sufficient to protect access to a resource.
The reason we think this was happening is that IIS was recycling the AppDomain before the lock was released, and kicking off a new AppDomain, so we were getting conflicts.
Changing to use a Mutex has resolved this for us (so far).
Related
A few months ago I was interviewing for a job inside the company I am currently in, I dont have a strong web development background, but one of the questions he posed to me was how could you improve this block of code.
I dont remember the code block perfectly but to sum it up it was a web hit counter, and he used lock on the hitcounter.
lock(HitCounter)
{
// Bla...
}
However after some discussion he said, lock is good but never use it in web applications!
What is the basis behind his statement? Why shouldnt I use lock in web applications?
There is no special reason why locks should not be used in web applications. However, they should be used carefully as they are a mechanism to serialize multi-threaded access which can cause blocking if lock blocks are contended. This is not just a concern for web applications though.
What is always worth remembering is that on modern hardware an uncontended lock takes 20 nanoseconds to flip. With this in mind, the usual practice of trying to make code inside of lock blocks as minimal as possible should be followed. If you have minimal code within a block, the overhead is quite small and potential for contention low.
To say that locks should never be used is a bit of a blanket statement really. It really depends on what your requirements are e.g. a thread-safe in-memory cache to be shared between requests will potentially result in less request blocking than on-demand fetching from a database.
Finally, BCL and ASP.Net Framework types certainly use locks internally, so you're indirectly using them anyway.
The application domain might be recycled.
This might result in the old appdomain still finishing serving some requests and the new appdomain also serving new requests.
Static variables are not shared between them, so locking on a static global would not grant exclusivity in this case.
First of all, you never want to lock an object that you actually use in any application. You want to create a lock object and lock that:
private readonly object _hitCounterLock = new object();
lock(_hitCounterLock)
{
//blah
}
As for the web portion of the question, when you lock you block every thread that attempts to access the object (which for the web could be hundreds or thousands of users). They will all be waiting until each thread ahead of them unlocks.
Late :), but for future readers of this, an additional point:
If the application is run on a web farm, the ASP's running on multiple machines will not share the lock object
So this can only work if
1. No web farm has to be supported AND 2. ASP is configured (non-default) NOT to use parallel instances during recycle until old requests are served (as mentioned by Andras above)
This code will create a bottleneck for your application since all incoming request will have to wait at this point before the previous went out of the lock.
lock is only intended to be used for multithreaded applications where multiple threads require access to the same shared variable, thus a lock is exclusively acquired by the requesting thread and all pending threads will block and wait until the lock is released.
in web applications, user requests are isolated so there is no need for locking by default
Couple reasons...
If you're trying to lock a database read/write operation, there's a really high risk of a race condition happening anyway because the database isn't owned by the process doing the lock, so it could be read from/written to by another process -- perhaps even a hypothetical future version of IIS that runs multiple processes per application.
Locks are typically used in client applications for non-UI threads, i.e. background/worker threads. Web applications don't have as much of a use for multithreaded processing unless you're trying to take advantage of multiple cores (in which case locks on request-associated objects would be acceptable), because each request can be assumed to run on its own thread, and the server can't respond until it's processed the entire output (or at least a sequential chunk) anyway.
As best I can tell when a worker process recycles:
a) a new one spins up before the old one shuts down
b) the old one shuts down once all the active requests its servicing completes
Is the above accurate?
If so, I have data that I store in SQL once Application_End() fires from the global.ascx file. I pull this data back in when Application_Start() fires.
The problem is based on my testing, the new worker process fires the Application_Start() before my old worker process gets a chance to complete its Application_End().
What are best practices for handling this situation?
cheers in advance
edit: I just noticed a feature on IIS 7 'Disabled Overlapped Recycle' - I'm guessing this is the best route
Your description of overlapped recycling is accurate, yes (1); and there is a setting for disabling it, but it's intended to prevent HTTP errors which you would be re-introducing. App pool recycles are a normal occurrence for managed apps (stems from, among other things, a CLR limitation that prevents the unloading of assemblies in the same memory space) that you must design for.
Your technique would be difficult to manage in a web-farm or web-garden scenario.
I think a better design would be to rely on out-of-process storage for the data (using distributed cache products like ScaleOut, App Fabric, and the like) so that all app pools have the same view of the cached data.
(1) -
http://mvolo.com/blogs/serverside/archive/2008/02/25/Starting_2C00_-stopping-and-recycling-IIS-7.0-Web-sites-and-application-pools.aspx
I’m having some trouble with understanding how IIS is handling static variables on its threads. My understanding has always been that if IIS has 4 worker processes that it can handle 4 requests simultaneously and that it would be the same as having 4 separate threads running the website. Any static variables would persist in each individual thread. The reason I’m a bit confused is that I have a scope that I’ve made which manages connections and caching transactions. When I’m testing the app I don’t notice any issues but after I’ve compiled it and hit it at the same time from two different locations I seem to get a sort of conflict. Now if these worker processes are separate why would this be? Can more than one request be processed on a single worker thread at the same time? This is tremendously important as there are unique ID’s that are held in these static members to handle escalation of the objects that manage these functions and it appears that they are trying to access the same object.
I'm running this on Vista's IIS server on an x64 machine.
EDIT
For values that need to persist through the thread on a single request, I put these values into Web.HttpContext.Current.Items which seems to do the trick.
<ThreadStatic()> can be used but it may not be available during the entirity of the request process. In one module that I have, is only used on a variable to indicate if that thread has already loaded the settings for the cahcing server. If true then the tread (not asp.net) is ready to fetch data from the caching server.
First concept to change: if you're using ASP.NET, they are ASP.NET threads, not IIS threads.
Second, this is a .NET issue. static variables are shared throughout the AppDomain in .NET. Since you'll have one AppDomain per IIS application (more or less), that means your static variables will be shared across all worker threads in the application.
There will be a lot more than four threads, and they'll all be sharing the same variables, which means you'll either need to do locking, or you'll need to not use static variables.
Whatever your understanding has always been, I suggest you go back and figure out where you got that understanding from; then update it, because it doesn't have much to do with ASP.NET.
EDIT: The subject has changed, so I'll change the answer a little.
You have to interlock access to these variables. Alternatively, you should consider reevaluating your design. Your design apparently assumed some different model for access to statics. This assumption has turned out not to be correct. It's possible that this assumption may have cascaded throughout your design. You should reevaluate your design in the light of reality.
Each worker process runs in its own AppDomain, so each WP will have its own instance of a static variable.
In the answer here it suggests the AppDomain is shared across WPs which is incorrect.
You should be using the .NET connection pooling though and you should investigate the using(IDisposable){} method of scoping your connections.
First of all to give you a bit of background on the current environment. We have a number of ASP.NET applications, all of which use session for certain aspects. We are "Load Balanced" over multiple servers due to traffic levels, however, our load balancing is set to use "Sticky Sessions" as currently all web applications are set to use "InProc" for session state.
We are looking at being able to remove the "Sticky Sessions" configuration on our load balancer, as due to our traffic loads servers can and do get overloaded. We want to go with a more balanced approach, but must be able to use session.
I know that SqlServer for session state will work, but for reasons beyond our control, we cannot use SqlServer to store our state. In researching it seems that StateServer is our best bet. We have an additional server, with loads of memory sitting around. This server could be our StateServer for the entire Web Cluster. We just want to know the following things.
1.) Besides any potential serialization issues with the switch from InProc to StateServer, are there any major known issues with losing session objects or generating errors with the above listed environment?
2.) Aside from the single point of failure, and slighly slower performance are there any other gotchas that we need to be aware of with using StateServer.
3.) Are there any metrics that show the performance differences between the three types of state storage?
Here is a decent FAQ on asp.net state: http://www.eggheadcafe.com/articles/20021016.asp
From that Article, here is some information on StateServer:
In a web farm, make sure you have the same MachineKey in all your web servers. See KB 313091 on how to do it.
Also, make sure your objects are serializable. See KB 312112 for details.
For session state to be maintained across different web servers in the web farm, the Application Path of the website (For example \LM\W3SVC\2) in the IIS Metabase should be identical in all the web servers in the web farm. See KB 325056 for details
I have only used sql and in-proc. But these 3 that apply when using sql server apply as well:
Avoid storing too much information in the session, as it affects both in serialization and data transmitted over the network.
Make sure you don't have anything that depends on the Session_onEnd. This is just not available for out of process sessions.
Turn off session on pages that doesn't uses it. This don't make a difference for in-process session, but for out of process it will save you a lot.
Make sure your server etag ids are synchronized across the web farm otherwise caching at client browsers will be upset.
Have you reviewed your code in detail to make sure everything can be serialized out of process and across a LAN efficiently?
Are you solving the main performance problem within your system? I ask because the database is the typical source of contention.
My main motivation for moving away from sticky sessions was operational flexibility i.e. cycle down a problematic server or to deploy a software upgrade. So having implemented a central session state service make sure you take full advantage from an operational stand point.
In my experience we've found out that native state server or even using SQL Server for sessions is a very scary scenario as both have issues (mainly performance). By the way, we are also using sticky sessions.
I think you can explore other products for this to achive the absolute best. A free option would be Velocity but it is still not released.
And another comprehensive but proven product will be (Very expensive actually) NCache. THis will even help in your serilizations with less cost, If you use their API's it will be even better results.
Take a look and see which looks best for you.
About SQL Server, you server will die very soon if you have enough number of hits coming in (I belive you have some hits already which yielded you to do Web Farm or you do it just for the sake of redundancy)
Bottom line: We are evaluating Velocity because NCAchce is really expensive. However advantages are huge.
We are using StateServer for a very small web farm with only two nodes for a few hundred users.
I'm not responsible for its operation but I remember only two issues in two years where the service had to be restarted because it crashed.
I would like to another one more point to the accepted answer:
Make sure the version of framework dlls is the same.
In my case the System.Web dll versions were different as a few windows updates were skipped on one of the servers of the farm.
What kind of multi-threading issues do you have to be careful for in asp.net?
It's risky to spawn threads from the code-behind of an ASP.NET page, because the worker process will get recycled occasionally and your thread will die.
If you need to kick off long-running processes as a result of user actions on web pages, your best bet is to drop a message off in MSMQ and have a separate background service monitoring the queue. The service could take as long as it wants to accomplish the task, and the web page would be finished with its work almost immediately. You could accomplish the same thing with an asynch call to a web method, but don't rely on getting the response when the web method is finished working. From code-behind, it needs to be a quick fire-and-forget.
One thing to watch out for at things that expire (I think httpContext does), if you are using it for operations that are "fire and forget" remember that all of a sudden if the asp.net cleanup code runs before your operation is done, you won't be able to access certain information.
If this is for a web service, you should definitely consider thread pooling. Too many threads will bring your application to a grinding halt because they will eventually start competing for CPU time.
Is this for file or network IO? If so, you should also consider using asynchronous IO. It can be a bit more of a pain to program, but you don't have to worry about spawning off too many threads at once.
Programmatic Caching is one area which immediately comes to my mind. It is a great feature which needs to be used carefully. Since it is shared across requests, you have to put locks around it before updating it.
Another place I would check is any code accessing filesystem like writing to log files. If one request has a read-write lock on a file, other concurrent requests will error out if not handled properly.
Isn't there a Limit of 25 Total Threads in the IIS Configuration? At least in IIS 6 i believe. If you exceed that limit, interesting things (read: loooooooong response times) may happen.
Depending on what you need, as far as multi threading is concerned, have you thought of spawning requests from the client. It's safe to spawn requests using AJAX, and then act on the results in a callback. Or use a service as a backgrounding mechanism, which runs every X minutes and processes in the background that way.