My question is about the fast storage into RAM. Actually my project is a big symfony2 app, and I use a lot session storage (Symfony\Component\HttpFoundation\Session\Session). I'd like to try something else because I'm not totally happy with symfony2 session since it stores only simple types in it, objects are not allowed. I did a search and redis seems to be the perfect choice BUT someone suggested memcache.
Could memcached be better than redis? Is redis worth a try (despite the learning time we've to spend on it)?
I store a large structure holding my application's reference data in a variable I access through HttpContext.Application. Every once in a while this data needs to change. When I update it in place, is there a danger that incoming requests will see the data in an inconsistent state? Is there a need (and a way) to lock some or all of this structure? Finally, are there other approaches to this problem other than querying the database every time you need this (mostly static) data?
There are also other solutions availiable, there are many caching providers that you can use.
First of all, there's the HttpRuntime.Cache (which is the same as the HttpContext cache). There's also the System.Runtime.Caching.MemoryCache in .NET 4.
You can set data expiry and other rules for the data in the cache.
http://wiki.asp.net/page.aspx/655/caching-in-aspnet/
http://msdn.microsoft.com/en-us/library/6hbbsfk6.aspx
http://msdn.microsoft.com/en-us/library/system.runtime.caching.memorycache.aspx
More advanced caching includes distributed caches.
Usually, they reside on another server but may also reside on a different process on the same server.
Such providers are AppFabric (from Microsoft) and MemCached and others that I can't recall currently.
appfabric: http://msdn.microsoft.com/en-us/magazine/ff714581.aspx
memcached: http://memcached.org/
You will not see the application variable in inconsistent state.
The MSDN page for HttpApplicationState says (Under the Thread Safety section):
This type is thread safe.
You may be looking for HttpContext.Items instead to store data in the request scope instead of the application scope. Check out this article to get a great overview of the different context scopes in ASP.NET.
Your solution to avoid querying the database for "mostly static data" is to leverage ASP.NET's caching.
I'm wondering if there exist any distributed ASP.Net State Service alternative.
Session can be stored in InProc, StateService or SQL. You can also write custom storage providers. For load balancing (without any form of sticky IP) only SQL/Custom will work.
Are there any alternative ASP.Net State Services that is distributed? I'm thinking so each server can synchronize with the others.
Edit: In response to answers: I am looking for Session() storage. Distributed cache is not a problem, Velocity will do fine for distributed caching. :)
Edit2: In response to Oded: State server has the problem of single point of failure. SQL server may not be directly accessible because of the model chosen (i.e. 3-layer), it is only accessible through the middle layer.
I thought thats what "velocity" was going to be bringing to the table? You might want to check out Microsoft AppFabric for your needs.
Edit:
Maybe I'm missing something about your post and your edit, but I'm still pretty sure AppFabric is what you need for your Session() storage. Maybe I'm wrong, but it sure seems dead on to me.
I would look at Memcache for .net
Take a look at this:
http://technet.microsoft.com/en-us/library/cc725582(WS.10).aspx
I'd recommend to stay with SQL server since it will be the easiest to implement.
Make sure the classes you store in session are marked as [Serializable] or you will get runtime errors.
I've done a search on this subject already, and have found the same data over and over-- a review of the three different types of sessions. (InProc, Sql, StateServer) However, my question is of a different nature.
Specifically, what is the advantages/disadvantages of using the built in .NET session in the first place?
Here is why I am asking: A fellow .NET developer has told me to NEVER use the built in Microsoft Session. Not at all. Not even create a custom Session State Provider. His reasoning for this is the following--that if you have the Session turned on in IIS it makes all of your requests happen synchronously. He says that enabling session degrades the performance of a web server.
His solution to this is to create a session yourself-- a class that stores all values you need and is serialized in and out of the database. He advises that you store the unique ID to reference this in a cookie or a querystring variable. In our environment, using a DB to store the sessions is a requirement because all the pages we make are on web farms, and we use Oracle-- so I agree with that part.
Does using the built in Session degrade performance more than a home-built Session? Are there any security concerns with this?
So to sum it all up, what are the advantages/disadvantages?
Thanks to all who answer!
My experience has been that the session is a good means of managing state when you use it appropriately. However, often times it's misused, causing the "never ever use the session" sentiment shared by many developers.
I and many other developers have ran into major performance issues when we mistakenly used the session to store large amounts of data from a database, so as to "save a trip." This is bad. Storing 2000 user records per session will bring the web server to its knees when more than a couple of users use the application. Session should not be used as a database cache.
Storing an integer, however, per session is perfectly acceptable. Small amounts of data representing how the current user is using your application (think shopping cart) is a good use of session state.
To me, it's really all about managing state. If done correctly, then session can be one of many good ways to manage state. It should be decided in the beginning on how to manage state though. Most often times, we've run into trouble when someone decides to just "throw something in the session".
I found this article to be really helpful when using out-of-process modes, and it contains some tips that I would have never thought of on my own. For example, rather than marking a class as serializable, storing its primitive datatype members in separate session variables, and then recreating the object can improve performance.
Firstly, you colleague is implementing his own DB backed session management system, I do not see what advantage this has over using built in session state stored on a database (MS SQL is the default, there is no reason not to use Oracle instead).
Is his solution better than the built in one? Unlikely. It's way more work for you for a start. Here's a simple illustration of why. Let's say you use cookies to store your ID, how do you cope with a user who turns off cookies? If you are using ASP.Net's session state there's no problem as it will fall back to using the query string. With your colleagues idea you have to roll your own.
There is a very valid question as to whether you shold have session state at all. If you can design your application not to need any session state at all you will have a much easier time scaling and testing. Obviously you may have application state which needs to live beyond a session anyway (simple case beign user names and passwords), but you have to store these data anyway regardless of whether you have session state.
The MS implementation of Session State is not evil in and of itself... it is how some developers use it. As mentioned above, using the built-in session state provider means that you don't have to reinvent the security, aging, and concurrency issues. Just don't start jamming lots of garbage in the session because you're too lazy to figure out a better way to manage state and page transitions. Session doesn't scale really well... if each user on your site stuffs a bunch of objects in the session, and those objects take up a tiny bit of the finite memory available to your app, you'll run into problems sooner than later as your app grows in popularity. Use session in the manner for which it was designed: a token to represent that a user is still "using" your site. When you start to venture beyond that, either because of ignorance or laziness, you're bound to get burned.
You should be judicious in your use of Session, since multiple requests to the same Session object will usually be queued: see "Concurrent requests and session state" http://msdn.microsoft.com/en-us/library/ms178581.aspx.
Note that you can set EnableSessionState to ReadOnly to allow concurrent read access to session state.
This queuing is a good thing, as it means developers can use Session without being concerned about synchronization.
I would not agree with your colleague's recommendation to "never" use Session and I certainly wouldn't consider rolling my own.
First, a browser will only make two requests, to a given hostname, at a given time. For the most part these requests are for static content (JS files, CSS, etc). So, the serializing of requests to dynamic content aren't nearly the issue that one might think. Also, I think this may be confused with Classic ASP, where pages that use Session are definitely serialized, I don't believe this is the case with ASP.Net.
With ASP.Net session state (SQL mode, state server, or custom) you have an implementation that is standard, and consistent throughout an application. If you don't need to share session information this is your best bet. If you need to share information with other application environments (php, swing/java, classic asp, etc.) it may be worth considering.
Another advantage/disadvantage is that there has been a lot of developer focus on the built-in methodology for sessions with regards to performance, and design over rolling your own, even with a different provider.
Are there any security concerns with this?
If you roll your own you'll have to handle Session Fixation and Hijacking attacks, whereas using the built-in Session I think they are handled for you (but I could be wrong).
the home made session as you have described is doing nothing different "SQL" state of .Net sessions and in my experience i dont think session degrades your performance in anyway. building your own session manager will require putting in several other plumbing tasks along - security, flushing it out, etc.
the advantage with in-built sessions is its easy to use with all this plumbing already been taken care of. with "SQL" mode you can persist the session data in database thus allowing you to run your app on web-farms without any issues.
we designed a b2b ecommerce app for fortune 57 company which processes over 100k transactions a day and used sessions [SQL mode] quite extensively without any problems whatsover at all.
Correct me if I am wrong:
The primary advantage of storing Session state in a db, e.g., SQL Server, is that you are not consuming memory resources, but instead storing it to disk in a db.
The disadvantage is that you take an IO hit to retrieve this info from the database each time you need it (or maybe SQL Sever even does some magic caching of the data for you based on recently executed queries?)
In any event, this the price an IO to retrieve the session info from a db per trip to the web server seems like a safer strategy for sites that encounter a lot of traffic.
Are there well-known best practices for synchronizing tasks across a server farm? For example if I have a forum based website running on a server farm, and there are two moderators trying to do some action which requires writing to multiple tables in the database, and the requests of those moderators are being handled by different servers in the server farm, how can one implement some locking functionality to ensure that they can't take that action on the same item at the same time?
So far, I'm thinking about using a table in the database to sync, e.g. check the id of the item in the table if doesn't exsit insert it and proceed, otherwise return. Also probably a shared cache could be used for this but I'm not using this at the moment.
Any other way?
By the way, I'm using MySQL as my database back-end.
Your question implies data level concurrency control -- in that case, use the RDBMS's concurrency control mechanisms.
That will not help you if later you wish to control application level actions which do not necessarily map one to one to a data entity (e.g. table record access). The general solution there is a reverse-proxy server that understands application level semantics and serializes accordingly if necessary. (That will negatively impact availability.)
It probably wouldn't hurt to read up on CAP theorem, as well!
You may want to investigate a distributed locking service such as Zookeeper. It's a reimplementation of a Google service that provides very high speed distributed resource locking coordination for applications. I don't know how easy it would be to incorporate into a web app, though.
If all the state is in the (central) database then the database transactions should take care of that for you.
See http://en.wikipedia.org/wiki/Transaction_(database)
It may be irrelevant for you because the question is old, but it still may be useful for others so i'll post it anyway.
You can use a "SELECT FOR UPDATE" db query on a locking object, so you actually use the db for achieving the lock mechanism.
if you use ORM, you can also do that. for example, in nhibernate you can do:
session.Lock(Member, LockMode.Upgrade);
Having a table of locks is a OK way to do it is simple and works.
You could also have the code as a Service on a Single Server, more of a SOA approach.
You could also use the the TimeStamp field with Transactions, if the timestamp has changed since you last got the data you can revert the transaction. So if someone gets in first they have priority.