There are certain tables that get called often but updated rarely. One of these tables is Departments. So to save DB trips, I think it is ok to cache this table taking into consideration that the table has very small size. However, once you cached it an issue of keeping the table data fresh occurs. So what is the best way to determine that the table is dirty and therefore requires a reload and how that code should be invoked. I look for solution that will be scalable. So updating the cache on single right after inserting will not resolve the issue. If one machine inserted the record all other on the farm should get notified to reload the cache. I was thinking for calling corresponding web service from T-SQL but don't really like the idea of consuming recourses on sql server. So what are the best practices to resolve this type of problems.
Thanks in advance
Eddy
There are some great distributed caching frameworks out there. Have a look at NCache and Velocity. NCache has some great features for keeping the cached data in sync between different cache nodes as well as the underlying database. But it comes at a price.
Have you tried using sql dependencies or cache dependencies? The library will pole the database every so often to see if the data has changed. An alternative is to use cache dependencies too. You can have a master cache object and have child caches depend on it. so if the master cache change the child caches will be updated.
Edit:
If the above is not a solution you can easily use memcached.net -- wikipedia. Geared toward large sites but it is a solution for your problem.
Here is an article that describes the thinking around setting up a cache.
http://www.javaworld.com/javaworld/jw-07-2001/jw-0720-cache.html?page=1
Generally speaking objects in a cache have lifetimes and when the lifetime expires they are re-fetched from the database. If the data is not so important this eventual consistency allows for a mixture of performance and accuracy of presented information.
Other cache tools add in additional techniques to keep data more accurate i.e. if a particular object is known to be updated then repopulate after the update command is executed.
Related
I need to access some data on my asp.net website. The data relates to around 50 loan providers.
I could simply build it into the web page at the moment, however I know that I will need to re-use it soon, so its probably better to make it more accessisble.
The data will probably only change once in a while, maybe once a month at most. I was looking at the best method of storing the data - database/xml file, and then how to persist that in my site (cache perhaps).
I have very little experience so would appreciate any advice.
It's hard to beat a database, and by placing it there, you could easily access it from anywhere you wanted to reuse it. Depending on how you get the updates and what DBMS you are using, you could use something like SSIS (for MS SQL Server) to automate updating the data.
ASP.NET also has a robust API for interacting with a database and using it as a datasource for many of it's UI structures.
Relational databases are tools for storing data when access to the data needs to be carefully controlled to ensure that it is atomic, consistent, isolated, and durable. (ACID). To accomplish this, databases include significant additional infrastructure overhead and processing logic. If you don't need this overhead why subject your system to it? There is a broad range of other data storage options at your disposal that might be more appropriate, but should at least be considered options in your decision process.
Using Asp.Net, you have access to several other options, including text files, custom configuration files (stored as Xml), custom Xml, and dotNet classes serialized to binary or Xml files. The fact that your data changes so infrequently may make one of these options more appropriate. Using one of these options also reduces system coupling. Functions dependent on this data are now no longer dependent on the existence of a functioning database.
I've done some searching and haven't found a specific answer. Anyhow, I was wondering how most medium sized ASP.NET based websites cache data so that they don't always have to database look ups for the same data on different pages when running on a web farm environment. I'm aware that you can use a sql server cache, but to me, that defeats the whole purpose of the cache. If I want to use a dedicated server to share cache (and even possible session data), what do most sites use for this? I've done searching and get a lot of 'guesses' when it comes to implementations, but it's really hard to believe that there isn't some standard way of doing this given that there are so many ASP.NET websites out there.
I am aware of AppFabric that seems like it might do the trick, but only runs on Windows server 2008+ and we're currently using 2003. Also, I've checked out NCache but it seems to be a little pricey.
Has anyone implemented a solution that worked for them? Database lookups can be quite painfully slow when 1000+ users are hitting your site simultaneously.
Thanks in advance!
Might want to take a look at HttpContext.Cache and HttpRuntime.Cache
You can get your information from the database then create datasets (or dictionaries, or whatever you want) and then store those in cache to be referenced site wide.
You can even specify how long you want the cache to persist. And when it expires the next request for that data will go to the database and recreate your cache.
This is related to a previous question I asked, regarding splitting a asp.net mvc web application into two apps - one public, and one admin. Both of them would be sharing the same database of course.
With two web apps I'll have two host applications and so two seperate Nihbernate session factories. My concern is that their seperate data caches will cause all sorts of problems.
While posting this, I looked at this question, which covers a similar problem (for a different reason). I really don't want to have to set up some kind of distributed cache just for a low use admin application.
How can this be resolved? Is separating the admin section into it's own application not possible with NHibernate without drastic measures?
Thanks.
We run this successfully although some discrepancy in the data is always a problem. However, since the 2nd level cache is configurable per site, you can disable as well as turn it down for specific cache areas on your manager.
The 2nd level cache will only be used for reading, since explicit updates will be flushed down and persisted directly.
If your concern is that content on the site will be "old" once modified, some sort of trigger will be needed to instruct the site to evict the cache. NHibernate will then evict all 2nd level cache for a specific entity type if I remember it correctly.
I think your problem with concurrency will be minimal if your site vs your admin will update different entities. For example in a webshop:
Site will create orders, modify customers etc but only read products, prices and categories
Admin will modify orders, products, prices and categories but only read customers
You can however instruct NHibernate to only update modified fields/properties on your objects for entities that you are concerned about concurrency issues with dynamic-update="true" on your mapping. This won't fully solve your problem, but minimize concurrency issues.
First, you should know that NHibernate doesn't enable second-level cache by default.
So, actually you don't even need any additional steps to complete to just not to use distributed cache. Just use your "Admin" ISessionFactory and don't enable any L2 cache for that.
It could be a sort of problem inside a single App/Factory but you already solved that problem by dividing them into 2 different physical apps.
Im getting to where i need to consider caching for my web application. Im a bit courious as to how long time i should cache the items, so i would like to make some sorts of statistics on how often i use my cache vs how often i load data from my server. How to go about doing this? Should i do this manually with data caching, like this:
Does the data exist in my cache
No - load it into cache and return it
Insert a row into my database for statistics
Yes - use the data from the cache
Insert a row into my database for statistics
To me it seems a bit odd to query the cache and then insert into my database, seems the performance gain is lost then - right?
What to do then?
Instead of looking at this problem as a sitewide problem, I would investigate those areas that are bottlenecks in your system, and consider caching those. The time you cache will depend on the data, and how frequently it changes.
Asp.net has some built in performance tools that will show you quite a bit of information about your cache
http://msdn.microsoft.com/en-us/library/ms972959.aspx
Don't focus on the cache hits vs. misses. Start off by focusing on the data that is retrieved often, but changes very little.
This article has some good information on implementing a caching policy
http://www.c-sharpcorner.com/UploadFile/vishnuprasad2005/ImplementingCachinginASP.NET11302005072210AM/ImplementingCachinginASP.NET.aspx
I would agree with Aaron. Why do you want to introduce caching? What problem are you trying to resolve? Are you having performance problems and are they directly related to database reads?
I have implemented ASP.Net/SQL caching on numerous projects and have found that the bottlenecks usually lie elsewhere i.e. page size.
I hope this helps.
B
I'm working on an ASP.NET MVC project and I've come to the point where I want to start considering my caching strategy. I've tried to leave my framework as open as possible for the use in caching.
From what I heard during Scott Hanselman's podcast StackOverflow.com uses page output caching and zips that content and puts it into RAM. This sounds like this would be great for user-wide cache but for something like personalized pages you would have to cache a version for each user and that could get out of control very quickly.
So, for a caching strategy. Which should be used, Output Caching, Data Caching or combined? My first thoughts are both but as far as cache dependencies it sounds like it could get a bit complex.
We're doing API and Output caching on a large scale (3 milion visits a day) web site (news portal). The site is primarily used by anonymous users, but we do have authenticated users and we cache a complete site just for them, due to some personalized parts of the site, and I must admit that we had absolutely no problems with memory pressure.
So, my advice would be cache everything you can in API cache so your Output cache rebuilding is even faster.
Of course, pay close attention to your cache ratio values in the performance counters. You should see numbers >95% of cached hits.
Another thing to pay attention is cache invalidation, this is a big issue if you have a lot of related content. For example, you cache music stuff and information about one album or song might be displayed and cached on few hundred pages. If anything changes in that song, you have to invalidate all of these pages which can be problematic.
Bottom line, caching is one of the best features of ASP.NET, it's done superbly and you can rely on it.
Be careful about over-aggressive caching. Although caching is a tool for helping performance, when used incorrectly, it can actually make performance worse.
I can't answer whether output caching or data caching would work for you better without knowing more details about your project.
I can help provide a couple examples of when to use one over another.
If you have a specific data set which you would use often in many different views, you'd be better off using data caching. You'd use this if your data fetch operation was very common and expensive relative to your data rendering. If you had multiple views which used the same data, you would save your data fetching time.
If you had a view which used a very specific data set and the rendering of the view was complicated and this view was requested very often (for example, stack overflow's home page), then you would benefit a lot from output caching.
So in the end, it really depends on your needs and be careful about using caching incorrectly.