I'm experienced with ASP.NET but some of my knowledge is a little shaky. I'm creating an application that uses services to get large portions of data. I then want to filter it with LINQ to what I need. This data rarely changes, but I know there's too much of it to fit in Session.
How can I store rarely-changing , large amounts of data in memory? Would this be suitable for Application variables?
Instead of caching the source data, consider using HTTP caching via ASP.NET Output Cache or memcached to store the rendered output itself. ASP.NET OutputCache can be tuned to work on specific ASP.NET resources and there are many ways to invalidate the cache explicitly if need be. See the following MSDN resources for more information.
ASP.NET Caching Overview
Output Cache Configuration
OutputCache Attribute for ASP.NET MVC
You can store large amounts of fairly static data:
In the HttpContext.Cache
In a private static field of a C# class. Wrap the variable with an public get property that can initialize the static field. The static field will need to be initialized when the app first starts, and any time the app domain is recycled by IIS.
HttpContext.Cache
Pro's
Configurable cache expiration policy.
Designed specifically for the ASP.Net environment
Con's
Might be evoked from cache if ASP.Net determines it is under-utilized.
Static Field
Pro's
Complete user control over cache expiration.
Con's
Any cache expiration must be explicitly programmed. No built-in expiration support.
Related
When i need to cache something in my application, i used to choose Web.Caching.Cache. But i ran into some legacy code that using HttpApplicationState instead.
Since Web.Caching.Cache is more powerful and flexible (seems MUCH more), is there a situation that better to use HttpApplicationState??
I will be very appreciate if you can show me some examples :-)
Both HttpApplicationState and Web.Caching.Cache can be used to store information that can be globally accessible throughout an ASP.Net application. However, they have very different uses.
HttpApplicationState is used to store application data that typically does not change. It is typically populated in Application_Start in Global.asax, when the application is starting. I personally have not used it much, but I believe it is typically used to store small pieces of application configuration that are global to all users of an application and which either do not change or change very infrequently. Something put into Application state will remain there indefinitely, until the app recycles. But when it recycles and restarts again, Application_Start will execute again and re-populate it.
It is important to note that HttpApplicationState is a singleton and is not thread safe. So when you make changes to it, you must lock and unlock the Application object via calls to Application.Lock() and Application.UnLock(). Read more
There are actually three different ways you can cache ASP.Net content: Page level, partial page, and data. I am going to talk about data caching, since I think that is most relevant to your question. The ASP.Net cache is used to store large quantities of application data that would be expensive to retrieve from a data store for every request. The key differences between HttpApplicationState and Cache are 1) Cache data is designed to expire and be purged from memory by a variety of triggers or conditions (time, other cache dependencies, etc), whereas HttpApplicationState will be there forever until the app recycles, and 2) Cache data can be purged from memory if a server is experiencing severe memory pressure, and you thus can never count on it being there and must always test to see if it is present, whereas HttpApplicationState will always be there.
Caching is used to store data closer to the application that does not need to be pulled from a database on every request. Caching is designed to store very large quantities of data, and an intelligent caching architecture can have an enormous positive impact on performance.
I store a large structure holding my application's reference data in a variable I access through HttpContext.Application. Every once in a while this data needs to change. When I update it in place, is there a danger that incoming requests will see the data in an inconsistent state? Is there a need (and a way) to lock some or all of this structure? Finally, are there other approaches to this problem other than querying the database every time you need this (mostly static) data?
There are also other solutions availiable, there are many caching providers that you can use.
First of all, there's the HttpRuntime.Cache (which is the same as the HttpContext cache). There's also the System.Runtime.Caching.MemoryCache in .NET 4.
You can set data expiry and other rules for the data in the cache.
http://wiki.asp.net/page.aspx/655/caching-in-aspnet/
http://msdn.microsoft.com/en-us/library/6hbbsfk6.aspx
http://msdn.microsoft.com/en-us/library/system.runtime.caching.memorycache.aspx
More advanced caching includes distributed caches.
Usually, they reside on another server but may also reside on a different process on the same server.
Such providers are AppFabric (from Microsoft) and MemCached and others that I can't recall currently.
appfabric: http://msdn.microsoft.com/en-us/magazine/ff714581.aspx
memcached: http://memcached.org/
You will not see the application variable in inconsistent state.
The MSDN page for HttpApplicationState says (Under the Thread Safety section):
This type is thread safe.
You may be looking for HttpContext.Items instead to store data in the request scope instead of the application scope. Check out this article to get a great overview of the different context scopes in ASP.NET.
Your solution to avoid querying the database for "mostly static data" is to leverage ASP.NET's caching.
I am a learner.I am learning Caching in ASP.NET.There are three types of caching in ASP.NET.
1.Page output caching.
2.Partial Output caching.
3.Data Caching.
In Page output caching, all the rendered content of the page saved in Cache and page every time re-execute.
In Partial Output caching, we can apply caching rules on different parts of pages.
But Data Caching, I didn't understand.
Could anyone please explain me Data Caching?
Thanx in advance.
In simple terms data caching is storing data in memory for quick access. Typically information that is costly to obtain (in terms of performance) is stored in the cache. One of the more common items stored in a cache in a Web application environment is commonly displayed database values; by caching such information, rather than relying on repeated database calls, the demand on the Web server and database server's system resources are decreased and the Web application's scalability increased. As Microsoft eloquently puts it, "Caching is a technique widely used in computing to increase performance by keeping frequently accessed or expensive data in memory. In the context of a Web application, caching is used to retain pages or data across HTTP requests and reuse them without the expense of recreating them."
Read more : .NET Data Caching
It is about caching application data (using the Cache class) - persistence of some objects (values).
We are working on a large legacy application and we're redesigning the business layer and the data layer. We believe that it is a good time to redesign the way cache is handled. Are there any patterns and best practices for implementing a caching layer (or building it in the business layer)?
The only things that I can think of are to use SQL Cache Dependency (which is hard because of we've got a big legacy database with a lot of tables that do not always correspond to the business objects) and implementing strongly typed CacheManager class to hide the string keys and eliminate class casting problems.
Is there anything more sophisticated that we can do? Is there any way to enforce cache invalidation when performing update/delete? Should we somehow maintain a single object in the cache and retrieve list of IDs from the DB that always work with the same objects?
Basically what techniques for caching do you use in ASP.NET? Too bad we cannot use IoC containers or ORM frameworks that support cache :(
Edit: I am more concerned about maintainability than performance.
Just save every queryresult to the database (with cache key: your query, value: your list of business objects)
Use distributed cache like memcached next to ASP.Net cache
Use a sophisticated cachemanager like https://github.com/enyim/memcached-providers; that can have cache-groups. Some data has to be stored for a long time, some short time. Some data has to be stored in ASP.Net cache, etc.
Do calls that has to be stored in the cache using a wrapper function like public T GetFromCache<T>(string key, Func<T> ifKeyNotFoundDelegate) to ensure that cache is always used the same. [1]
Think of when to use ASP.Net cache, and when to use distributed cache. Data that is read every request should be stored in ASP.Net, large data like search results; with a lot of different keys and data etc. should be in memcached.
Add versioning. Prefix all keys with a versionnumber, so you won't get in trouble when updating your web application, and some objectcontracts change.
Ah well, that covers most of what we do in our website (20GB memcached cluster spread over 20 servers).
[1] By making such a function the only interface to store stuff in cache, you can achieve the following. Let's say I want to use something from the cache, like the result from a function. Normally you would do something like
CacheManager cm = new CacheManager(CacheGroups.Totals);
object obj = cm.GetFromCache("function1result");
if(obj == null)
{
obj = (object)DAO.Foo();
cm.StoreInCache("function1result", obj);
}
return (List<MyEntity>)obj;
By using a different interface you can ensure that users won't make a mistake here.
Like
public T GetFromCache<T>(string key, Func<T> ifnotfound)
{
T obj = this.GetFromCache(key) as T;
if(obj == default(T))
{
obj = ifnotfound.Invoke();
this.StoreInCache(key, obj);
}
return obj;
}
This ensures that
We always work with the correct type
That your user always work with cache the same way
Ergo: less probable that they make a mistake. Furthermore: you get nicer, more clear, code, like:
List<MyEntity> list = new CacheManager(CacheGroups.Total).GetFromCache<List<MyEntity>>("function1result", ()=>DAO.Foo());
This is a big subject, but here are a few suggestions:
Don't cache data that's unlikely to be reused, such as user-specific data
Cache at all tiers: client, Silverlight (isolated storage), proxies, http.sys, IIS, ASP.NET cache object, ASP.NET per-request cache, SQL Server
Use SqlDependency / SqlCacheDependency when you can, but don't over-use
Avoid session state; use cookies instead when you can
Leverage page and control (fragment) output caching
Consider using cache validation when needed
Consider light-weight alternatives to the ASP.NET cache object, such as weak memory refs
When used correctly, SQL Server can act as a large cache
In case it helps, I cover this subject in detail in my book: Ultra-Fast ASP.NET.
The MS Patterns and Practices team created Enterprise Library as their response to this question for a host of commone scenarios. EntLib includes Caching as well as Data Access, Validation, Logging, Exception handling, etc. We've used it for years and wouldn't think of starting a new project without it.
http://www.codeplex.com/entlib
As well as the P&P home page, http://msdn.microsoft.com/en-us/practices/default.aspx
I'm using the SqlProfileProvider on one of my websites and in one page I need to fetch the whole list of profiles (it is an intranet).
The method that I use is the ProfileManager.GetAllProfiles(). The problem is that its performance is really bad and it slows down the website considerably.
Therefore, I was thinking of caching the result of the method call in the Application scope as a DataTable (so I could filter/search on it as well).
My problem is that I have several servers running this webapp, and I would like the cache to be in sync. I started using memcached but I was put off by some problems (hence going back to thinking in caching in the Application scope).
So, here are my questions:
Would it be efficient to store the DataTable containing the profiles in the Application object? Or, is it possible to store objects in the Cache and have them available for all clients/browsers?
Is it possible to add a (SQL) Cache Depedency to this cache?
You could cache portions of the web page which will depend on the list of profiles by putting them in a user control and marking it as cacheable. SqlCacheDependency cache policy expiration could be defined as well. As for the cache location, every web server in the farm will have it's own version in memory but using cache expiration will make sure that this version is not out of sync with the data in the DB.
Page or fragment caching is the most effective caching technique because contrary to caching your model (a DataTable or whatever) you don't pay the price of HTML rendering.