I have an ASP.NET web application and I want to be able to take items from a master list and store them temporarliy into one of four other lists. The 'other' lists need to survive post backs so that more items can be added to them. What direction would you suggest going with?
I have thought of using a generic list stored in memory, temporarliy storing the items into the database and calling them back on PostBack, or storing them into the viewstate, but I have a feeling that there is some solution that I'm missing that might be easier or better.
Josh laid out the states pretty well. My recommendation for a smaller list like he said would be using Session state. Using the DB would be a little messy because you have to maintain those temp tables and worry about multi-session access to the tables. Likewise, cache have the same problem. Viewstate gives you this with extra client traffic and insecure data. So if you're talking less than a few thousand instances on a low traffic server, then session is likely fine.
To make session easier to work with (and you can do this with caching and application state as well) is setup a container object that manages the lists.
//To use it in your page, you can easily access it via:
ListManagerContext.Current.MasterList.Add(4);
[Serializable]
public class ListManagerContext
{
public List<int> MasterList { get; set; }
public List<int> SubList1 { get; set; }
public List<int> SubList2 { get; set; }
public List<int> SubList3 { get; set; }
/// <summary>
/// Key used for the list manager context session variable.
/// </summary>
public const string ListManagerContextKey = "ListManagerContext";
/// <summary>
/// Gets the current ListManagerContext for this session.
/// If none exists, it returns a brand new one.
/// </summary>
[XmlIgnore]
public static ListManagerContext Current
{
get
{
HttpContext context = HttpContext.Current;
if (context != null && context.Session != null)
{
ListManagerContext data = null;
if (context.Session[ListManagerContextKey] == null)
{
data = new ListManagerContext();
context.Session[ListManagerContextKey] = data;
}
else
data = context.Session[ListManagerContextKey]
as ListManagerContext;
return data;
}
throw new ApplicationException("
No session available for list manager context.");
}
}
}
The first thing I would suggest is to see if you can remove the need of keeping state across postbacks.
If you can't do so (and ViewState is not applicable for some reason like bandwidth limitations or requiring data preservation even without a postback from a server form), I suggest consider using Session. You can configure session state to use a SQL Server database backend whenever you want without worrying about changing source code.
The database Idea is likely a poor one (assuming you're not dealing with large amounts of data).
Perhaps your best method would be to store the main list in ViewState, and have the other lists be lists of indexes to the first list.
The lists should automatically store the values they have in the viewstate. If they don't, you probably need to turn the viewstate on for these controls.
If you manually want to make the data survive the round trip, you can either store them in the session or in the viewstate yourself. Technically the viewstate makes the most sense, but if there's a lot of data, it can make the viewstate very large and take a long time to do a round trip. The only issue with the session is you'll have to make sure you clear it once you leave the page.
Don't use the database, that's not what its for.
You could store the list in ViewState or Session and assign it to a property. Here's simple example using a generic list of string, but can be any serializable type.
private List<String> MyTempList
{
get{return Session["mylist"] as List<String>;}
set{Session["mylist"] = value;}
}
protected void Page_Load(object source, EventArgs e)
{
if(!IsPostBack)
{
MyTempList = new List<String>();
}
else
{
MyTempList.Add("Something");
}
}
All of those are options and all have pro's and cons:
Database:
Storing items in the database is a fairly easy and consistent option. You do have to worry about making a round trip call to the database, but at least you have a centralized location to story the data that will scale easily with your web load. However, if this is short lived data, then you will have to worry about cleaning up your database as it might begin to get unwieldy.
Session/Cache:
Session affords a quick solution for in memory storage, but scaling can become problematic if the amount of data is very large. The more information you store in memory the less capacity you have for concurrent users. Also, if you start to add multiple web servers, then you will have to look into some sort of session state server to make sure users don't spontaneously lose their session.
Cache has basically all the same pros/cons except that there is the additional complexity of having to make sure you expire cache items, and manage concurrency issues.
Again, these are both easy to implement solutions, but don't scale as well under heavy load, or large amounts of data.
ViewState:
Viewstate is also an easy to implement solution and gets the load off the server and into the client, but can result in longer load times for the end user. Also it is important to remember that ViewState can be hacked, so if security is a concern then you want to take extra precautions to ensure data integrity.
Conclusion:
All in all, figure out what you want to accomplish and choose the solution that best fits your needs. Shove it behind some abstraction layer like an interface so you can easily change the details later, and then you won't have to worry as much. It's all about knowing what will work best in your particular scenario.
Related
Some background:
Working with:
.NET 4.5 (thinking of migrating to 4.5.1 if it's painless)
Web Forms
Entity Framework 5, Lazy Loading enabled
Context Per Request
IIS 8
Windows 2012 Datacenter
Point of concern: Memory Usage
Over the project we are currently on, and probably our first bigger project, we're often reading some bigger chunks of data, coming from CSV imports, that are likely to stay the same for very long periods of time.
Unless someone explicitly re-imports the CSV data, they are guaranteed to be the same, this happens in more than one places in our project and similar approach is used for some regular documents that are often being read by the users. We've decided to cache this data in the HttpRuntime cache.
It goes like this, and we pull about 15,000 records consisting mostly of strings.
//myObject and related methods are placeholders
public static List<myObject> GetMyCachedObjects()
{
if (CacheManager.Exists(KeyConstants.keyConstantForMyObject))
{
return CacheManager.Get(KeyConstants.keyConstantForMyObject) as List<myObject>;
}
else
{
List<myObject> myObjectList = framework.objectProvider.GetMyObjects();
CacheManager.Add(KeyConstants.keyConstantForMyObject, myObjectList, true, 5000);
return myObjectList;
}
}
The data retrieving for the above method is very simple and looks like this:
public List<myObject> GetMyObjects()
{
return context.myObjectsTable.AsNoTracking().ToList();
}
There are probably things to be said about the code structure, but that's not my concern at the moment.
I began profiling our project as soon as I saw high memory usage and found many parts where our code could be optimized. I never faced 300 simultaneous users before and our internal tests, done by ourselves were not enough to show the memory issues. I've highlighted and fixed numerous memory leaks but I'd like to understand some Entity Framework related unknowns.
Given the above example, and using ANTS Profiler, I've noticed that 'myObject', and other similar objects, are referencing many System.Data.Entity.DynamicProxies.myObject, additionally there are lots of EntityKeys which hold on to integers. They aren't taking much but their count is relatively high.
For instance 124 instances of 'myObject' are referencing nearly 300 System.Data.Entity.DynamicProxies.
Usually it looks like this, whatever the object is:
Some cache entry, some object I've cached and I now noticed many of them have been detached from dbContext prior caching, the dynamic proxies and the objectContext. I've no idea how to untie them.
My progress:
I did some research and found out that I might be caching something Entity Framework related together with those objects. I've pulled them with NoTracking but there are still those DynamicProxies in the memory which probably hold on to other things as well.
Important: I've observed some live instances of ObjectContext (74), slowly growing, but no instances of my unitOfWork which is holding the dbContext. Those seem to be disposed properly per request basis.
I know how to detach, attach or modify state of an entry from my dbContext, which is wrapped in a unitOfWork, and I often do it. However that doesn't seem to be enough or I am asking for the impossible.
Questions:
Basically, what am I doing wrong with my caching approach when it comes to Entity Framework?
Is the growing number of Object Contexts in the memory a concern, I know the cache will eventually expire but I'm worried of open connections or anything else this context might be holding.
Should I be detaching everything from the context before inserting it into the cache?
If yes, what is the best approach. Especially with List I cannot think of anything else but iterating over the collection and call detach one by one.
Bonus question: About 40% of the consumed memory is free (unallocated), I've no idea why .NET is reserving so much free memory in advance.
You can try using non entity class with specific properties with SELECT method.
public class MyObject2 {
public int ID { get; set; }
public string Name { get; set; }
}
public List<MyObject2> GetObjects(){
return framework.provider.GetObjects().Select(
x=> new MyObject2{
ID = x.ID ,
Name = x.Name
}).ToList();
);
}
Since you will be storing plain c# objects, you will not have to worry about dynamic proxies. You will not have to call detach on anything at all. Also you can store only few properties.
Even if you disable tracking, You will see dynamic proxy because EF uses dynamic class derived from your class which stores extra meta data information (relation e .g. name of foreign key etc to other entities) for the entity.
steps to reduce memory here:
Re new the context, often
Dont try and delete content from the Context. Or Set it to detached.
It hangs around like a fart in a phone box
eg context = new MyContext.
But if possible you should be
using (var context = new Mycontext){ .... }
//short lived contexts is best practice
With your Context you can set Configurations
this.Configuration.LazyLoadingEnabled = false;
this.Configuration.ProxyCreationEnabled = false; //<<<<<<<<<<< THIS one
this.Configuration.AutoDetectChangesEnabled = false;
you can disable proxies if you still feel they are hogging memory.
But that may be unecesseary if you apply using to the context in the first place.
I would redesign the solution a bit:
You are storing all data as a single entry in cache
I would move this and have an entry per cache item.
You are using HTTPRuntime cache
I would use Appfabric Caching, also MS, also free.
Not sure where you are calling that code from
I would Call it on Application start, then all data is in memory when the user needs it
You are using Entity SQL
For this I would use an Entity Data Reader http://msdn.microsoft.com/en-us/library/system.data.entityclient.entitydatareader(v=vs.110).aspx
See also:
http://msdn.microsoft.com/en-us/data/hh949853.aspx
I'm still a relative noob, however I was surprised at the results of a small test I did.
I want to store a list of string in viewstate. To date when I want to modify the list I retrieved it from viewstate, performed a List.Add and saved it back to viewstate.
However, I then decided to do a simple test, here it is below:
protected void Page_Load(object sender, EventArgs e)
{
if (!IsPostBack)
{
List<string> s = new List<string>();
s.Add("abc");
s.Add("def");
ViewState.Add("test", s);
s.Add("one");
s.Add("two");
}
var t = (List<string>)ViewState["test"];
foreach (var str in t)
{
Response.Write(str + "<br>");
}
}
As you can see, I create a list, add it to viewstate, then modify the list. To my suprise the list is modified in viewstate , even after postback.
The question is, is it safe to do this, and if so is it considered bad practice.
Thanks in advance.
ViewState is only serialized to the client page at the time of postback.
Previous to that, it is held in memory and safe for editing.
I often use ViewState as a backer to a property:
public Class1 MyClass1
{
get { return (Class1)ViewState["MyClass1"]; }
set { ViewState["MyClass1"] = value; }
}
In general, I would not consider it bad practice to do so, except for the following:
Storing sensitive data - Since data is serialized to the client, it is susceptible to being changed.
Large amounts of data - Since the data is persisted to the client, it will increase the page load times significantly for large amounts.
The reason you're seeing this is due to the Asp.Net WebForms page life cycle. At the point you're modifying the view state, Asp.Net has not yet rendered the page output as HTML. Since you're adding a reference to a dictionary, and changes you make prior to rending will show up. There are other points in the life cycle where such changes might not show up, although I've not investigated to see if that's the case or not.
I would not consider this good practice however. ViewState is serialized as a hidden form field in your web page, and thus any data in it is part of your page. This can at best just increase the size of the response sent to the client (sometimes significantly, if you store a lot of data there). Since it's a hidden form field, the request back to the server is also larger.
It can also open up security holes. You cannot trust necessarly trust view state, although there are ways you can secure it, but I would say its better not to send any data at all you don't absolutely need to render the page.
As an alternative, you may consider storing state that is specific to the user in Session, although you do need to take some extra effort if you're application will be behind a load balancer in a web farm. Storing the data in session will keep it isolated from other users and won't require exposing the data at all to the client.
Kind of subjective, but I don't see anything wrong with this by practice. This is something that happens all the time within different form controls on the page. The controls will edit viewstate information pertaining to themselves and persist that over postback. The reason the list changes in the viewstate when you modify it after adding it to the viewstate is that it doesn't get immediately serialized. The list is passed by reference to the viewstate collection where it sits until the page serializes the viewstate shortly before completing the request.
In ASP.NET MVC 2, the lifespan of an entry in the TempDataDictionary was just one HTTP Request.
That translated to setting a value in one request, redirecting, and having access to the same item at the other end of the line. After this the entry would be no longer available, regardless of whether you read the value out of the dictionary at the latter end of the line or not.
Since ASP.NET MVC 3 (I believe), this implementation detail has changed quite significantly.
Entries in the TempDataDictionary are now only removed once they've been read.
MVC 4
public object this[string key]
{
get
{
object obj;
if (!this.TryGetValue(key, out obj))
return (object) null;
this._initialKeys.Remove(key);
return obj;
}
}
and
public bool TryGetValue(string key, out object value)
{
this._initialKeys.Remove(key);
return this._data.TryGetValue(key, out value);
}
MVC 2:
public object this[string key] {
get {
object value;
if (TryGetValue(key, out value)) {
return value;
}
return null;
}
and
public bool TryGetValue(string key, out object value) {
return _data.TryGetValue(key, out value);
}
Since most people seem to put items in the TempData collection in one request and immediately read them back out in the immediate next request, the functionality seems roughtly the same.
In scenarios where this is not the case such as wanting to read the TempData entry if redirected to one place, and expecting it to have been removed if requesting other resources and navigating back, this change has quite an impact.
No longer is the entry available for one http request but is available over many HTTP requests, be it only available to one single get on the dictionary.
I'd like to know more about this implimentation change, what were the reasons for the change, was this simply to cater for multiple redirects or are there deeper benefits?
Secondary to that, I'm intrigued to know if there's anything built in that now caters for single HTTP request sharing of data in the same way that TempData used to cater for?
You're correct that TempData keys are only cleared if they’ve been read (or after the user’s session expires) but this has been the case since MVC2, (http://forums.asp.net/post/3692286.aspx)
I'd like to know more about this implimentation change, what were the
reasons for the change, was this simply to cater for multiple
redirects or are there deeper benefits?
This change prevented problems that arose in MVC 1, such as TempData keys being deleted before they were read. So yes, the primary benefit is in avoiding these problems when you have multiple re-directs, or interleaved requests. In addition, the RedirectToRouteResult or RedirectResult methods now automatically call TempData.Keep() to prevent clearing of keys, even after they've been read so keep that in mind as well.
In scenarios where this is not the case such as wanting to read the
TempData entry if redirected to one place, and expecting it to have
been removed if requesting other resources and navigating back, this
change has quite an impact.
You’re correct, if you've been coding under the assumption that the TempData keys are cleared automatically you could run into unexpected problems. You can call TempData.Clear() to manually remove all keys from the TempDataDictionary, or TempData.Remove(key) to remove a specific key. You can also use TempData.Peek() to read the value of a TempData key without flagging it for removal from the TempDataDictionary.
Secondary to that, I'm intrigued to know if there's anything built in
that now caters for single HTTP request sharing of data in the same
way that TempData used to cater for?
I'm not aware of any new objects or functions that replicate the original implementation of TempData. Essentially we still use TempData but have to be mindful that the data persists until read and clear the dictionary manually if needed.
Say I have a page in my web application that lets a user update their contact information. Pretend in order to retrieve or save this information I have the following class:
public class User
{
DataAccesClass dataAccesClass = new DataAccesClass()
public string UserName {get;set;}
public string Address {get;set;}
public string EmailAddress {get;set;}
public User(){}
public static User GetUser(int userID)
{
User user = dataAccesClass.GetUser(userID); //
return user;
}
public void Save()
{
dataAccesClass.SaveUser(this);
}
}
Say that on my Page_Load event I create a new instance of my User class (wrapped in a !isPostBack). I then use it's public properties to populate text fields on said page in my web application. Now the question is... When the page is posted back, what is the correct way to rebuild this class to then save the updated information? Because the class was created on Page_Load !isPostBack event it is not available. What is the correct way to handle this? Should I store it in a Session? ViewState? Should I simply rebuild it every post back? The User class in this example is small so that might influence the correct way to do it but I'd like to be able to take the same approach for much larger and more complex classes. Also, would this class be considered an acceptable business object?
what is the correct way to rebuild this class to then save the updated information?
I would say the best practice would be do not rebuild the class on every postback. You should build the data on the first request, set values on controls, then let the viewstate on those controls persist the data.
If there is a potential for the data to need to be updated, tie re-generation of the object to an event indicating there is actual need to update.
Should I store it in a Session? ViewState? Should I simply rebuild it every post back?
Selecting whether to store the value in session or re-pull from the data layer should be based on the memory footprint of the object, the scalability requirements of the application, the costliness of the database operation, and the likelihood that the object will need to be accessed on any particular request. So I believe that is highly situational.
Also, would this class be considered an acceptable business object?
I don't have a lot of experience with BLL's but it looks like you're on the right track.
Unless profiling indicates otherwise, it's okay to just reconstruct the object with every request. You can also implement some kind of caching in your data access code. Your class is an acceptable business object.
Given that User object might have info you wouldn't want to expose through ViewState, it can be stored in Session.
This is the "standard" way of doing this in ASP.NET.
In the case of your example, reconstructing the object looks fine as it is small. But if you have a small object you inevitably store for a while, I would use session. If the object is large, I would directly use database or session with database connection.
Depending how complex you are thinking of getting a JavaScript framework called knockout.js might be a good fit. You could create a json object to bind to a jQuery template that would build the HTML depending on the object, it handles complex objects very well.
I'm beginning to work on the caching infrastructure for my ASP.NET MVC site. The problem is, I can't seem to find a reasonable place for data caching (other than 'everywhere')
Right now my architecture looks like this:
Controller -> Service Layer -> Repository. The repository uses Linq to SQL for data access.
The repository exposes generic methods like Insert, GetById, and GetQueryable, which returns an IQueryable that the service layer can further refine.
I like the idea of putting caching in the repository layer, since the service layer shouldn't really care where the data comes from. The problem though is with cache invalidation. The service layer has more information about when data becomes stale than the repository. For instance:
Suppose we have a Users table and an Orders table (the canonical example). The service layer offers methods like GetOrder(int id), which would call the repository layer:
public Order GetOrder(int id)
{
using(var repo = _repoFactory.Create<Order>())
{
return repo.GetById(id)
}
}
or
repo.GetQueryable(order => order.Id == id && order.HasShipped == false).Single();
If we cache in the repository layer, it seems like it would be very limited in knowing when that order data has changed. Suppose the user was deleted, causing all their orders to be deleted with a CASCADE. The service layer could invalidate the Orders cache, since it knew the user was just removed. The repository though (since it's a Unit of Work), wouldn't be aware. (Ignore the fact that we shouldn't be querying orders for a deleted user, since it's just an example).
There's other situations where I think this shows itself. Suppose we want to fetch all the users orders:
repo.GetQueryable(order => order.UserId == userId).ToList()
The repository can cache the results of this query, but, if another order is added, this query is no longer valid. Only the service layer is aware of this though.
It's also possible my understanding of the repository layer is wrong. I sort of view it as a facade around the data source (i.e. changing from L2SQL to EF to whatever, the service layer is unaware of the underlying source).
Realistically, you will need another layer; the data caching layer. It will be used by your service layer when requesting data. Upon such a request, it will decide if it has the data in cache or if it needs to query the appropriate repository. Likewise, your service layer can tell this new data caching layer of an invalidation (the deletion of a particular user, etc.).
What this can mean for your architecture though, is that your data caching layer will implement the same interface(s) your repositories do. A fairly simple implementation would cache the data by entity type and key. However, if you are using a more sophisticated ORM behind the scenes (NHibernate, EF 4, etc.), it should have caching as an option for you.
You could put an event on the objects returned by your repositories, and have the repository subscribe the cache invalidation to a handler.
For example,
public class SomethingRepository{
public Something GetById(int id){
var something = _table.Single(x=>x.id==id);
something.DataChanged += this.InvalidateCache;
return something;
}
public void InvalidateCache(object sender, EventArgs e){
// invalidate your cache
}
}
And your Something object needs to have a DataChanged event and some public method for your service layer to call to trigger it. Like,
public class Something{
private int _id;
public int Id{
get { return _id; }
set {
if( _id != value )
{
_id = value;
OnDataChanged();
}
}
}
public event EventHandler DataChanged;
public void OnDataChanged(){
if(DataChanged!=null)
DataChanged(this, EventArgs.Empty);
}
}
So, all your service layer needs to know is that the data is being changed, and the repository handles the cache invalidation.
I also suggest you take ventaur's advice and put the cache invalidation logic in a separate service. You don't need to go so far as to create a separate "data caching layer", but the logic would be cleaner if kept in a different class.