I'm writing a webcontrol in asp.net which has to go and fetch information from an API. It's not critical to be up-to-date, and retrieving the information is quite slow, so I'd rather cache the information and update it every 5 minutes.
It strikes me as potentially risky to use the Context.Cache as potentially someone could use the same name, but I can't find another way to do caching within a control.
Does anyone have any other ideas?
(using asp.net 2.0).
It sounds like the Context.Cache is exactly what you need to store this kind of information. The cache object is localized to your application so it would only be changeable by other code in your application.
You can always give your cache key a very long and presumably useless name etc and store that key within your class...
Private const CacheKey as String = "e92a627b-3a9f-46da-a182-d73b44fe87ad" ' A random guid
Cache.Item(CacheKey) = TheRefreshedData ' etc
As I was writing this answer I realised maybe you were talking about your conntrol being re-used within someone else's application...as in your control is redistributed or provided for download. In this case, you could always prefix the cache key with your control to be "sensibly" unique...
Private const CacheKey As String = "MyCompany.MyControl.MyData"
Cache.Item(CacheKey) = TheRefreshedData ' etc
Either way, you should be able to come up with some fairly length string that has zero chance of being reused...
Related
Some background:
Working with:
.NET 4.5 (thinking of migrating to 4.5.1 if it's painless)
Web Forms
Entity Framework 5, Lazy Loading enabled
Context Per Request
IIS 8
Windows 2012 Datacenter
Point of concern: Memory Usage
Over the project we are currently on, and probably our first bigger project, we're often reading some bigger chunks of data, coming from CSV imports, that are likely to stay the same for very long periods of time.
Unless someone explicitly re-imports the CSV data, they are guaranteed to be the same, this happens in more than one places in our project and similar approach is used for some regular documents that are often being read by the users. We've decided to cache this data in the HttpRuntime cache.
It goes like this, and we pull about 15,000 records consisting mostly of strings.
//myObject and related methods are placeholders
public static List<myObject> GetMyCachedObjects()
{
if (CacheManager.Exists(KeyConstants.keyConstantForMyObject))
{
return CacheManager.Get(KeyConstants.keyConstantForMyObject) as List<myObject>;
}
else
{
List<myObject> myObjectList = framework.objectProvider.GetMyObjects();
CacheManager.Add(KeyConstants.keyConstantForMyObject, myObjectList, true, 5000);
return myObjectList;
}
}
The data retrieving for the above method is very simple and looks like this:
public List<myObject> GetMyObjects()
{
return context.myObjectsTable.AsNoTracking().ToList();
}
There are probably things to be said about the code structure, but that's not my concern at the moment.
I began profiling our project as soon as I saw high memory usage and found many parts where our code could be optimized. I never faced 300 simultaneous users before and our internal tests, done by ourselves were not enough to show the memory issues. I've highlighted and fixed numerous memory leaks but I'd like to understand some Entity Framework related unknowns.
Given the above example, and using ANTS Profiler, I've noticed that 'myObject', and other similar objects, are referencing many System.Data.Entity.DynamicProxies.myObject, additionally there are lots of EntityKeys which hold on to integers. They aren't taking much but their count is relatively high.
For instance 124 instances of 'myObject' are referencing nearly 300 System.Data.Entity.DynamicProxies.
Usually it looks like this, whatever the object is:
Some cache entry, some object I've cached and I now noticed many of them have been detached from dbContext prior caching, the dynamic proxies and the objectContext. I've no idea how to untie them.
My progress:
I did some research and found out that I might be caching something Entity Framework related together with those objects. I've pulled them with NoTracking but there are still those DynamicProxies in the memory which probably hold on to other things as well.
Important: I've observed some live instances of ObjectContext (74), slowly growing, but no instances of my unitOfWork which is holding the dbContext. Those seem to be disposed properly per request basis.
I know how to detach, attach or modify state of an entry from my dbContext, which is wrapped in a unitOfWork, and I often do it. However that doesn't seem to be enough or I am asking for the impossible.
Questions:
Basically, what am I doing wrong with my caching approach when it comes to Entity Framework?
Is the growing number of Object Contexts in the memory a concern, I know the cache will eventually expire but I'm worried of open connections or anything else this context might be holding.
Should I be detaching everything from the context before inserting it into the cache?
If yes, what is the best approach. Especially with List I cannot think of anything else but iterating over the collection and call detach one by one.
Bonus question: About 40% of the consumed memory is free (unallocated), I've no idea why .NET is reserving so much free memory in advance.
You can try using non entity class with specific properties with SELECT method.
public class MyObject2 {
public int ID { get; set; }
public string Name { get; set; }
}
public List<MyObject2> GetObjects(){
return framework.provider.GetObjects().Select(
x=> new MyObject2{
ID = x.ID ,
Name = x.Name
}).ToList();
);
}
Since you will be storing plain c# objects, you will not have to worry about dynamic proxies. You will not have to call detach on anything at all. Also you can store only few properties.
Even if you disable tracking, You will see dynamic proxy because EF uses dynamic class derived from your class which stores extra meta data information (relation e .g. name of foreign key etc to other entities) for the entity.
steps to reduce memory here:
Re new the context, often
Dont try and delete content from the Context. Or Set it to detached.
It hangs around like a fart in a phone box
eg context = new MyContext.
But if possible you should be
using (var context = new Mycontext){ .... }
//short lived contexts is best practice
With your Context you can set Configurations
this.Configuration.LazyLoadingEnabled = false;
this.Configuration.ProxyCreationEnabled = false; //<<<<<<<<<<< THIS one
this.Configuration.AutoDetectChangesEnabled = false;
you can disable proxies if you still feel they are hogging memory.
But that may be unecesseary if you apply using to the context in the first place.
I would redesign the solution a bit:
You are storing all data as a single entry in cache
I would move this and have an entry per cache item.
You are using HTTPRuntime cache
I would use Appfabric Caching, also MS, also free.
Not sure where you are calling that code from
I would Call it on Application start, then all data is in memory when the user needs it
You are using Entity SQL
For this I would use an Entity Data Reader http://msdn.microsoft.com/en-us/library/system.data.entityclient.entitydatareader(v=vs.110).aspx
See also:
http://msdn.microsoft.com/en-us/data/hh949853.aspx
A problem appears when two users are logged on to our service system at the same time and looking at the service list gridview. If user1 does a search to filter the gridview and user2 happens to click to another page user2 sees the results from the search performed by user1. That means one company can see another company's data.
It's an ASP.NET application that was developed in house with C#/ASP.NET 3.5. The data is stored in a SQL 2000 database and relies very heavily on stored procedures to update, select, and delete data. There are multiple user types that are restricted to what data they can see. For example, we have a company use that can only see data relavant to that company.
From what I've seen, the security is handled through If statements in the front end. Example, if userlevel = 1 then do this, if userlevel = 2 do this. These statments are used to show or hide columns in a grid, run queries to return data, and any other restrictions needed. For a company user the code behind gets the companyid assigned to the user and uses that in a query to return the results of all the data associated with that companyid (services, ships, etc).
Any recommendations for fixing this will be highly appreciated.
It's hard to say without seeing any implementation details, but on the surface it appears that there maybe some company level caching. Check for OutputCache settings, DataSource caching, explicit caching with Page.Cache, etc.
This article is a little dated, but at a glance it looks like most information is still relevant in ASP.NET 4.0.
ASP.NET Caching: Techniques and Best Practices
In addition to jrummerll's answer, check the Data Acces Layer of our app and make sure that you don't have any static variables defined. Having a static variable defined could cause this sort of issue too, since 2 contending requests may overwrite the value of the CompanyID, for example.
You basic model should work. What you've told us is not enough to diagnose the problem. But, I've got a few guesses. Most likely your code is confusing UserID or CompanyID values.
Are you mistakenly storing the CompanyID in the Cache, rather than the session?
Is the CompanyID stored in a static variable? A common (and disastrous!) pitfall in web applications is that a value stored in a static variable will remain the same for all users! In general, don't use static variables in asp.net apps.
Maybe your db caching or output caching doesn't vary properly by session or other variables. So, a 2nd user will see what was created for the previous user. Stop any caching that's happening and see if that fixes it, but debug from there.
Other variations on the above themes: maybe the query is stored in a static variable. Maybe these user-related values are stored in the cache or db, but the key for that record (UserID?) is stored in a static variable?
You can put that if statements in a thread. Threading provides you the option that only 1 user can access the application or gridview in your case.
See this link: http://msdn.microsoft.com/en-us/library/ms173179.aspx
Here is some sample code that is throughout the entire application that is used for filtering results. What is the best way to fix this so that when one user logs on, the other user doesn't see those results?
protected void PopulategvServiceRequestListing(string _whereclause)
{
_dsGlobalDatasource = new TelemarServiceRequestListing().GetServiceRequestListingDatasource(_whereclause);
if(_dsGlobalDatasource.Tables[0].Rows.Count!=0)
{
gv_ServiceRequest.DataSource = _dsGlobalDatasource;
gv_ServiceRequest.DataBind();
}
else
{
gv_ServiceRequest.DataSource=new TelemarServiceRequestListing().DummyDataset();
gv_ServiceRequest.DataBind();
gv_ServiceRequest.Rows[0].Visible = false;
gv_ServiceRequest.HeaderStyle.Font.Bold = true;
}
}
I have an ASP.NET c# project.
I have to pass a list of values (id numbers such as "23,4455,21,2,765,...) from one form to another. Since QueryString is not possible because the list could be long, which is the best way to do it?
Thanks in advance.
Thanks for all your answers, you are helping a lot !!!
I decided to do this:
On the first form:
List lRecipients = new List();
.....
Session["Recipients"] = lRecipients;
On the final form:
List lRecipients = (List)Session["Recipients"];
Session.Remove("Recipients");
You could use Session collection.
In the first page, use:
List<int> listOfInts = new List<int>();
...
Session["someKey"] = listOfInts
And in the second page, retrieve it like this:
List<int> listOfInts = Session["someKey"] as List<int>;
If your using asp.net webforms you can put it into a session variable to pass stuff from page to page. You've got to be concise of the potential performance issues of putting lots of stuff into session mind.
Session["ListOfStff"] = "15,25,44.etc";
There are any number of ways to pass this data. Which you choose will depend on your environment.
Session state is useful, but is constrained by the number of concurrent users on the system and the amount of available memory on the server. Consider this when deciding whether or not to use Session state. If you do choose session state for this operation, be sure to remove the data when you're done processing the request.
You could use a hidden input field, with runat="server" applied to it. This will make its data available server-side, and it will only last for the duration of the request. The pros of this technique are that it's accessible to both the server code and the client-side JavaScript. However, it also means that the size of your request is increased, and it may take more work to get the data where you want it (and back out).
Depending on how much data's involved, you could implement a web service to serialize the data to a temporary storage medium (say, a database table), and get back a "request handle." Then, you could pass the request handle on the query string to the next form and it could use the "handle" to fetch the data from your medium.
There are all kinds of different ways to deal with this scenario, but the best choice will depend on your environment, time to develop, and costs.
For Asp.NET MVC you can use ViewData.
ViewData["ID"] = "";
I think I have a solution to this, but is there a better way, or is this going to break on me?
I am constructing a localized web site using global/local resx files. It is a requirement that non-technical users can edit the strings and add new languages through the web app.
This seems easy enough -- I have a form to display strings and the changes are saved with code like this snippet:
string filename = MapPath("App_GlobalResources/strings.hu.resx");
XmlDocument xDoc = new XmlDocument();
XmlNode xNode;
xDoc.Load(filename);
xNode = xDoc.SelectSingleNode("//root/data[#name='PageTitle']/value");
xNode.InnerText = txtNewTitle.Text;
xDoc.Save(filename);
Is this going to cause problems on a busy site? If it causes a momentary delay for recompilation, that's no big deal. And realistically, this form won't see constant, heavy use. What does the community think?
I've used a similar method before for a very basic "CMS". The site wasn't massively used but it didn't cause me any problems.
I don't think changing a resx will cause a recycle.
We did something similar, but used a database to store the user modified values. We then provided a fallback mechanism to serve the overridden value of a localized key.
That said, I think your method should work fine.
Have you considered creating a Resource object? You would need to wrap your settings into a single object that all the client code would use. Something like:
public class GuiResources
{
public string PageTitle
{
get return _pageTitle;
}
// Fired once when the class is first created.
void LoadConfiguration()
{
// Load settings from config section
_pageTitle = // Value from config
}
}
You could make it a singleton or a provider, that way the object is loaded only one time. Also you could make it smart to look at the current thread to get the culture info so you know what language to return.
Then in your web.config file you can create a custom section and set restartOnExternalChanges="true". That way, your app will get the changed when they are made.
I have object A which in turn has a property of type Object B
Class A
property x as Object B
End Class
On my ASP.NET page when I select a gridview item which maps to an object of type A I serialize the object onto the QueryString and pass it to the next page.
However I run into problems if property x actually has some value as it looks like I exceed the QueryString capacity length of 4k (although I didn't think the objects were that large)
I have already considered the following approaches to do this
Session Variables
Approach not used as I have read that this is bad practice.
Using a unique key for the object and retrieving it on the next page.
Approach not used as the objects do not map to a single instance in a table, they arte composed of data from different databases.
So I guess my question is two fold
Is it worth using GKZip to compress the querystring further (is this possible??)
What other methods would people suggest to do this?
If displaying the url of the next page in the browser does not matter, you could use the context.items collection.
context.items.add("keyA", objectA)
server.transfer("nextPage.aspx")
Then on the next page:
public sub page_load(...)
dim objectA as A = ctype(context.items("keyA"), objectA)
dim objectB as B = objectA.B
end sub
One reason to use this is if you want the users to believe that the next page is really a part of the first page. To them, it only appears as if a PostBack has occurred.
Also, you don't really need a unique key using this approach if the only way to use "next page" is if you first came from "first page". The scope for the context items collections is specific to just this particular request.
I agree with the other posters who mentioned that serialized objects on the querystring is a much worse evil than using session state. If you do use session state, just remember to clear the key you use immediately after using it.
I don't understand why you wouldn't use session state but...
Option 1: Viewstate
Option 2: Form parameters instead of querystring
But also be aware that you do not get the same object back when you serialize/deserialize. You get a new object initialized with the values of the original that were serialized out. You're going to end up with two of the object.
EDIT: You can store values in viewstate using the same syntax as Session state
ViewState["key"] = val;
The value has to be serializeable though.
While storing objects in session might be considered bad practice, it's lightyears better than passing them via serialized querystrings.
Back in classic asp, storing objects in session was considered bad practice because you created thread-affinity, and you also limited your ability to scale the site by adding other web servers. This is no longer a problem with asp.net (as long as you use an external stateserver).
There are other reasons to avoid session variables, but in your case I think that's the way to go.
Another option is to combine the 2 pages that need access to this object into one page, using panels to hide and display the needed "sub-pages" and use viewstate to store the object.
I don't think passing it in the query string, or storing it in the session, is a good idea.
You need one of the following:
a) A caching layer. Something like Microsoft Velocity would work, but I doubt you need something on that scale.
b) Put the keys to each object in the databases that you need in the query string and retrieve them the next time around. (E.g. myurl.com/mypage.aspx?db1objectkey=123&db2objectkey=345&db3objectkey=456)
Using session state seems like the most practical way to do this, its exactly what its designed for.
Cache is probably not the answer here either. As Telos mentioned, I'm not sure why you're not considering session.
If you have a page that depends on this data being available, then you just throw a guard clause in the page load...
public void Page_Load()
{
if(!IsPostBack)
{
const string key = "FunkyObject";
if(Session[key] == null)
Response.Redirect("firstStep.aspx");
var obj = (FunkyObject)Session[key];
DoSomething(obj);
}
}
If session is absolutely out of the quesiton, then you'll have to re-materialize this object on the other page. Just send the unique identifier in the querystring so you can pull it back again.
Session isn't always available. For instance when XSS (cross-site-scripting) security settings on IE prevent the storage of third-party cookies. If your site is being called within an IFrame from a site that's not your DNS domain, your cookies are going to be blocked by default. No cookies = no session.
Another example is where you have to pass control to another website that will make the callback to your site as a pure URL, not a post. In this case you have to store your session parameters in a querystring parameter, something that's tough to do given the 4k size constraint and URL encoding, not to mention encryption, etc.
The issue is that most of the built-in serialisation methods are pretty verbose, thus one has to resort to a roll-your-own method, probably using reflection.
Another reason for not using sessions is simply to give a better user experience; sessions get cleared after N minutes and when the server restarts. OK, in this case a viewstate is preferable, but sometimes it's not possible to use a form. OK, one could rely on JavaScript to do a postback, but again, that's not always possible.
These are the problems I'm currently coding around.
Here is what I do:
Page1.aspx - Add a public property of an instance of my object. Add a button (Button1) with the PostBackURL property set to ~/Page2.aspx
Private _RP as ReportParameters
Public ReadOnly Property ReportParams() as ReportParameters
Get
Return _RP
End Get
End Property
Protected Sub Button1_Click(ByVal sender As Object, ByVal e As EventArgs) Handles Button1.Click
_RP = New ReportParameters
_RP.Name = "Report 1"
_RP.Param = "42"
End Sub
Now, on the second page, Page2.aspx add the following to the Markup at the top of the page under the first directive:
<%# PreviousPageType VirtualPath="~/Default.aspx" %>
Then for the Page_Load in the code behind for Page2.aspx, add the following
If Not Page.PreviousPage is Nothing Then
Response.write (PreviousPage.ReportParams.Name & " " & PreviousPage.ReportParams.Param)
End If
Faced with a similar situation what I did, is to XML serialize the object and pass it around as query string parameter. The difficulty with this approach was that despite encoding, the receiving form throws exception saying "potentially dangerous request...". The way I got around was to encrypt the serialized object and then encode to pass it around as query string parameter. Which in turn made the query string tamper proof (bonus wandering into the HMAC territory)!
FormA XML serializes an object > encrypts the serialized string > encode > pass as query string to FormB FormB decrypts the query parameter value (as request.querystring decodes also) > deserialize the resulting XML string to object using XmlSerializer.
I can share my VB.NET code upon request to howIdidit-at-applecart-dot-net