I have form that displays several keywords (standard set of choice lists that changes rarely). There are about 4 such fields and each have about 20 choices or so.
I'm thinking if caching the keywords will be helpful for performance / best practice? Is there a strategy for determining when to cache?
To start, you ought to look at the cost for those keywords.
Are you querying them from the database, each one individually?
Are you querying them as a group?
Are they constants that you're simply writing out?
In general when optimizing (IE caching) look for items that are going to return the most bang for your buck.
Also look at the old 80-20 rule; ~80 items of data are a small drop in the bucket, whereas a list of 800,000 items is worth looking at.
You could use the ASP.NET Output Cache in your Page Directive.
<%# OutputCache Duration="60" VaryByParam="Keyword" %>
This will create a server-side cache of the page for each Keyword GET/POST request, and each cache will last 1 minute.
So if someone visits my-page.aspx?Keyword=Cards asp.net will render the page and save it in memory as HTML for 60 seconds. If someone visits my-page.aspx?Keyword=Books it will create a separate version of the page in HTML and cache it as well.
Use a lazy initialized singeton with appropriate field to store your data. eg of this singleton at
http://www.yoda.arachsys.com/csharp/singleton.html
You can have the property to be of type
"IDictionary<string, IList<string>"
if you want them organised by Category and iterable by keyword.
You can use
"IDIctionary<string, IDictionary<string, string>"
IDIctionary
if you want to be search able by category and keyword. If I read your question correctly, this would be an appropriate choice for you
Related
I have a legacy website that needs a little optimization because of poor performance. It is an asp.net shopping website with linq to sql as data layer and MVP pattern as UI pattern.
The most costly entities in the db are product and category tables that have a one to many relationship. These two entities might not change regularly unless a user of admin group decides to add a product or category… etc. i was wondering how resource costly would it be to create and fetch everything from these two entities for each request! so if i could have had a way to keep my data alive…
first I thought well let’s use AJAX for data retrievals so I will create only those entities that I need to query or bind to, but wait, how can I do that without creating a new DataContext instance?!!
At the other side, using cache for whole DataContext is considered a bad decision because of memory cost. So what would be the best option here? How can I improve things?
UPDATE
1) doing what #HatSoft suggested.
Cons: those approaches will not help your code, only the database. beside this, there might be memory issues since we're putting data in memory instead of rendered html, however this might be the best option regarding de-coupling.
2) using output caching we have this code in an http handler with *.aspx wildcard:
string pagePath = Context.Request.Url.AbsolutePath;
object cacheKey = application[pagePath];
if(cacheKey == null)
return; //application restarted/first run so cache the stuff
else
Context.Response.RemoveOutputCacheItem(pagePath);
Cons: now we should link the pagePath to each database entity that the page uses, but if i do so then i'm coupling things instead of de-coupling them. this approach also will run into a little hard coding.
3) another solution would be output caching in post-cache mode instead of control cache mode. using Subsituation element and setting the OutPutCache Duration to 86400 so the page will be re-created every 24 hours.
Cons: hard coding user controls to produce the html output for Subsituation element dynamically.
so what do you suggest?
I would suggest you look in to SqlDependency class please read this article http://www.asp.net/web-forms/tutorials/data-access/caching-data/using-sql-cache-dependencies-cs
Also I would suggest you look in to loading data in the cache at application startup if it suits your application. Please see a good example here http://www.asp.net/web-forms/tutorials/data-access/caching-data/caching-data-at-application-startup-cs
With Linq2SQL you can use LinqToCache which offers a SqlDependency powered cache for your LINQ queries. It transforms the IQueryable<Products> into IEnumerable<Products> and enumerates form memmory after first access (first iteration of the underlying IQueryable). Based on SqlDependency data change notifications it invalidates the list and subsequent access will query again from DB, and cache the result.
My recommendation would be to cache the Products list and Categories in memory, since they change seldom and I expect them to be of a fairly constrained size.
I'm developing a website (using asp.net-mvc) with a SqlServer 2005 database.
I have numerous database tables which drive content pages for the site e.g. I have a table called Activity:
Activity
-----------
ID
Name
So for each activity record, there would be a corresponding 'Activity' page. The same applies for other tables e.g. Location and Person etc.
For the purposes of SEO I want to allow storing of additional info for pages such as html meta title/descritpion/keyword info, and perhaps even page content.
I'm considering two main options to do this:
1) Modify Activity, Location and Person tables to include the relevant fields to hold this additional info
or
2) Create a PageInfo table to hold all this info in the same place, then simply add a PageInfoID field to the above mentioned tables
What are the pros and cons of these approaches and are there any better ways of doing this?
(One con I can think of for option 2 is that you can't really enforce the 1:1 relationship, so you could, theoretically, have a PageInfo record used by an Activity record and a Person record.
If i may suggest.. you may be going about this SEO thing the wrong way.
Rather than trying to pack each page with additional meta-data per 'Activity' object to get better search results from the page, i think you should concentrate on simply putting the 'Activity' data into the page with clean, semantic, valid XHTML and CSS. This way, search engines will worry about finding out which part of each page is of the most importance and index/rank that accordingly.
Also, trying to add these 'page meta' objects to the domain model of your system will create all sorts of conceptual (and indeed practical) problems for your design and it will certainly be confusing to users who will struggle to understand what the importance of the difference between say an Activity Title verses a Page Title is. Let the bots figure our keywords from your content, dont try to do it yourself - this type of over-optimisation will actually end up resulting in worse page rankings than better ones.
Here's my scenario:
1) User runs search to retrieve values for display in ListView via LinqDataSource.
2) They click on one of the items which takes them to another page where the details can be examined, further drill-down can happen, etc.
3) User wants to go back to the original ListView results to select another item for inspection.
I can see it's possible to pass the querystring params around, allowing the querying to be duplicated each time the user comes back to the ListView, but it seems like there ought to be a way to cache the results.
Since I'm using the LinqDataSource, though, I believe the actual results are fetched each time the query is run. I'm currently feeding a "select new {blah, blah}" type of IEnumerable to the e.Results, which can't be turned into a List since it's populated with anonymous types.
In short:
1) Does it make sense to try to place potentially large query results in the users session?
2) If it does, is a List the reasonable data structure?
3) Do I need to resort to something like creating a class with the correct properties to hold the anonymous data, enumerate the query return, populate the List?
4) Is there a better option than the LinqDataSource for this type goal?
5) Or, does it just make more sense to run the query each time they hit the ListView?
I apologize if this wasn't clear. I would really appreciate it if someone can set me straight before I nuke a bunch of my free time headed down the wrong path :)
First, I would suggest that you look into the caching mechanism that comes with ASP.NET, unless the data is private for a certain user.
Second, I would suggest that you design your application in a way so that you create natural points where you could try to get data from a cache before querying the database (and insert data into the cache, with expiration rules), but don't start putting stuff into the cache until you have verified that it will actually make a difference.
Measure how much time that is actually spent on retrieving data and use caching in the cases where it makes a difference.
I'm not sure if resurrecting threads from the dead is cool on SO, but here is what I found to answer this question:
http://weblogs.asp.net/pwelter34/archive/2007/08.aspx
I'm using ASP.net and an SQL database. I have a blog like system where a number of comments are made against a post and I want to display the number of those comments next to the post. To get that number I could either hold it in the post record and add/subtrack when a comment is added or deleted or I could use the SQL to calculate the number of comments using a query each time a user hits the page. The latter seems to be a bad idea as its going to hit my SQL database harder however holding the number against the record feels like it could be error prone. What do you think is best coding practice in this case?
Always start with a normalized database (your second option). Only denormalize if you have an absolute necessity for performance reasons. Designing it in the denormalized way (which is error-prone as you guessed) is premature optimization. With proper indexes it should be fine calculating the number on the fly.
I think the SQL statement should be fine. The other is duplication of data you already have. A count query should be quick.
Don't optimize prematurely. Use the simple solution and pagefault in optimizations only when they're needed.
I would query the database each time you want the information. I would revisit it later if you find that performance is lacking (optimize later). For the traffic most blog type applications will get, that should be sufficient.
Perhaps get the count back as part of the main thread query so as to limit the number of hits on the actual DB from the webserver. But I would always query the actual count and not try and keep it in a field, data will eventually get out of sync as that is reality.
To increase performance, you could keep a flag in the main table to indicate if the item has any comments but only use this as a 'hint' as to whether or not to perform an additional query to count and retrieve comments at a later time.
Imagine a photo gallery that returns 50 photos to rotate through. Each photo could have its own comments.
The initial page load would return a list of photos plus a flag indicating if a photo has comments.
When a photo is displayed, if the comments flag is set to True, your app would make an ajax request to count and fetch the comments for that photo.
If only 3 out of the 50 photos have comments, you just saved yourself 47 additional requests!
This does denormalize the data, but on a limited level.
Creating hints can really help improve performance for very busy sites.
Depending on how your data model looks...Don't add the total post count to the main thread record, it is error prone, you should calculate the comment count when needed based on the thread ID, IMHO
Caching the pages and updating that cache as comments are added/removed would be a good option a long with the SQL count query if you are that worried about the number of queries happening against the db..
I usually use an indexed view for this kind of thing. This allows you to denormalize the data for quick retrieval, but there is no way for it to get out of sync. Folks will also not be confused and think the view is the master of the data. I have mostly used the standard sku of SS2K5, so I have to specify the (noexpand) hint to get it to actually use the index on the view (enterprise will do it automatically). So for standard sku, I always create a wrapper view that everyone hits so I know the hint is always in place.
Coding this on the web page, so hopefully no syntax errors ;)
create view postCount__
as
select
threadId
,postCount=count_big(*)
from thread
group by threadId
go
create unique clustered index postCount__xpk_threadid on postCount__(threadId)
go
create view postCount
as
select
threadId
,postCount=cast(postCount as int)
from postCount__ with (noexpand)
go
So I use a nomenclature on the actual indexed view to let everyone know not to query it directly. Instead they look for the associated wrapper view that enforces the noexpand hint. Using an indexed view forces you to do count_big, so I often cast down to int in the wrapper view to be able to keep our asp.net code lazily using 32 bit ints. It would be better to omit the cast, but it hasn't been of any significant impact for me.
EDIT - I can tell you that forum software always denormalizes the post count to the thread table. It kills the DB to continually count the post count on every page view if you have an active forum. I love that mssql has indexed views so you can define the denormalization declaratively rather than maintain it yourself.
What does VaryByParam do on an asp.net directive? I have tried looking on the web, but can't actually understand what it does.
Thanks
Please see OutputCacheParameters.VaryByParam:
Gets a comma-delimited list of query
string or form POST parameters that
the output cache uses to vary the
cache entry.
Basically what this means is that the output caching of a given page will use certain values to determine which cache should be returned since a dynamic page may have different redered representations of itself.
Those different representations tend to be driven by user-provided data and this property allows you to configure which values from the query string or a POST payload will control that.