Say I have 2 tables in a database, each with 100,000's of rows in detail and 10,000's of rows in the master 5 columns each, properly indexed and related. I want to have master/detail nested gridviews. What's the fastest architecture I could design in .net to do this? The largest result set on the master could be in the thousands (usually in the tens though) and the detail per records could be in the hundreds (usually in the single digits though). I'd like to be able to do a display all masters if possible.
The bottom line: bind to DataReaders, use Repeaters instead of GridViews, and turn off ViewState.
The design you're proposing is going to be pretty hard on your users (thousands of records, yikes), but you probably already know that.
If this is just for display and you want the absolute fastest architecture for Asp.Net, you should obtain an IDataReader for each data segment (master and child), sorted such that you can manually match child records while reading both resultsets in a forward-only fashion. (See Scott Mitchell's Why I don't Use DataSets in my ASP.NET Applications for some details about DataReader performance - as long as you aren't optimizing prematurely, it's quite good advice.)
Instead of using a GridView, I'd use a Repeater which has less overhead and lets you write more compact HTML (for a smaller payload to the client): bind the master IDataReader to that repeater. (I'm not sure whether you meant the GridView control, or just a conceptual grid - the basic architecture would be the same for a GridView)
Then add a handler to the Repeater.ItemDataBound that checks if the child data reader's current record matches. If it does, manually render your detail data for that master record.
Finally, you should also turn ViewState off, at least for the Repeater (preferably for as much of the page as possible), again so that the HTML payload is smaller.
If you're totally committed to nested GridViews, particularly to using a GridView to render the detail data, it's going to hurt the performance one way or another, because you'll have to either make many more database calls (to obtain discrete resultsets you can bind to) or you'll have to massage the detail data into an intermediary container.
Related
I'm relatively new to .NET and the controls it contains. I realize that this question is really open to opinion, but would appreciate the input of those of you who've had to support .NET pages where your database record sets were in the millions of records.
I have an Oracle table that will grow to several hundred thousand rows and likely to over 5-10 million over the next few years. Our current system that this is replacing is 6 million records. Not massive, but enough to impact page performance. We want to be able to display the data via an ASPX page with typical Next Page/Previous Page navigation. The user would have the ability to search/filter the data to limit their record set, but could still potentially have two to three pages (100-300 records) of results.
It is my understanding that the ListView object in .NET retrieves all of the records in your data source's SELECT statement and then the filtering/paging controls manipulate the results at the .NET server instead of at the database (until you issue a new databind()). This seems to be a potential performance issue in larger record sets that we are already seeing with just a few thousand records where the filter controls don't appear to filter until the entire underlying tables records are returned to .NET. If this is an incorrect assumption or understanding, please correct me.
Given the potential growth of our system, we're concerned about performance down the road. We're considering that we might be better off using a SELECT ... FROM ... where rownum>1 and rownum<=50 to work through our data pagination in our databind forcing the pagination to happen on the database side and executing on every page change, so that we're only working with a few hundred records at a time on the .NET side. Is this a false assumption regarding performance? Is there a performance issue with .NET performing the paging on large record sets?
My final question is if we're better off doing our pagination in the database rather than .NET, would we be better off storing our initial search results in a session specific temporary table and then using that as the basis of our paginated data rather than running the above query repeatedly against the master table as we move through the records?
Thank you in advance for everyone's input.
whats are methods to keeping datasource for a gridview.
situation :
I have a query which can take 5 to 10 seconds (a lot of link on a lot of data).
The result is too big for a page so I have a paging on my grid.
But, every time I use the pageIndexChanged, I need to get the datasource again.
So I want to know how I can keep my datasource.
Is it possible? NOT by session.
It is possible, I have used a custom (server-side) viewstate provider to cache your datasource.
*Updated: There is a pretty good article on custom viewstate providers (with sample code) here: http://www.codeproject.com/Articles/8001/ViewState-Provider-an-implementation-using-Provide
However, I would strongly recommend improving your query to limit the data to that which the user really wants to see. Large grids that have many more rows than the user is truly interested in are unwieldy from memory, processor, bandwidth, and user-experience perspectives. Try to find a better way.
I used bind all GridViews, DetailViews etc. on my page using an ObjectDataSource (unless it wasn't possible to do so). Recently, I've started binding all my contols programatically. I find this a lot cleaner and easier, though some may disagree.
Binding with a ObjectDataSource obviously has it advantages and disadvantages, as does doing it programatically.
Say I bind a GridView programatically (e.g. GridView1.DataSource = SomeList), when I change page on the GridView, I have to also code this. Each time the page changes I have to call GridView1.DataSource = SomeList again. Obviously with a ObjectDataSource I don't need to do this. I normally stick my SomeList object into the ViewState so when I change page I don't need to hit the database each and every time.
My question is: Is this how the ObjectDataSource works? Does it store it's data in the ViewState and not hit the database again unless you call the .Select method? I like to try and get the best performance out of my applications and hit the database as few times as possible but I don't really like the idea of storing a huge list in the ViewState. Is there a better way of doing this? Is caching per user a good idea (or possible)? Shall I just hit the database everytime instead of storing my huge list in the ViewState? Is it sometimes better to hit the database than to use ViewState?
Does it store it's data in the ViewState and not hit the database again unless you call the .Select method?
No its not save the data in ViewState. In the view state gridview and other similar lists, saves the General Status, eg the sort column, the page, the total pages, the control state, but not the Data.
Is caching per user a good idea
The caching per user on server side is not so good idea except if the caching is last for few minutes only or/and the data that you going to cache is very small. If you cache per user large amount of data for long time they going to grow too much especial if a user starts to read a lot of pages, that at the end you have the same problem.
Now you have to show a large amount of data that come from the relation of many tables, then maybe is better to cache the full relation of the tables to "one flat table".
Shall I just hit the database everytime instead of storing my huge list in the ViewState?
This is also depend from how fast you have design the reading of your data. For me is better to keep the ViewState small, and keep there only informations that you need to make the actions on your page, and not data.
I have a gridview with about 300-400 rows that I use for reporting; it needs paging and requires sorting. My choice is between these two options: a) load the 300-400 in one query and let the gridview do the paging and sorting; b) handle the paging/sorting at the data source level. I know that b) is going to be better/faster/more efficient... In my context, I'm looking to get something done relatively fast; if I choose a), will the page seem incredibly/painfully slow?
Thanks.
Use the builtin functions of the GridView and load the whole data at one go. It wouldn't be worth the effort to implement paging in database(f.e. with RowNumber) when your number of records is such small, especially because you mentioned that you are looking for a fast solution. When you enable paging in GridView the performance will suffice.
read Scott Guthrie's excellent article about paging.
How to implement a data-result search page built with ASP.NET 2.0 and Atlas that can present hundreds of thousands of row results using the ASP.NET 2.0 GridView control. The results are formatted using a “paging” based UI model – where 15 results per page are displayed, and the user can skip from page to page to see their data. For kicks I also added support for editing and deleting each individual row.
Although 300-400 rows isn't a particularly large set of data, I would certainly vote for option B since it will definitely be more scalable. Once you start getting massive sets of data, loading all of it with one query and having the GridView handle the paging will end up being incredibly slow. The better option is to, like you said, only query the data you need for each page.
However, if you aren't going to have data sets larger than 300-400 records, you probably don't need to be too concerned about load times, but again, the key is in scalability.
How much difference will there be if I bind data to a gridview in comparison to a loop through the data and build out the Html?
I am currently using an html table in the ItemTemplate of a gridview and have <%#Eval("ID")%> in that table to bind the data from iQueryable
What if i loop through the IQueryable and build out html from the code behind instead. How much is the performance difference if someone has done that comparison or has good knowledge of which should be the way to go ?
Thanks.
I am using Asp.net /C#
Generally speaking the performance benefit of avoiding complex controls and binding is not measurable on an individual page level, and thus inconsequential. The developer time saved in using existing controls and simpler api's, like data binding, greatly outweigh the small performance hit.
In our main application, we use complex controls and data binding throughout the ASP.NET page. The data binding portion of the full page life cycle is under 2% of the time to process the whole page. It's much less than the I/O for the page and particularly the DB calls.
One exception is in reports. The reporting engine we use supports directly setting data in a loop or using data binding. Data binding is much easier. However, with some reports hitting 200+ pages with over 300,000 bound data items, the performance hit of data binding was noticeable in this case. In our reports, we don't use data binding.