I'm writing an internal web application that is used to look up credit checks. I'm using ASP.NET for this website.
I have a SQL Query that I have to query the credit check database and it returns a datatable of results. After the person clicks search, I run the query and bind the returned datatable to the Gridview every time the user clicks the search button.
This works fine, and it populates my gridview like it should.
How do I maintain the data that is bound to my gridview beyond postback? For example, lets say a person clicks Page 2 of the paginated gridview... how do I keep from losing the data of the gridview during that postback?
Currently I am storing the datatable in session[] but I don't think I should be doing this, because the size of this table is very large (sometimes hundreds of thousands of results!)
Do I need to re-query the database for each postback? The query takes a decent bit and I'd rather not if I could help it.
What is the common solution here?
If you are storing the data in the session I would make sure viewstate is turned off and you are rebinding it on each postback so at least your not storing all that data in 2 places.
You could also considered leaving the viewstate on and changing logic that grabs the session data to bring back one page of data at a time? You would take in the current page and JUST return that page's data so that you could get all the data at first and store it in the session. Then if any other postback actions occurred during that page view where nothing was really going on with the grid you wouldn't have to rebind each time.
I have found that most of the issues I have run across in the past due to too much data being displayed in a grid were more design issues and I would step back and look at the overall problem that was trying to be solved and implement another solution to not have to display so much data. Usually a customer/user asks for everything because they don't consider any alternatives or the impact of their request.
Several things:
Clicking on "Page 2" should not result in a postback - it should be a simple GET request.
Do not allow GridView to store its data in the ViewState: cache data manually and "rebind" datasource on each request
Use ASP.NET MVC
Related
I'm populating a GridView with code, setting datasource to the dataset returned by a query. So apparently sorting and paging don't just work magically like if I use a datasourceid= some sqldatasource.
I've found a number of examples of how to do this on the web, but they all appear to re-execute the query every time. Shouldn't the contents of the query be saved in the view state? Is there some way to just get the previous query results and re-sort without having to go back to the database? (Is it saving all the data in the view state? If so, why can't I get to it? It seems pretty dumb to send it all to the user's browser and send it all back, wasting all the bandwidth, if there's no way to get to it.)
Also, if I try to allow paging, it appears I again have to re-execute the query every time the user goes to another page. And if the user sorts and then pages, then I have to remember what the sort order was in a hidden field or some such, so I can re-read the data, re-sort, and then go to the right page.
Given that when you use a data source control all this behavior is built in, I think I'm missing something here. But given all the examples out there that do it this slow, hard way, if I'm missing something, a lot of other programmers are missing it, too.
If you're using an ASP.NET GridView control every time you sort a column or page through the data set then you're making a server postback. Sorting and paging with this particular control has never worked 'magically' and has long been a bugbear of mine.
You can speed things up by storing the data source that you're building the grid from in memory, either as a session or through the ViewState. Both have pros and cons and I suggest you read up.
If at all possible I suggest forgetting the ASP.NET GridView and looking at a client side solution such as the jQuery jqGrid. It uses AJAX calls to sort and page and is much, much faster and less of a headache. The only drawback is the learning curve but believe me it's worth it in the long run.
Yes the gridview re-execute the query every time.
If the query takes too long, you can manually store data in the session, or ViewState. And in the Algorithm that populates the grid just read them directly for it, instead of running the query.
you can, in the page load event, run the query one time when there is no postback (you can check for postback with
if (!Page.IsPostBack){
//Run the query and save it to the session
}
and the the method that populate the grid, should read from the session directly. no need to run the query again
I would like to fetch a number of records from a database to fill a DropDownList that is filled in a OnItemDataBound of a DataList.
I think it might be a bit much to fetch all the records every time a DropDownList is bound since they are the same.
How can this be done?
If I understand your question correctly, you could put your drop down list into a user control, and then output cache it, which would mean the resulting drop down list would not keep making new requests to the database each time it is loaded.
MSDN has quite a good article on output caching:
http://msdn.microsoft.com/en-us/library/hdxfb6cy.aspx
I used bind all GridViews, DetailViews etc. on my page using an ObjectDataSource (unless it wasn't possible to do so). Recently, I've started binding all my contols programatically. I find this a lot cleaner and easier, though some may disagree.
Binding with a ObjectDataSource obviously has it advantages and disadvantages, as does doing it programatically.
Say I bind a GridView programatically (e.g. GridView1.DataSource = SomeList), when I change page on the GridView, I have to also code this. Each time the page changes I have to call GridView1.DataSource = SomeList again. Obviously with a ObjectDataSource I don't need to do this. I normally stick my SomeList object into the ViewState so when I change page I don't need to hit the database each and every time.
My question is: Is this how the ObjectDataSource works? Does it store it's data in the ViewState and not hit the database again unless you call the .Select method? I like to try and get the best performance out of my applications and hit the database as few times as possible but I don't really like the idea of storing a huge list in the ViewState. Is there a better way of doing this? Is caching per user a good idea (or possible)? Shall I just hit the database everytime instead of storing my huge list in the ViewState? Is it sometimes better to hit the database than to use ViewState?
Does it store it's data in the ViewState and not hit the database again unless you call the .Select method?
No its not save the data in ViewState. In the view state gridview and other similar lists, saves the General Status, eg the sort column, the page, the total pages, the control state, but not the Data.
Is caching per user a good idea
The caching per user on server side is not so good idea except if the caching is last for few minutes only or/and the data that you going to cache is very small. If you cache per user large amount of data for long time they going to grow too much especial if a user starts to read a lot of pages, that at the end you have the same problem.
Now you have to show a large amount of data that come from the relation of many tables, then maybe is better to cache the full relation of the tables to "one flat table".
Shall I just hit the database everytime instead of storing my huge list in the ViewState?
This is also depend from how fast you have design the reading of your data. For me is better to keep the ViewState small, and keep there only informations that you need to make the actions on your page, and not data.
how does it work when i have a data grid that i want page able and sortable, do i use view state? does the data get pulled once and then the view state keeps all the data and grid working, sortable, page able?
or do i need to go to the db server each time i page and sort?
help clarify this point for me please.
If you use the built in paging without a SqlDataSource object, your data will be paged in memory, which can be a massive overhead.
ViewState will by default retain the information in the grid, however the data item for each row won't be available after a postback.
If you are dealing with large volumes of data, then paging and sorting should be done on the database so as littel data as possible is transferred to the web app.
I have a web app, that consumes a web service. The main page runs a search - by passing parameters to a particular web service method, and I bind the results to a gridview.
I have implemented sorting and paging on the grid. By putting the datatable that the grid is bound to in the viewstate and then reading / sorting / filtering it as necessary - and rebinding to the grid.
As the amount of data coming back from the web service has increased dramatically, when I try to page/sort etc I receive the following errors.
The connection was reset
The connection to the server was reset while the page was loading.
I have searched around a bit, and it seems that a very large viewstate is to blame for this.
But surely the only other option is to
Limit the results
Stick the datatable in the session rather than the viewstate
Something else I am unaware of
Previously I did have the datatable in the session, as some of this data needed to persist from page to page - (not being posted however so viewstate was not an option). As the amount of data rose and the necessity to persist it was removed, I used the viewstate instead. Thinking this was a better option than the session because of the amount of data the session would have to hold and the number of users using the app.
It appears maybe not.
I thought that when the viewstate got very big, that .net split it over more than one hidden viewstate field, but it seems all I'm getting is one mammoth viewstate that I have trouble viewing in the source.
Can anyone enlighten me as to how to avoid the error I'm getting? If it is indeed to do with the amount of data in the viewstate?
It sounds like your caching the whole dataset for all pages even though you are only presenting one page of that data. I would change your pagination to only require the data for the current page the user is on.
If the query is heavy and you don't want to have to be constantly calling it over and over because there is a lot of paging back and forth (you should test typical useage pattern) then I would implement some type of caching on the web service end to cache page by page (by specific user if the data is specific to a user) and have it expire rather quick (eg a few minuites).
I think you need to limit the total amount of data your dealing with. Change your code to not pass back extra data that might never be needed is a good place to start.
EDIT: Based on your comments:
You can't change the web service
The user can manipulate the query by filtering or sorting
There is a large amount of data returned by the web service
The data is user specific
Well I think you have a perfect case for using the Session then. This can be taxing the the server with large amounts of users and data so you might want to implement some logic to clear the data from the Session and not wait for it to expire (like on certain landing pages you know the user will go when they are done, clear the session data).
You really want to get it out of the ViewState beacuse it is a huge bandwidth hog. Just look at your physical page size and that data is being passed back and forth with every action. Moving it to the Session would eliminate that bandwidth useage and allow for you to do everything you need.
You could also look at the data the web service is bringing back and store it in a custom object that you make as 'thin' as possible. If your storing a DataSet or a DataTable in your Session, those objects have some extra overhead you probably don't need so store the data as an array of some custom thin object and just bind to that. You would need to map the result from the WS to your custom object but this is a good option you cut down on memory useage.
Let me know if there is something else I am missing.
I wouldn't put the data in either the view state or the session. Instead store the bare minimum information to re-request the dataset from the web service and store that (in either view state or session, or even on the URL). Then call the web service using that data and reaction the data on each request. If necessary, look to use some form of caching (memCache) to improve performance.