I have a client who wants to return an ENORMOUS query from a web service, and will not budge. They are adamant that the query must be able to return absolutely everything, and they also will not allow us to do any sort of server side paging.
We have it working so that it returns all the data just fine, and we have it displayed in an ASP.NET GridView. In order to speed up the rendering of the huge grid, we decided to implement client-side paging, but I'm not exactly sure how to go about this.
Currently, we store the whole of the data in the ViewState, and then page and sort based off of that so as to avoid hitting the server again.
However, with the default set up, when trying to select any page but the first, it returns a 404 error. I'm guessing this is because it times out while trying to send the enormous data set via the ViewState.
Is there any sort of way to return the data set, and then do all of the paging and sorting all on the client without having to do any sort of post back? Or is there some other way to do this that we haven't thought of? (besides server side paging. We would LOVE to do it this way because it is so obviously the right way, but the client won't budge...)
EDIT: I would like to stick to the ASP.NET GridView control if at all possible. We've found a couple different options like jQuery and the like, but we have a lot of things to change if we change to a different type of control. So I'd like to avoid changing that if at all possible.
If the time is not a big problem for you the time it takes to get all the records from the database and I believe that on the client side the grid is typically renders as an HTML table than you can use a jquery plugin that will paginate the table on the client side.
http://www.codeproject.com/KB/webforms/clientside_gridviewpaging.aspx
We've run into situations where the Viewstate itself becomes a performance issue, and had to come up with ways to address a similar situation...
Your options depend greatly on some details which aren't included in the question.
For example? How often does the data from the webservice change? You may be able to do one of the following:
Save the results in Cache or Session (not a good idea if the results are that large...)
Serialize the result to an Xml file and read from that while paging
Import the data periodically into a SQL database, or into a temp table in a SQL database, with some identifier to tie the data to the user.
I've had good luck using the jquery tablesorter plugin. It gives some great options.
http://tablesorter.com/docs/
Related
I'm going to display a product detail from a database. But I can't decide whether to use an $.ajax post to a WebMethod that returns a JSON string, or a traditional Page.Load with sqldatareader in codebehind.
I know how to do both, coding is not the issue. I'm wondering what would be faster and more secure?
It depends on what kind of data you are showing on the page. (more of howmuch data)
Lets say if you are showing a fixed amount of data like, summary, product detail then its better to fetch data from server side and bind in label in Page Load event.
But if you are about to display the list of item (which may vary depending on the user input) for example product list, order list, employee list. in that case it is good to fetch the rows using ajax. reason is to display the fixed number of rows on the page (lets say 10) and there should be a pagination to jump to the next/pre page. now when user click next or previous it should not post back the page and should get the rows using ajax.
Each method has its advantages and disadvantages.
AJAX advantages and disadvantages
Advantages:
Forces you to separate the concerns in the code a bit more, you will have the data gathering and the data display in different places
The code will be more testable due to this additional modularity
Disadvantages:
Slower due to the additional HTTP request
Harder to deal with the back button and with bookmarking
Search engine optimization will be harder due to data not being right in the html
I'm populating a GridView with code, setting datasource to the dataset returned by a query. So apparently sorting and paging don't just work magically like if I use a datasourceid= some sqldatasource.
I've found a number of examples of how to do this on the web, but they all appear to re-execute the query every time. Shouldn't the contents of the query be saved in the view state? Is there some way to just get the previous query results and re-sort without having to go back to the database? (Is it saving all the data in the view state? If so, why can't I get to it? It seems pretty dumb to send it all to the user's browser and send it all back, wasting all the bandwidth, if there's no way to get to it.)
Also, if I try to allow paging, it appears I again have to re-execute the query every time the user goes to another page. And if the user sorts and then pages, then I have to remember what the sort order was in a hidden field or some such, so I can re-read the data, re-sort, and then go to the right page.
Given that when you use a data source control all this behavior is built in, I think I'm missing something here. But given all the examples out there that do it this slow, hard way, if I'm missing something, a lot of other programmers are missing it, too.
If you're using an ASP.NET GridView control every time you sort a column or page through the data set then you're making a server postback. Sorting and paging with this particular control has never worked 'magically' and has long been a bugbear of mine.
You can speed things up by storing the data source that you're building the grid from in memory, either as a session or through the ViewState. Both have pros and cons and I suggest you read up.
If at all possible I suggest forgetting the ASP.NET GridView and looking at a client side solution such as the jQuery jqGrid. It uses AJAX calls to sort and page and is much, much faster and less of a headache. The only drawback is the learning curve but believe me it's worth it in the long run.
Yes the gridview re-execute the query every time.
If the query takes too long, you can manually store data in the session, or ViewState. And in the Algorithm that populates the grid just read them directly for it, instead of running the query.
you can, in the page load event, run the query one time when there is no postback (you can check for postback with
if (!Page.IsPostBack){
//Run the query and save it to the session
}
and the the method that populate the grid, should read from the session directly. no need to run the query again
I'm working on an asp.net website with telerik controls. Im using multiple conditional grids (Show data based on selection in a grid.)
Every time I do new selection it is kinda slow (I'm using ajax call). Is it possible to preload all data to the client and then instantly show it to user.
I mean, is there any simple way of doing so?
There is a good chance that the slowness comes from the amount of data being rendered on your page. Keep in mind that AJAX still goes through the entire life-cycle of the page; the savings come from not having to render the entire page, just the updated parts.
Are your AJAX settings correctly updating the controls, or do you have a massive 'pnlAllControls' updating 'pnlAllControls'?
For example -- if you have Grid1, Grid2, Grid3; and Grid1 updates (Grid2, Grid3) while Grid2 updates only (Grid3), you should set your AJAX accordingly.
If your controls are getting data from the server, it does not make sense to cache the data on the client. I am not sure how much control you have over configuring them [controls].
You can store/cache data on server side instead (e.g. Cache, Session etc.). The data retrieval from there should be fast unless you send tons of data back and forth. But, caching data (on either client or server) should only be considered if the amount is 1)predictable and 2)relatively small.
Another technique to consider is paging/sorting on the server side. From my experience, you can get a real performance boost using just this.
There is no simple answer to your question. It all depends on data volume, security requirements. Moreover, your controls may not be able to pull data from the client-side.
I have a web app, that consumes a web service. The main page runs a search - by passing parameters to a particular web service method, and I bind the results to a gridview.
I have implemented sorting and paging on the grid. By putting the datatable that the grid is bound to in the viewstate and then reading / sorting / filtering it as necessary - and rebinding to the grid.
As the amount of data coming back from the web service has increased dramatically, when I try to page/sort etc I receive the following errors.
The connection was reset
The connection to the server was reset while the page was loading.
I have searched around a bit, and it seems that a very large viewstate is to blame for this.
But surely the only other option is to
Limit the results
Stick the datatable in the session rather than the viewstate
Something else I am unaware of
Previously I did have the datatable in the session, as some of this data needed to persist from page to page - (not being posted however so viewstate was not an option). As the amount of data rose and the necessity to persist it was removed, I used the viewstate instead. Thinking this was a better option than the session because of the amount of data the session would have to hold and the number of users using the app.
It appears maybe not.
I thought that when the viewstate got very big, that .net split it over more than one hidden viewstate field, but it seems all I'm getting is one mammoth viewstate that I have trouble viewing in the source.
Can anyone enlighten me as to how to avoid the error I'm getting? If it is indeed to do with the amount of data in the viewstate?
It sounds like your caching the whole dataset for all pages even though you are only presenting one page of that data. I would change your pagination to only require the data for the current page the user is on.
If the query is heavy and you don't want to have to be constantly calling it over and over because there is a lot of paging back and forth (you should test typical useage pattern) then I would implement some type of caching on the web service end to cache page by page (by specific user if the data is specific to a user) and have it expire rather quick (eg a few minuites).
I think you need to limit the total amount of data your dealing with. Change your code to not pass back extra data that might never be needed is a good place to start.
EDIT: Based on your comments:
You can't change the web service
The user can manipulate the query by filtering or sorting
There is a large amount of data returned by the web service
The data is user specific
Well I think you have a perfect case for using the Session then. This can be taxing the the server with large amounts of users and data so you might want to implement some logic to clear the data from the Session and not wait for it to expire (like on certain landing pages you know the user will go when they are done, clear the session data).
You really want to get it out of the ViewState beacuse it is a huge bandwidth hog. Just look at your physical page size and that data is being passed back and forth with every action. Moving it to the Session would eliminate that bandwidth useage and allow for you to do everything you need.
You could also look at the data the web service is bringing back and store it in a custom object that you make as 'thin' as possible. If your storing a DataSet or a DataTable in your Session, those objects have some extra overhead you probably don't need so store the data as an array of some custom thin object and just bind to that. You would need to map the result from the WS to your custom object but this is a good option you cut down on memory useage.
Let me know if there is something else I am missing.
I wouldn't put the data in either the view state or the session. Instead store the bare minimum information to re-request the dataset from the web service and store that (in either view state or session, or even on the URL). Then call the web service using that data and reaction the data on each request. If necessary, look to use some form of caching (memCache) to improve performance.
We use ViewState to store data that is being edited by users. However, ViewState just got too large so we really want something that is faster.
We considered session, but it has to be manually collected when users travel among pages.
Does anybody has suggestions?
Update:
Actually I am using Asp.net. They reason we do not want to use session is:
1. We don't need to carry our data among pages.
2. When developer put something into session, he has to remember to delete it if it is no longer useful. Otherwise, the session will get bigger and bigger. This is kind of trival.
You say you want the data stored server side and its should be automatically available?
You could trick the viewstate into storing its data in session rather than a hidden field by using this technique:
http://aspadvice.com/blogs/robertb/archive/2005/11/16/13835.aspx
You might find this article interesting as well which shows another technique to store your viewstate server side:
http://msdn.microsoft.com/en-us/magazine/cc188774.aspx#S6
Despite the initial complexity of getting this set up I think it would be the best solution because then you don't have to change your code throughout, it can still use ViewState as normal without realising this is now saved on the server.
You can look into Conversations.
http://evolutionarygoo.com/blog/?p=68
http://code.google.com/p/google-guice/issues/detail?id=5
You can use the database as a alternate to the session store. This would scale in terms of the size of the data stored, and if you use an appropriate caching strategy you can reduce the overhead of retrieving the data a great deal.