I have an application that holds data referencing 300,000 customers. When a user did a search the result was often bigger than our MaxRequestlength would allow, we have dealt with this in two ways: We have increased our MaxRequestLength to 102400 (KB) and required the user to supply two letters of the first Name and two letters of the last name, to limit the sheer # of customer records returned. This keeps us from exceeding the MaxRequestLength limit.
I was just wondering if anyone had any insight in to whether this was a particularly good approach, whether there is a limit to how big MaxRequestLength could be or should be, and what other options might be useful in this situation.
Most web applications I have seen deal with this by returning a paginated list, and displaying only the first page of results.
In modern implementations using ORM's, "Skip" and "Take" operators are used to retrieve only those records which are required for a given page.
So any given request is no longer than the number of records on one page.
I would recommend paging the results instead of displaying everything. I would also suggest adding multiple search fields allowing your users to filter their results even further. This will allow your user to find what they are looking for faster.
As you can guess from my comment, I think MaxRequestLength only restricts the size of the request (-> the amount of data sent from the client/browser to the server).
If you are exceeding this limit, then this probably means that you have a huge ViewState which is sent with every response. ViewState is stored in a hidden field on the page and is sent back to the server with every PostBack (and that's where the MaxRequestLength setting could come into play). You can easily check this by looking at the source of your page in the web browser and looking for a hidden INPUT element with the name "__VIEWSTATE" and a large string-value.
If this is the case, the you should try to reduce the size of the ViewState, e.g. by
setting ViewState="false" on your controls (GridView or whatever) and re-binding the control on every PostBack (this is the recommended approach)
storing the ViewState on the server side
compressing the ViewState
If your requirements allow it, I would suggest implementing server-side paging. That way you only send one page worth of records over the wire rather than the entire record set.
300,000 records is a completely unusable result set from a human perspective.
As others have said, page the results to something like the top 50 or 100 records. Let them sort it and provide a way to narrow the search criteria.
For perspective, look at google. They default to 10 records per page. Part of the reason for this is that people would rather provide more criteria than go spelunking through a large result set.
Related
I want to use viewstate variable's value which is saved in one page on another page. But while doing so it shows NullReferenceException. I am new to ASP.net. Please Help me out.
in register.aspx
ViewState("name")=textbox1.text
in success.aspx
dim a as string
a=ViewState("name").toString
Use Session["name"] = textbox1.text...
If I am not mistaken, you don't have direct control or should have direct control over viewstate.
ViewState is the technique to persist state across postbacks and is lost when you load another page. therefore you will need to use another way to send the data to the next page. Read up more here on MSDN
The common choices for sending data across pages include:
QueryString
By setting up the previous page property
Session variables
ViewState is restricted to a page, so it cannot be used on another page. The reason is that ViewState is serialized in the page output in a hidden field that is transferred to the client and back to the server upon a postback.
If you want to transfer data to another pages you have several other alternatives:
As #AnastacioGianareas also pointed out, Session memory is one of them. It is located on the server, but sessions will expire after a certain time of user inactivity. Being located on the server, it reduces application scalability if you have many users and lots of memory allocated in the session.
Hand over the data as a query string parameter to the other page, e.g. redirecting to "Success.aspx?name=". This will work for smaller amounts of data that you can put into the query string (e.g. ids or names). It is important that a client can also request Success.aspx with that query string parameter, so it should be reserved for uncritical data and be validated carefully.
Use a cookie to transfer it to the client and back again.
This link gives a good overview of the alternatives.
From MSDN:
View state is the method that the ASP.NET page framework uses by
default to preserve page and control values between round trips. When
the HTML for the page is rendered, the current state of the page and
values that need to be retained during postback are serialized into
base64-encoded strings and output in the view state hidden field or
fields.
Options to persist data between pages are varied and depend upon your requirements, this page on MSDN describes each of these options and the considerations you should make.
For your requirement either QueryStrings, SessionState seem like the best solution.
As a side note, always validate your 'State' variables (regardless of which method you chose) to ensure that they aren't null and are of the expected type. i.e. all these values will be stored as Strings, if you are going to use a variable as another type (an int, double etc) you should make sure this is valid. Also, should you go down the querystring route, take into account any security considerations as users may / will modify these values.
I have a this asp.net page which upon first time load:
1: Make a DB call and get data - XML string (this chunk can go beyond 100kb). And this DB call is a bit expensive takes about 5-10 secs.
2: I loop through this XML and create a Custom Collection with values from XML. Then Bind it to a Repeater Control.
Now the repeater control has one text input. User is free to enter values in one or more or all TBs or leave all blank. Then then hit Save button.
On Save Postback, I will have to loop through all rows in the Repeater, Collect all the rows that has some input in the and generate an XML using the initial values and this new input value and Save it to DB.
Problem:
So I will need reference to all the initial XML values. I can think of these options and looking for inputs on selecting a best one.
1: ViewState: Store my Collection or XML string in ViewState - I'm sure it is will be too huge
2: Session: Use Session to store Collection of xml string - Again
3: DB Call: Make a DB call to get the data again - as I said it is kind of expensive call and my DBA is asking me to avoid this
4: HiddenField: Store the essential data from XML in to HiddenField and use that for Save manipulation. i.e. in each repeater item find all the hiddenfields
Which one is best in terms of better request response and less resource utilization on server?
Or is there a better way I am missing?
PS: Have to use ASP.NET 2.0 WebForms only.
Update1:
I tried the following with ViewState:
1: Store entire xml string: ViewState length = 97484 and FireBug shows pagesize - 162Kb
2:Store stripped down version of Collection with required only data: ViewState length = 27372 and FireBug shows pagesize - 94Kb and with gzip compression it reduces to 13kb.
With the existing website FireBug shows Size 236Kb.
So definitely option 2 is better and my new version is better then current website.
So any inputs?
A quick question - who is your target audience for this page? If it's an internal website for a company then just storing the data in viewstate might be acceptable. If it's for external people, e.g. potential customers, then speed and performance probably matter to you more.
Viewstate - have you tried adding your XML to viewstate? How much did it increase the page size by? If you're gzipping all of your pages rather than sending them over the wire uncompressed then you could see about a 70% reduction in size - 30kb isn't that much these days...
Session - it is worth remembering that the server can and will drop data from sessions if it runs out of space. They can also expire. Do you trust your users not to log in again in a new tab and then submit the page that they've had open for the last 10 hours? While using session results in less data on the wire you might need to re-pull the data from the db if the value does end up being dropped for whatever reason. Also, if you're in a web farm environment etc there are complications involving synchronizing sessions across servers.
DB Call - can the query be optimised in any way? Are the indices on all the fields that need them? Maybe you and your DBA can make it less painful to pull. But then again, if the data can change between you pulling it the first time and the user submitting their changes then you wouldn't want to re-pull it, I suspect.
Hidden Fields - With these you'd be saving less data than if you put the whole string in Viewstate. The page wouldn't be depending on the state of the webserver like with session and nor would you be racing against other users changing the state of the db.
On the whole, I think 4 is probably the best bet if you can afford to slow your pages down a little. Use Firebug/YSlow and compare how much data is transmitted before and after implementing 4.
One final thought - how are things like this persisted between postbacks in the rest of your webapp? Assuming that you haven't written the whole thing on your own/only just started it you might be able to find some clues as to how other developers in a similar situation solved the problem.
Edit:
there is a load-balancer, not sure how it will play with Session
If you have a load balancer then you need to make sure that session state is stored in a state server or similar and not in the process ("inproc"). If the session is stored on the webserver then option 2 will play very badly with the load balancer.
While I'm not a huge fan of overusing session, this will probably be your best bet as it will be your fastest option from the user's standpoint.
Since session state does have it's own inherit issues, you could load the data you need into session, and if your session drops for whatever reason, just do another database hit and reload it.
I would really stay away from options 1 and 4 just because of the amount of unnecessary data you will be sending to the client, and potentially slowing down their experience.
Option 3 will also slow down the user experience, so I would stay away from that if at all possible unless you can speed up your query time.
I have a web app, that consumes a web service. The main page runs a search - by passing parameters to a particular web service method, and I bind the results to a gridview.
I have implemented sorting and paging on the grid. By putting the datatable that the grid is bound to in the viewstate and then reading / sorting / filtering it as necessary - and rebinding to the grid.
As the amount of data coming back from the web service has increased dramatically, when I try to page/sort etc I receive the following errors.
The connection was reset
The connection to the server was reset while the page was loading.
I have searched around a bit, and it seems that a very large viewstate is to blame for this.
But surely the only other option is to
Limit the results
Stick the datatable in the session rather than the viewstate
Something else I am unaware of
Previously I did have the datatable in the session, as some of this data needed to persist from page to page - (not being posted however so viewstate was not an option). As the amount of data rose and the necessity to persist it was removed, I used the viewstate instead. Thinking this was a better option than the session because of the amount of data the session would have to hold and the number of users using the app.
It appears maybe not.
I thought that when the viewstate got very big, that .net split it over more than one hidden viewstate field, but it seems all I'm getting is one mammoth viewstate that I have trouble viewing in the source.
Can anyone enlighten me as to how to avoid the error I'm getting? If it is indeed to do with the amount of data in the viewstate?
It sounds like your caching the whole dataset for all pages even though you are only presenting one page of that data. I would change your pagination to only require the data for the current page the user is on.
If the query is heavy and you don't want to have to be constantly calling it over and over because there is a lot of paging back and forth (you should test typical useage pattern) then I would implement some type of caching on the web service end to cache page by page (by specific user if the data is specific to a user) and have it expire rather quick (eg a few minuites).
I think you need to limit the total amount of data your dealing with. Change your code to not pass back extra data that might never be needed is a good place to start.
EDIT: Based on your comments:
You can't change the web service
The user can manipulate the query by filtering or sorting
There is a large amount of data returned by the web service
The data is user specific
Well I think you have a perfect case for using the Session then. This can be taxing the the server with large amounts of users and data so you might want to implement some logic to clear the data from the Session and not wait for it to expire (like on certain landing pages you know the user will go when they are done, clear the session data).
You really want to get it out of the ViewState beacuse it is a huge bandwidth hog. Just look at your physical page size and that data is being passed back and forth with every action. Moving it to the Session would eliminate that bandwidth useage and allow for you to do everything you need.
You could also look at the data the web service is bringing back and store it in a custom object that you make as 'thin' as possible. If your storing a DataSet or a DataTable in your Session, those objects have some extra overhead you probably don't need so store the data as an array of some custom thin object and just bind to that. You would need to map the result from the WS to your custom object but this is a good option you cut down on memory useage.
Let me know if there is something else I am missing.
I wouldn't put the data in either the view state or the session. Instead store the bare minimum information to re-request the dataset from the web service and store that (in either view state or session, or even on the URL). Then call the web service using that data and reaction the data on each request. If necessary, look to use some form of caching (memCache) to improve performance.
How would I store the value of a GridView's selected rows to a session variable?
From the codebehind file you will want to use something like this to access the underlying data item (MyDataItem) from the selected row.
MyDataItem item = (MyDataItem)GridView1.Rows[GridView1.SelectedIndex].DataItem;
Session["myItem"] = item;
Remember though, the gridview is already storing this data for you, so you may just want to access it from the GridView directly whenever you need it.
On a side note: can I stronly advise you NOT to use the session state.
Unless you are using it as a store where data is cached for the current user, which you can retrieve back at any time from e.g. a database.
If not, the "session" will come back and bite you. At some point there will be a user that leaves the browser open for longer time than your session lives (e.g. they get a telephone call, go out to lunch in a hurry, rush of to a meeting...). And when they return, they wish to complete what they are doing. And if you cannot restore all of your session data back at that point, you will either have to redirect your user to start over again (very annoying for your users), or you will have lost some information (very embarrasing), or the worst case, and most common case: your application will no longer work and crash (just plain: very bad).
It is a better approach to define small serializable objects that store your state (query parameters, selected items, etc) and use ASP.NET Viewstate to store that state across page requests. Note that most ASP.NET controls already use the viewstate to store their data. Then disable the Viewstate of your grids in the page, to vastly reduce the amount of data in your viewstate, and request the data upon each request (here it is safe to use the session or ASP.NET cache to improve performance of your application). You will have a much more robust and much more scalable application.
It is more work, but it will pay back very fast, and many times over.
ViewState only scope within one page. It's useful for postback problem, but not for cross-page problems. Session can handle both, but it has some limitations of security, lifetime, transmittion time... Depend on particular sittuation you can pick your right choice.
I have a dataset that contains around 500 records with 7 columns in each record. I would like to use objectdatasource and caching. I am sure that the data doesn't change. But there will be lots of users accessing the data. My question is whether it would be a good idea to cache the data that has 500 records. is it optimal or not? I think the objectdatasource caches the data per user. so I am wondering whether the data with this size would have performance issues if I cache the data.
Thanks,
sridhar.
The answer really depends on the size of the records, not just the number of them. In any event, ASP.NET's caching algorithm uses a Least Recently Used (LRU) algorithm to evict out items that have not been used recently to make room for additional items. Thus, unless your site is either extremely busy or your result set extremely large, you should be fine to cache the result set and let ASP.NET's cache handle the details of keeping the most used data in the cache and expiring the least used data.
You might also consider using Page level Output Caching if nothing else on the page is user-specific, as this would likely give you slightly more performance for minimal additional memory usage. Other areas to investigate are the control, SqlDependencies, and of course if the data is read-only, be sure you are disabling ViewState on whatever control you're using to display the data.