In my project I have a page with some RDLC graphs. They used to run on some stored procedures and an xsd. I would pass a string of the ID's of my results should include to restrict my data set. I had to change this because I started running into the 1000 character limit on object data set parameters.
I updated my graphs to run on a list of business objects instead and it seems that the page loads significantly slower than before. By significantly slower I mean page loads take about a minute now.
Does anybody know if object data sources are known to run slow when pulling business objects? If not is there a good way to track down what exactly is causing the issue? I put break points in my method that actually retrieves the business objects before and after it gets them; that method doesn't seem to be the cause of the slowdown.
I did some more testing and it seems that the dang thing just runs significantly slower when binding to business objects instead of a data table.
When I was binding my List< BusinessObject> to the ReportViewer the page took 1 minute 9 seconds to load.
When I had my business logic use the same function that returns the List and build a DataTable from the list with only the required columns for the report, then bound the DataTable to the report the page loads in 20 seconds.
Are you using select *? If so try selecting each field individually if you aren't using the entire table. That will help a bit.
#William: I experienced the same problem. I noticed though, when I flatten the business object, the reports runs significantly faster. You don't even have to map the business object to a new flattened one, you can simply set the nested object to null. I.e.:
foreach(var employee in employees)
{
employee.Department = null;
employee.Job = null;
}
It seems that the reportwriter does some weired things traversing the object graph.
This seems to be the case only in VS 2010. VS 2008 doesn't seem to suffer the same problems.
Related
I've just started a new job and inherited a nightmare WebForms .net 4.0 project that's going wrong in lots of places.
I've got a user control that is storing a DataTable in ViewState. The code is referencing the columns throughout the code by index number which is making it totally unreadable. I know I can reference columns by name to make it more readable but I'd prefer to break it out into a List.
The previous developer is storing the list in ViewState because changes can't be persisted back to the database, we read in data, sales people can make modifications to pricing and then the information is then pushed into XML format to generate a sales order form in PDF. So we need a mechanism for storing temporarily.
If I had the luxury of starting from scratch I'd pull the data down in JSON and do it all client side but I don't have that luxury just yet.
It's been a long, long time since I worked with DataTables and I'm
pretty sure it's not good to be placing this in ViewState, am I
correct? What kind of load would it be placing on the ViewState;
we're looking at 44 columns and normally around 25 rows. :s
Secondly, if there a big difference between placing a List in
ViewState as opposed to a DataTable or are they both as bad as each
other - in which case I'm doing no harm from my current position to
switch out?
Third question, updates to rows are automatically updated in the
ViewState DataTable, would that be the same with a List?
Last one, more probing for advice. Where would the best place to
store this information (preferably the List) server side - would
Session or Cache be better?
You are correct. It's a terrible idea to store the DataTable in ViewState.
Storing a List in ViewState is still bad and perhaps marginally better than storing a DataTable.
It's not clear what you mean by "updates occur automatically on the DataTable." Unless you retrieve the DataTable from ViewState and apply any updates programmatically, I doubt that this is the case. You'd have to do the same if you use a List.
Session is probably better but then you may run into scalability issues. Frankly, I would store it in Session but would opt for using one of the Out of Proc state providers. This way your serialized Datatable is not sent to the client on every request making your page size horrendously big while giving you the option to store this serialized data on different places, SQL server or ASP.NET State Server, for example.
I have a web application that talks to R using plr when doing adaptive testing.
I would need to find a way to store static data persistently between calls.
I have an expensive calculation creating an item bank than a lot of cheap ones getting the next item after each response submission. However currently I can't find a way to store the result of the expensive calculation persistently.
Putting it into the db seems to be a lot of overhead.
library(catR)
data(tcals)
itembank <- createItemBank(tcals) --this is the expensive call
nextItem(itembank, 0) # item 63 is selected
I tried to save and load the result, like this, but it doesn't seem to work, the result of the second NOTICE is 'itembank'.
save(itembank, file="pltrial.Rdata")
pg.thrownotice(itembank)
aaa=load("pltrial.Rdata")
pg.thrownotice(aaa)
I tried saving and loading the workspace as well, but didn't succeed with that either.
Any idea how to do this?
The load function directly loads objects into your workspace. You don't have to assign the return value (which is just the names of the objects loaded, as you discovered). If you do a ls() after loading, you should find your itembank object sitting there.
I am working on a large project where I have to present efficient way for a user to enter data into a form.
Three of the fields of that form require a value from a subset of a common data source (SQL Table). I used JQuery and JQuery UI to build an autocomplete, which posts to a generic HttpHandler.
Internally the handler uses Linq-to-sql to grab the data required from that specific table. The table has about 10 different columns, and the linq expression uses the SqlMethods.Like() to match the single search term on each of those 10 fields.
The problem is that that table contains some 20K rows. The autocomplete works flawlessly, accept the sheer volume of data introduces deleays, in the vicinity of 6 seconds or so (when debugging on my local machine) before it shows up.
The JqueryUI autocomplete has 0 delay, queries on the 3 key, and the result of the post is made in a Facebook style multi-row selectable options. (I almost had to rewrite the autocomplete plugin...).
So the problem is data vs. speed. Any thoughts on how to speed this up? The only two thoughts I had were to cache the data (How/Where?); or use straight up sql data reader for data access?
Any ideas would be greatly appreciated!
Thanks,
<bleepzter/>
I would look at only returning the first X number of rows using the .Take(10) linq method. That should translate into a sensbile sql call, which will put much less load on your database. As the user types they will find less and less matches, so they will only see that data they require.
I'm normally reckon 10 items is enough for the user to understand what is going on and still get to the data they need quickly (see the amazon.com search bar for an example).
Obviously if you can sort the data in a meaningful fashion then the 10 results will be much more likely to give the user what they are after quickly.
Returning the top N results is a good idea for sure. We found (querying a potential list of 270K) that returning the top 30 is a better bet for the user finding what they're looking for, but that COMPLETELY depends on the data you are querying.
Also, you REALLY should drop the delay to something sensible like 100-300 ms. When you set delay to ZERO, once you hit the 3-character trigger, effectively EVERY. SINGLE. KEY. STROKE. is sent as a new query to your server. This could easily have the unintended and unwelcome effect of slowing down the response even MORE.
I’m currently reading a book on website programming and author mentions that he will code DLL objects to use lazy load pattern. I think that conceptually I somewhat understand lazy load pattern, but I’m not sure if I understand its usefulness in the way author implemented it
BTW - Here I’m not asking for usefulness of lazy load patterns in general, but whether it is useful in the way this particular book implements it:
1) Anyways, when DLL object is created, a DB query is performed (via DAL), which retrieves data from various columns and with it populates the properties of our DLL object. Since one of the fields ( call it "L" ) may contain quite a substantial amount of text, author decided to retrieve that field only when that property is read for the first time.
A) In our situation, what exactly did we gain by applying lazy load pattern? Just less of memory usage?
B) But on the other hand, doesn’t the way author implemented lazy load pattern cause for CPU to do more work and thus longer time to complete, since if L is retrieved separately from other fields, then that will require of our application to make an additional call to Sql Server in order to retrieve "L", while without lazy load pattern only one call to Sql Server would be needed, since we would get all the fields at once?!
BTW – I realize that lazy load pattern may be extremely beneficial in situations where retrieving particular piece of data would require heavy computing, but that’s not the case in the above example
thanx
It makes sense if the DLL objects can be used without the L field (most of the time). If that is the case your program can work with the available data while waiting for L to load. If L is always needed then the pattern just increases complexity. I do not think it will significantly slow things down especially if loading L takes more time than anything else. But that is just a guess. Write both with lazy loading and without then see which is better.
I think that this is pretty useful when applied to the correct columns. For instance let's say that you have a table in your database, Customers, and in that table you have a column CustomerPublicProfile, which is a text column which can be pretty big.
If you have a screen (let's call it Customers.aspx) which displays a list of the customers (but not the CustomerPublicProfile column), then you should try and avoid populating that column.
For instance if your Customers.aspx page shows 50 customers at a time you shouldn't have to get the CustomerPublicProfilecolumn for each customer. If the user decides to drill down into a specific customer then you would go and get the CustomerPublicProfile column.
About B, yes this does make N extra calls, where N is the number of customers that the user decided to drill down into. But the advantage is that you saved a lot of extra un-needed overhead in skipping the column in the first place. Specifically you avoided getting M-N values of the CustomerPublicProfile column, where M is the number of customers that were retrieved on the Customers.aspx page to begin with.
If in your scenario M has a value close to N then it is not worth it. But in the situation I described M is usually much larger than N so it makes sense.
Sayed Ibrahim Hashimi
I had a situation like this recently where I was storing large binary objects in the database. I certainly didn't want these loading into the DLL object every time it was initialised, especially when the object was part of a collection. So there are cases when lazy loading a field would make sense. However, I don't think there is any general rule you can follow - you know your data and how it will be accessed. If you think it's more efficient to make one trip to the database and use a little more memory, then that is what you should do.
I've got an ASP.NET page that has a bunch of controls that need to be populated (e.g. dropdown lists).
I'd like to make a single trip to the db and bring back multiple recordsets instead of making a round-trip for each control.
I could bring back multiple tables in a DataSet, or I could bring back a DataReader and use '.NextResult' to put each result set into a custom business class.
Will I likely see a big enough performance advantage using the DataReader approach, or should I just use the DataSet approach?
Any examples of how you usually handle this would be appreciated.
If you have more than 1000 record to bring from your DataBase.
If you are not very interested with
custom storing and custom paging
"For GridView"
If your server have a memory stress.
If there is no problem to connect to
your DataBase every time that page
called.
Then i think the better is to use DataReader.
else
If you have less than 1000 record to bring from your DataBase.
If you are interested with
storing and paging "For
GridView"
If your server haven't a memory
stress.
If you want to connect to your
DataBase just one time and get the
benefits of Caching.
Then i think the better is to use DataSet.
I hop that i'm right.
Always put your data into classes defined for the specific usage. Don't pass DataSets or DataReaders around.
If your stored proc returns multiple sets, use the DataReader.NextResult to advance to the next chunk of data. This way you can get all your data, load it to your objects, and close the reader as soon as possible. This will be the fastest method to get your data.
Seeing that no answer has been marked yet even though there are plently of good answers already, I thought I'd add by two bits as well.
I'd use DataReaders, as they are quite a bit faster (if performance is your thing or you need as much as you can get). Most projects I've worked on have millions of records in each of the tables and performance is a concern.
Some people have said it's not a good idea to send DataReader across layers. I personally don't see this as a problem since a "DbDataReader" is not technically tied (or doesn't have to be) to a database. That is you can create an instance of a DbDataReader without the need for a database.
Why I do it is for the following reasons:
Frequently (in a Web Application) you are generating either Html, or Xml or JSON or some other transformation of your data. So why go from a DaraReader to some POCO object only to transform it back to XML or JSON and send it down the wire. This kind of process typically requires 3 transformations and a boot load of object instantiations only to throw them away almost instantly.
In some situations that's fine or can't be helped. My Data layer typically surfaces two methods for each stored procedure I have in the system. One returns a DbDataReader and the other returns a DataSet/DataTable. The methods that return a DataSet/DataTable call the method returning a DbDataReader and then uses either the "Load" method of the DataTable or an adapter to fill the dataset. Sometimes you need DataSets because you probably have to repurpose the data in some way or you need to fire another query and unnless you have MARS enabled you can't have a DbDataReader open and fire another query.
Now there are some issues with using DbDataReader or DataSet/DataTable and that is typically code clarity, compile time checking etc. You can use wrapper classes for your datareader and in fact you can use your DataReaders a IEnumerable with them. Really cool capability. So not only do you get strong typing and code readability you also get IEnumerable!
So a class might look like this.
public sealed class BlogItemDrw : BaseDbDataReaderWrapper
{
public Int64 ItemId { get { return (Int64)DbDataReader[0]; } }
public Int64 MemberId { get { return (Int64)DbDataReader[1]; } }
public String ItemTitle { get { return (String)DbDataReader[2]; } }
public String ItemDesc { get { if (DbDataReader[3] != DBNull.Value) return (String)DbDataReader[3]; else return default(String); } }
public DateTime ItemPubdate { get { return (DateTime)DbDataReader[4]; } }
public Int32 ItemCommentCnt { get { return (Int32)DbDataReader[5]; } }
public Boolean ItemAllowComment { get { return (Boolean)DbDataReader[6]; } }
public BlogItemDrw()
:base()
{
}
public BlogItemDrw(DbDataReader dbDataReader)
:base(dbDataReader)
{
}
}
DataReader Wrapper
I have a blog post (link above) that goes into a lot more detail and I'll be making a source code generator for these and other DataAccess layer code.
You can use the same technique for DataTables (the code generator produces the code) so you can treat them as strongly typed DataTable without the overhead of what VS.NET provides out of the box.
Keep in mind that there is only one instance of the wrapper class. So you're not creating hundreds of instances of a class only to throw it away.
Map the DataReader to intermediate objects and then bind your controls using those objects. It can be ok to use DataSets in certain circumstances, but those are few and far between when you have strong reasons for "just getting data". Whatever you do, don't pass a DataReader to your controls to bind off of (not that you said that you were considering that).
My personal preference would be to use an ORM, but if you are going to hand roll your data access, by all means I think you should prefer mapping DataReaders to objects over using DataSets. Using the .NextResult as a way to limit yourself from hitting the database multiple times is a double edged sword however so choose wisely. You will find yourself repeating yourself if you try to create procs that always grab exactly what you need using only one call to the database. If your application is only a few pages, it is probably OK, but things can get out of control quickly. Personally I'd rather have one proc per object type and then hit the database multiple times (once for each object type) in order to maximize maintainability. This is where an ORM shines because a good one will generate Sql that will get you exactly what you want with one call in most cases.
If you are not interested in updating or deleting the records you fetched from database, I would suggest using DataReader. Basically DataSet internally uses multiple Datareaders, so DataReader should give you good performance advantage.
In almost every situation DataReaders are the best solution for reading from a database. DataReaders are faster and require less memory than DataTables or DataSets.
Also, DataSets can often lead to situations in which the OO model is broken. It's not very object oriented to be passing around relational data/schemata instead of objects that know how to manipulate that data.
So, for extensibility, scalability, modularity, and performance reasons, always use DataReaders if you consider yourself a Real Programmer™ ;)
Check the links for facts and discussion about the two in practice and theory.
Irrespective of whether you're fetching a single result-set or multiple result-sets, the consensus seems to be to use a DataReader instead of a DataSet.
In regards to whether you should ever bother with multiple result-sets, the wisdom is that you shouldn't, but I can conceive of a reasonable class of exceptions to that rule: (tightly) related result-sets. You certainly don't want to add a query to return the same set of choices for a drop-down list that's repeated on hundreds or even dozens of pages in your app, but several sets narrowly-used may be sensibly combined. As an example, I'm currently creating a page to display several sets of 'discrepancies' for a batch of ETL data. None of those queries are likely to be used elsewhere, so it would be convenient to encapsulate them as a single 'get discrepancies' sproc. On the other hand, the better performance of doing so may be insignificant compared to working-around the natural one-sproc-one-result-set architecture of your ORM or hand-rolled data-access code.
I have gone to a method that uses DataReaders for all calls, I have noticed a marked performance impovement, especially in cases when I am loading drop down lists, and other simple items like that.
Personally with multiple drop downs, I typically go to pullling individual chunks of data to get it, rather than say a stored procedure that returns 5 result sets.
Take a look into the TableAdapters that are available with .NET 2.0 and up. What they do is give you the strength of a strongly-typed DataTable and allow you to map a Fill method to it that will use a DataReader to load it up. Your fill method can be existing stored procedures, your own AdHoc SQL, or even let the wizard generate the AdHod or Stored Procedure for you.
You can find this by starting up a new XSD DataSet object within your project. For tables that are used for more than just lookup, you can also map insert/update/delete methods to the TableAdapter.