In looking up how to perform an equivalent to SELECT TOP 5 with LINQ-to-SQL, all the answers I've seen suggest using .Take(), like so:
var myObject = (
from myObjects in repository.GetAllMyObjects()
select myObject)
.Take(10);
I don't yet understand most of how LINQ works behind-the-scenes but to my understanding of C-like languages this would resolve by first assigning a temporary array containing ALL records, then copying the first 10 elements in the array to var. Not such a problem if you're working on a small dataset or without any performance constraints but it seems horribly inefficient to me if you're, for example, selecting the most recent 5 log entries from a table which can contain millions of records.
Is my understanding of how this works wrong? If so, could someone explain what actually happens? Otherwise, what (if any) better (ie more efficient) way is there of only selecting x records through LINQ-to-SQL?
[edit]
I have the hypothetical myObject class sending LINQ-to-SQL output to the debug output as per the suggestion in the accepted answer. I ended up using the DebuggerWriter from here: http://www.u2u.info/Blogs/Kris/Lists/Posts/Post.aspx?ID=11
Your assumption is incorrect. With Linq to SQL, it evaluates to an Expression<Func<...>> which can be evaluated and the proper SQL generated. You do not need to worry about it loading all the records.
Also, see this following question. You can attach a stream to your DataContext and see the SQL generated.
How to get the TSQL Query from LINQ DataContext.SubmitChanges()
LINQ uses deferred execution, and, for LINQ-to-SQL, expression trees.
No query will be executed until you enumerate the result of the Take call, so you don't need to worry about anything.
I just went through this last week! I opened the SQL profiler on my dev data base and stepped through the code. It was very interesting to see the generated SQL for the various queries. I recommend you do the same. It may not be an exact answer to your question but it was certainly enlightening to see how your various components generate entirely different SQL statements depending on the contents of the call.
I believe the "deferred query resolution" or something (?) reading on MSDN would be enlightening as well.
Related
I got a complex data structure like this:
leve1Model->1:level2Model->n:level3Model->1:value1
->1:level4Model->1:value2
What I need is to find all level1Model instances which's level2Model instance contains one level3Model instance which has a certain value1 and which's level4Model instance also has a certain value2.
I am relatively new to Extbase and it's Database abstraction, but I figured I could do something like:
$query->contains('level2Model.level3Model.value1', val1),
but this gives me an error about an unsupported property value1. But even if it worked I don't know how I would make sure that the contained instance also meets the required level4Model->1:value2condition.
So is there a way to properly use the abstraction layer for this kind of query (and how would I go about that?) or do I have to resort to plain sql?
The query builder will not help you much here. If the second level model is stored within an object storage you could query against the uids of these objects. But that's as far as the query builder will take you. Your query is too complex. You would have to split the query into multiple parts and re-filter the results but the performance might be bad. I would suggest using the helper functions for sanitation etc. but passing it the finished sql statement parts (fields, from, where etc. ). Alternatively you could rethink the data structure of this is an essential part of your app to make it more extbase compliant.
I am trying to get the CostCenter, MainAccount and Department starting from the LedgerDimension field in the LedgerJournalTrans table.
I found this but I am lost.
http://ax2009developer.blogspot.ro/2014/02/how-create-customize-look-up-for.html
In fact, for this task, I have implemented only queries in AOT. Is there any way to join some tables and get there without taking the X++ approach?
Financial dimensions in AX 2012 are far more complicated than in previous versions.
You should start with this white paper: http://download.microsoft.com/download/4/E/3/4E36B655-568E-4D4A-B161-152B28BAAF30/Implementing_the_Account_and_Financial_Dimensions_Framework_AX2012.pdf
You'll find the tables involved and their relations.
By the way, I recommend you not to build your own queries. As the model is really versatile, it will be first tricky to build your query, then they will not be performant.
You need to use the APIs as they are already built and also as they use the system global object cache to cache data, as the model is not set for fast queries.
Unfortunately, I don't believe there is an easy way to do what you want with queries only and X++ is the way to go.
You could, in theory, create a view that you would use in your query objects. It would have tables DimensionAttribute, DimensionAttributeValueSet DimensionAttributeValueSetItem, and DimensionAttributeValue I think. And multiple instances of each in some cases.
Then in your view, you'd set ranges with your different Attribute names. This is fairly complex, but you could repeatedly use it on any query. I could see value in it for sure, but if you've not worked much with dimensions, you have some learning to do to get that working.
Simple SQLite SELECT query on Windows Phone is very slow on a high-end device (Lumia 930).
select * from tableName
It's fetching around 15000 records (yeah, I need them all) and normally I'd expect it to not be this slow. However, it takes around 12-13 seconds to get all the records. I'm using SQLite.Net-PCL client.
What could be causing it? Is it true that it's due to the very slow wrapper? Is there a workaround, any way to improve it?
EDIT: I tried using SQLite PCL from Microsoft Open Technologies and I manually mapped property by property and I got much better results. So it seems that the count of rows, count of columns and the reflection, all combined, cause things to slow down. I am now working on trying to expose a similar functionality through SQLite.NET-PCL, the library which I'm using, to see how that would go.
EDIT2: I marked Peter's answer as answer to my question as I was able to improve performance dramatically by manually mapping type by type using Prepare call and stepping through row by row.
SQLite can easily return 15,000 records from a simple table in a fraction of a second on a Windows Phone (tested on a Lumia 920).
There are other things causing your poor performance. If you have a huge number of columns, that might be a problem. Depending on how the SQLite wrapper is implemented (I don't know), two possible culprits are use of Reflection to fill your result objects or per-row Async overhead. But again, I don't know how that wrapper is implemented specifically.
The way to speed it up (other than to return less data) is to write your code in C++ and wrap it in a WinRT component to be called by your managed app.
Depends on what information you want from your entities you may try using Query<>() method that alows you to write raw SQL query and then you can select only fields that you are interested in and map it to lighter entities if that is possible, even if you getting all fields for your entitiy Query<>() still should be quicker.Also check if you are using lates SQLite driver for WP
This is entity framework:
var department = _context.Departments
.Include(dep => dep.Employees.Select(emp => emp.ContactTypes))
.SingleOrDefault(d => d.Id == departmentId);
Here I expect one department to be returned containing all related employees and all contact types for each employee.
This is ormlite servicestack:
I have no idea. When I look at the docu/samples: https://github.com/ServiceStack/ServiceStack.OrmLite
They write:
Right now the Expression support can satisfy most simple queries with a strong-typed API. For anything more complex (e.g. queries with table joins) you can still easily fall back to raw SQL queries as seen below.
I have seen there is a JoinSqlBuilder class but I do not think it can return nested collections.
Maybe what I want is not possible but maybe I can do a compromise like get all employees for the departmentId. Then I inmemory foreach the employees and fetch all contact types for a certain employeeId. Creating the hierarchy and assigning the lists would still be my job.
But I hope there is a shorter solution.
What would also be fine is when the query however it might look like return an object (Dynamic?) with 3 flat properties: Department, Employees, ContactTypes and assign thoese properties to my DTO.
Ok, please don't take this as a definitive answer, but more just my take on the situation (I don't use service stack very much) however...
When I first started to use EF many years ago, I came across a similar situation, where the references just would not load. Like you I was faced with the likely hood of having to enumerate the individual collections myself and write a lot of extra code for an operation the ORM should be able to handle easily.
What I ended up doing, was to use auto-mapper, which basically reduce all the multiline loops I had everywhere to a single line mapping statement.
Granted, I still had to do one mapping statement for each linked property, but it reduced the code I had to write, and more importantly got me up and running until EF improved, or I found a better way of doing things.
Let me stress, I'm not proposing this as an answer, and it's a bit big for a comment, I'm simply suggesting shifting your thought in a different direction, that may help a better solution come to the surface.
I'm a former classic ASP programmer and sometimes PHP programmer writing my first ASP.NET application. I'm loving the much-improved ADO.NET functions, but I'm feeling the need to write a database class. Partly I want to consolidate the code that actually interacts with the database, and partly I want to reduce what feels like repetition.
A simple database class is not hard to do, but I'm using parameterized queries as one of my security measures. I'm struggling with how to incorporate this into a class. I wrote a function to return the datatype of a column in the database by passing in the table and column name, but I can't think of a robust way to obtain the table and column name from the SQL query.
My design for the class was to have a Query() function for selecting, and an Execute() function for insert/update/delete. (Not opposed to having more public functions, but didn't want to get ahead of myself.) Both functions take a SQL string and a SortedList for the parameters. It might be possible to get the column name by finding the parameter name in the SQL string and looking in front of the equal sign. Likewise, it should be fairly simple to get the table name when the query is insert, update, or delete, because you only work with one table at a time. The big concern is selecting, because there could be one or more joins, inner selects, etc.
Am I headed in the wrong direction? Anything I'm not thinking of that could make my life easier or more difficult? Anybody written a class for this in any language that could offer some advice?
Don't reinvent the wheel. Look into nHibernate or LINQToSQL (or LINQToEntities) for your ORM needs.
Would second the call to find a tried and tested wheel that works for you, especially if this is your first foray into aspnet... there will be plenty else to keep you busy.
Would add a suggestion for SubSonic, which is perhaps a little lighter than nHibernate, but it really depends on the nature of your project, they are both great tools, and both have saved me months of work over the last few years.
I think since this is your first experience in ASP.NET you would be well advised to look into Linq to SQL. Do some tutorials so you get a feel for how it works before you try to code any Linq queries.
The only reasons I can think of to NOT use Linq to SQL in your case would be if you are not using SQL Server (or need to support other DBs either now or in future), or you cannot use .NET 3.5 runtime for some reason.
Good luck
It sounds to me like your "simple database class" is hiding too many details from the classes that need to use it.
I've written classes that contain a SqlCeEngine and expose methods like "LookupDescription(String Code)" ... I think that kind of design is something you should be looking into. And, consider looking into LINQ. It has a lot to offer.