I'm using LINQ to SQL to search a fairly large database and am unsure of the best approach to perform paging with a DataPager. I am aware of the Skip() and Take() methods and have those working properly. However, I'm unable to use the count of the results for the datapager, as they will always be the page size as determined in the Take() method.
For example:
var result = (from c in db.Customers
where c.FirstName == "JimBob"
select c).Skip(0).Take(10);
This query will always return 10 or fewer results, even if there are 1000 JimBobs. As a result, the DataPager will always think there's a single page, and users aren't able to navigate across the entire result set. I've seen one online article where the author just wrote another query to get the total count and called that.
Something like:
int resultCount = (from c in db.Customers
where c.FirstName == "JimBob"
select c).Count();
and used that value for the DataPager. But I'd really rather not have to copy and paste every query into a separate call where I want to page the results for obvious reasons. Is there an easier way to do this that can be reused across multiple queries?
Thanks.
In situations like this I sometimes return the total record count as a field in my result set from the db.
Basically you only have the two options, write another query specifically for the count, or return it as column in the results.
Remember that linq provides defferred query execution...
var qry = from c in db.Customers
where c.FirstName == "JimBob"
select c;
int resultCount = qry.Count();
var results = qry.Skip(0).Take(10);
Related
I have to build a specialized form which queries the InventTrans table for some purchase transactions.
The transactions must be selected, either with a non-empty Voucher, either with a non-empty Packing slip Id (PackingSlipId).
While looking how to specify this in a query range, I found that it was possible to associate this query range to the RecId of the table. Is this the usual and classic way to do this ? See my example (which seem to work).
QueryBuildDataSource qbds;
;
qbds = this.query().dataSourceTable(tableNum(inventTrans));
sysQuery::findOrCreateRange(qbds, fieldNum(InventTrans, RecId)).value(
strfmt('(%1 != "") || (%2 != "")',
fieldstr(InventTrans, Voucher),
fieldstr(InventTrans, PackingSlipId)));
Those are called range expressions and is the only way to state more complex queries with a Query object - see this MSDN page
In addition have a look at this Axaptapedia page (thanks to FH-Inway)
Regarding if it is the classical way: No - only use it if necessary
I'm experiencing a really odd result when I do a count in X++, something I've not experienced before. I am performing what I thought was a really simply count query, but I can't seem to get the result I am after.
WMSOrderTrans orderTrans;
WMSOrderTrans orderTransChk;
;
select count(RecId) from orderTrans group by shipmentid where orderTrans.inventTransRefId == 'XXXXXX';
info(strFmt('Count is %1', orderTrans.RecId));
while select orderTransChk group by shipmentid where orderTransChk.inventTransRefId == 'XXXXXX' {
info(strFmt('Shipment is %1', orderTransChk.shipmentId));
}
The data set that I am selecting all have only 1 shipmentid, so the first select I am expecting a count of 1, instead I get 4 (which is how many lines for that transrefid exist). If I change the count from 'RecId' to 'ShipmentId', then instead of the count, I get actual shipmentId. I simply want it to return the count of the records, which is what I believe I've asked it to do.
I really can't see what I am missing.
In the while select, I get what I expect (the shipmentid), only 1 infolog message for the loop. This tells me that the group by with the where clause is working, but it doesn't explain why the first count select statement isn't behaving as I would expect.
For reference, this is AX2012 R1 system.
For anyone who might be interested in knowing my answer, it's tied up with Jeff's response. At the end of the day, I didn't look at data well enough, and the query returned the correct results. I initially thought there were a number of unique shipments, but I was wrong. My expected result was erroneous. There were 4 lines in the file, but the lines were unique for the item, not the shipment. They were all on the same shipment. So really, my own fault, it goes to show that one really needs to look at the data closely.
Thanks to all that responded, greatly appreciated.
I would try a database sync, then restarting the AOS. I don't see anything obviously wrong, so it points to bouncing everything.
Try getting the select statement via this method (from memory so check syntax) and then review the query against SQL directly. It uses generateOnly.
select generateOnly count(RecId) from orderTrans group by shipmentid where orderTrans.inventTransRefId == 'XXXXXX';
info(orderTrans.getSQLStatement());
If I understand what you try to achieve, you'd like to get something like this SQL query:
select count(distinct shipmentid) from orderTrans
where inventTransRefId = 'XXXXXX'
The 'distinct' keyword is not available in AX select command. The group by clause will allow you to iterate all distinct values but not to use an aggregate on top of it.
You may use a sql connection to push the exact sql command you want.
In AX, aggregate values are stores in the field used: Count(RecId), the count will go in the recid field otherwise the system may need to add new field on the buffer on the fly. I don't think you can aggregate on the group by clause as it's important to have its value.
You can try (I don't have an AX to test it) to use a query:
Query query = new Query();
QueryRun queryRun;
QueryBuildDataSource qbd;
qbd = query.addDataSource(tablenum(OrderTrans));
qbd.addRange(fieldNum(OrderTrans, InventTransId)).value("xxxx");
qbd.addSortField(fieldNum(OrderTrans, ShipmentId));
qbd.SortOrder(SortOrder::GroupBy);
queryRun = new QueryRun(query);
info(strfmt("Total Records in Query %1",SysQuery::countTotal(queryRun)));
I have been using Linq to SQL for a while now on one of my sites and over time the code I am using to query the database has gotten a little messy so I decided to re-write, originally my queries were all handled exclusively by Linq but recently there has been a demand for more advanced search features which has led me more towards using ExecuteQuery and handwriting my SQL statements the problem is that I cannot for the life of me get the Join statement to work properly.
I have two tables in my databases, t_events and t_clients. The only thing similar between the two tables is that they both have a clientid field (the id of the client the event is for). What I need to be able to do is pull all of the events into the page (which works fine) but I dont want to show the clientid on the page I need to show the client name. Originally I had a join clause that handled this nicely:
var eve = from p in db.t_events
join c in db.Clients on p.clientid equals c.clientid
where p.datedue <= thisDay && p.status != "complete"
select new { p.eventname, p.datedue, p.details, p.eventid, p.status, c.clientname };
With the redesign of the page however I am having issues recreating what linq has done here with the join. My current code:
StringBuilder sqlQuery = new StringBuilder("SELECT * FROM dbo.t_events JOIN dbo.t_clients ON dbo.t_events.clientid=dbo.t_clients.clientid");
var query = db.ExecuteQuery<t_events>(sqlQuery.ToString());
foreach (var c in query)
{
counter = counter + 1;
MyStringBuilder.Append("<tr class='"+c.status+"'><td><a href='searchdetails.aspx?id="+c.eventid+"'>"+c.eventname+"</a></td><td>" +c.clientname+ "</td></tr>");
}
in the foreach loop I have you can see I am trying to pull in c.clientname (which doesnt work) as it is not on the t_events database, changing this to c.clientid makes the code work, I am not sure what the issue is as taking that same SQL and running the query directly off the sql management tool works like a charm. Any ideas on this issue would be greatly appreciated!
FIXED!
DaveMarkle suggested using a view, which was by far a much easier way of doing this. I created a view that joins the two tables together with the fields I need and run my queries against it, simple and effective, I thank you!
Erm - so maybe we should have an answer here then so the question drops off the 'unanswered' list.
As Dave Markle stated.
Use a view.
Another option!
Execute the query twice: once with db.ExecuteQuery<t_events> and once db.ExecuteQuery<t_clients>. Now that you have both events and clients you can re-join them client-side by matching client_id.
I have this query:
var newComponents = from ic in importedComponents
where !existingComponents.Contains(ic)
select ic;
importedComponents and existingComponents are of type List<ImportedComponent>, and exist only in memory (are not tied to a data context). In this instance, importedComponents has just over 6,100 items, and existingComponents has 511 items.
This statement is taking too long to complete (I don't know how long, I stop the script after 20 minutes). I've tried the following with no improvement in execution speed:
var existingComponentIDs = from ec in existingComponents
select ec.ID;
var newComponents = from ic in importedComponents
where !existingComponentIDs.Contains(ic.ID)
select ic;
Any help will be much appreciated.
The problem is quadratic complexity of this algorithm. Put the IDs of all existingComponentIDs into a HashSet and use the HashSet.Contains method. It has O(1) lookup cost compared to O(N) for Contains/Any on a list.
The morelinq project contains a method that does all of that in one convenient step: ExceptBy.
You could use Except to get the set difference:
var existingComponentIDs = existingComponents.Select(c => c.ID);
var importedComponentIDs = importedComponents.Select(c => c.ID);
var newComponentIDs = importedComponentIDs.Except(existingComponentIDs);
var newComponents = from ic in importedComponents
join newID in newComponentIDs on ic.ID equals newID
select ic;
foreach (var c in newComponents)
{
// insert into database?
}
Why is LINQ JOIN so much faster than linking with WHERE?
In short: Join method can set up a hash table to use as an index to quicky zip two tables together
Well based on the logic and numbers you provided that means you are basically performing 3117100 comparisons when you run that statement. Obviously that is not entirely accurate because your condition may be satisfied before running through the entire array but you get my point.
With collections this large you are going to want use a collection where you can index your key (in this case your component ID) to help reduce the overhead of the search. The thing to remember is that even though LINQ looks like SQL there are no magic indexes here; it is mainly for convenience. In fact, I have seen articles where a link lookup is actually a slight bit slower than a brute force lookup.
EDIT: If it is possible I would suggest trying a Dictionary or SortedList for your values. I believe either one would have slightly better lookup performance.
I have a trivial issue that I can't resolve. Currently our app uses Linq to retrieve data and get a basic integer value of the row count. I can't form a query that gives back a count without a 'select i'. I don't need the select, just the count(*) response. How do I do this? Below is a sample:
return (from io in db._Owners
where io.Id == Id && io.userId == userId
join i in db._Instances on io.Id equals i.Id **select i**).Count()
;
The select i is fine - it's not actually going to be fetching any data back to the client, because the Count() call will be translated into a Count(something) call at the SQL side.
When in doubt, look at the SQL that's being generated for your query, e.g. with the DataContext.Log property.
Using the LINQ query syntax requires a select statement. There's no way around that.
That being said, the statement will get transformed into a COUNT()-based query; the select i is there only to satisfy the expression system that underlies the LINQ query providers (otherwise the type of the expression would be unknown).
Including the select will not affect the performance here because the final query will get translated into SQL. At this point it will be optimized and will be like select (*) from ......