I'm trying to write a LINQ equivalent of the following SQL:
SELECT i.ItemID, i.ItemName
FROM Items AS i
LEFT JOIN BillOfMaterials AS bom ON bom.ItemID = i.ItemID
WHERE bom.ItemID IS NULL
This returns a set of items where the itemid does not appear in the BillofMaterials.ItemID column.
I've tried the following which (not surprisingly) doesn't work:
from i in ctx.Items
join b in ctx.BillOfMaterials on i.ItemID equals b.ItemID
into joinedTable
from j in joinedTable.DefaultIfEmpty().Where(w => w.ItemID == null)
select new
{
i.ItemID,
i.ItemName
};
Since you don't seem to be interested here in the actual BOM objects, only if they exist or not, I would recommend using an expression with just an existence check:
from i in ctx.Items
where !ctx.BillOfMaterials.Any(b => i.ItemID == b.ItemID)
select new
{
i.ItemID,
i.ItemName
};
It is shorter, and it describes better what you are checking for. Also depending on how smart EF is (or not), it could even be more efficient because the BillOfMaterials data will never be loaded into .NET memory as BillOfMaterial objects.
WHERE bom.ItemID IS NULL in sql corresponds to j == null in Linq.
Where(w => w.ItemID == null) will not work here, because the w was null and an exception was thrown.
The right syntax for the left join Linq is like :
from i in ctx.Items
join b in ctx.BillOfMaterials on i.ItemID equals b.ItemID into joinedTable
from j in joinedTable.DefaultIfEmpty()
where j == null
select new
{
i.ItemID,
i.ItemName
};
I hope you find this helpful.
Related
Apparently I'm missing something with how LINQ to entities works. Hopefully one of you all can educate me.
Please try the below locally and let me know if you are seeing the same results. Something is really strange here...
Lets look at a very simple LINQ expression using navigation properties.
This was generated in LinqPad in a C# statement.
var result = (from ge in group_execution
where ge.automation_sequences.project.client_id == 1 && ge.parent_group_exec_id != null
select new
{
ge.id,
ge.parent_group_exec_id,
ge.automation_sequences.project.client_id
});
result.Dump();
OR, we can use joins...which will lead to the same bad results, but lets continue...
var result = (from ge in group_execution
join aseq in automation_sequences on ge.automation_sequence_id equals aseq.id
join p in project on aseq.project_id equals p.id
where p.client_id == 1 && ge.parent_group_exec_id != null
select new
{
ge.id,
ge.parent_group_exec_id,
p.client_id
});
result.Dump();
These very simple LINQ expressions generate the following SQL:
SELECT
[Filter1].[id1] AS [id],
[Filter1].[parent_group_exec_id] AS [parent_group_exec_id],
[Extent5].[client_id] AS [client_id]
FROM (SELECT [Extent1].[id] AS [id1], [Extent1].[automation_sequence_id] AS [automation_sequence_id], [Extent1].[parent_group_exec_id] AS [parent_group_exec_id]
FROM [dbo].[group_execution] AS [Extent1]
INNER JOIN [dbo].[automation_sequences] AS [Extent2] ON [Extent1].[automation_sequence_id] = [Extent2].[id]
INNER JOIN [dbo].[project] AS [Extent3] ON [Extent2].[project_id] = [Extent3].[id]
WHERE ([Extent1].[parent_group_exec_id] IS NOT NULL) AND (1 = [Extent3].[client_id]) ) AS [Filter1]
LEFT OUTER JOIN [dbo].[automation_sequences] AS [Extent4] ON [Filter1].[automation_sequence_id] = [Extent4].[id]
LEFT OUTER JOIN [dbo].[project] AS [Extent5] ON [Extent4].[project_id] = [Extent5].[id]
This baffles me. For the life of me I can't understand why LINQ is doing this. It's horrible, just look at the execution plan:
Now lets manually clean this up in SSMS and view the correct SQL and execution plan:
Much better, but how do we get LINQ to act this way?
Is anyone else seeing this? Has anyone else ever saw this and corrected it and if so how?
Thanks for looking into this.
UPDATE, attempting Chris Schaller fix:
var result = (from ge in group_execution
select new
{
ge.id,
ge.parent_group_exec_id,
ge.automation_sequences.project.client_id
}).Where(x=>x.client_id == 1 && x.parent_group_exec_id != null);
result.Dump();
Just so you all know I'm monitoring the SQL through SQL Server Profiler. If anyone knows of any issues doing it this way let me know.
UPDATE, a fix for JOINS, but not nav properties, and a cause, but why?
Here's your solution:
var result = (from ge in group_execution.Where(x=>x.parent_group_exec_id != null)
join aseq in automation_sequences on ge.automation_sequence_id equals aseq.id
join p in project on aseq.project_id equals p.id
where p.client_id == 1// && ge.parent_group_exec_id != null
select new
{
ge.id,
ge.parent_group_exec_id,
p.client_id
});
result.Dump();
Null checks shouldn't cause the framework to mess up like this. Why should I have to write it this way? This just seems like a defect to me in the framework. It will make my dynamic expressions a little bit more difficult to write, but maybe I can find a way.
Navigation Properties still mess up...so I'm still really sad. Picture below:
var result = (from ge in group_execution.Where(x=>x.parent_group_exec_id != null)
where ge.automation_sequences.project.client_id == 1// && ge.parent_group_exec_id != null
select new
{
ge.id,
ge.parent_group_exec_id,
ge.automation_sequences.project.client_id
});
result.Dump();
move your where clause to after you have defined the structure of the select statement
var result = (from ge in group_execution
select new
{
ge.id,
ge.parent_group_exec_id,
ge.automation_sequences.project.client_id
}).Where(x => x.client_id == 1 && x.parent_group_exec_id != null)
result.Dump();
Remember that Linq-to-entities flattens the results of queries to execute as SQL and then hydrates the object graph from those results.
When your query uses navigation properties or joins the query parser has to allow for zero results from those sub queries (Extents) to make sure that all columns that are required in the output and any interim processing are represented. By explicitly specifying a filter on a table for != null early in the query the parser knows that there is no further possibility that the field and any relationships linked by that field will be null, until then the parser prepares the query as if the joins will return null results
It is worth checking, but i wonder if UseDatabaseNullSemantics has anything to do with this?
Try:
dbContext.Configuration.UseDatabaseNullSemantics = false
In Linq, we can specify Where clauses as often as we like, improve the resulting SQL we should filter early in the query and granularly.
The parser engine is optimized to follow and implement your query sequentially, and generate good SQL at the end of it. Don't try to write linq-to-entities the same way that you structure your SQL, I know it's counter intuitive because the syntax is similar
A good technique is to assume that before each clause all the records from the previous statements have been loaded into memory, and that the next operation will affect all of those records. So you want to reduce the records before each additional operation by specifying a filter before moving on to the next clause
In general, if you have a filter condition based on the root table, apply this to the query before define all other joins and filters and even selects, you will get much cleaner sql.
I am comming to ASP .NET Core from PHP w/ MySQL.
The problem:
For the illustration, suppose the following two tables:
T: {ID, Description, FK} and States: {ID, ID_T, Time, State}. There is 1:n relationship between them (ID_T references T.ID).
I need all the records from T with some specific value of FK (lets say 1) along with the related newest record in States (if any).
In terms of SQL it can be written as:
SELECT T.ID, T.Description, COALESCE(s.State, 0) AS 'State' FROM T
LEFT JOIN (
SELECT ID_T, MAX(Time) AS 'Time'
FROM States
GROUP BY ID_T
) AS sub ON T.ID = sub.ID_T
LEFT JOIN States AS s ON T.ID = s.ID_T AND sub.Time = s.Time
WHERE FK = 1
I am struggling to write an efficient equivalent query in LINQ (or the fluent API). The best working solution I've got so far is:
from t in _context.T
where t.FK == 1
join s in _context.States on t.ID equals o.ID_T into _s
from s in _s.DefaultIfEmpty()
let x = new
{
id = t.ID,
time = s == null ? null : (DateTime?)s.Time,
state = s == null ? false : s.State
}
group x by x.id into x
select x.OrderByDescending(g => g.time).First();
When I check the resulting SQL query in the output window when executed it is just like:
SELECT [t].[ID], [t].[Description], [t].[FK], [s].[ID], [s].[ID_T], [s].[Time], [s].[State]
FROM [T] AS [t]
LEFT JOIN [States] AS [s] ON [T].[ID] = [s].[ID_T]
WHERE [t].[FK] = 1
ORDER BY [t].[ID]
Not only it selects more columns than I need (in the real scheme there are more of them). There is no grouping in the query so I suppose it selects everything from the DB (and States is going to be huge) and the grouping/filtering is happening outside the DB.
The questions:
What would you do?
Is there an efficient query in LINQ / Fluent API?
If not, what workarounds can be used?
Raw SQL ruins the concept of abstracting from a specific DB technology and its use is very clunky in current Entity Framework Core (but maybe its the best solution).
To me, this looks like a good example for using a database view - again, not really supported by Entity Framework Core (but maybe its the best solution).
What happens if you try to do a more straight forward translation to LINQ?
var latestState = from s in _context.States
group s by s.ID_T into sg
select new { ID_T = sg.Key, Time = sg.Time.Max() };
var ans = from t in _context.T
where t.FK == 1
join sub in latestState on t.ID equals sub.ID_T into subj
from sub in subj.DefaultIfEmpty()
join s in _context.States on new { t.ID, sub.Time } equals new { s.ID, s.Time } into sj
from s in sj.DefaultIfEmpty()
select new { t.ID, t.Description, State = (s == null ? 0 : s.State) };
Apparently the ?? operator will translate to COALESCE and may handle an empty table properly, so you could replace the select with:
select new { t.ID, t.Description, State = s.State ?? 0 };
OK. Reading this article (almost a year old now), Smit's comment to the original question and other sources, it seems that EF Core is not really production ready yet. It is not able to translate grouping to SQL and therefore it is performed on the client side, which may be (and in my case would be) a serious problem. It corresponds to the observed behavior (the generated SQL query does no grouping and selects everything in all groups). Trying the LINQ queries out in Linqpad it always translates to a single SQL query.
I have downgraded to EF6 following this article. It required some changes in my model's code and some queries. After changing .First() to .FirstOrDefault() in my original LINQ query it works fine and translates to a single SQL query selecting only the needed columns. The generated query is much more complex than it is needed, though.
Using a query from NetMage's answer (after small fixes), it results in a SQL query almost identical to my own original SQL query (there's only a more complex construct than COALESCE).
var latestState = from s in _context.States
group s by s.ID_T into sg
select new { ID = sg.Key, Time = sg.Time.Max() };
var ans = from t in _context.T
where t.FK == 1
join sub in latestState on t.ID equals sub.ID into subj
from sub in subj.DefaultIfEmpty()
join s in _context.States
on new { ID_T = t.ID, sub.Time } equals new { s.ID_T, s.Time }
into sj
from s in sj.DefaultIfEmpty()
select new { t.ID, t.Description, State = (s == null ? false : s.State) };
In LINQ it's not as elegant as my original SQL query but semantically it's the same and it does more or less the same thing on the DB side.
In EF6 it is also much more convenient to use arbitrary raw SQL queries and AFAIK also the database views.
The biggest downside of this approach is that full .NET framework has to be targeted, EF6 is not compatible with .NET Core.
I am having trouble with the AX2012 class (default AX2012 class and code, no modifications have been made on it): CustVendTransDetails in the method calcCashDiscounts
The following query is giving me the error The select list for the INSERT statement contains fewer items than the insert list. The number of SELECT values must match the number of INSERT columns.:
if (TaxParameters::canApplyCashDiscOnInvoice_ES())
{
insert_recordset tmpValue
(CustVendTransRefRecId, AmountMST)
select CustVendTransRefRecId
from _custVendAccountStatementIntTmpProcessing
exists join custVendTransLoc
where
custVendTransLoc.RecId == _custVendAccountStatementIntTmpProcessing.CustVendTransRefRecId
exists join firstOnly subledgerVoucherGeneralJournalEntry
where
subledgerVoucherGeneralJournalEntry.Voucher == custVendTransLoc.Voucher &&
subledgerVoucherGeneralJournalEntry.AccountingDate == custVendTransLoc.TransDate
exists join generalJournalEntry
where
generalJournalEntry.RecId == subledgerVoucherGeneralJournalEntry.GeneralJournalEntry &&
generalJournalEntry.Ledger == Ledger::current()
join AccountingCurrencyAmount from generalJournalAccountEntry
where
generalJournalAccountEntry.GeneralJournalEntry == generalJournalEntry.RecId &&
(generalJournalAccountEntry.PostingType == LedgerPostingType::CustCashDisc ||
generalJournalAccountEntry.PostingType == LedgerPostingType::VendCashDisc);
update_recordSet _custVendAccountStatementIntTmpProcessing setting
UtilizedCashDisc = tmpValue.AmountMST,
PossibleCashDisc = tmpValue.AmountMST
join tmpValue
where
tmpValue.CustVendTransRefRecId == _custVendAccountStatementIntTmpProcessing.CustVendTransRefRecId;
}
I understand why, but I am not sure how to solve this problem. Will it be a problem to replace the exist join with a normal join?
Replacing the exist join with join, does solve my problem, but I am not sure what difference it will make to the data? Because it is only is selecting 1 field?
You can try to switch the order of joins:
insert_recordset tmpValue (CustVendTransRefRecId, AmountMST)
select CustVendTransRefRecId
from _custVendAccountStatementIntTmpProcessing
join AccountingCurrencyAmount from generalJournalAccountEntry // Moved up
where generalJournalAccountEntry.PostingType == LedgerPostingType::CustCashDisc ||
generalJournalAccountEntry.PostingType == LedgerPostingType::VendCashDisc
exists join custVendTransLoc
where
custVendTransLoc.RecId == _custVendAccountStatementIntTmpProcessing.CustVendTransRefRecId
exists join firstOnly subledgerVoucherGeneralJournalEntry
where
subledgerVoucherGeneralJournalEntry.Voucher == custVendTransLoc.Voucher &&
subledgerVoucherGeneralJournalEntry.AccountingDate == custVendTransLoc.TransDate
exists join generalJournalEntry
where
generalJournalEntry.RecId == subledgerVoucherGeneralJournalEntry.GeneralJournalEntry && &&
generalJournalEntry.RecId == generalJournalAccountEntry.GeneralJournalEntry && // Moved from join
generalJournalEntry.Ledger == Ledger::current();
Replacing the exist join with join will not fix your issue. Exist is a way to join to essentially inner join to a table without returning any fields.
The query should return CustVendTransRefRecId from _custVendAccountStatementIntTmpProcessing and AccountingCurrencyAmount from generalJournalAccountEntry which is exactly what the insert is expecting.
I expect the query isn't actually returning anything. Check the criteria it is using and check the data.
I've got a complex query written in SQL which works.
SELECT Instruments.Id as [Id], Instruments.ShareCode as [Share Code], Instruments.Name AS [Short Name], Instruments.Description as [Share Name],
InstrumentGames.Instrument_Id,
InstrumentGames.Game_Id, Games.Name AS [Game Name],
Entries.Name AS [Entry Name], AspNetUsers.UserName, AspNetUsers.Id as [User_Id],
Sectors.Name AS Sector_Id, Sectors.ShortName AS Sector
FROM AspNetUsers INNER JOIN
Entries ON AspNetUsers.Id = Entries.User_Id INNER JOIN
Games ON Entries.Game_Id = Games.Id INNER JOIN
InstrumentGames ON Games.Id = InstrumentGames.Game_Id INNER JOIN
Instruments ON InstrumentGames.Instrument_Id = Instruments.Id INNER JOIN
Sectors ON Instruments.Sector_Id = Sectors.Id
WHERE Instruments.Listed = 'true' and InstrumentGames.Game_Id = 2 and Entries.User_Id = 'd28d6552-7d98-476c-82cb-063e7ef45cb6'
I'm using Entity code first models and trying to convert what I have in SQL to a linq query.
I've come up with:
public static Models.Instrument GetShare(string shareSearchCriteria,
Models.Game selectedGame,
string userId)
{
var _db = new JSEChallenge.Models.ApplicationDbContext();
var records = (from instru in _db.Instruments
from e in _db.Entries
where (instru.ShareCode.Contains(shareSearchCriteria) ||
instru.Name.Contains(shareSearchCriteria) ||
instru.Description.Contains(shareSearchCriteria))
where (instru.Listed == true &&
instru.Games.Contains(selectedGame) &&
e.User_Id == userId)
select instru).ToList();
return records.FirstOrDefault();
}
But I keep getting this error:
Unable to create a constant value of type 'X.Models.Game'. Only primitive types or enumeration types are supported in this context.
I think the issue is the m2m table InstrumentGames. In my SQL query I can join it easily but in my C# I cannot. The way I usually find m2m records is syntax like instru.Games.Contains(selectedGame)
Unfortunately I still cannot get this to work.
How do I implement this kind of query in Linq?
Not sure if it will work, but try instru.Games.Any(q => q.Id == selectedGame.Id) instead of instru.Games.Contains(selectedGame).
Hope this helps!
I have two tables:
Phase :
long ID
string Name
and another Activity :
long ID
string Name
long PhaseID
I already know the name of the phases and I want to get the activity for those particular phases. Do i add PhaseName to the activity table or do I do it through join in LINQ?
Maybe something like this?
var query = from a in entities.Activities
join p in entities.Phases on a.PhaseId equals p.Id
where p.Name == "Preplanning"
... and here im not sure how to finish this query..
Thanks for your help!
The code that you've provided will use an Inner Join to find all Activities where the Phase with Name "Preplanning" exists.
To finish your query you need to add a select clause.
var query = from a in entities.Activities
join p in entities.Phases on a.PhaseId equals p.Id
where p.Name == "Preplanning"
select a.Name
will return IEnumerable<string> of all activity names.
Just select activity, as you want:
var query = from a in entities.Activities
join p in entities.Phases on a.PhaseId equals p.Id
where p.Name == "Preplanning"
select a;
Here is how query expression should look like:
A query expression must begin with a from clause and must end with a select or group clause. Between the first from clause and the last select or group clause, it can contain one or more of these optional clauses: where, orderby, join, let and even additional from clauses. You can also use the into keyword to enable the result of a join or group clause to serve as the source for additional query clauses in the same query expression.
Same as puzzling image:
With method syntax you don't need to end query with something special:
var query = entities.Phases
.Where(p => p.Name == "Preplanning")
.Join(entities.Activities, p => p.Id, a => a.PhaseId, (p,a) => a);
No need to do a join if you only need data from one of the tables. You can apply a filter instead:
var q = entities.Activities.Where(a =>
entities.Phases.Any(p => a.PhaseId == p.Id && p.Name == "Preplanning"));