I have been trying to solve this problem and I can't seem to figure it out. I'm not sure if it's because of my db design and LINQ, but I'm hoping for some direction here.
My db table:
Id Name ParentId
1 Data1 null
2 Data2 null
3 Data3 null
4 Data4 1
5 Data5 1
6 Data6 2
7 Data7 2
Basically Data1 and Data2 are the top levels that I want to use for headings and their children will be related based on their ParentID.
I am trying to use a listview to present the data like the following:
Data1
-----
Data4
Data5
Data2
-----
Data6
Data7
I am trying to use a combination of LINQ and listview to accomplish this.
The following is the code for the linq query:
var query = from data in mydb.datatable
where data.ParentId == null
select data;
But this only gives the heading level... and unfortunately listview only takes in 1 datasource.
While it's possible with some databases (like SQL Server post 2005) to write recursive queries, I don't believe those get generated by LINQ. On the other hand, if the number of records is sufficiently small, you could materialize the data (to a list) and write a LINQ query that uses a recursive function to generate your list.
This is from memory, but it would look something like this:
Func<int?,IEnumerable<data>> f = null;
f = parentId => {
IEnumerable<data> result = from data in mydb.datatable
where data.ParentId = parentId
select data;
return result.ToList().SelectMany(d=>f(d.Id));
};
That should get you the hierarchy.
If your hierarchy has only two levels you can use a group join and anonymous objects:
var query = from data in mydb.datatable.Where(x => x.ParentId == null)
join child in mydb.datatable.Where(x => x.ParentId != null)
on data.Id equals child.ParentId into children
select new { data, children };
Edit: You will have to convert the data to a collection that can be bound to a ListView. One hack would be to have a list that is only one level deep with spacing in front of the subitems:
var listViewItems = (from item in query.AsEnumerable()
let dataName = item.data.Name
let childNames = item.children.Select(c => " " + c.Name)
from name in dataName.Concat(childNames)
select new ListViewItem(name)).ToArray();
You could also try to find a control that fits better, like a TreeView. You might want to ask a separate question about this issue.
I just wrote up a blog post describing a solution to build a graph from a self-referencing table with a single LINQ query to the database which might be of use. See http://www.thinqlinq.com/Post.aspx/Title/Hierarchical-Trees-from-Flat-Tables-using-LINQ.
Related
Apparently I'm missing something with how LINQ to entities works. Hopefully one of you all can educate me.
Please try the below locally and let me know if you are seeing the same results. Something is really strange here...
Lets look at a very simple LINQ expression using navigation properties.
This was generated in LinqPad in a C# statement.
var result = (from ge in group_execution
where ge.automation_sequences.project.client_id == 1 && ge.parent_group_exec_id != null
select new
{
ge.id,
ge.parent_group_exec_id,
ge.automation_sequences.project.client_id
});
result.Dump();
OR, we can use joins...which will lead to the same bad results, but lets continue...
var result = (from ge in group_execution
join aseq in automation_sequences on ge.automation_sequence_id equals aseq.id
join p in project on aseq.project_id equals p.id
where p.client_id == 1 && ge.parent_group_exec_id != null
select new
{
ge.id,
ge.parent_group_exec_id,
p.client_id
});
result.Dump();
These very simple LINQ expressions generate the following SQL:
SELECT
[Filter1].[id1] AS [id],
[Filter1].[parent_group_exec_id] AS [parent_group_exec_id],
[Extent5].[client_id] AS [client_id]
FROM (SELECT [Extent1].[id] AS [id1], [Extent1].[automation_sequence_id] AS [automation_sequence_id], [Extent1].[parent_group_exec_id] AS [parent_group_exec_id]
FROM [dbo].[group_execution] AS [Extent1]
INNER JOIN [dbo].[automation_sequences] AS [Extent2] ON [Extent1].[automation_sequence_id] = [Extent2].[id]
INNER JOIN [dbo].[project] AS [Extent3] ON [Extent2].[project_id] = [Extent3].[id]
WHERE ([Extent1].[parent_group_exec_id] IS NOT NULL) AND (1 = [Extent3].[client_id]) ) AS [Filter1]
LEFT OUTER JOIN [dbo].[automation_sequences] AS [Extent4] ON [Filter1].[automation_sequence_id] = [Extent4].[id]
LEFT OUTER JOIN [dbo].[project] AS [Extent5] ON [Extent4].[project_id] = [Extent5].[id]
This baffles me. For the life of me I can't understand why LINQ is doing this. It's horrible, just look at the execution plan:
Now lets manually clean this up in SSMS and view the correct SQL and execution plan:
Much better, but how do we get LINQ to act this way?
Is anyone else seeing this? Has anyone else ever saw this and corrected it and if so how?
Thanks for looking into this.
UPDATE, attempting Chris Schaller fix:
var result = (from ge in group_execution
select new
{
ge.id,
ge.parent_group_exec_id,
ge.automation_sequences.project.client_id
}).Where(x=>x.client_id == 1 && x.parent_group_exec_id != null);
result.Dump();
Just so you all know I'm monitoring the SQL through SQL Server Profiler. If anyone knows of any issues doing it this way let me know.
UPDATE, a fix for JOINS, but not nav properties, and a cause, but why?
Here's your solution:
var result = (from ge in group_execution.Where(x=>x.parent_group_exec_id != null)
join aseq in automation_sequences on ge.automation_sequence_id equals aseq.id
join p in project on aseq.project_id equals p.id
where p.client_id == 1// && ge.parent_group_exec_id != null
select new
{
ge.id,
ge.parent_group_exec_id,
p.client_id
});
result.Dump();
Null checks shouldn't cause the framework to mess up like this. Why should I have to write it this way? This just seems like a defect to me in the framework. It will make my dynamic expressions a little bit more difficult to write, but maybe I can find a way.
Navigation Properties still mess up...so I'm still really sad. Picture below:
var result = (from ge in group_execution.Where(x=>x.parent_group_exec_id != null)
where ge.automation_sequences.project.client_id == 1// && ge.parent_group_exec_id != null
select new
{
ge.id,
ge.parent_group_exec_id,
ge.automation_sequences.project.client_id
});
result.Dump();
move your where clause to after you have defined the structure of the select statement
var result = (from ge in group_execution
select new
{
ge.id,
ge.parent_group_exec_id,
ge.automation_sequences.project.client_id
}).Where(x => x.client_id == 1 && x.parent_group_exec_id != null)
result.Dump();
Remember that Linq-to-entities flattens the results of queries to execute as SQL and then hydrates the object graph from those results.
When your query uses navigation properties or joins the query parser has to allow for zero results from those sub queries (Extents) to make sure that all columns that are required in the output and any interim processing are represented. By explicitly specifying a filter on a table for != null early in the query the parser knows that there is no further possibility that the field and any relationships linked by that field will be null, until then the parser prepares the query as if the joins will return null results
It is worth checking, but i wonder if UseDatabaseNullSemantics has anything to do with this?
Try:
dbContext.Configuration.UseDatabaseNullSemantics = false
In Linq, we can specify Where clauses as often as we like, improve the resulting SQL we should filter early in the query and granularly.
The parser engine is optimized to follow and implement your query sequentially, and generate good SQL at the end of it. Don't try to write linq-to-entities the same way that you structure your SQL, I know it's counter intuitive because the syntax is similar
A good technique is to assume that before each clause all the records from the previous statements have been loaded into memory, and that the next operation will affect all of those records. So you want to reduce the records before each additional operation by specifying a filter before moving on to the next clause
In general, if you have a filter condition based on the root table, apply this to the query before define all other joins and filters and even selects, you will get much cleaner sql.
I am comming to ASP .NET Core from PHP w/ MySQL.
The problem:
For the illustration, suppose the following two tables:
T: {ID, Description, FK} and States: {ID, ID_T, Time, State}. There is 1:n relationship between them (ID_T references T.ID).
I need all the records from T with some specific value of FK (lets say 1) along with the related newest record in States (if any).
In terms of SQL it can be written as:
SELECT T.ID, T.Description, COALESCE(s.State, 0) AS 'State' FROM T
LEFT JOIN (
SELECT ID_T, MAX(Time) AS 'Time'
FROM States
GROUP BY ID_T
) AS sub ON T.ID = sub.ID_T
LEFT JOIN States AS s ON T.ID = s.ID_T AND sub.Time = s.Time
WHERE FK = 1
I am struggling to write an efficient equivalent query in LINQ (or the fluent API). The best working solution I've got so far is:
from t in _context.T
where t.FK == 1
join s in _context.States on t.ID equals o.ID_T into _s
from s in _s.DefaultIfEmpty()
let x = new
{
id = t.ID,
time = s == null ? null : (DateTime?)s.Time,
state = s == null ? false : s.State
}
group x by x.id into x
select x.OrderByDescending(g => g.time).First();
When I check the resulting SQL query in the output window when executed it is just like:
SELECT [t].[ID], [t].[Description], [t].[FK], [s].[ID], [s].[ID_T], [s].[Time], [s].[State]
FROM [T] AS [t]
LEFT JOIN [States] AS [s] ON [T].[ID] = [s].[ID_T]
WHERE [t].[FK] = 1
ORDER BY [t].[ID]
Not only it selects more columns than I need (in the real scheme there are more of them). There is no grouping in the query so I suppose it selects everything from the DB (and States is going to be huge) and the grouping/filtering is happening outside the DB.
The questions:
What would you do?
Is there an efficient query in LINQ / Fluent API?
If not, what workarounds can be used?
Raw SQL ruins the concept of abstracting from a specific DB technology and its use is very clunky in current Entity Framework Core (but maybe its the best solution).
To me, this looks like a good example for using a database view - again, not really supported by Entity Framework Core (but maybe its the best solution).
What happens if you try to do a more straight forward translation to LINQ?
var latestState = from s in _context.States
group s by s.ID_T into sg
select new { ID_T = sg.Key, Time = sg.Time.Max() };
var ans = from t in _context.T
where t.FK == 1
join sub in latestState on t.ID equals sub.ID_T into subj
from sub in subj.DefaultIfEmpty()
join s in _context.States on new { t.ID, sub.Time } equals new { s.ID, s.Time } into sj
from s in sj.DefaultIfEmpty()
select new { t.ID, t.Description, State = (s == null ? 0 : s.State) };
Apparently the ?? operator will translate to COALESCE and may handle an empty table properly, so you could replace the select with:
select new { t.ID, t.Description, State = s.State ?? 0 };
OK. Reading this article (almost a year old now), Smit's comment to the original question and other sources, it seems that EF Core is not really production ready yet. It is not able to translate grouping to SQL and therefore it is performed on the client side, which may be (and in my case would be) a serious problem. It corresponds to the observed behavior (the generated SQL query does no grouping and selects everything in all groups). Trying the LINQ queries out in Linqpad it always translates to a single SQL query.
I have downgraded to EF6 following this article. It required some changes in my model's code and some queries. After changing .First() to .FirstOrDefault() in my original LINQ query it works fine and translates to a single SQL query selecting only the needed columns. The generated query is much more complex than it is needed, though.
Using a query from NetMage's answer (after small fixes), it results in a SQL query almost identical to my own original SQL query (there's only a more complex construct than COALESCE).
var latestState = from s in _context.States
group s by s.ID_T into sg
select new { ID = sg.Key, Time = sg.Time.Max() };
var ans = from t in _context.T
where t.FK == 1
join sub in latestState on t.ID equals sub.ID into subj
from sub in subj.DefaultIfEmpty()
join s in _context.States
on new { ID_T = t.ID, sub.Time } equals new { s.ID_T, s.Time }
into sj
from s in sj.DefaultIfEmpty()
select new { t.ID, t.Description, State = (s == null ? false : s.State) };
In LINQ it's not as elegant as my original SQL query but semantically it's the same and it does more or less the same thing on the DB side.
In EF6 it is also much more convenient to use arbitrary raw SQL queries and AFAIK also the database views.
The biggest downside of this approach is that full .NET framework has to be targeted, EF6 is not compatible with .NET Core.
i am bit confused by the nature and working of query , I tried to access database which contains each name more than once having same EMPid so when i accessed it in my DROP DOWN LIST then same repetition was in there too so i tried to remove repetition by putting DISTINCT in query but that didn't work but later i modified it another way and that worked but WHY THAT WORKED, I DON'T UNDERSTAND ?
QUERY THAT DIDN'T WORK
var names = (from n in DataContext.EmployeeAtds select n).Distinct();
QUERY THAT WORKED of which i don't know how ?
var names = (from n in DataContext.EmployeeAtds select new {n.EmplID, n.EmplName}).Distinct();
why 2nd worked exactly like i wanted (picking each name 1 time)
i'm using mvc 3 and linq to sql and i am newbie.
Both queries are different. I am explaining you both query in SQL that will help you in understanding both queries.
Your first query is:
var names = (from n in DataContext.EmployeeAtds select n).Distinct();
SQL:-
SELECT DISTINCT [t0].[EmplID], [t0].[EmplName], [t0].[Dept]
FROM [EmployeeAtd] AS [t0]
Your second query is:
(from n in EmployeeAtds select new {n.EmplID, n.EmplName}).Distinct()
SQL:-
SELECT DISTINCT [t0].[EmplID], [t0].[EmplName] FROM [EmployeeAtd] AS
[t0]
Now you can see SQL query for both queries. First query is showing that you are implementing Distinct on all columns of table but in second query you are implementing distinct only on required columns so it is giving you desired result.
As per Scott Allen's Explanation
var names = (from n in DataContext.EmployeeAtds select n).Distinct();
The docs for Distinct are clear – the method uses the default equality comparer to test for equality, and the default comparer sees 4 distinct object references. One way to get around this would be to use the overloaded version of Distinct that accepts a custom IEqualityComparer.
var names = (from n in DataContext.EmployeeAtds select new {n.EmplID, n.EmplName}).Distinct();
Turns out the C# compiler overrides Equals and GetHashCode for anonymous types. The implementation of the two overridden methods uses all the public properties on the type to compute an object's hash code and test for equality. If two objects of the same anonymous type have all the same values for their properties – the objects are equal. This is a safe strategy since anonymously typed objects are essentially immutable (all the properties are read-only).
Try this:
var names = DataContext.EmployeeAtds.Select(x => x.EmplName).Distinct().ToList();
Update:
var names = DataContext.EmployeeAtds
.GroupBy(x => x.EmplID)
.Select(g => new { EmplID = g.Key, EmplName = g.FirstOrDefault().EmplName })
.ToList();
Actually I spend whole day on the EntityFramework for foreign key.
assume we have two table.
Process(app_id,process_id)
LookupProcessId(process_id, process_description)
you can understand two tables with names, first table ,use process_id to indicate every application, and description is in the seoncd table.
Actually i try many times and figure out how to do inquery: it was like
Dim result = (from x in db.Processes where x.LookupProcess is (from m in db.LookupProcessIds where descr = "example" select m).FirstOrDefault() select x).FirstOrDefault()
First I want to ask is there easier way to do it.
Second i want to ask question is about insert
p As New AmpApplication.CUEngData.Process
p.app_id=100
p.LookupProcess = (from m in db.LookupProcessIds where descr = "example" select m).FirstOrDefault()
db.AddToProcesses(p)
db.SaveChanges()
from appearance it looks fine, but it give me error says
Entities in 'AmpCUEngEntities.Processes' participate in the 'FK_Process_LookupProcess' relationship. 0 related 'LookupProcess' were found. 1 'LookupProcess' is expected.
can i ask is that insert wrong? and is that my query correct?
For your first question:
Dim result = (from x in db.Processes
where x.LookupProcess.descr = "example"
select x).FirstOrDefault()
Actually, you missed some concepts from DataEntityModel, and its Framework. To manipulate data, you have to call object from contextual point of view. Those allow you to specify to the ObjectStateManager the state of an DataObject. In your case, if you have depending data from FK, you will have to add/update any linked data from leaf to root.
This example demonstrate simple (no dependances) data manipulation. A select if existing and an insert or update.
If you want more info about ObjectStateManager manipulation go to http://msdn.microsoft.com/en-us/library/bb156104.aspx
Dim context As New Processing_context 'deseign your context (this one is linked to a DB)
Dim pro = (From r In context.PROCESS
Where r.LOOKUPPROCESS.descr = LookupProcess.descr
Select r).FirstOrDefault()
If pro Is Nothing Then 'add a new one
pro = New context.PROCESS With {.AP_ID = "id", .PROCESS_ID = "p_id"}
context.PROCESS.Attach(pro)
context.ObjectStateManager.ChangeObjectState(pro, System.Data.EntityState.Added)
Else
'update data attibutes
pro.AP_ID = "id"
pro.PROCESS_ID = "p_id"
context.ObjectStateManager.ChangeObjectState(pro, System.Data.EntityState.Modified)
'context.PROCESS.Attach(pro)
End If
context.SaveChanges()
I hope this will help. Have a nice day!
For your first question, to expand on what #jeroenh suggested:
Dim result = (from x in db.Processes.Include("LookupProcess")
where x.LookupProcess.descr = "example"
select x).FirstOrDefault()
The addition of the Include statement will hydrate the LookupProcess entities so that you can query them. Without the Include, x.LookupProcess will be null which would likely explain why you got the error you did.
If using the literal string as an argument to Include is not ideal, see Returning from a DbSet 3 tables without the error "cannot be inferred from the query" for an example of doing this using nested entities.
For your second question, this line
p.LookupProcess = (from m in db.LookupProcessIds
where descr = "example" select m).FirstOrDefault()
Could cause you problems later on because if there is no LookupProcessId with a process_description of "example", you are going to get null. From MSDN:
The default value for reference and nullable types is null.
Because of this, if p.LookupProcess is null when you insert the entity, you will get the exception:
Entities in 'AmpCUEngEntities.Processes' participate in the 'FK_Process_LookupProcess' relationship. 0 related 'LookupProcess' were found. 1 'LookupProcess' is expected.
To avoid this kind of problem, you will need to check that p.LookupProcess is not null before it goes in the database.
If Not p.LookupProcess Is Nothing Then
db.AddToProcesses(p)
db.SaveChanges()
End If
I have a table called Quiz that have these fields
id As Int
created As DateTime
header As Sring
body As String
fk_parent As int
url As String
All the items without parent key would be the question, and ones that have the parent key would be the answer. I am having the problem to get all the latest active questions (based both on questions created time and and answer created time).
I am struggling to write a Linq query that can do the above task.
Here's a start:
IQueryable<int> keys =
from x in dc.Quiz
let masterID = ParentId.HasValue ? ParentId.Value : Id
group x by masterID into g
order g by g.Max(x => x.created) descending
select g.Key
List<Quiz> = dc.Quiz.Where(x => keys.Take(20).Contains(x.Id)).ToList();
This assumes answers aren't parents of answers... If they can - you have an arbitrary depth tree walk on your hands, which is a wrong shaped nail for Linq and for SQL.
You could try joining the table on itself in LINQ and creating a new object that holds the Questions and answers:
var QandA = from q in tbl
join a in tbl on q.id equals a.fk_parent
select new {
QHeader = q.header,
QBody = q.body,
QUrl = q.url,
AHeader = a.header,
ABody = a.body,
AUrl = a.url
};
I think this is how your table is setup, but I might have the join wrong.