Simple.Data join without where clause including any primary table columns - simple.data

I am using Simple.Data and have been trying to find an example that will let me do a join with the only condition in the WHERE clause be from the joined table. All of the examples I have seen always have at least one column in the primary table included in the WHERE. Take for example the following data:
private void TestSetup()
{
var adapter = new InMemoryAdapter();
adapter.SetKeyColumn("Events", "Id");
adapter.SetAutoIncrementColumn("Events", "Id");
adapter.SetKeyColumn("Doors", "Id");
adapter.SetAutoIncrementColumn("Doors", "Id");
adapter.Join.Master("Events", "Id").Detail("Doors", "EventId");
Database.UseMockAdapter(adapter);
db.Events.Insert(Id: 1, Code: "CodeMash2013", Name: "CodeMash 2013");
db.Events.Insert(Id: 2, Code: "SomewhereElse", Name: "Some Other Conf");
db.Doors.Insert(Id: 1, Code: "F7E08AC9-5E75-417D-A7AA-60E88B5B99AD", EventID: 1);
db.Doors.Insert(Id: 2, Code: "0631C802-2748-4C63-A6D9-CE8C803002EB", EventID: 1);
db.Doors.Insert(Id: 3, Code: "281ED88F-677D-49B9-84FA-4FAE022BBC73", EventID: 1);
db.Doors.Insert(Id: 4, Code: "9DF7E964-1ECE-42E3-8211-1F2BF7054A0D", EventID: 2);
db.Doors.Insert(Id: 5, Code: "9418123D-312A-4E8C-8807-59F0A63F43B9", EventID: 2);
}
I am trying to figure out the syntax I need to use in Simple.Data to get something similar to this T-SQL:
SELECT d.Code FROM Doors AS d INNER JOIN Events AS e ON d.EventID = e.Id WHERE e.Code = #EventCode
The final result should be only the three Door rows for EventId 1 when I pass in an event code of "CodeMash2013". Thanks!

First, a general point: since you've got criteria against the joined Events table, the LEFT OUTER is redundant; only rows with matching Event Codes will be returned, which implies only those rows where the join from Doors to Events was successful.
If you've got referential integrity set up in your database, with a foreign key relationship from Doors to Events, then Simple.Data can handle joins automatically. With that in mind, this code will work, both with the InMemoryAdapter and SQL Server:
List<dynamic> actual = db.Doors.FindAll(db.Doors.Events.Code == "CodeMash2013")
.Select(db.Doors.Id, db.Events.Name)
.ToList();
Assert.AreEqual(3, actual.Count);
If you don't have referential integrity set up then you should, but if you can't for some reason, then the following will work with SQL Server, but will trigger a bug in the InMemoryAdapter that I've just fixed but haven't done a release for yet:
dynamic eventAlias;
List<dynamic> actual = db.Doors.All()
.Join(db.Events, out eventAlias)
.On(db.Doors.EventID == eventAlias.Id)
.Select(db.Doors.Id, db.Events.Name)
.Where(eventAlias.Code == eventCode)
.ToList();
Assert.AreEqual(3, actual.Count);

UPDATE : This answer applies when using the Simple.Data SQL Server Provider, not the InMemoryAdapter
You could probably use the following for this:
db.Doors.All()
.Select(
db.Doors.Code)
.LeftJoin(db.Events).On(db.Doors.EventID == db.Events.Id)
.Where(db.Events.Code == eventCode);
You might need to experiment between using LeftJoin and OuterJoin depending on your provider. If you're using the ADO provider for example, the two functions both generate LEFT JOIN statements as LEFT JOIN and LEFT OUTER JOIN are synonymous in t-sql.
If you need to use aliases for some reason, the syntax is slightly different.
dynamic EventAlias;
db.Doors.All()
.LeftJoin(db.Events.As"e", out EventAlias).On(db.Doors.EventID == db.EventAlias.Id)
.Select(
db.Doors.Code)
.Where(db.EventAlias.Code == eventCode);
There's no reason why a where clause must contain a field from the primary key table. You can find more examples of Joins here on the Simple.Data doc site. Click Explicit Joins when you get there.

Related

Entity Framework using SQL Server datastore modelling a one-to-many relationship

Can I get some help diagnosing the following?
IQueryable<MyComponentsViewModel> vcvm = from v1 in pg.MyComponents
join item in Items on v1.Item equals item into vc
select new MyComponentsViewModel
{ Cost = item.Cost };
The following error appears:
Error CS1941: The type of one of the expressions in the join clause is
incorrect. Type inference failed in the call to 'GroupJoin'.
Is this error state occuring as item is not the same type as v1.Item?
The keys are shown below
ALTER TABLE [dbo].[MyComponents] WITH CHECK ADD CONSTRAINT
[FK_MyComponents_Items] FOREIGN KEY([ItemId])
REFERENCES [dbo].[Items] ([Itemid])
GO
ALTER TABLE [dbo].[MyComponents] CHECK CONSTRAINT [FK_MyComponents_Items]
I assume like, you need inner join and while you doing equals you need to use their matching column Ids to filter records like below
IQueryable<MyComponentsViewModel> vcvm =
from myComp in pg.MyComponents
join item in Items on myComp.ItemId equals item.Id
select new MyComponentsViewModel { Cost = item.Cost };
I was able to get it to work.
There was an ambiguity between an action named Items. Fully specify the namespace resolved this.
Additionally I had to add the statement "from subitem in vc.DefaultIfEmpty"
I'm still working to learn the full semantics of this
IQueryable<MyComponentsViewModel> vcvm =
from v1 in pg.MyComponents
join item in pg.Items on v1.Item equals item into vc
from subitem in vc.DefaultIfEmpty()
select new MyComponentsViewModel { Cost = subitem.Cost };

LINQ Entities Where clause not in correct place

Apparently I'm missing something with how LINQ to entities works. Hopefully one of you all can educate me.
Please try the below locally and let me know if you are seeing the same results. Something is really strange here...
Lets look at a very simple LINQ expression using navigation properties.
This was generated in LinqPad in a C# statement.
var result = (from ge in group_execution
where ge.automation_sequences.project.client_id == 1 && ge.parent_group_exec_id != null
select new
{
ge.id,
ge.parent_group_exec_id,
ge.automation_sequences.project.client_id
});
result.Dump();
OR, we can use joins...which will lead to the same bad results, but lets continue...
var result = (from ge in group_execution
join aseq in automation_sequences on ge.automation_sequence_id equals aseq.id
join p in project on aseq.project_id equals p.id
where p.client_id == 1 && ge.parent_group_exec_id != null
select new
{
ge.id,
ge.parent_group_exec_id,
p.client_id
});
result.Dump();
These very simple LINQ expressions generate the following SQL:
SELECT
[Filter1].[id1] AS [id],
[Filter1].[parent_group_exec_id] AS [parent_group_exec_id],
[Extent5].[client_id] AS [client_id]
FROM (SELECT [Extent1].[id] AS [id1], [Extent1].[automation_sequence_id] AS [automation_sequence_id], [Extent1].[parent_group_exec_id] AS [parent_group_exec_id]
FROM [dbo].[group_execution] AS [Extent1]
INNER JOIN [dbo].[automation_sequences] AS [Extent2] ON [Extent1].[automation_sequence_id] = [Extent2].[id]
INNER JOIN [dbo].[project] AS [Extent3] ON [Extent2].[project_id] = [Extent3].[id]
WHERE ([Extent1].[parent_group_exec_id] IS NOT NULL) AND (1 = [Extent3].[client_id]) ) AS [Filter1]
LEFT OUTER JOIN [dbo].[automation_sequences] AS [Extent4] ON [Filter1].[automation_sequence_id] = [Extent4].[id]
LEFT OUTER JOIN [dbo].[project] AS [Extent5] ON [Extent4].[project_id] = [Extent5].[id]
This baffles me. For the life of me I can't understand why LINQ is doing this. It's horrible, just look at the execution plan:
Now lets manually clean this up in SSMS and view the correct SQL and execution plan:
Much better, but how do we get LINQ to act this way?
Is anyone else seeing this? Has anyone else ever saw this and corrected it and if so how?
Thanks for looking into this.
UPDATE, attempting Chris Schaller fix:
var result = (from ge in group_execution
select new
{
ge.id,
ge.parent_group_exec_id,
ge.automation_sequences.project.client_id
}).Where(x=>x.client_id == 1 && x.parent_group_exec_id != null);
result.Dump();
Just so you all know I'm monitoring the SQL through SQL Server Profiler. If anyone knows of any issues doing it this way let me know.
UPDATE, a fix for JOINS, but not nav properties, and a cause, but why?
Here's your solution:
var result = (from ge in group_execution.Where(x=>x.parent_group_exec_id != null)
join aseq in automation_sequences on ge.automation_sequence_id equals aseq.id
join p in project on aseq.project_id equals p.id
where p.client_id == 1// && ge.parent_group_exec_id != null
select new
{
ge.id,
ge.parent_group_exec_id,
p.client_id
});
result.Dump();
Null checks shouldn't cause the framework to mess up like this. Why should I have to write it this way? This just seems like a defect to me in the framework. It will make my dynamic expressions a little bit more difficult to write, but maybe I can find a way.
Navigation Properties still mess up...so I'm still really sad. Picture below:
var result = (from ge in group_execution.Where(x=>x.parent_group_exec_id != null)
where ge.automation_sequences.project.client_id == 1// && ge.parent_group_exec_id != null
select new
{
ge.id,
ge.parent_group_exec_id,
ge.automation_sequences.project.client_id
});
result.Dump();
move your where clause to after you have defined the structure of the select statement
var result = (from ge in group_execution
select new
{
ge.id,
ge.parent_group_exec_id,
ge.automation_sequences.project.client_id
}).Where(x => x.client_id == 1 && x.parent_group_exec_id != null)
result.Dump();
Remember that Linq-to-entities flattens the results of queries to execute as SQL and then hydrates the object graph from those results.
When your query uses navigation properties or joins the query parser has to allow for zero results from those sub queries (Extents) to make sure that all columns that are required in the output and any interim processing are represented. By explicitly specifying a filter on a table for != null early in the query the parser knows that there is no further possibility that the field and any relationships linked by that field will be null, until then the parser prepares the query as if the joins will return null results
It is worth checking, but i wonder if UseDatabaseNullSemantics has anything to do with this?
Try:
dbContext.Configuration.UseDatabaseNullSemantics = false
In Linq, we can specify Where clauses as often as we like, improve the resulting SQL we should filter early in the query and granularly.
The parser engine is optimized to follow and implement your query sequentially, and generate good SQL at the end of it. Don't try to write linq-to-entities the same way that you structure your SQL, I know it's counter intuitive because the syntax is similar
A good technique is to assume that before each clause all the records from the previous statements have been loaded into memory, and that the next operation will affect all of those records. So you want to reduce the records before each additional operation by specifying a filter before moving on to the next clause
In general, if you have a filter condition based on the root table, apply this to the query before define all other joins and filters and even selects, you will get much cleaner sql.

Efficient joining the most recent record from another table in Entity Framework Core

I am comming to ASP .NET Core from PHP w/ MySQL.
The problem:
For the illustration, suppose the following two tables:
T: {ID, Description, FK} and States: {ID, ID_T, Time, State}. There is 1:n relationship between them (ID_T references T.ID).
I need all the records from T with some specific value of FK (lets say 1) along with the related newest record in States (if any).
In terms of SQL it can be written as:
SELECT T.ID, T.Description, COALESCE(s.State, 0) AS 'State' FROM T
LEFT JOIN (
SELECT ID_T, MAX(Time) AS 'Time'
FROM States
GROUP BY ID_T
) AS sub ON T.ID = sub.ID_T
LEFT JOIN States AS s ON T.ID = s.ID_T AND sub.Time = s.Time
WHERE FK = 1
I am struggling to write an efficient equivalent query in LINQ (or the fluent API). The best working solution I've got so far is:
from t in _context.T
where t.FK == 1
join s in _context.States on t.ID equals o.ID_T into _s
from s in _s.DefaultIfEmpty()
let x = new
{
id = t.ID,
time = s == null ? null : (DateTime?)s.Time,
state = s == null ? false : s.State
}
group x by x.id into x
select x.OrderByDescending(g => g.time).First();
When I check the resulting SQL query in the output window when executed it is just like:
SELECT [t].[ID], [t].[Description], [t].[FK], [s].[ID], [s].[ID_T], [s].[Time], [s].[State]
FROM [T] AS [t]
LEFT JOIN [States] AS [s] ON [T].[ID] = [s].[ID_T]
WHERE [t].[FK] = 1
ORDER BY [t].[ID]
Not only it selects more columns than I need (in the real scheme there are more of them). There is no grouping in the query so I suppose it selects everything from the DB (and States is going to be huge) and the grouping/filtering is happening outside the DB.
The questions:
What would you do?
Is there an efficient query in LINQ / Fluent API?
If not, what workarounds can be used?
Raw SQL ruins the concept of abstracting from a specific DB technology and its use is very clunky in current Entity Framework Core (but maybe its the best solution).
To me, this looks like a good example for using a database view - again, not really supported by Entity Framework Core (but maybe its the best solution).
What happens if you try to do a more straight forward translation to LINQ?
var latestState = from s in _context.States
group s by s.ID_T into sg
select new { ID_T = sg.Key, Time = sg.Time.Max() };
var ans = from t in _context.T
where t.FK == 1
join sub in latestState on t.ID equals sub.ID_T into subj
from sub in subj.DefaultIfEmpty()
join s in _context.States on new { t.ID, sub.Time } equals new { s.ID, s.Time } into sj
from s in sj.DefaultIfEmpty()
select new { t.ID, t.Description, State = (s == null ? 0 : s.State) };
Apparently the ?? operator will translate to COALESCE and may handle an empty table properly, so you could replace the select with:
select new { t.ID, t.Description, State = s.State ?? 0 };
OK. Reading this article (almost a year old now), Smit's comment to the original question and other sources, it seems that EF Core is not really production ready yet. It is not able to translate grouping to SQL and therefore it is performed on the client side, which may be (and in my case would be) a serious problem. It corresponds to the observed behavior (the generated SQL query does no grouping and selects everything in all groups). Trying the LINQ queries out in Linqpad it always translates to a single SQL query.
I have downgraded to EF6 following this article. It required some changes in my model's code and some queries. After changing .First() to .FirstOrDefault() in my original LINQ query it works fine and translates to a single SQL query selecting only the needed columns. The generated query is much more complex than it is needed, though.
Using a query from NetMage's answer (after small fixes), it results in a SQL query almost identical to my own original SQL query (there's only a more complex construct than COALESCE).
var latestState = from s in _context.States
group s by s.ID_T into sg
select new { ID = sg.Key, Time = sg.Time.Max() };
var ans = from t in _context.T
where t.FK == 1
join sub in latestState on t.ID equals sub.ID into subj
from sub in subj.DefaultIfEmpty()
join s in _context.States
on new { ID_T = t.ID, sub.Time } equals new { s.ID_T, s.Time }
into sj
from s in sj.DefaultIfEmpty()
select new { t.ID, t.Description, State = (s == null ? false : s.State) };
In LINQ it's not as elegant as my original SQL query but semantically it's the same and it does more or less the same thing on the DB side.
In EF6 it is also much more convenient to use arbitrary raw SQL queries and AFAIK also the database views.
The biggest downside of this approach is that full .NET framework has to be targeted, EF6 is not compatible with .NET Core.

nature of SELECT query in MVC and LINQ TO SQL

i am bit confused by the nature and working of query , I tried to access database which contains each name more than once having same EMPid so when i accessed it in my DROP DOWN LIST then same repetition was in there too so i tried to remove repetition by putting DISTINCT in query but that didn't work but later i modified it another way and that worked but WHY THAT WORKED, I DON'T UNDERSTAND ?
QUERY THAT DIDN'T WORK
var names = (from n in DataContext.EmployeeAtds select n).Distinct();
QUERY THAT WORKED of which i don't know how ?
var names = (from n in DataContext.EmployeeAtds select new {n.EmplID, n.EmplName}).Distinct();
why 2nd worked exactly like i wanted (picking each name 1 time)
i'm using mvc 3 and linq to sql and i am newbie.
Both queries are different. I am explaining you both query in SQL that will help you in understanding both queries.
Your first query is:
var names = (from n in DataContext.EmployeeAtds select n).Distinct();
SQL:-
SELECT DISTINCT [t0].[EmplID], [t0].[EmplName], [t0].[Dept]
FROM [EmployeeAtd] AS [t0]
Your second query is:
(from n in EmployeeAtds select new {n.EmplID, n.EmplName}).Distinct()
SQL:-
SELECT DISTINCT [t0].[EmplID], [t0].[EmplName] FROM [EmployeeAtd] AS
[t0]
Now you can see SQL query for both queries. First query is showing that you are implementing Distinct on all columns of table but in second query you are implementing distinct only on required columns so it is giving you desired result.
As per Scott Allen's Explanation
var names = (from n in DataContext.EmployeeAtds select n).Distinct();
The docs for Distinct are clear – the method uses the default equality comparer to test for equality, and the default comparer sees 4 distinct object references. One way to get around this would be to use the overloaded version of Distinct that accepts a custom IEqualityComparer.
var names = (from n in DataContext.EmployeeAtds select new {n.EmplID, n.EmplName}).Distinct();
Turns out the C# compiler overrides Equals and GetHashCode for anonymous types. The implementation of the two overridden methods uses all the public properties on the type to compute an object's hash code and test for equality. If two objects of the same anonymous type have all the same values for their properties – the objects are equal. This is a safe strategy since anonymously typed objects are essentially immutable (all the properties are read-only).
Try this:
var names = DataContext.EmployeeAtds.Select(x => x.EmplName).Distinct().ToList();
Update:
var names = DataContext.EmployeeAtds
.GroupBy(x => x.EmplID)
.Select(g => new { EmplID = g.Key, EmplName = g.FirstOrDefault().EmplName })
.ToList();

linq to entity join on string equals int

I saw this post Linq int to string, and tried it
var personalInfoQuery = from t in crnnsupContext.Tombstones
join i in crnnsupContext.InitialEducations on t.InitialEducation equals SqlFunctions.StringConvert((double)i.InitalEducationID)
where t.RegNumber == 25952
select new CPersonalInfo
{
Tombstone = t,
InitialEducation = i
};
in the database t.InitialEducation is char, i.InitalEducationID is int, but the retrieved result is null. I am pretty sure the value is not empty in the SQL server. So I think the problem is SqlFunctions.StringConvert((double)i.InitalEducationID)
when i remove the join statement, it got this person's information.
Does anyone know why. thanks
Finally find the reason!!
t.InitialEducation is nvarchar(1) in the database, i.InitalEducationID is int, after I modified to SqlFunctions.StringConvert((double)i.InitialEducationID, 1) it works!
"1" is the length of the returned string, the default length is 10, I guess there are some extra space.
Since you're using the SqlFunctions.StringConvert method, I'm assuming you're using EF as the underlying LINQ provider, no?
From the information given, it would appear that you're looking to do a 1 to many join on those properties. The code you've written could coalesce (not enough info on the context to be certain) as an INNER JOIN in SQL, so to force the LEFT JOIN behavior, you can add a .DefaultIfEmpty() call:
crnnsupContext.InitialEducations.DefaultIfEmpty()

Resources