what's the problem with this ?
int folderid = (from p in db.folder where p.isDefault == true select p.id).Last();
i get this error
LINQ to Entities does not recognize the method 'Int32 Last[Int32]
(System.Linq.IQueryable`1[System.Int32])' method, and this method cannot be
translated into a store expression.
Linq can not translate the Last() to any valid sql statment. So my suggestion would be to orderby decending and Take(1)
Maybe something like this:
int? folderid =(
from p in db.folder
where p.isDefault == true
orderby p.id descending
select p.id
).Take(1).SingleOrDefault();
I don't know which to take so you might have to change the orderby p.id descending to something that suites you.
Related
Apparently I'm missing something with how LINQ to entities works. Hopefully one of you all can educate me.
Please try the below locally and let me know if you are seeing the same results. Something is really strange here...
Lets look at a very simple LINQ expression using navigation properties.
This was generated in LinqPad in a C# statement.
var result = (from ge in group_execution
where ge.automation_sequences.project.client_id == 1 && ge.parent_group_exec_id != null
select new
{
ge.id,
ge.parent_group_exec_id,
ge.automation_sequences.project.client_id
});
result.Dump();
OR, we can use joins...which will lead to the same bad results, but lets continue...
var result = (from ge in group_execution
join aseq in automation_sequences on ge.automation_sequence_id equals aseq.id
join p in project on aseq.project_id equals p.id
where p.client_id == 1 && ge.parent_group_exec_id != null
select new
{
ge.id,
ge.parent_group_exec_id,
p.client_id
});
result.Dump();
These very simple LINQ expressions generate the following SQL:
SELECT
[Filter1].[id1] AS [id],
[Filter1].[parent_group_exec_id] AS [parent_group_exec_id],
[Extent5].[client_id] AS [client_id]
FROM (SELECT [Extent1].[id] AS [id1], [Extent1].[automation_sequence_id] AS [automation_sequence_id], [Extent1].[parent_group_exec_id] AS [parent_group_exec_id]
FROM [dbo].[group_execution] AS [Extent1]
INNER JOIN [dbo].[automation_sequences] AS [Extent2] ON [Extent1].[automation_sequence_id] = [Extent2].[id]
INNER JOIN [dbo].[project] AS [Extent3] ON [Extent2].[project_id] = [Extent3].[id]
WHERE ([Extent1].[parent_group_exec_id] IS NOT NULL) AND (1 = [Extent3].[client_id]) ) AS [Filter1]
LEFT OUTER JOIN [dbo].[automation_sequences] AS [Extent4] ON [Filter1].[automation_sequence_id] = [Extent4].[id]
LEFT OUTER JOIN [dbo].[project] AS [Extent5] ON [Extent4].[project_id] = [Extent5].[id]
This baffles me. For the life of me I can't understand why LINQ is doing this. It's horrible, just look at the execution plan:
Now lets manually clean this up in SSMS and view the correct SQL and execution plan:
Much better, but how do we get LINQ to act this way?
Is anyone else seeing this? Has anyone else ever saw this and corrected it and if so how?
Thanks for looking into this.
UPDATE, attempting Chris Schaller fix:
var result = (from ge in group_execution
select new
{
ge.id,
ge.parent_group_exec_id,
ge.automation_sequences.project.client_id
}).Where(x=>x.client_id == 1 && x.parent_group_exec_id != null);
result.Dump();
Just so you all know I'm monitoring the SQL through SQL Server Profiler. If anyone knows of any issues doing it this way let me know.
UPDATE, a fix for JOINS, but not nav properties, and a cause, but why?
Here's your solution:
var result = (from ge in group_execution.Where(x=>x.parent_group_exec_id != null)
join aseq in automation_sequences on ge.automation_sequence_id equals aseq.id
join p in project on aseq.project_id equals p.id
where p.client_id == 1// && ge.parent_group_exec_id != null
select new
{
ge.id,
ge.parent_group_exec_id,
p.client_id
});
result.Dump();
Null checks shouldn't cause the framework to mess up like this. Why should I have to write it this way? This just seems like a defect to me in the framework. It will make my dynamic expressions a little bit more difficult to write, but maybe I can find a way.
Navigation Properties still mess up...so I'm still really sad. Picture below:
var result = (from ge in group_execution.Where(x=>x.parent_group_exec_id != null)
where ge.automation_sequences.project.client_id == 1// && ge.parent_group_exec_id != null
select new
{
ge.id,
ge.parent_group_exec_id,
ge.automation_sequences.project.client_id
});
result.Dump();
move your where clause to after you have defined the structure of the select statement
var result = (from ge in group_execution
select new
{
ge.id,
ge.parent_group_exec_id,
ge.automation_sequences.project.client_id
}).Where(x => x.client_id == 1 && x.parent_group_exec_id != null)
result.Dump();
Remember that Linq-to-entities flattens the results of queries to execute as SQL and then hydrates the object graph from those results.
When your query uses navigation properties or joins the query parser has to allow for zero results from those sub queries (Extents) to make sure that all columns that are required in the output and any interim processing are represented. By explicitly specifying a filter on a table for != null early in the query the parser knows that there is no further possibility that the field and any relationships linked by that field will be null, until then the parser prepares the query as if the joins will return null results
It is worth checking, but i wonder if UseDatabaseNullSemantics has anything to do with this?
Try:
dbContext.Configuration.UseDatabaseNullSemantics = false
In Linq, we can specify Where clauses as often as we like, improve the resulting SQL we should filter early in the query and granularly.
The parser engine is optimized to follow and implement your query sequentially, and generate good SQL at the end of it. Don't try to write linq-to-entities the same way that you structure your SQL, I know it's counter intuitive because the syntax is similar
A good technique is to assume that before each clause all the records from the previous statements have been loaded into memory, and that the next operation will affect all of those records. So you want to reduce the records before each additional operation by specifying a filter before moving on to the next clause
In general, if you have a filter condition based on the root table, apply this to the query before define all other joins and filters and even selects, you will get much cleaner sql.
I've got a complex query written in SQL which works.
SELECT Instruments.Id as [Id], Instruments.ShareCode as [Share Code], Instruments.Name AS [Short Name], Instruments.Description as [Share Name],
InstrumentGames.Instrument_Id,
InstrumentGames.Game_Id, Games.Name AS [Game Name],
Entries.Name AS [Entry Name], AspNetUsers.UserName, AspNetUsers.Id as [User_Id],
Sectors.Name AS Sector_Id, Sectors.ShortName AS Sector
FROM AspNetUsers INNER JOIN
Entries ON AspNetUsers.Id = Entries.User_Id INNER JOIN
Games ON Entries.Game_Id = Games.Id INNER JOIN
InstrumentGames ON Games.Id = InstrumentGames.Game_Id INNER JOIN
Instruments ON InstrumentGames.Instrument_Id = Instruments.Id INNER JOIN
Sectors ON Instruments.Sector_Id = Sectors.Id
WHERE Instruments.Listed = 'true' and InstrumentGames.Game_Id = 2 and Entries.User_Id = 'd28d6552-7d98-476c-82cb-063e7ef45cb6'
I'm using Entity code first models and trying to convert what I have in SQL to a linq query.
I've come up with:
public static Models.Instrument GetShare(string shareSearchCriteria,
Models.Game selectedGame,
string userId)
{
var _db = new JSEChallenge.Models.ApplicationDbContext();
var records = (from instru in _db.Instruments
from e in _db.Entries
where (instru.ShareCode.Contains(shareSearchCriteria) ||
instru.Name.Contains(shareSearchCriteria) ||
instru.Description.Contains(shareSearchCriteria))
where (instru.Listed == true &&
instru.Games.Contains(selectedGame) &&
e.User_Id == userId)
select instru).ToList();
return records.FirstOrDefault();
}
But I keep getting this error:
Unable to create a constant value of type 'X.Models.Game'. Only primitive types or enumeration types are supported in this context.
I think the issue is the m2m table InstrumentGames. In my SQL query I can join it easily but in my C# I cannot. The way I usually find m2m records is syntax like instru.Games.Contains(selectedGame)
Unfortunately I still cannot get this to work.
How do I implement this kind of query in Linq?
Not sure if it will work, but try instru.Games.Any(q => q.Id == selectedGame.Id) instead of instru.Games.Contains(selectedGame).
Hope this helps!
I have a Product Entity which has a ManyToMany relationship with a Taxon Entity. I want to find all the products that belong to the intersection of all Taxons. For instance I want to find all products that belong to taxons with IDs 1 and 2.
Products
{1,2,3}
Taxons
{1,2,3,4,5}
ProductsToTaxons
{{p1,t1},{p1,t2}, {p2,t2}}
I want to retrieve the following set ONLY when querying products for taxons 1 and 2:
Product
{1}
which is from {{p1,t1}, {p1,t2}}
Okay, So here is the DQL that i tried... but it doesn't work?
SELECT p FROM SRCProductBundle:Product p
JOIN p.taxons t
WHERE t.id = 1 AND t.id = 2
(P.S. I would also do this with QueryBuilder with as well)
EDIT
To clarify, here is the SQL that I would like to translate into DQL/QueryBuilder.
select p.id
from product p
where exists (select product_id
from product_to_taxon
where taxon_id = 1
and product_id = p.id)
and exists (select product_id
from product_to_taxon
where taxon_id = 4
and product_id = p.id);
You can use the MEMBER OF statement to achieve this although in my experience it hasn't produced very performant results with a ManyToMany relationship:
In DQL:
SELECT p FROM SRCProductBundle:Product p
WHERE 1 MEMBER OF p.taxons OR 2 MEMBER OF p.taxons
Or with Query builder
$this->createQueryBuilder('p')
->where(':taxon_ids MEMBER OF p.taxons')
->setParameter('taxon_ids', $taxonIdsArray)
->getQuery()
->getResult();
This will create SQL similar to the example provided although in my experience it still had a join in the EXISTS subqueries. Perhaps future versions of Doctrine can address this.
I think you want something like this:
$qb = $this
->createQueryBuilder('p')
->select('p.id')
;
$qb
->leftJoin('p.taxons', 'taxon1', Join::WITH, 'taxon1.id = :taxonId1')
->setParameter('taxonId1', 1)
->andWhere($qb->expr()->isNotNull('taxon1'))
->leftJoin('p.taxons', 'taxon2', Join::WITH, 'taxon2.id = :taxonId2')
->setParameter('taxonId2', 2)
->andWhere($qb->expr()->isNotNull('taxon2'))
;
Which is equivalent to the SQL:
SELECT p.id
FROM products p
LEFT JOIN taxons t1 ON (p.id = t1.product_id AND t1.id = 1)
LEFT JOIN taxons t2 ON (p.id = t2.product_id AND t2.id = 2)
WHERE t1.id IS NOT NULL
AND t2.id IS NOT NULL
;
Your DQL has wrong logic. You can't have a taxon with both id=1 and id=4. You could do it like this:
SELECT p FROM SRCProductBundle:Product p
JOIN p.taxons t
WHERE t.id = 1 OR t.id = 4
But I would prefer this way:
SELECT p FROM SRCProductBundle:Product p
JOIN p.taxons t
WHERE t.id IN (1, 4)
Using query builder that would look something like this, assuming you're in EntityRepository class:
$this->createQueryBuilder('p')
->join('p.taxons', 't')
->where('t.id IN :taxon_ids')
->setParameter('taxon_ids', $taxonIdsArray)
->getQuery()
->getResult();
For lack of a clean way to do this with DQL, and after a considerable amount of research, I resorted to doing this in Native SQL. Doctrine allows Native SQL via the EntityManager with createNativeQuery().
So in short, I utilized this ability and constructed the SQL query included in my question as a string and then passed it to the createNativeQuery() function.
This does appear to have some drawbacks as it appears I will be unable to use the KnpPaginatorBundle with it... So I might end up just filtering the results in PHP rather than SQL, which I'm hesitant to do as I think there are performance drawbacks.
I have two tables:
Phase :
long ID
string Name
and another Activity :
long ID
string Name
long PhaseID
I already know the name of the phases and I want to get the activity for those particular phases. Do i add PhaseName to the activity table or do I do it through join in LINQ?
Maybe something like this?
var query = from a in entities.Activities
join p in entities.Phases on a.PhaseId equals p.Id
where p.Name == "Preplanning"
... and here im not sure how to finish this query..
Thanks for your help!
The code that you've provided will use an Inner Join to find all Activities where the Phase with Name "Preplanning" exists.
To finish your query you need to add a select clause.
var query = from a in entities.Activities
join p in entities.Phases on a.PhaseId equals p.Id
where p.Name == "Preplanning"
select a.Name
will return IEnumerable<string> of all activity names.
Just select activity, as you want:
var query = from a in entities.Activities
join p in entities.Phases on a.PhaseId equals p.Id
where p.Name == "Preplanning"
select a;
Here is how query expression should look like:
A query expression must begin with a from clause and must end with a select or group clause. Between the first from clause and the last select or group clause, it can contain one or more of these optional clauses: where, orderby, join, let and even additional from clauses. You can also use the into keyword to enable the result of a join or group clause to serve as the source for additional query clauses in the same query expression.
Same as puzzling image:
With method syntax you don't need to end query with something special:
var query = entities.Phases
.Where(p => p.Name == "Preplanning")
.Join(entities.Activities, p => p.Id, a => a.PhaseId, (p,a) => a);
No need to do a join if you only need data from one of the tables. You can apply a filter instead:
var q = entities.Activities.Where(a =>
entities.Phases.Any(p => a.PhaseId == p.Id && p.Name == "Preplanning"));
I have this SQL query:
SELECT Sum(ABS([Minimum Installment])) AS SumOfMonthlyPayments FROM tblAccount
INNER JOIN tblAccountOwner ON tblAccount.[Creditor Registry ID] = tblAccountOwner.
[Creditor Registry ID] AND tblAccount.[Account No] = tblAccountOwner.[Account No]
WHERE (tblAccountOwner.[Account Owner Registry ID] = 731752693037116688)
AND (tblAccount.[Account Type] NOT IN
('CA00', 'CA01', 'CA03', 'CA04', 'CA02', 'PA00', 'PA01', 'PA02', 'PA03', 'PA04'))
AND (DATEDIFF(mm, tblAccount.[State Change Date], GETDATE()) <=
4 OR tblAccount.[State Change Date] IS NULL)
AND ((tblAccount.[Account Type] IN ('CL10','CL11','PL10','PL11')) OR
CONTAINS(tblAccount.[Account Type], 'Mortgage')) AND (tblAccount.[Account Status ID] <> 999)
I have created a Linq query:
var ownerRegistryId = 731752693037116688;
var excludeTypes = new[]
{
"CA00", "CA01", "CA03", "CA04", "CA02",
"PA00", "PA01", "PA02", "PA03", "PA04"
};
var maxStateChangeMonth = 4;
var excludeStatusId = 999;
var includeMortgage = new[] { "CL10", "CL11", "PL10", "PL11" };
var sum = (
from account in context.Accounts
from owner in account.AccountOwners
where owner.AccountOwnerRegistryId == ownerRegistryId
where !excludeTypes.Contains(account.AccountType)
where account.StateChangeDate == null ||
(account.StateChangeDate.Month - DateTime.Now.Month)
<= maxStateChangeMonth
where includeMortgage.Contains(account.AccountType) ||
account.AccountType.Contains("Mortgage")
where account.AccountStatusId != excludeStatusId
select account.MinimumInstallment).ToList()
.Sum(minimumInstallment =>
Math.Abs((decimal)(minimumInstallment)));
return sum;
Are they equal/same ? I dont have records in db so I cant confirm if they are equal. In SQL there are brackets() but in Linq I didnt use them so is it ok?
Please suggest.
It is not possible for us to say anything about this, because you didn't show us the DBML. The actual definition of the mapping between the model and the database is important to be able to see how this executes.
But before you add the DBML to your question: we are not here to do your work, so here are two tips to find out whether they are equal or not:
Insert data in your database and run the queries.
Use a SQL profiler and see what query is executed by your LINQ provider under the covers.
If you have anything more specific to ask, we will be very willing to help.
The brackets will be generated by LINQ provider, if necessary.
The simplest way to check if the LINQ query is equal to the initial SQL query is to log it like #Atanas Korchev suggested.
If you are using Entity Framework, however, there is no Log property, but you can try to convert your query to an ObjectQuery, and call the ToTraceString method then:
string sqlQuery = (sum as ObjectQuery).ToTraceString();
UPD. The ToTraceString method needs an ObjectQuery instance for tracing, and the ToList() call already performs materialization, so there is nothing to trace. Here is the updated code:
var sum = (
from account in context.Accounts
from owner in account.AccountOwners
where owner.AccountOwnerRegistryId == ownerRegistryId
where !excludeTypes.Contains(account.AccountType)
where account.StateChangeDate == null ||
(account.StateChangeDate.Month - DateTime.Now.Month)
<= maxStateChangeMonth
where includeMortgage.Contains(account.AccountType) ||
account.AccountType.Contains("Mortgage")
where account.AccountStatusId != excludeStatusId
select account.MinimumInstallment);
string sqlQuery = (sum as ObjectQuery).ToTraceString();
Please note that this code will not perform the actual query, it is usable for testing purposes only.
Check out this article if you are interested in ready-for-production logging implementation.
There can be a performance difference:
The SQL query returns a single number (SELECT Sum...) directly from the database server to the client which executes the query.
In your LINQ query you have a greedy operator (.ToList()) in between:
var sum = (...
...
select account.MinimumInstallment).ToList()
.Sum(minimumInstallment =>
Math.Abs((decimal)(minimumInstallment)));
That means that the query on the SQL server does not contain the .Sum operation. The query returns a (potentially long?) list of MinimumInstallments. Then the .Sum operation is performed in memory on the client.
So effectively you switch from LINQ to Entities to LINQ to Objects after .ToList().
BTW: Can you check the last proposal in your previous question here which would avoid .ToList() on this query (if the proposal should work) and would therefore be closer to the SQL statement.