As far as I know, doctrine dql does not allow subqueries inside a join.
I have a table and a trasnlation table. The relation is one to many. One record has many translations.
In order to get the right translation row from the translatiosn table I did subselects in the select clause:
$query = $this->getDoctrine()->getEntityManager()
->createQuery('
SELECT w.id, w.pastid, w.name, w.jsonParameters as params, m.id as milestone_id, m.name as milestone_name,
m.slug as milestone_slug, m.startdate as milestone_start, m.enddate as milestone_end,
w.expand as expand, w.backgroundcolor as background, w.colorschema as colorschema, w.headline as headline, w.subheadline as subheadline, w.text as text, w.expandheight as expandheight,
w.url as url, w.created as created, w.updated as updated, wt.name as wtname, ws.weight as width, ws.height as height
, (SELECT t.headline FROM AdminBundle:widgetTranslation t WHERE t.widget = w.id and t.locale = :published) AS headline_trans
, (SELECT t2.subheadline FROM AdminBundle:widgetTranslation t2 WHERE t2.widget = w.id and t2.locale = :published) AS subheadline_trans
, (SELECT t3.text FROM AdminBundle:widgetTranslation t3 WHERE t3.widget = w.id and t3.locale = :published) AS text_trans
FROM AdminBundle:Widget w
JOIN w.milestone m
JOIN w.widgetType wt
JOIN w.widgetShape ws
WHERE w.published = 1
ORDER BY m.order, w.order
')->setParameter('published', $currentLocale);
$result = $query->getArrayResult();
This query does the job but Iam worried about performance, is there a better query to do this?
First, I need to clarify a little thing.
There is something really wrong in your question.
You are using an ORM, right? But you're talking about tables.
An ORM doesn't care about how your tables are organized or linked together. It only matters on the relationship between the objects themselves.
However, if you are worried about performance, the best thing to do, as you would do it with any SQL query, is to EXPLAIN it.
Since you're using Symfony 2.0, you may use the profiler (accessible in dev env) to see the EXPLAIN results of your query.
Make sure that is using index, and the number of rows to lookup is not too high.
Depending on your result you may need to use another strategy to fetch your translation or re-write your query in native SQL to allow more flexibility.
Related
I am comming to ASP .NET Core from PHP w/ MySQL.
The problem:
For the illustration, suppose the following two tables:
T: {ID, Description, FK} and States: {ID, ID_T, Time, State}. There is 1:n relationship between them (ID_T references T.ID).
I need all the records from T with some specific value of FK (lets say 1) along with the related newest record in States (if any).
In terms of SQL it can be written as:
SELECT T.ID, T.Description, COALESCE(s.State, 0) AS 'State' FROM T
LEFT JOIN (
SELECT ID_T, MAX(Time) AS 'Time'
FROM States
GROUP BY ID_T
) AS sub ON T.ID = sub.ID_T
LEFT JOIN States AS s ON T.ID = s.ID_T AND sub.Time = s.Time
WHERE FK = 1
I am struggling to write an efficient equivalent query in LINQ (or the fluent API). The best working solution I've got so far is:
from t in _context.T
where t.FK == 1
join s in _context.States on t.ID equals o.ID_T into _s
from s in _s.DefaultIfEmpty()
let x = new
{
id = t.ID,
time = s == null ? null : (DateTime?)s.Time,
state = s == null ? false : s.State
}
group x by x.id into x
select x.OrderByDescending(g => g.time).First();
When I check the resulting SQL query in the output window when executed it is just like:
SELECT [t].[ID], [t].[Description], [t].[FK], [s].[ID], [s].[ID_T], [s].[Time], [s].[State]
FROM [T] AS [t]
LEFT JOIN [States] AS [s] ON [T].[ID] = [s].[ID_T]
WHERE [t].[FK] = 1
ORDER BY [t].[ID]
Not only it selects more columns than I need (in the real scheme there are more of them). There is no grouping in the query so I suppose it selects everything from the DB (and States is going to be huge) and the grouping/filtering is happening outside the DB.
The questions:
What would you do?
Is there an efficient query in LINQ / Fluent API?
If not, what workarounds can be used?
Raw SQL ruins the concept of abstracting from a specific DB technology and its use is very clunky in current Entity Framework Core (but maybe its the best solution).
To me, this looks like a good example for using a database view - again, not really supported by Entity Framework Core (but maybe its the best solution).
What happens if you try to do a more straight forward translation to LINQ?
var latestState = from s in _context.States
group s by s.ID_T into sg
select new { ID_T = sg.Key, Time = sg.Time.Max() };
var ans = from t in _context.T
where t.FK == 1
join sub in latestState on t.ID equals sub.ID_T into subj
from sub in subj.DefaultIfEmpty()
join s in _context.States on new { t.ID, sub.Time } equals new { s.ID, s.Time } into sj
from s in sj.DefaultIfEmpty()
select new { t.ID, t.Description, State = (s == null ? 0 : s.State) };
Apparently the ?? operator will translate to COALESCE and may handle an empty table properly, so you could replace the select with:
select new { t.ID, t.Description, State = s.State ?? 0 };
OK. Reading this article (almost a year old now), Smit's comment to the original question and other sources, it seems that EF Core is not really production ready yet. It is not able to translate grouping to SQL and therefore it is performed on the client side, which may be (and in my case would be) a serious problem. It corresponds to the observed behavior (the generated SQL query does no grouping and selects everything in all groups). Trying the LINQ queries out in Linqpad it always translates to a single SQL query.
I have downgraded to EF6 following this article. It required some changes in my model's code and some queries. After changing .First() to .FirstOrDefault() in my original LINQ query it works fine and translates to a single SQL query selecting only the needed columns. The generated query is much more complex than it is needed, though.
Using a query from NetMage's answer (after small fixes), it results in a SQL query almost identical to my own original SQL query (there's only a more complex construct than COALESCE).
var latestState = from s in _context.States
group s by s.ID_T into sg
select new { ID = sg.Key, Time = sg.Time.Max() };
var ans = from t in _context.T
where t.FK == 1
join sub in latestState on t.ID equals sub.ID into subj
from sub in subj.DefaultIfEmpty()
join s in _context.States
on new { ID_T = t.ID, sub.Time } equals new { s.ID_T, s.Time }
into sj
from s in sj.DefaultIfEmpty()
select new { t.ID, t.Description, State = (s == null ? false : s.State) };
In LINQ it's not as elegant as my original SQL query but semantically it's the same and it does more or less the same thing on the DB side.
In EF6 it is also much more convenient to use arbitrary raw SQL queries and AFAIK also the database views.
The biggest downside of this approach is that full .NET framework has to be targeted, EF6 is not compatible with .NET Core.
I need help to convert a valid SQL query into createQueryBuilder. The problem I have is I don't know how to LEFT JOIN on a SELECT in the createQueryBuilder.
SELECT username, count(c.user_owner_id) as num_contact, a_g.name as
group_name
FROM `oro_user` as u
INNER JOIN `oro_user_access_group` as u_g on u.id=u_g.user_id
INNER JOIN `oro_access_group` as a_g on u_g.group_id=a_g.id
LEFT JOIN
(SELECT cc.user_owner_id
FROM `orocrm_contact` as cc
INNER JOIN`orocrm_contact_to_contact_grp` as cc_g on cc_g.contact_id=cc.id
INNER JOIN `orocrm_contact_group`
as c_g on cc_g.contact_group_id=c_g.id
WHERE c_g.label='New One' and cc.semester_contacted='2017A')
as c on u.id=c.user_owner_id
WHERE a_g.name='Full-timer' and u.enabled = 1 and u.gender='male'
GROUP BY u.id
ORDER BY num_contact
I have two queries below, I want user to LEFT JOIN the results from contact
$user = $this->em->getRepository('OroUserBundle:User')->createQueryBuilder('u')
->select('u.username')
->innerJoin('u.groups','g')
->andWhere('g.name = :group')
->setParameter('group', 'Full-timer')
->getQuery();
$contacts = $this->em->getRepository('OroContactBundle:Contact')->createQueryBuilder('c')
->select('c')
->innerJoin('c.groups','g')
->andWhere('g.label = :group')
->andWhere('c.semester_contacted = :sem')
->setParameter('group', 'New One')
->setParameter('sem', '2017A')
->setMaxResults(1)
->getQuery();
This is a pretty complex queries and since the ORM QueryBuilder works closer to your entities than the database I'm not sure if you can just "dump" the DQL into a ->join(). The good news with the DBAL QueryBuilder that works:
$dbalQueryBuilder
->from('user_table as u')
...
->join('u', '('.$otherDbalQueryBuilder->getSQL().')', 'c')
This is from memory so it might be a little different, but something like that.
With that you can get all the fields you require, but you won't get any entities. Luckily Doctrine provides ways to build entities from Native SQL using ResultSetMapping.
$userWithContacts = $entityManager->createNativeQuery(
$dbalQueryBuilder->getSQL(),
$yourResultSetMapping
);
I know this is will require more code and is probably not as nice as just using the ORM QueryBuilder, but I find it oftentimes to be the best way to deal with existing queries that need to be ported to ORM somehow.
Here is the SQL equivalent of what I expect to get from Doctrine:
SELECT c.* FROM comments c
LEFT JOIN articles a
ON a.id = c.articles_id OR a.translation = c.articles_id
WHERE c.published = 1 AND c.language = a.language
The problem is that I cannot make Doctrine to generate the JOIN operation with OR as it is supposed to be. If we execute query from the following QueryBuilder object:
$qb->select('c')
->from('Project:Comment', 'c')
->leftJoin('c.article', 'a', 'WITH', 'a = c.article OR a.translation = c.article')
->where('c.published = true AND c.language = a.language');
we receive the following SQL statement:
SELECT
...
FROM comments c0_
LEFT JOIN articles a0_ ON c0_.articles_id = a0_.id
AND (
a0_.id = c0_.articles_id OR
a0_.translation = c0_.profiles_id
)
WHERE c0_.published = 1 AND c0_.language = a0_.language
which is obviously not the same as the initial query, as WITH operator seems to add additional conditions to the basic one instead of replacing the whole condition.
Is there any way to force Doctrine to output exactly what I need? I know that I may use native SQL but I doubt that it will be as convenient as QueryBuilder. Maybe there is a way to extend Doctrine with normal JOIN ON implementation instead of this odd JOIN WITH?
Doctrine doesn't implement this because it is (as far as I understand at least) not considered optimized enough for SQL.
See this SO post for precisions.
What you intend to do could appearantly be done using Union and other types of Join.
Okay, so i've got a query that i've researched and researched how to get this to work and for the life of me i cant!... perhaps i'm just doing this incorrectly and the minimal information ive found..
I've got a table named timeclock setup.. which has a field: noteBy_id in it which is an id to the user the record belongs to...
What I need to do now, is for the management side of things in the system.. I anticipate more than 1 company using this timeclock system, and as such I need to filter the results based on the company id.. In the user table, i have a field named parentcompany_id
So, lets see if I can express in words what I need to do..
I need to Select * from timeclock and left join user.parentcompany_id where timeclock.daydate < :start and where u.parentcompany = :pid
where start is: `date('Y-m-d 00:00:00');
I've setup this query:
$em = $this->getDoctrine()->getEntityManager();
$start = date('Y-m-d 00:00:00');
$qb = $em->getRepository('EcsCrmBundle:TimeClock');
$qb = $qb->createQueryBuilder('t');
$query = $qb->select('t, u.parentcompany_id')
->from('timeclock', 't')
->leftJoin('Ecs\AgentManagerBundle\Entity\User', 'u', 'ON' 'u.id = t.noteBy_id AND u.parentcompany_id = :pid')
->where('t.daydate < :start')
->andWhere("t.noteBy_id != ''")
->setParameter('start', $start)
->setParameter('pid', $user->getParentcompany())
->getQuery();
$entities = $query->getArrayResult();
I've looked and looked and can't find a solution to the error that I get which is:
An exception has been thrown during the rendering of a template ("[Semantical Error] line 0, col 112 near 'u ON u.id = t.noteBy_id': Error: Identification Variable Ecs\AgentManagerBundle\Entity\User used in join path expression but was not defined before.") in EcsCrmBundle:TimeClock:manager.html.twig at line 5.
and the query that gets output is:
SELECT t, u.parentcompany_id FROM Ecs\CrmBundle\Entity\TimeClock t LEFT JOIN Ecs\AgentManagerBundle\Entity\User u ON u.id = t.noteBy_id AND u.parentcompany_id = :pid, timeclock t LEFT JOIN Ecs\AgentManagerBundle\Entity\User u ON u.id = t.noteBy_id AND u.parentcompany_id = :pid WHERE t.daydate < :start AND t.noteBy_id != ''
which under normal circumstances would work perfectly... but in this, it just doesn't... Any ideas?
I've recently had to do it like this.. I'm guessing in this that your noteBy is a ManyToOne in the user table and you are wanting to have it filter the results by the company of the admin that is currently logged into your system..
So, adapting a join I had to write myself for such a task is easy enough. I personally like to use the QueryBuilder so this will be done in query builder..
Your first mistake in your query is the ->from('timeclock', 't') line. Because you have previously created your object with $qb = $em->getRepository('EcsCrmBundle:TimeClock'); $qb = $qb->createQueryBuilder('t'); you don't need the from in the query builder, as it will be generated for you.
The next issue, is the leftJoin and I'll explain why when I've shown you a working version.
And the last issue, preventing this from working how you want it - is a missing andWhere clause. So, lets take a look at a working query.
$query = $qb->select('t, u')
->leftJoin('t.noteBy', 'u', 'WITH', 'u.id = t.noteBy')
->where('t.daydate < :start')
->andWhere('u.parentcompany = :pid')
->setParameter('start', $start)
->setParameter('pid', $user->getParentcompany())
->getQuery();
So because we've already created the object by using $qb = $qb->createQueryBuilder('t') we just select t and u
For the join, we're joining the timeclock table by the noteBy column, which is the user id from the user table. So, the first argument being the "from" alias. So, since we've aliased the timeclock table with t we use t.noteBy. The next argument in the leftjoin is the alias of the 2nd table, which is u in this case but can be anything.. The third argument for a leftJoin anyway - is the way you join it.. either a WITH or ON will work here. and the 4th argument, is the match you wish it to have.. in this case u.id must equal t.noteBy
You will see that I got rid of one of the andWhere, I did this because with the properly structured query you shouldn't need it. I did however add in the andWhere for the u.parentcompany since that is afterall what you are looking to filter by you should have it in a WHERE instead of as a match in the join itself.
The documentation is very limited in this, and it took me a while to figure it all out as well.. You, undoubtedly - like me, came to using doctrine from writing your queries by hand. And So since you seem to be just starting with Symfony (i am myself as well about 2 months in now), you're still in the hand-coding mindset. But with further time, you'll start understanding the DQL way of life. Try this query out and see what happens.
Ok, first you would need to relate entity Timeclock to Company. Whenever you want to join two entities in Doctrine they need to be related by some attribute (that is, table column).
I don't see any need for User entity in this query as all info is available through Company entity and you are not filtering down results based on any user properties.
You desired query should look something like this (more or less). I took liberty and ditched _id suffixes from entity attributes as they tend to cloud what is really going on. ;)
$query = $this->getEntityManager()->createQuery("SELECT t, c.id FROM EcsCrmBundle:TimeClock t JOIN t.company c WHERE c.id = :pid AND t.daydate < :start AND t.noteBy != ''");
$query->setParameter('start', $start);
$query->setParameter('pid', $user->getParentcompany());
return $query->getArrayResult();
Also, I did inner-join (JOIN) as I think there could not be timeclock without it's company but feel free to change that to LEFT JOIN if that suits you better.
Is this what you were trying to achieve?
I've got two un-related (no FK's defined) tables. The first table contains some tasks for which a user may not have access. I need to find all those tasks - in this case, the joined table would contain nulls. How do I get them?
Here's the setup:
TimeData table
- userID
- taskID
- hours
ApprovedTasks table (the one that should contain nulls)
- taskID
- userID
The SQL query would look like this:
select * from TimeData td
left join ApprovedTasks at
on at.taskID = td.taskID and at.userID = td.userID
where at.taskID is null
Any way to pull that off using a LINQ to Entity query?
TIA
Check out... Disjoint Union in LINQ
This should work...
var approvedTaks = from at in ApprovedTasks.Except(
from at2 in ApprovedTasks
where at2.userID == userId and at2.taskID==taskId
select at2)
where at.userID == userId and at.taskID==taskId
select at;
but sorry don't have the database handy to test it.