Mapping and DQL - symfony

I have 3 tables - user, area, and contacts. A contact can belong to a user or an area. A user can belong to many areas.
I want to pull all the contacts that belong to a user (as specifically defined in the DB), as well as all contacts that belong to the same area as the user.
Can I get a fresh set of eyes on my Database mapping, and the query I need to write in DQL to get what I want. Am I doing something wrong in my database mapping?
I'm definitely a SQL person, and am able to easily fetch what I want in plain SQL. In plain SQL, here's what I want to do:
select c.* from contact c LEFT JOIN user_area ua ON c.area_id=ua.area_id where (ua.user_id=XXX OR c.user_id=XXX);
USER
/**
* #ORM\ManyToMany(targetEntity="area", inversedBy="areas")
* #ORM\JoinTable(name="user_area",
* joinColumns={#ORM\JoinColumn(name="user_id", referencedColumnName="id")},
* inverseJoinColumns={#ORM\JoinColumn(name="area_id", referencedColumnName="id")}
* )
*/
private $areas;
/**
* #ORM\OneToMany(targetEntity="Contact", mappedBy="user")
*/
private $contacts;
CONTACT
/**
* #ORM\ManyToOne(targetEntity="Area")
* #ORM\JoinColumn(name="area_id", referencedColumnName="id")
*/
private $area;
/**
* #ORM\ManyToOne(targetEntity="User", inversedBy="Contacts")
* #ORM\JoinColumn(name="user_id", referencedColumnName="id")
*/
private $user;
AREA
/**
* #ORM\ManyToMany(targetEntity="User", mappedBy="users")
*/
private $users;
/**
* #ORM\OneToMany(targetEntity="Contact", mappedBy="area")
*/
private $contacts;
The main problem I'm running into is that DQL really wants you to query an object, and it's just plain easier in SQL to query the user/area relationship table to get what I want. I tried to write an query that pulls areas from contacts, then users from contacts, and then users from areas but I get an error message that "users" isn't a defined index in my areas object. Again, I'm a Doctrine newbie, so I'm probably doing something wrong.
Here's my attempt at a query, from the User object in Symfony:
$qb = $em->createQueryBuilder()
->addSelect('c')
->from('MyBundle:Contact', 'c')
->leftJoin('c.area', 'ca')
->leftJoin('c.user', 'cu')
->leftJoin('ca.users', 'cau')
->add('where', 'c.user = ?1 OR cau.id = ?1')
->add('orderBy', 'c.name')
->setParameter(1, $this->getId());

Someone should have slapped me for providing that previous answer. While it got the job done, I was absolutely right, it was not optimized. Queries using that method were taking 3 seconds to go back and forth to the database (3 seconds!). Clearly, there were plenty of other things going on in my world that took away from performance as a requirement for getting this done, but things have changed. I managed to break down this query into two smaller (Doctrine generated) ones, each taking maybe 0.2 or 0.3s.
$areas = $user->getAreas();
$qb = $this->getEntityManager()->createQueryBuilder();
$qb->select('c')
->from('MyBundle:Contact', 'c')
->where($qb->expr()->in('c.area', '?1'))
->orWhere('c.user = ?2')
->setParameter(1, $areas->toArray())
->setParameter(2, $user);
$query = $qb->getQuery();
$result = $query->getResult();
return $result;
The fact that I have to call $user->getAreas() adds a database query (if Doctrine doesn't already have that information), but this code, using Query Builder expressions, works much better (0.3s vs. 3s is 10% of the original query time!).
I think the main concept I was missing back then was that the Query Builder wants to work with your objects (Entities), and the properties you've defined in your entities. Coming from a strong SQL background, and knowing the specific SQL query I wanted Doctrine to produce, I wasn't approaching the problem properly.
Hope this update to an 8-month old question helps somebody!

So it turns out you can't fetch objects of objects in DQL. I needed to fetch all "Area"s (an object of "Contact") and then fetch all of that Area's "User"s.
In DQL, you can specify multiple "from()" helper methods, and this was what I needed to get the job done.
$qb = $em->createQueryBuilder()
->addSelect('c')
->from('MyBundle:Contact', 'c')
->from('MyBundle:User', 'u')
->leftJoin('c.area', 'ca')
->leftJoin('c.user', 'cu')
->leftJoin('u.areas', 'ua')
->add('where', 'c.user = ?1 OR (ua.id=ca.id AND u.id = ?1')
->add('orderBy', 'c.name')
->setParameter(1, $this->getId());
The resulting SQL generated from Doctrine doesn't seem particularly optimized, but it gets the job done. If anyone has any thoughts on getting Doctrine to better optimize the following query, I'd love to hear opinions.
SELECT m0_.id AS id0, m0_.name AS name1, m0_.email AS email2, m0_.media_area_id AS media_area_id3, m0_.user_id AS user_id4 FROM contact m0_ LEFT JOIN user u1_ ON m0_.user_id = u1_.id LEFT JOIN area m2_ ON m0_.area_id = m2_.id, user u3_ LEFT JOIN user_area u5_ ON u3_.id = u5_.user_id LEFT JOIN area m4_ ON m4_.id = u5_.area_id WHERE u1_.id = ? OR (m4_.id = m2_.id AND u3_.id = ? );

Related

Doctrine OneToOne always loaded on querybuilder

I have Partner entity with two relation:
/**
* #var PartnerSettings
* #ORM\OneToOne(targetEntity="PartnerSettings", mappedBy="partner", cascade={"persist", "remove"}, fetch="LAZY")
*/
private $settings;
/**
* #var PartnerRating
* #ORM\OneToOne(targetEntity="PartnerRating", mappedBy="partner", cascade={"persist", "remove"}, fetch="LAZY")
*/
private $rating;
...getRepository(Partner::class)->findAll() work correctly, one query was made,but when I create queryBuilder:
return $this->createQueryBuilder('p')
->getQuery()
->getResult();
doctrine make 31 queries(i have 10 partners)... in debug toolbar i saw select queries to settings and rating for every partner. I don't want it in this case.
Additionally, in every querybuilder where I used join to partners, setting and rating are selected too.
answer
->setHint(Query::HINT_FORCE_PARTIAL_LOAD, true);
did the job
This behaviour is known as the N + 1 selects problem. To reduce database queries, you might want to consider the following approach.
First, retrieve all partners:
$partners = $em->createQueryBuilder()
->select("p")
->from("Parent", "p")
->where(/*...*/)
->setParameter(/*...*/)
->indexBy("p.id")
->getQuery()->getResult();
Now load all children at once, in two queries:
$settings = $em->createQueryBuilder()
->select("s")
->from("PartnerSetting", "s")
->where("IDENTITY(s.partner) IN (?1)")
->setParameter(1, array_keys($partners))
->getQuery()->getResult();
$ratings = $em->createQueryBuilder()
->select("r")
->from("PartnerRating", "r")
->where("IDENTITY(r.partner) IN (?1)")
->setParameter(1, array_keys($partners))
->getQuery()->getResult();
Doctrine will now have all of the retrieved entities are stored in memory. So when, for example, you do a $parnter->getRatings(), you don’t trigger a new DB query, instead the entity is filled from memory.
just define the table AND the relation in the select
$qb->select('g', 'gi');
and now it works
Just mapped relation: #ORM\OneToOne(targetEntity="ENTITY", mappedBy="MAPPEDBY", fetch="EAGER")

Doctrine get all entities from collection

I don't know if what I am trying is really possible. So I thought to ask it to you guys.
What I am trying to do:
get a set of companies
get all the users associated with the given companies
In code:
$companyIds = array(1,2,3);
$companies = $this->em->getRepository('AppBundle:Company')->findById($companyIds);
dump($companies->getUsers()); // this will not work, but I like it to work
Where they are associated as follows:
class User implements AdvancedUserInterface, \Serializable
{
/**
* #ORM\ManyToOne(targetEntity="AppBundle\Entity\Company", inversedBy="users")
* #ORM\JoinColumn(name="company_id", referencedColumnName="id", nullable=false)
*/
private $company;
}
class Company
{
/**
* #ORM\OneToMany(targetEntity="AppBundle\Entity\User", mappedBy="company")
*/
private $users;
}
Repository returns you an ArrayCollection of entities, not a single entity, therefore you need to access each of them separately.
This should work:
foreach($companies as $company) {
$company->getUsers();
}
The problem with the above is that by default it will fetch (lazy load) users from database for each company in separate query (on demand when calling getUsers), which would be very inefficient on larger scale.
There are couple possible solution depending on your needs.
You could configure doctrine to always fetch users with companies, which is called eager fetch.
Having fetched users, you can merge ArrayCollections (and remove duplicates if needed), to achieve single collection containing all users.
Other way could be to fetch companies with users by creating sufficient DQL query in a custom method of you company's repository. If you need only users and don't need companies, then it could be a query that only fetches users without companies.
Try something like this in your User-Repository:
public function getAllUsersFromCompanies($ids)
{
$qb = $this->createQueryBuilder('u');
$qb->leftJoin('u.company', 'c')
->where('c.id IN :ids')
->setParameter('ids', $ids)
->getQuery();
return $query->getResult();
}
We are joining the user table with the company table here, which gives us the company for each user. Then, we filter out every user, that has the wrong company.
You can e.g. fetch all Comapany entities with users with one query:
$companies = $em->createQueryBuilder()
->select('c, u')
->from('AppBundle:Company', 'c')
// or left join based on you needs
->innerJoin('c.users', 'u')
->getQuery()
->getResult();
This will not result in queries when fetching company users.

Doctrine 2: Cache in One-to-Many associations

I'm trying to use doctrine cache from Common package, but I can't get it working with one-to-many, many-to-one accosiations. I'll explain later what I want to do.
My configuration:
'configuration' => array(
'orm_default' => array(
'metadata_cache' => 'filesystem',
'query_cache' => 'filesystem',
'result_cache' => 'filesystem',
'hydration_cache' => 'filesystem',
)
),
My entity
class Category
{
/**
* #ORM\Id
* #ORM\GeneratedValue
* #ORM\Column(type="integer")
*
* #var string
*/
protected $id;
/**
* #var string
*
* #ORM\Column(name="name", type="string", length=100, nullable=false)
*/
protected $name;
/**
* #var integer
*
* #ORM\ManyToOne(targetEntity="Category", inversedBy="childrenId", fetch="EAGER")
* #ORM\JoinColumn(name="parent_id", referencedColumnName="id")
*/
protected $parentId;
/**
* #ORM\OneToMany(targetEntity="Category", mappedBy="parentId", fetch="EAGER")
*/
protected $childrenId;
}
My DQL
$result = $this->em->createQueryBuilder()->select('c')
->from('App\Entity\Category', 'c')
->where('c.parentId IS NULL')
->orderBy('c.priority', 'ASC')
->getQuery()
->setFetchMode("App\Entity\Category", "parentId", \Doctrine\ORM\Mapping\ClassMetadata::FETCH_EAGER);
->useResultCache(true, 900, 'categories')
->getResult();
I have 28 categories, 15 of them have parentId.
Above query executes 29 SQL queries, but Doctrine store in cache only 1, So when I run again this query, it executes 28 queries.
Any idea what am I doing wrong? missing some cache configuration? missing some methods in DQL? I would like to cache all queries not only one- main query.
Edit
I would like to use query result in loop, like this:
foreach($result as $row)
{
$categories[]['attr'] = $row->getAttribute()->getName();
$categories[]['value'] = $row->getAttribute()->getValue();
}
but this way cache won't work, so currently I'm using:
foreach($result as $row)
{
$attributes = $this->em->createQueryBuilder()->select('c, a.name, a.value')
->from('App\Entity\Category', 'c')
->innerJoin('App\Entity\Attribute', 'a', 'WITH', 'a.id = c.attribute')
->where('c.id = :catId')
->setParameter('catId', $row['id'])
->getQuery()
->useResultCache(true, 900, $categoryName.'-attributes')
->getArrayResult();
}
But I would rather work on objects then on arrays, but I can't cuz if I use object and it has association then this association will not be cached. So ideally would be some way to cache object + ALL his associations.
Association fetch-modes
The query you present only fetches "parent" Category entities, which get hydrated with an uninitialized collection for the children. When accessing that collection (by iterating over those children for example), Doctrine will load the collection, thus perform another query. It will do that for all parent categories hydrated by the first query.
Setting fetch-mode to EAGER only changes the moment these queries are done. Doctrine will do them right after hydrating the parent categories, it won't wait until you access the collection (like with fetch-mode LAZY). But it will still do those queries.
Fetch-join query
The simplest way to tell Doctrine to query and hydrate the categories with their children is to do a "fetch join" query:
$queryBuilder = $this->em->createQueryBuilder();
$queryBuilder
->select('p', 'c') // [p]arent, [c]hild
->from('App\Entity\Category', 'p')
->leftJoin('p.children', 'c')
->where('p.parent IS NULL')
->orderBy('p.priority', 'ASC');
$query = $queryBuilder->getQuery();
$query
->useResultCache(true, 900, 'categories')
$result = $query->getResult();
Note the select() and leftJoin() calls here.
You also don't need to alter the fetch-mode of the association (by calling setFetchMode()), because the query itself will tell Doctrine to do what you want.
The result of this is that Doctrine will perform 1 query if it isn't cached yet (or the cache is stale), or 0 queries if it is cached (and still fresh).
Assumptions
The property $parentId (in Category) is renamed to $parent. This property will contain the parent Category entity, or null, but never an id.
The property $childrenId is renamed to $children. This property will contain a collection of Category entities (which might be empty), but never a collection (or array) of ids, and certainly never a single id.
The query I suggest above takes these renames into account.
I'm completely ignoring the fact that right after your "edit" a new Attribute entity has sprung into existence. It isn't relevant to your question or this answer IMHO.
More levels
It looks/sounds like your Categories only use 2 levels (parents and children). When you introduce more levels (grandchildren, etc), reading this model can become very inefficient very quickly.
When going for 3 or more levels, you might want to look into the Nested Set model. It's heavier on the writes, but highly optimized for reads.
The DoctrineExtensions library has support for this, and there's also a Symfony Bundle.
According to the docs: http://doctrine-orm.readthedocs.io/en/latest/reference/dql-doctrine-query-language.html#temporarily-change-fetch-mode-in-dql I think that setFetchMode - EAGER will not work for you as this will produce extra queries that will not be cached.
Why don't you load your categories with attributes?
$result = $this->em->createQueryBuilder()
->select('c, a.name, a.value')
->from('App\Entity\Category', 'c')
->innerJoin('App\Entity\Attribute', 'a', 'WITH', 'a.id = c.attribute')
->where('c.parentId IS NULL')
->orderBy('c.priority', 'ASC')
->getQuery()
->useResultCache(true, 900, 'categories')
->getResult();
And it should work as expected.

limit columns returned in relationnal entity symfony2

Is it possible to filter an entity and display only few columns in symfony2?
I think I can do a custom query for this, but it seems a bit dirty and I am sure there is a better solution.
For example I have my variable $createdBy below, and it contains few data that shouldnt be displayed in this parent entity such as password etc...
/**
* #var Customer
*
* #ORM\ManyToOne(targetEntity="MyCompany\Bundle\CustomerBundle\Entity\Customer")
* #ORM\JoinColumns({
* #ORM\JoinColumn(name="created_by", referencedColumnName="id", nullable=false)
* })
*/
protected $createdBy;
So I need to display my Customer entity, but only containing fields like id and name for example.
EDIT :
I already have an instance of Project, the entity with my createdBy field, and I want to grab my customer data 'formatted' for this entity and not returning too much fields like password ...
Thanks
It sounds like expected behavior to me. The doctrine documentation seems to imply that eager fetching is only one level deep.
According to the docs:
Whenever you query for an entity that has persistent associations and
these associations are mapped as EAGER, they will automatically be
loaded together with the entity being queried and is thus immediately
available to your application.
http://doctrine-orm.readthedocs.org/en/latest/reference/working-with-objects.html#by-eager-loading
The entity being queried has eager on createdBy so it will be populated.
to bypass you can create a method in your entity repository as following :
// join entities and load wanted fields
public function findCustom()
{
return $this->getEntityManager()
->createQuery(
'SELECT p FROM AppBundle:Product p ORDER BY p.name ASC'
)
->getResult();
}
hope this helps you
try this and let me know if it works, you should fill the right repository name
'XXXXBundle:CustomerYYYY', 'c'
public function findUser($user_id){
$qb = $this->_em->createQueryBuilder('c')
->select(array('c', 'cb.id', 'cb.name'))
->from('XXXXBundle:Customer', 'c')
->where('c.id <> :id')
->leftJoin('c.createdBy', 'cb')
->setParameter('id', $user_id)->getQuery();
if ($qb != null)
return $qb->getOneOrNullResult();
return null;
}

DQL to join on property of a property

I am trying to create a DQL query by testing on the property of a property. The objective is to create a DQL query that will indicate in what role a user is fulfilling a job, returning no rows if the user does not have the proper role.
My Entities are Role, Job, and User. User's have roles, Jobs require a role (to fulfill them), and Roles have 'alternates' that link to another role that can fill in for that role.
Stubbed versions of the entites look like:
class User {
//Annotation not needed for question
protected $id;
/**
* #var SystemBundle\Entity\Role
*
* #ORM\ManyToMany(targetEntity="SystemBundle\Entity\Role")
* #ORM\JoinTable(name="User_User__Role",
* joinColumns={
* #ORM\JoinColumn(name="User_ID", referencedColumnName="ID")
* },
* inverseJoinColumns={
* #ORM\JoinColumn(name="Role_ID", referencedColumnName="ID")
* }
* )
*/
protected $roles;
}
class Job {
//Annotation not needed for question
protected $id;
/**
* #var SystemBundle\Entity\Role $jobRole
*
* #ORM\ManyToOne(targetEntity="Role")
* #ORM\JoinColumns({
* #ORM\JoinColumn(name="Role_ID", referencedColumnName="ID")
* })
*/
protected $jobRole;
}
class Role {
//Annotation not needed for question
protected $id;
/**
* #var SystemBundle\Entity\Role $jobRole
*
* #ORM\OneToOne(targetEntity="Role")
* #ORM\JoinColumn(name="BackupRole_ID", referencedColumnName="ID")
*/
protected $backUpRole;
}
The conceptual logic is to join Roles with Jobs, testing if the current Role IS job.JobRole or job.JobRole.backUpRole. I've tried this, and doctrine really doesnt seem to like the job.jobRole.backUpRole and throws this error at the second period:
[Syntax Error] line 0, col 210: Error: Expected =, <, <=, <>, >, >=, !=, got '.'
My DQL attempt looks like this
SELECT r
FROM SystemBundle:Role r
INNER JOIN ProcessBundle:Job j
WITH j = :j AND j.jobRole = r OR j.jobRole.backUpRole = r
LEFT JOIN UserBundle:User u
WITH r MEMBER OF u.roles
WHERE u = :emp AND u IS NOT NULL
ORDER BY r.id
I can accomplish this task with pure SQL, as well as just using php to walk the associations, but I am looking to stay true to use DQL (because it's vexing me and I want to know if it can be done).
If it helps, here is my pure SQL:
#get all roles
select r.*
from Sys_Role as r
inner join
#get the role assigned to the job
(select r.*
FROM Sys_Role as r
INNER JOIN Sys_Job as j
ON j.JobRole_ID = r.ID
WHERE j.ID = ?) as jr
#join roles based on the job-role's id or backup id
# this should filter the list of roles down to the role and backup role for the job
on ar.ID = aar.ID OR ar.ID = aar.BackUpRole_ID
# and filter by the roles the user is assigned
INNER JOIN User_User__Role as uur
ON r.ID = uur.Role_ID
WHERE uur.User_ID = ?
EDIT: My apologies. While editing the question layout i accidently deleted the DQL. I've added that back the question!
UPDATE: I am exploring making the $backUpRole a bidirectional self-join and attacking from that direction. Doctrine does not like the naive attempt, so it looks like if this tact is used, fresh DQL is needed.
Changing the DQL to WITH j = :j AND j.jobRole = r OR j.jobRole = r.roleIBackUp yeilds a new error: A single-valued association path expression to an inverse side is not supported in DQL queries. Use an explicit join instead.
I figured it out. It IS possible... you need to join the sub-objects before you can use them. I had to switch up the order of my joins and I'm not terribly happy with my filters being in WITH clauses instead of WHERE, but it's hard to argue with results!
SELECT r
FROM SystemBundle:Role r
LEFT JOIN UserBundle:User u
WITH u = :emp
INNER JOIN ProcessBundle:Job j
WITH j = :job
JOIN j.jobRole jr
JOIN jr.backUpRole jrbr
WHERE (jr = ar OR jrbr = ar) AND r MEMBER OF u.role
The last 2 Joins let us alias the objects that belong to the linked entities so we can use them to compare.
The "30,000 foot view" of how this DQL works is such:
Foreach Role
Does this Role belong to the user?
Get the Role for the Job we want
Alias That Role
Alias That ROles Back Up
Is this role the Job's Role or Job's Role's Backup Role?
Kind of weird, but fairly simple once you get it.

Resources