Doctrine 2: Cache in One-to-Many associations - symfony

I'm trying to use doctrine cache from Common package, but I can't get it working with one-to-many, many-to-one accosiations. I'll explain later what I want to do.
My configuration:
'configuration' => array(
'orm_default' => array(
'metadata_cache' => 'filesystem',
'query_cache' => 'filesystem',
'result_cache' => 'filesystem',
'hydration_cache' => 'filesystem',
)
),
My entity
class Category
{
/**
* #ORM\Id
* #ORM\GeneratedValue
* #ORM\Column(type="integer")
*
* #var string
*/
protected $id;
/**
* #var string
*
* #ORM\Column(name="name", type="string", length=100, nullable=false)
*/
protected $name;
/**
* #var integer
*
* #ORM\ManyToOne(targetEntity="Category", inversedBy="childrenId", fetch="EAGER")
* #ORM\JoinColumn(name="parent_id", referencedColumnName="id")
*/
protected $parentId;
/**
* #ORM\OneToMany(targetEntity="Category", mappedBy="parentId", fetch="EAGER")
*/
protected $childrenId;
}
My DQL
$result = $this->em->createQueryBuilder()->select('c')
->from('App\Entity\Category', 'c')
->where('c.parentId IS NULL')
->orderBy('c.priority', 'ASC')
->getQuery()
->setFetchMode("App\Entity\Category", "parentId", \Doctrine\ORM\Mapping\ClassMetadata::FETCH_EAGER);
->useResultCache(true, 900, 'categories')
->getResult();
I have 28 categories, 15 of them have parentId.
Above query executes 29 SQL queries, but Doctrine store in cache only 1, So when I run again this query, it executes 28 queries.
Any idea what am I doing wrong? missing some cache configuration? missing some methods in DQL? I would like to cache all queries not only one- main query.
Edit
I would like to use query result in loop, like this:
foreach($result as $row)
{
$categories[]['attr'] = $row->getAttribute()->getName();
$categories[]['value'] = $row->getAttribute()->getValue();
}
but this way cache won't work, so currently I'm using:
foreach($result as $row)
{
$attributes = $this->em->createQueryBuilder()->select('c, a.name, a.value')
->from('App\Entity\Category', 'c')
->innerJoin('App\Entity\Attribute', 'a', 'WITH', 'a.id = c.attribute')
->where('c.id = :catId')
->setParameter('catId', $row['id'])
->getQuery()
->useResultCache(true, 900, $categoryName.'-attributes')
->getArrayResult();
}
But I would rather work on objects then on arrays, but I can't cuz if I use object and it has association then this association will not be cached. So ideally would be some way to cache object + ALL his associations.

Association fetch-modes
The query you present only fetches "parent" Category entities, which get hydrated with an uninitialized collection for the children. When accessing that collection (by iterating over those children for example), Doctrine will load the collection, thus perform another query. It will do that for all parent categories hydrated by the first query.
Setting fetch-mode to EAGER only changes the moment these queries are done. Doctrine will do them right after hydrating the parent categories, it won't wait until you access the collection (like with fetch-mode LAZY). But it will still do those queries.
Fetch-join query
The simplest way to tell Doctrine to query and hydrate the categories with their children is to do a "fetch join" query:
$queryBuilder = $this->em->createQueryBuilder();
$queryBuilder
->select('p', 'c') // [p]arent, [c]hild
->from('App\Entity\Category', 'p')
->leftJoin('p.children', 'c')
->where('p.parent IS NULL')
->orderBy('p.priority', 'ASC');
$query = $queryBuilder->getQuery();
$query
->useResultCache(true, 900, 'categories')
$result = $query->getResult();
Note the select() and leftJoin() calls here.
You also don't need to alter the fetch-mode of the association (by calling setFetchMode()), because the query itself will tell Doctrine to do what you want.
The result of this is that Doctrine will perform 1 query if it isn't cached yet (or the cache is stale), or 0 queries if it is cached (and still fresh).
Assumptions
The property $parentId (in Category) is renamed to $parent. This property will contain the parent Category entity, or null, but never an id.
The property $childrenId is renamed to $children. This property will contain a collection of Category entities (which might be empty), but never a collection (or array) of ids, and certainly never a single id.
The query I suggest above takes these renames into account.
I'm completely ignoring the fact that right after your "edit" a new Attribute entity has sprung into existence. It isn't relevant to your question or this answer IMHO.
More levels
It looks/sounds like your Categories only use 2 levels (parents and children). When you introduce more levels (grandchildren, etc), reading this model can become very inefficient very quickly.
When going for 3 or more levels, you might want to look into the Nested Set model. It's heavier on the writes, but highly optimized for reads.
The DoctrineExtensions library has support for this, and there's also a Symfony Bundle.

According to the docs: http://doctrine-orm.readthedocs.io/en/latest/reference/dql-doctrine-query-language.html#temporarily-change-fetch-mode-in-dql I think that setFetchMode - EAGER will not work for you as this will produce extra queries that will not be cached.
Why don't you load your categories with attributes?
$result = $this->em->createQueryBuilder()
->select('c, a.name, a.value')
->from('App\Entity\Category', 'c')
->innerJoin('App\Entity\Attribute', 'a', 'WITH', 'a.id = c.attribute')
->where('c.parentId IS NULL')
->orderBy('c.priority', 'ASC')
->getQuery()
->useResultCache(true, 900, 'categories')
->getResult();
And it should work as expected.

Related

Doctrine OneToOne always loaded on querybuilder

I have Partner entity with two relation:
/**
* #var PartnerSettings
* #ORM\OneToOne(targetEntity="PartnerSettings", mappedBy="partner", cascade={"persist", "remove"}, fetch="LAZY")
*/
private $settings;
/**
* #var PartnerRating
* #ORM\OneToOne(targetEntity="PartnerRating", mappedBy="partner", cascade={"persist", "remove"}, fetch="LAZY")
*/
private $rating;
...getRepository(Partner::class)->findAll() work correctly, one query was made,but when I create queryBuilder:
return $this->createQueryBuilder('p')
->getQuery()
->getResult();
doctrine make 31 queries(i have 10 partners)... in debug toolbar i saw select queries to settings and rating for every partner. I don't want it in this case.
Additionally, in every querybuilder where I used join to partners, setting and rating are selected too.
answer
->setHint(Query::HINT_FORCE_PARTIAL_LOAD, true);
did the job
This behaviour is known as the N + 1 selects problem. To reduce database queries, you might want to consider the following approach.
First, retrieve all partners:
$partners = $em->createQueryBuilder()
->select("p")
->from("Parent", "p")
->where(/*...*/)
->setParameter(/*...*/)
->indexBy("p.id")
->getQuery()->getResult();
Now load all children at once, in two queries:
$settings = $em->createQueryBuilder()
->select("s")
->from("PartnerSetting", "s")
->where("IDENTITY(s.partner) IN (?1)")
->setParameter(1, array_keys($partners))
->getQuery()->getResult();
$ratings = $em->createQueryBuilder()
->select("r")
->from("PartnerRating", "r")
->where("IDENTITY(r.partner) IN (?1)")
->setParameter(1, array_keys($partners))
->getQuery()->getResult();
Doctrine will now have all of the retrieved entities are stored in memory. So when, for example, you do a $parnter->getRatings(), you don’t trigger a new DB query, instead the entity is filled from memory.
just define the table AND the relation in the select
$qb->select('g', 'gi');
and now it works
Just mapped relation: #ORM\OneToOne(targetEntity="ENTITY", mappedBy="MAPPEDBY", fetch="EAGER")

limit columns returned in relationnal entity symfony2

Is it possible to filter an entity and display only few columns in symfony2?
I think I can do a custom query for this, but it seems a bit dirty and I am sure there is a better solution.
For example I have my variable $createdBy below, and it contains few data that shouldnt be displayed in this parent entity such as password etc...
/**
* #var Customer
*
* #ORM\ManyToOne(targetEntity="MyCompany\Bundle\CustomerBundle\Entity\Customer")
* #ORM\JoinColumns({
* #ORM\JoinColumn(name="created_by", referencedColumnName="id", nullable=false)
* })
*/
protected $createdBy;
So I need to display my Customer entity, but only containing fields like id and name for example.
EDIT :
I already have an instance of Project, the entity with my createdBy field, and I want to grab my customer data 'formatted' for this entity and not returning too much fields like password ...
Thanks
It sounds like expected behavior to me. The doctrine documentation seems to imply that eager fetching is only one level deep.
According to the docs:
Whenever you query for an entity that has persistent associations and
these associations are mapped as EAGER, they will automatically be
loaded together with the entity being queried and is thus immediately
available to your application.
http://doctrine-orm.readthedocs.org/en/latest/reference/working-with-objects.html#by-eager-loading
The entity being queried has eager on createdBy so it will be populated.
to bypass you can create a method in your entity repository as following :
// join entities and load wanted fields
public function findCustom()
{
return $this->getEntityManager()
->createQuery(
'SELECT p FROM AppBundle:Product p ORDER BY p.name ASC'
)
->getResult();
}
hope this helps you
try this and let me know if it works, you should fill the right repository name
'XXXXBundle:CustomerYYYY', 'c'
public function findUser($user_id){
$qb = $this->_em->createQueryBuilder('c')
->select(array('c', 'cb.id', 'cb.name'))
->from('XXXXBundle:Customer', 'c')
->where('c.id <> :id')
->leftJoin('c.createdBy', 'cb')
->setParameter('id', $user_id)->getQuery();
if ($qb != null)
return $qb->getOneOrNullResult();
return null;
}

Doctrine ORM : Calculated related entity count in one shot

I've a User entity and a Product entity.
class User{
/*
* #ORM\OneToMany(targetEntity="Product", mappedBy="User")
*/
private $Products;
}
class Product{
/**
* #ORM\ManyToOne(targetEntity="User", inversedBy="Products")
* #ORM\JoinColumn(name="user_id", referencedColumnName="id")
*/
private $User;
}
Now I'm trying to display a html table of users, but I want to show each user's product count too.
By using following code I'm able to obtain the users objects.
$qb = $this->_em->createQueryBuilder();
$qb->select('usr')
->from('User', 'usr');
$query = $qb->getQuery();
But I don't know how to get the products count in one shot. Any help?
First, you should really create a repository class for your entities if you want to create custom queries. Then you can simply run that query by injecting the entity repository as a service wherever you need it and then running the query method.
Second, you need to return a result from a doctrine query to retrieve anything from the database. If you want to determine the count of the objects returned, simply do this:
$qb = $this->_em->createQueryBuilder();
$qb->select('usr')
->from('User', 'usr');
$query = $qb->getQuery();
$count = count($query->getResult());
A doctrine query will return an array of objects matching your query. If you just want to return a count of the matching records, try something like this:
$qb = $this->_em->createQueryBuilder();
$qb->select('count(id)')
->from('User', 'usr');
$query = $qb->getQuery();
$count = $query->getSingleScalarResult();
Or to just get a count of Product objects for that user, from within the User repository class:
$qb = $this->_em->createQueryBuilder('usr');
$qb->select('count(p.id)')
->from('usr.Products', 'p');
return $qb->getQuery()->getSingleScalarResult();

OneToMany relationship not persisting along with new entities

I'm facing some issue when I try to persist a collection of entities using a symfony form. I followed the official documentation but I can't make it work becouse of this error:
Entity of type ProductItem has identity through a
foreign entity Product, however this entity has no identity itself. You have to call
EntityManager#persist() on the related entity and make sure that an identifier was
generated before trying to persist ProductItem. In case of Post Insert ID
Generation (such as MySQL Auto-Increment or PostgreSQL SERIAL) this means you
have to call EntityManager#flush() between both persist operations.
I have to entities linked with a OneToMany relation:
Product
/**
* #ORM\Column(name="id", type="integer", nullable=false)
* #ORM\Id
* #ORM\GeneratedValue(strategy="IDENTITY")
*/
protected $id;
/**
* #ORM\OneToMany(targetEntity="ProductItem", mappedBy="product",cascade={"persist"})
*/
protected $items;
And ProductItem
/**
* #ORM\Id()
* #ORM\ManyToOne(targetEntity="Product", inversedBy="items")
*/
protected $product;
/**
* #ORM\Id()
* #ORM\ManyToOne(targetEntity="Item")
*/
protected $item;
This is how it is added to the form:
->add('items','collection',array(
'label' => false,
'type' => new ProductItemType(),
'allow_add' => true,
'allow_delete' => true,
'by_reference' => false))
And this is the controller action:
public function newAction()
{
$product= new Product();
$form = $this->createForm(new ProductType(), $product);
if($request->isMethod("POST"))
{
$form->handleRequest($request);
if($form->isValid())
{
$em = $this->getDoctrine()->getManager();
$em->persist($product);
$em->flush();
}
}
}
I'm doing something wrong for sure in the controller because, as the error message says, I have to persist $product before adding $productItems, but how can I do that?
I only get this error when trying to persist a new entity, if the entity has been persisted before, I can add as may items as I want successfully
I had exact same problem last week, here is a solution I found after some reading and testing.
The problem is your Product entity has cascade persist (which is usually good) and it first try to persist ProductItem but ProductItem entities cannot be persisted because they require Product to be persisted first and its ID (Composite key (product, item).
There are 2 options to solve this:
1st I didn't use it but you could simply drop a composite key and use standard id with foreign key to the Product
2nd - better This might look like hack, but trust me this is the best what you can do now. It doesn't require any changes to the DB structure and works with form collections without any problems.
Code fragment from my code, article sections have composite key of (article_id, random_hash). Temporary set one to many reference to an empty array, persist it, add you original data and persist (and flush) again.
if ($form->isValid())
{
$manager = $this->getDoctrine()->getManager();
$articleSections = $article->getArticleSections();
$article->setArticleSections(array()); // this won't trigger cascade persist
$manager->persist($article);
$manager->flush();
$article->setArticleSections($articleSections);
$manager->persist($article);
$manager->flush();
You didn't follow the docs completely. Here is something you can do to test a single item, but if you want to dynamically add and delete items (it looks like you do), you will also need to implement all the javascript that is included in the docs that you linked to.
$product= new Product();
$productItem = new ProductItem();
// $items must be an arraycollection
$product->getItems()->add($productItem);
$form = $this->createForm(new ProductType(), $product);
if($request->isMethod("POST"))
{
$form->handleRequest($request);
if($form->isValid())
{
$em = $this->getDoctrine()->getManager();
$em->persist($productItem);
$em->persist($product);
$em->flush();
}
}
So this should work for a single static item, but like I said, the dynamic stuff is a bit more work.
The annotation is wrong... the cascade persist is on the wrong side of the relation
/**
* #ORM\OneToMany(targetEntity="ProductItem", mappedBy="product")
*/
protected $items;
/**
* #ORM\Id()
* #ORM\ManyToOne(targetEntity="Product", inversedBy="items", cascade={"persist"})
*/
protected $product;
Another way to achieve this (e.g. annotation not possible) is to set the form by_reference
IMO, your problem is not related to your controller but to your Entities. It seems your would like to make a ManyToMany between your Product and Item and not creating a ProductItem class which should behave as an intermediate object for representing your relation. Additionally, this intermediate object have no id generation strategy. This is why Doctrine explains you, you must first persist/flush all your new items and then persist/flush your product in order to be able to get the ids for the intermediate object.
Also faced this issue during the work with form to which CollectionType field was attached. The other one approach which could solve this problem and also mentioned in doctrine official documentation is following:
public function newAction()
{
$product= new Product();
$form = $this->createForm(new ProductType(), $product);
if($request->isMethod("POST"))
{
$form->handleRequest($request);
if($form->isValid())
{
foreach ($product->getItems() as $item)
{
$item->setProduct($product);
}
$em = $this->getDoctrine()->getManager();
$em->persist($product);
$em->flush();
}
}
}
In simple words, you should provide product link to linked items manually - this is described in "Establishing associations" section of following article: http://docs.doctrine-project.org/en/latest/reference/working-with-associations.html#working-with-associations

Mapping and DQL

I have 3 tables - user, area, and contacts. A contact can belong to a user or an area. A user can belong to many areas.
I want to pull all the contacts that belong to a user (as specifically defined in the DB), as well as all contacts that belong to the same area as the user.
Can I get a fresh set of eyes on my Database mapping, and the query I need to write in DQL to get what I want. Am I doing something wrong in my database mapping?
I'm definitely a SQL person, and am able to easily fetch what I want in plain SQL. In plain SQL, here's what I want to do:
select c.* from contact c LEFT JOIN user_area ua ON c.area_id=ua.area_id where (ua.user_id=XXX OR c.user_id=XXX);
USER
/**
* #ORM\ManyToMany(targetEntity="area", inversedBy="areas")
* #ORM\JoinTable(name="user_area",
* joinColumns={#ORM\JoinColumn(name="user_id", referencedColumnName="id")},
* inverseJoinColumns={#ORM\JoinColumn(name="area_id", referencedColumnName="id")}
* )
*/
private $areas;
/**
* #ORM\OneToMany(targetEntity="Contact", mappedBy="user")
*/
private $contacts;
CONTACT
/**
* #ORM\ManyToOne(targetEntity="Area")
* #ORM\JoinColumn(name="area_id", referencedColumnName="id")
*/
private $area;
/**
* #ORM\ManyToOne(targetEntity="User", inversedBy="Contacts")
* #ORM\JoinColumn(name="user_id", referencedColumnName="id")
*/
private $user;
AREA
/**
* #ORM\ManyToMany(targetEntity="User", mappedBy="users")
*/
private $users;
/**
* #ORM\OneToMany(targetEntity="Contact", mappedBy="area")
*/
private $contacts;
The main problem I'm running into is that DQL really wants you to query an object, and it's just plain easier in SQL to query the user/area relationship table to get what I want. I tried to write an query that pulls areas from contacts, then users from contacts, and then users from areas but I get an error message that "users" isn't a defined index in my areas object. Again, I'm a Doctrine newbie, so I'm probably doing something wrong.
Here's my attempt at a query, from the User object in Symfony:
$qb = $em->createQueryBuilder()
->addSelect('c')
->from('MyBundle:Contact', 'c')
->leftJoin('c.area', 'ca')
->leftJoin('c.user', 'cu')
->leftJoin('ca.users', 'cau')
->add('where', 'c.user = ?1 OR cau.id = ?1')
->add('orderBy', 'c.name')
->setParameter(1, $this->getId());
Someone should have slapped me for providing that previous answer. While it got the job done, I was absolutely right, it was not optimized. Queries using that method were taking 3 seconds to go back and forth to the database (3 seconds!). Clearly, there were plenty of other things going on in my world that took away from performance as a requirement for getting this done, but things have changed. I managed to break down this query into two smaller (Doctrine generated) ones, each taking maybe 0.2 or 0.3s.
$areas = $user->getAreas();
$qb = $this->getEntityManager()->createQueryBuilder();
$qb->select('c')
->from('MyBundle:Contact', 'c')
->where($qb->expr()->in('c.area', '?1'))
->orWhere('c.user = ?2')
->setParameter(1, $areas->toArray())
->setParameter(2, $user);
$query = $qb->getQuery();
$result = $query->getResult();
return $result;
The fact that I have to call $user->getAreas() adds a database query (if Doctrine doesn't already have that information), but this code, using Query Builder expressions, works much better (0.3s vs. 3s is 10% of the original query time!).
I think the main concept I was missing back then was that the Query Builder wants to work with your objects (Entities), and the properties you've defined in your entities. Coming from a strong SQL background, and knowing the specific SQL query I wanted Doctrine to produce, I wasn't approaching the problem properly.
Hope this update to an 8-month old question helps somebody!
So it turns out you can't fetch objects of objects in DQL. I needed to fetch all "Area"s (an object of "Contact") and then fetch all of that Area's "User"s.
In DQL, you can specify multiple "from()" helper methods, and this was what I needed to get the job done.
$qb = $em->createQueryBuilder()
->addSelect('c')
->from('MyBundle:Contact', 'c')
->from('MyBundle:User', 'u')
->leftJoin('c.area', 'ca')
->leftJoin('c.user', 'cu')
->leftJoin('u.areas', 'ua')
->add('where', 'c.user = ?1 OR (ua.id=ca.id AND u.id = ?1')
->add('orderBy', 'c.name')
->setParameter(1, $this->getId());
The resulting SQL generated from Doctrine doesn't seem particularly optimized, but it gets the job done. If anyone has any thoughts on getting Doctrine to better optimize the following query, I'd love to hear opinions.
SELECT m0_.id AS id0, m0_.name AS name1, m0_.email AS email2, m0_.media_area_id AS media_area_id3, m0_.user_id AS user_id4 FROM contact m0_ LEFT JOIN user u1_ ON m0_.user_id = u1_.id LEFT JOIN area m2_ ON m0_.area_id = m2_.id, user u3_ LEFT JOIN user_area u5_ ON u3_.id = u5_.user_id LEFT JOIN area m4_ ON m4_.id = u5_.area_id WHERE u1_.id = ? OR (m4_.id = m2_.id AND u3_.id = ? );

Resources