Check existence in Doctrine for persisted and not persisted entities - symfony

Question

My question could be a duplicate of this one, but I can't find any satisfying answer so I will try to make this one more precise.
I am building an import service from an other API. And I don't want any duplicate in my new database.
So here an example of my current implementation:
The Controller:
public function mainAction ()
{
$em = $this->getDoctrine()->getManager();
$persons_data = [
[
'first_name' => 'John',
'last_name' => 'Doe'
],
[
'first_name' => 'John',
'last_name' => 'Doe'
]
];
$array = [];
foreach($persons_data as $person_data)
{
$person = $this->get('my_service')->findOrCreatePerson($person_data);
$array[] = $person;
}
$em->flush();
return new Response();
}
A service function:
public function findOrCreatePerson ($data)
{
$em = $this->em;
$person = $em->getRepository('AppBundle:Person')->findOneBy([
'first_name' => $data['first_name'],
'last_name' => $data['last_name']
]);
if(is_null($person)) {
$person = new Person();
$person->setFirstName($data['first_name']);
$person->setLastName($data['last_name']);
$em->persist($person);
}
return $person
}
I tried to make it as simple as possible.
As you can see, I would like to make only one DB transaction to get some performance improvements.
Problem is, if I don't flush at the end of the findOrCreatePerson() method, the query to the Person repository won't find the first object and will create duplicates in the database.
My question is simple: How should I implement such a thing?

Answer 1

This is a job for memoize!
// Cache
private $persons = [];
public function findOrCreatePerson ($data)
{
// Need unique identifier for persons
$personKey = $data['first_name'] . $data['last_name'];
// Already processed ?
if (isset($this->persons[$personKey])) {
return $this->persons[$personKey];
}
$em = $this->em;
$person = $em->getRepository('AppBundle:Person')->findOneBy([
'first_name' => $data['first_name'],
'last_name' => $data['last_name']
]);
if(is_null($person)) {
$person = new Person();
$person->setFirstName($data['first_name']);
$person->setLastName($data['last_name']);
$em->persist($person);
}
// Cache
$this->persons[$personKey] = $person;
return $person
}

Answer 2

Cerad's answer (memoization) is a good one, but I'd encourage you to reconsider something.
As you can see, I would like to make only one DB transaction to get some performance improvements.
And there are a few things wrong with that sentence.
The main one is that you're conflating flush() with a single, atomic transaction. You can manually manage transaction boundaries, and it's often very advantageous to do so.
The second thing is that when you're talking about bulk imports, you'll quickly learn that the first performance issue you hit is not the database at all. It's the the EntityManager itself. As the EM's internal identity map gets swollen, computing changes to persist to the DB gets very, very slow.
I'd consider rewriting your core loop as follows, and see if it's fast enough. Only then consider memoization if necessary.
$em->beginTransaction();
foreach($persons_data as $person_data)
{
$person = $this->get('my_service')->findOrCreatePerson($person_data);
$em->flush();
$em->clear(); // don't keep previously inserted entities in the EM.
}
$em->commit();