Using Doctrine Entity Manager on same database in two daemons - symfony

Question

I'm running two daemons that interrogate an external service basically almost every second 24/7. Each of them inserts or updates things in the same local database after every loop, but they work on different objects of the same entities.
Since they run 24/7, after some tests I decided to clear the entity manager after every loop to avoid having a huge number of managed entities and a lot of memory usage.
So, in both of them, I run something like this after every loop:
$this->entityManager->flush();
....
$this->entityManager->clear(MyClass:class);
$this->entityManager->clear(MyOtherClass:class);
....
What I want to ask is: if DaemonA clears the entities and DaemonB hasn't flushed the persisted changes yet, what happens? When DaemonA flushes, does it affect in any way the entities in DaemonB? Could some objects get lost? Could some get duplicated? If so, what can I do to avoid this kind of things?
As I said, they work on different objects of the same entities, e.g DaemonA works on MyOtherClass objects 1, 2, 3 and DaemonB on MyOtherClass objects 4, 5, 6.
Both daemons are Symfony commands constructed like this:
class DaemonA extends Command
{
private $entityManager;
public function __construct(EntityManagerInterface $entityManager)
{
$this->entityManager = $entityManager;
parent::__construct();
}
...
}

Answer 1

That’s a lot of questions here, so let’s go through them step-by-step.
Before we start, remember how Doctrine works internally: If an entity, or a set of entities, is requested through a query or a repository, Doctrine loads the entity data from the database, creates entities, populates them with data, tracks changes and syncs changes back to the database. Doctrine entities have states, usually they are in a managed state unless you detach them. When you clear the entity manager, entities become detached.
Now, to answer your questions:
if DaemonA clears the entities and DaemonB hasn't flushed the persisted changes yet, what happens?
Clearing the entity manager only means that entities become detached and, if they aren’t referenced any more, are garbage-collected (I think). On a database level, this is irrelevant.
When DaemonA flushes, does it affect in any way the entities in DaemonB?
Yes, but not while DaemonB is running and doesn’t reload the entities from the database. Should DaemonA modify entities and DaemonB modify the same entities before reloading them in the meantime, the modifications by DaemonB will persist.
Could some get duplicated?
Only if you persist detached entities (they would have a new ID, though). However, persisting detached entities doesn’t make sense anyway.
If so, what can I do to avoid this kind of things?
Locks and transactions!
Every modification of your database which contains more than one query, must be wrapped in a transaction. This avoids inconsistencies if something goes wrong or a concurrent request modifies the data. On the PHP level, the transaction again should be wrapped in a try/catch block.
If you are modifying an entity, lock it. Doctrine supports different types of locking; pick the one that suits your scenario best.
The code in one of your daemons might look like this:
try
{
$em->beginTransaction();
$entity = $em->find($entityClassName, $id);
// lock the entity for all reading and writing.
$em->lock($entity, LockMode::PESSIMISTIC_WRITE);
$em->flush();
$em->commit();
}
catch (Exception $e)
{
$em->rollback();
throw $e;
}
Note that depending on your locking strategy and the general implementation of your system, the daemons may block each other by locking the database, up to the point where your system runs out of resources.
For example, pessimistic write locking is more secure (it makes sure that other processes don’t read the data until the modification is complete), but other processes will have to wait until the lock is released.
Be mindful and do test under heavy-load scenarios!