Handling doctrine transactions on multiple entity managers - symfony

Both transactions should be rollback-ed if;
$em1 fails but $em2 succeeds.
$em1 succeeds but $em2 fails.
So, is my example below correct way of dealing with transactions when more than one EMs are involved? I've come up with it after reading Transactions and Concurrency documentation.
$em1->getConnection()->beginTransaction();
$em2->getConnection()->beginTransaction();
try {
$em1->persist($object1);
$em1->flush();
$em1->getConnection()->commit();
$em2->persist($object2);
$em2->flush();
$em2->getConnection()->commit();
} catch (Exception $e) {
$em1->getConnection()->rollback();
$em2->getConnection()->rollback();
}
The reason I'm trying to implement this because I'm getting ....resulted in a Doctrine\ORM\ORMException exception (The EntityManager is closed.) error somewhere along the line in the application. I can probably handle it with the method below but I think using transaction for the business logic above is better.
private function getNewEntityManager($em)
{
if (!$em->isOpen()) {
$em = $em->create($em->getConnection(), $em->getConfiguration());
}
return $em;
}

Your example code actually does work, which surprises me because Francesco Panina is (or should be) correct that $em1->getConnection()->commit()
will commit the first transaction and you will loose [sic] the privilege to rollback such transaction should an error arise from the second transaction.
However, something in the way that Doctrine handles transaction nesting levels means that you actually can still rollback the first transaction when an error arises from the second transaction.
Nonetheless, best practice would be to not depend on this behavior and instead put both commits at the very end of your try block, as so:
$em1->getConnection()->beginTransaction();
$em2->getConnection()->beginTransaction();
try {
$em1->persist($object1);
$em1->flush();
$em2->persist($object2);
$em2->flush();
$em1->getConnection()->commit();
$em2->getConnection()->commit();
} catch (Exception $e) {
$em1->getConnection()->rollback();
$em2->getConnection()->rollback();
throw $e;
}
With this small change, your example does demonstrate the correct way to deal with transactions that span multiple entity managers.

I'd like to point out a couple of things that may clear your mind on the matter:
I'm not aware of the process you use to create the second entity manager, keep in mind that 2 completely different entity manager will not share the same connection. Can you point out your use case for 2 different entity manager?
Consider that the operation:
$em1->getConnection()->commit();
will commit the first transaction and you will loose the privilege to rollback such transaction should an error arise from the second transaction.
Doctrine\ORM\ORMException exception (The EntityManager is closed.)
It's typical when you try to operate any commit/flush operation after a DBAL (database related) exception has been thrown; in this case Doctrine default behaviour is to close the entity manager.
And it is common practice to do so after any rollback:
$em1->getConnection()->rollback();
$em1->close();
Hope it helps,
Regards.

Related

Doctrine: atomic updates and exceptions in a loop

We are migrating a project from a more basic ORM to using Symfony+Doctrine. In the project we have a lot of cron jobs looking like this:
$rows = $someRepository->getRows();
foreach ($rows as $row) {
try {
$db->beginTransaction(); //simple begin transaction in db
//do some handling of data
// Maybe load some other entities and update those
// ...
$db->commit();
} catch (Throwable $t) {
//log error
//clear entity cache
$db->rollback(); //simple rollback in db
}
}
When we did it this way, all changes within the try catch was atomic while it at the same time was possible to recover from an error and continue on the next $row.
In Symfony+Doctrine, I simply cannot figure out how to mimic this behaviour. The recommendation from Doctrine to handle an exception is closing the EntityManager, but how do you recover?
The ORM does this implicitly on flush, so most of the time you can avoid the hassle of doing so on your own.
However, if you want clear demarcation you can still do it explicitly, in a similar manner you did so far.
More reading and examples here: https://www.doctrine-project.org/projects/doctrine-orm/en/2.7/reference/transactions-and-concurrency.html
EDIT related to the comment below:
Instead of injecting the manager, you should inject the registry.
After that on catch, you can check if the $em->isOpen(), and call $registry->resetManager() if not.
I suspect this will also reset the unit of work, so you might encounter detached entities. In that case you should do $em->merge();
One thing to note here is that an expection is not considered normal in doctrine, so they are closing the manager because of that. You might think that this is overcompicated - yes it is, because you are working against the philosophy here. Validate your data if you can. Read this section: https://www.doctrine-project.org/projects/doctrine-orm/en/2.7/reference/transactions-and-concurrency.html#exception-handling
As for the why: (This is not offical, just based on my knowledge) The managers internal unit of work is a stateful object. When an exception occures during a transaction that state will remain the same, but couln't be persisted to the database. If they let this go that would mean the EM would try to apply all state changes again, and would encounter the same exception again. So no point in leaving it open in the same state, a reset is needed.

In Disassembler pipeline component - Send only last message out from GetNext() method

I have a requirement where I will be receiving a batch of records. I have to disassemble and insert the data into DB which I have completed. But I don't want any message to come out of the pipeline except the last custom made message.
I have extended FFDasm and called Disassembler(), then we have GetNext() which is returning every debatched message out and they are failing as there is subscribers. I want to send nothing out from GetNext() until Last message.
Please help if anyone have already implemented this requirement. Thanks!
If you want to send only one message on the GetNext, you have to call on Disassemble method to the base Disassemble and get all the messages (you can enqueue this messages to manage them on GetNext) as:
public new void Disassemble(IPipelineContext pContext, IBaseMessage pInMsg)
{
try
{
base.Disassemble(pContext, pInMsg);
IBaseMessage message = base.GetNext(pContext);
while (message != null)
{
// Only store one message
if (this.messagesCount == 0)
{
// _message is a Queue<IBaseMessage>
this._messages.Enqueue(message);
this.messagesCount++;
}
message = base.GetNext(pContext);
}
}
catch (Exception ex)
{
// Manage errors
}
Then on GetNext method, you have the queue and you can return whatever you want:
public new IBaseMessage GetNext(IPipelineContext pContext)
{
return _messages.Dequeue();
}
The recommended approach is to publish messages after disassemble stage to BizTalk message box db and use a db adapter to insert into database. Publishing messages to message box and using adapter will provide you more options on design/performance and will decouple your DB insert from receive logic. Also in future if you want to reuse the same message for something else, you would be able to do so.
Even then for any reason if you have to insert from pipeline component then do the following:
Please note, GetNext() method of IDisassembler interface is not invoked until Disassemble() method is complete. Based on this, you can use following approach assuming you have encapsulated FFDASM within your own custom component:
Insert all disassembled messages in disassemble method itself and enqueue only the last message to a Queue class variable. In GetNext() message then return the Dequeued message, when Queue is empty return null. You can optimize the DB insert by inserting multiple rows at a time and saving them in batches depending on volume. Please note this approach may encounter performance issues depending on the size of file and number of rows being inserted into db.
I am calling DBInsert SP from GetNext()
Oh...so...sorry to say, but you're doing it wrong and actually creating a bunch of problems doing this. :(
This is a very basic scenario to cover with BizTalk Server. All you need is:
A Pipeline Component to Promote BTS.InterchageID
A Sequential Convoy Orchestration Correlating on BTS.InterchangeID and using Ordered Delivery.
In the Orchestration, call the SP, transform to SOAP, call the SOAP endpoint, whatever you need.
As you process the Messages, check for BTS.LastInterchagneMessage, then perform your close out logic.
To be 100% clear, there are no practical 'performance' issues here. By guessing about 'performance' you've actually created the problem you were thinking to solve, and created a bunch of support issues for later on, sorry again. :( There is no reason to not use an Orchestration.
As noted, 25K records isn't a lot. Be sure to have the Receive Location and Orchestration in different Hosts.

intershop ORMException could not update - refresh ORMObject

In a clustered intershop environment, we see a lot of error messages. I'm suspecting the communication between the application servers is not reliable.
Caused by: com.intershop.beehive.orm.capi.common.ORMException:
Could not UPDATE object: com.intershop.beehive.bts.internal.orderprocess.basket.BasketPO
Is there safe way to for the local application server, to load the latest instance.
BasketPO basket = null;
try{
BasketPOFactory factory = (BasketPOFactory) NamingMgr.getInstance().lookupFactory(BasketPOFactory.FACTORY_NAME);
try(ORMObjectCollection<BasketPO>baskets = factory.getObjectsBySQLWhere("uuid=?", new Object[]{basketID},CacheMode.NO_CACHING);){
if(null != baskets && !baskets.isEmpty()){
basket = baskets.stream().findFirst().get();
}
}
}
catch(Throwable t){
Logger.error(this, t.getMessage(),t);
}
Does the ORMObject#refresh method help ?
try{
if(null != basket)
basket.refresh();
}
catch(Throwable t){
Logger.error(this, t.getMessage(),t);
}
You experience that error because an optimistic lock "fails". To understand the problem better I'll try to explain how the optimistic locking works in particular in the Intershop ORM layer.
There is a column named OCA in the PO tables (OCA == optimistic control attribute?). Imagine that two servers (or two different threads/transactions) try to update the same row in a table. For performance reasons there is no DB locking involved by default (e.g. by issuing select for update). Instead the first thread/server increments the OCA by one when it updates the row successfully within its transaction.
The second thread/server knows the value of the OCA from the time that it created its own state. It then tries to update the row by issuing a similar query:
UPDATE ... OCA = OCA + 1 ... WHERE UUID = <uuid> AND OCA = <old_oca>
Since the OCA is already incremented by the first thread/server this update fails (in reality - updates 0 rows) and the exception that you posted above is thrown when the ORM layer detects that no rows were updated.
Your problem is not the inter-server communication but rather the fact that either:
multiple servers/threads try to update the same object;
there are direct updates in the database that bypass the ORM layer (less likely);
To solve this you may:
Avoid that situation altogether (highly recommended by me :-) );
Use the ISH locking framework (very cumbersome imHo);
Use pesimistic locking supported by the ISH ORM layer and Oracle (beware of potential performance issues, deadlocks, bugs);
Use Java locking - but since the servers run in different JVM-s this is rarely an option;
OFFTOPIC remarks: I'm not sure why you use getObjectsBySQLWhere when you know the primary key (uuid). As far as I remember ORMObjectCollection-s should be closed if not iterated completely.
UPDATE: If the cluster is not configured correctly and the multicasts can't be received from the nodes you won't be able to resolve the problems programatically.
The "ORMObject.refresh()" marks the cached shared state as invalid. Next access to the object reloads the state from the database. This impacts the performance and increase the database server load.
BUT:
The "refresh()" method does not reload the PO instance state if it already assigned to the current transaction.
Would be best to investigate and fix the server communication issues.
Other possibility is that it isn't a communication problem (multicast between node in the cluster i assume), but that there are simply two request trying to update the basket at the same time. Example two ajax request to update something on the basket.
I would avoid trying to "fix" the orm, it would only cause more harm than good. Rather investigate further and post back more information.

Entity Framework delete object - concurrency error (how to disable concurrency)?

Is there a way to disable concurrency error thrown in EntityFramework?
For example:
using (xEntities5 entities = new xEntities5())
{
entities.Domains.MergeOption = System.Data.Objects.MergeOption.NoTracking;
Domain domain = new Domain() { DomainId = id };
EntityKey key = entities.CreateEntityKey(entities.CreateObjectSet<Domain>().EntitySet.Name, domain);
domain.EntityKey = key;
entities.Attach(domain);
//entities.AttachTo(entities.CreateObjectSet<Domain>().EntitySet.Name, domain);
entities.DeleteObject(domain);
return entities.SaveChanges(); // returns affected rows... must catch error?
}
Is there a way to not have to do try/catch around SaveChanges in order to detect if nothing was deleted?
As I know, you can't turn off concurrency error. Concurrency error is based on number of affected rows so if you want to delete a row and it is not deleted (for example because it doesn't exist any more) concurrency exception is fired. Moreover SaveChanges works in transaction so if you want to delete 5 rows and only 4 rows are deleted the exception is fired and all deletes are rolled back.
Concurrency test can be even more restrictive if you use columns marked as ConcurrencyMode.Fixed. These columns are used in where condition of SQL statements so only unmodified database records can be processed.
Once you get concurrency exception you are supposed to solve it.
SaveChanges() will throw exceptions and doesn't catch them internally, so if you want to continue execution, you have to use try/catch. SaveChanges() also returns a method with the number of entities committed to the database, that is, if no error occurred :-)
HTH.

Which type exception checking required in linq inside try/catch block while performing CRUD OPERATION

Which type exception checking required in linq inside try/catch block while performing CRUD(create,read,update,delete) OPERATIONS
for eg:
try {
db.SubmitChanges(ConflictMode.ContinueOnConflict);
}
catch (ChangeConflictException e) {
foreach (ObjectChangeConflict occ in db.ChangeConflicts) {
// All database values overwrite current values.
occ.Resolve(RefreshMode.OverwriteCurrentValues);
}
}
If it's not well-documented what exceptions will be thrown in normal use-case scenarios (and I can't say off the top of my head what will be thrown in your situation), I suggest trying to break it by performing operations you know will fail, and then add the exception that's thrown to the try/catch block.
If anything, trying to break your own code is a good debugging exercise, as it exposes problems that are likely to occur giving you a chance to recover gracefully.

Resources