Doctrine setAutoCommit(false) method and "there is no active transaction" message - symfony

EDIT: The question is about why using setAutoCommit(false) is a solution for the "there is no active transaction" exception. Forget this question since this is not the correct solution (at least in my case). I will leave the question here in case somebody encounters the same problem. See my answer below for more details.
================
The following code worked fine in Symfony 2.7, but after an update to Symfony 2.8 (and to the latest DoctrineBundle version), a There is no active transaction exception is thrown:
private function getSynchronization() {
$lock_repo = $this->entityManager->getRepository('MyAppBundle\Entity\DBLock');
$this->entityManager->getConnection()->beginTransaction();
try {
$sync = $lock_repo->findOneByUser($this->getUser());
if (!$lock) {
$lock = new DBLock();
} else {
if ($lock->isActive()) {
// ... Exception: Process already running
}
$expected_version = $lock->getVersion();
$this->entityManager->lock($lock, LockMode::OPTIMISTIC, $expected_version);
}
$sync->setActive(false);
$this->entityManager->persist($sync);
$this->entityManager->flush();
$this->entityManager->getConnection()->commit();
// EXCEPTION on this line
$this->entityManager->lock($lock, LockMode::NONE);
}
catch(\Exception $e) {
$this->entityManager->getConnection()->rollback();
throw new ProcessException($e->getMessage());
}
...
}
After some searching I found a solution in another post. After adding the following line everything works fine:
private function getSynchronization() {
$lock_repo = $this->entityManager->getRepository('MyAppBundle\Entity\DBLock');
$this->entityManager->getConnection()->beginTransaction();
// ADDED LINE
$this->entityManager->getConnection()->setAutoCommit(false);
try {
...
So, the question is not how to solve the problem, but how the solution works...
I am quite confused by the Doctrine docs of the setAutoCommit() method:
To have a connection automatically open up a new transaction on
connect() and after commit() or rollBack(), you can disable
auto-commit mode with setAutoCommit(false)
I do not understand this.
Does this mean, that the transaction that was started/created with beginTransaction() is now automatically closed when using commit()? So in order to be able to use lock(...) after using commit() I have to begin a new transaction first. I can either do this manually by calling beginTransaction() again, or auotmatically by setting setAutoCommit(false) before. Is that correct?
Is this a change in on of the latest Doctrine versions? I did not found anything about in in the updates notes and before the update of Symfony/Doctrine the code worked just fine.
Thank you very much!

As described before I encountered the problem, that calling lock($lock, LockMode::NONE) suddenly threw a There is no active transaction exception after the Update from Doctrine 2.4 to 2.5.
My solution was to add setAutoCommit(false), which automatically created a new transaction after calling commit(). It worked and the exception did not occur again. However, this is not the real/correct solution, it creates other problems as side effects.
After re-reading the Doctrine Update Notes I found out, that the correct solution is to use lock($lock, null) instead of lock($lock, LockMode::NONE). This is BC Break between Doctrine 2.4 and 2.5.
Maybe my question and answer helps someone else who encounters the same problem.

Related

BackoffExceptions are logged at error level when using RetryTopicConfiguration

I am a happy user of the recently added RetryTopicConfiguration there is however a small issue that is bothering me.
The setup I use looks like:
#Bean
public RetryTopicConfiguration retryTopicConfiguration(
KafkaTemplate<String, String> template,
#Value("${kafka.topic.in}") String topicToInclude,
#Value("${spring.application.name}") String appName) {
return RetryTopicConfigurationBuilder
.newInstance()
.fixedBackOff(5000L)
.maxAttempts(3)
.retryTopicSuffix("-" + appName + ".retry")
.suffixTopicsWithIndexValues()
.dltSuffix("-" + appName + ".dlq")
.includeTopic(topicToInclude)
.dltHandlerMethod(KAFKA_EVENT_LISTENER, "handleDltEvent")
.create(template);
}
When the a listener throws an exception that triggers a retry, the DefaultErrorHandler will log a KafkaBackoffException at error level.
For a similar problem it was suggested to use a ListenerContainerFactoryConfigurer yet this does not remove all error logs, since I still see the following in my logs:
2022-04-02 17:34:33.340 ERROR 8054 --- [e.retry-0-0-C-1] o.s.kafka.listener.DefaultErrorHandler : Recovery of record (topic-spring-kafka-logging-issue.retry-0-0#0) failed
org.springframework.kafka.listener.ListenerExecutionFailedException: Listener failed; nested exception is org.springframework.kafka.listener.KafkaBackoffException: Partition 0 from topic topic-spring-kafka-logging-issue.retry-0 is not ready for consumption, backing off for approx. 4468 millis.
Can the log-level be changed, without adding a custom ErrorHandler?
Spring-Boot version: 2.6.6
Spring-Kafka version: 2.8.4
JDK version: 11
Sample project: here
Thanks for such a complete question. This is a known issue of Spring for Apache Kafka 2.8.4 due to the new combine blocking and non-blocking exceptions feature and has been fixed for 2.8.5.
The workaround is to clear the blocking exceptions mechanism such as:
#Bean(name = RetryTopicInternalBeanNames.LISTENER_CONTAINER_FACTORY_CONFIGURER_NAME)
public ListenerContainerFactoryConfigurer lcfc(KafkaConsumerBackoffManager kafkaConsumerBackoffManager,
DeadLetterPublishingRecovererFactory deadLetterPublishingRecovererFactory,
#Qualifier(RetryTopicInternalBeanNames
.INTERNAL_BACKOFF_CLOCK_BEAN_NAME) Clock clock) {
ListenerContainerFactoryConfigurer lcfc = new ListenerContainerFactoryConfigurer(kafkaConsumerBackoffManager, deadLetterPublishingRecovererFactory, clock);
lcfc.setBlockingRetriesBackOff(new FixedBackOff(0, 0));
lcfc.setErrorHandlerCustomizer(eh -> ((DefaultErrorHandler) eh).setClassifications(Collections.emptyMap(), true));
return lcfc;
}
Please let me know if that works for you.
Thanks.
EDIT:
This workaround disables only blocking retries, which since 2.8.4 can be used along non-blocking as per the link in the original answer. The exception classification for the non-blocking retries is in the DefaultDestinationTopicResolver class, and you can set FATAL exceptions as documented here.
EDIT: Alternatively, you can use the Spring Kafka 2.8.5-SNAPSHOT version by adding the Spring Snapshot repository such as:
repositories {
maven {
url 'https://repo.spring.io/snapshot'
}
}
dependencies {
implementation 'org.springframework.kafka:spring-kafka:2.8.5-SNAPSHOT'
}
You can also downgrade to Spring Kafka 2.8.3.
As Gary Russell pointed out, if your application is already in production you should not use the SNAPSHOT version, and 2.8.5 is out in a couple of weeks.
EDIT 2: Glad to hear you’re happy about the feature!

Realm doesn’t work with xUnite and .net core

I’m having issues running realm with xUnite and Net core. Here is a very simple test that I want to run
public class UnitTest1
{
[Scenario]
public void Test1()
{
var realm = Realm.GetInstance(new InMemoryConfiguration("Test123"));
realm.Write(() =>
{
realm.Add(new Product());
});
var test = realm.All<Product>().First();
realm.Write(() => realm.RemoveAll());
}
}
I get different exceptions on different machines (Windows & Mac) on line where I try to create a Realm instace with InMemoryConfiguration.
On Mac I get the following exception
libc++abi.dylib: terminating with uncaught exception of type realm::IncorrectThreadException: Realm accessed from incorrect thread.
On Windows I get the following exception when running
ERROR Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. at
System.Net.Sockets.NetworkStream.Read(Span1 destination) at
System.Net.Sockets.NetworkStream.ReadByte() at
System.IO.BinaryReader.ReadByte() at
System.IO.BinaryReader.Read7BitEncodedInt() at
System.IO.BinaryReader.ReadString() at
Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.LengthPrefixCommunicationChannel.NotifyDataAvailable() at
Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.TcpClientExtensions.MessageLoopAsync(TcpClient client, ICommunicationChannel channel, Action1 errorHandler, CancellationToken cancellationToken) Source: System.Net.Sockets HResult: -2146232800 Inner Exception: An existing connection was forcibly closed by the remote host HResult: -2147467259
I’m using Realm 3.3.0 and xUnit 2.4.1
I’ve tried downgrading to Realm 2.2.0, and it didn’t work either.
The solution to this problem was found in this Github post
The piece of code from that helped me to solve the issue
Realm GetInstanceWithoutCapturingContext(RealmConfiguration config)
{
var context = SynchronizationContext.Current;
SynchronizationContext.SetSynchronizationContext(null);
Realm realm = null;
try
{
realm = Realm.GetInstance(config);
}
finally
{
SynchronizationContext.SetSynchronizationContext(context);
}
return realm;
}
Though it took a while for me to apply this to my solution.
First and foremost, instead of just setting the context to null I am using Nito.AsyncEx.AsyncContext. Because otherwise automatic changes will not be propagated through threads, as realm needs a non-null SynchronizationContext for that feature to work. So, in my case the method looks something like this
public class MockRealmFactory : IRealmFactory
{
private readonly SynchronizationContext _synchronizationContext;
private readonly string _defaultDatabaseId;
public MockRealmFactory()
{
_synchronizationContext = new AsyncContext().SynchronizationContext;
_defaultDatabaseId = Guid.NewGuid().ToString();
}
public Realm GetRealmWithPath(string realmDbPath)
{
var context = SynchronizationContext.Current;
SynchronizationContext.SetSynchronizationContext(_synchronizationContext);
Realm realm;
try
{
realm = Realm.GetInstance(new InMemoryConfiguration(realmDbPath));
}
finally
{
SynchronizationContext.SetSynchronizationContext(context);
}
return realm;
}
}
Further, this fixed a lot of failing unit tests. But I was still receiving that same exception - Realm accessed from incorrect thread. And I had no clue why, cause everything was set correctly. Then I found that the tests that were failing were related to methods where I was using async realm api, in particular realm.WriteAsync. After some more digging I found the following lines in the realm documentation.
It is not a problem if you have set SynchronisationContext.Current but
it will cause WriteAsync to dispatch again on the thread pool, which
may create another worker thread. So, if you are using Current in your
threads, consider calling just Write instead of WriteAsync.
In my code there was no direct need of using the async API. I removed and replaced with sync Write and all the tests became green again! I guess if I find myself in a situation that I do need to use the async API because of some kind of bulk insertions, I'd either mock that specific API, or replace with my own background thread using Task.Run instead of using Realm's version.

Gitkit Java client library - Trouble verifying token signature: Invalid audience

We're struggling with an issue during the token verification. We have the following exception:
java.security.SignatureException: Invalid audience: xxx-platform. Should be: 787384428332-32charsofidxxxxxxxx.apps.googleusercontent.com
at com.google.identitytoolkit.JsonTokenHelper$AudienceChecker.check(JsonTokenHelper.java:67)
at net.oauth.jsontoken.JsonTokenParser.verify(JsonTokenParser.java:156)
at net.oauth.jsontoken.JsonTokenParser.verify(JsonTokenParser.java:103)
at net.oauth.jsontoken.JsonTokenParser.verifyAndDeserialize(JsonTokenParser.java:116)
at com.google.identitytoolkit.JsonTokenHelper.verifyAndDeserialize(JsonTokenHelper.java:46)
at com.google.identitytoolkit.GitkitClient.validateToken(GitkitClient.java:126)
at com.google.identitytoolkit.GitkitClient.validateTokenInRequest(GitkitClient.java:154)
at com.some.package.user.GitKitUserService.getGitkitUserFromRequest(GitKitUserService.groovy:25)
We have checked many times the gitkit-server-config.json file, he seems to correct and points to a valid .p12 file. The p12 is correctly found and opened (since we have a FileNotFoundException when we remove it, or parsing error when we alter it...) but the validation fails because of a null verifier...
Here it is:
{
"clientId": "707385568332-32charsofidxxxxxxxx.apps.googleusercontent.com",
"projectId": "xxx-platform",
"serviceAccountEmail": "xxx#xxx-platform.iam.gserviceaccount.com",
"serviceAccountPrivateKeyFile": "/an/existing/path/xxx-platform-44d0379d237c.p12",
"widgetUrl": "https://example.com/authentication/authenticate",
"cookieName": "gtoken"
}
Of course we can provide any additional information that might be required, we're really stuck with this issue!
Thank in advance for any clue!
I think DFB's answer is correct.
But we don't recommend hard-coded json config in Java code. There's a static method called createFromJson you can use to read json file and then initialize GitkitClient.
We'll also need to update the README in identity-toolkit-java-client. Thanks for your question.
I'll just share my experience from setting up earlier today incase it can help you:
String token = cookie.getValue();
try {
GitkitClient gitkitClient = GitkitClient.newBuilder()
.setGoogleClientId("206268081687-u5mg1cl3teeeo635vrsuj8uotdi7meqq.apps.googleusercontent.com")
//.setGoogleClientId("effortless-edge-119904")
.setServiceAccountEmail("tables#effortless-edge-119904.iam.gserviceaccount.com")
.setCookieName("gtoken")
.setWidgetUrl("http://localhost:8080/gitkit")
.setKeyStream(new ClassPathResource("tables-8271416a8e0c.p12").getInputStream()).build();
GitkitUser gitkitUser = gitkitClient.validateToken(token);
Gives me
java.security.SignatureException: Gitkit token audience(effortless-edge-119904)
doesn't match projectId or clientId in server configuration
This works:
try {
GitkitClient gitkitClient = GitkitClient.newBuilder()
.setGoogleClientId("effortless-edge-119904")
.setServiceAccountEmail("tables#effortless-edge-119904.iam.gserviceaccount.com")
.setCookieName("gtoken")
.setWidgetUrl("http://localhost:8080/gitkit")
.setKeyStream(new ClassPathResource("tables-8271416a8e0c.p12").getInputStream()).build();
GitkitUser gitkitUser = gitkitClient.validateToken(token);
logger.info("Validated gitkit token");
I was getting the same error and stumbled upon this thread. I was using gitclient-1.2.3.jar. I updated it to gitkitclient-1.2.5.jar (latest) and the problem went away.
UPDATE: I'm adding the code snippet below. I'm setting both setGoogleClientId and setProjectId as shown in the sample https://github.com/google/identity-toolkit-java-client/blob/master/src/main/java/com/google/identitytoolkit/GitkitClient.java
GitkitClient gitkitClient = new GitkitClient.Builder()
.setGoogleClientId("654028407702-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.apps.googleusercontent.com")
.setProjectId("my-project")
.setServiceAccountEmail("my-project#my-project.iam.gserviceaccount.com")
.setKeyStream(context.getResourceAsStream("/WEB-INF/identity/my-project-xxxxxxxxxxxx.p12"))
.setWidgetUrl("https://my-project.appspot.com/oauth2callback")
.setCookieName("gToken")
.setServerApiKey("AIzaSyAxQ7z5Dxxxxxxxxxxxxxx-xxxxxxxx")
.build();
I had a look at the gitkitclient.js source code and both projectId and clientId are added to the same audiences array.
After more tests I found out that you must only put the project ID ('my-project-name') in the gitkit-server-config.json file.
The nasty thing is that if you add it with a 'clientId' property name it is also working...
As far as I can see, the client ID (like 654028407702-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.apps.googleusercontent.com) can be removed.

symfony2 can't catch PDOException

I want to catch PDOException in symfony 2.6, especially ConnectionException.
For instance if I stop my MySQL server I want to catch that exception and return a customised message to the user, but it seem that it's uncatchable in customised kernel.exception listner, and either in try catch block, i don't know if it's a symfony problem or something must be done.
I also tried out to customise error page like said in documentation but usselssly, I seached in web for solution, but I found nothing except something about redefining a controller in frameworkbundle whish is responsable of converting Exception into error page.
But I really don't want to go for that solution since i'm new with symfony.
You can do this by creating an exception listener and catch Pdo exception :
service.yml:
kernel.listener.your_pdo_listener:
class: Acme\AppBundle\EventListener\YourExceptionListener
tags:
- { name: kernel.event_listener, event: kernel.exception, method: onPdoException }
Then the listener class :
YourExceptionListener
UPDATED
use Symfony\Component\HttpKernel\Event\GetResponseForExceptionEvent;
class YourExceptionListener
{
public function onPdoException(GetResponseForExceptionEvent $event)
{
$exception = $event->getException();
if ($exception instanceof \PDOException || $exception->getPrevious() instanceof \PDOException) {
//now you can do whatever you want with this exception
}
}
}
Snippets from : Catching database exceptions in Symfony2
I have done more test, so the test i made first was making a query to the database, that's why i got pdoexception as first exception but sometime it can be a twig exception as you know twig throw runtime exception if it couldn't contact database but hopefully we can get the previous exception also and this can work with other exceptions which can be thrown after the PDO ones, so hopefully it will work for you as expected so i edited the code to check if previous exception is a PDOException also.

Extending phpunit error message

I want to add log output to all test messages.
$this->assertTrue(FALSE, "This assertion is False.". myLog::GetErrors());
I tried extending the PHPUnit_Framework_TestCase::assertThat function with my appended message, but it doesn't seem to have an effect on it. I know the extended PHPUnit_Framework_TestCase works because I have several helper functions (random string generation) in there as well that I use for testing.
Any other thoughts? Could it be a customer listener?
All of the assertions are static methods from PHPUnit_Framework_Assert and cannot be overridden by a subclass of TestCase. You can define your own assertions that call the originals with an amended message, however.
public static function assertTrue($value, $message='') {
PHPUnit_Framework_Assert::assertTrue($value, $message . myLog::GetErrors());
}
All failed tests call onNotSuccessfulTest() with the exception as the only parameter. You could override this method and in some cases add your log errors to the exception's message. Some of the PHPUnit exceptions provide a second description in addition to the error message contained in the exception.
public function onNotSuccessfulTest(Exception $e) {
if ($e instanceof PHPUnit_Framework_ExpectationFailedException) {
$e->setCustomMessage(implode(PHP_EOL,
array($e->getCustomMessage(), myLog::GetErrors())));
}
parent::onNotSuccessfulTest($e);
}
Update: As Gregory Lo noted in a comment below, setCustomMessage() and getCustomMessage() were removed in PHPUnit 3.6. :(
In recent PHPUnit Versions it is not necessary anymore.
When you specify the error parameter in an $this->assertXXXX() it preceeds PHPUnit's original message.
Screenshot:

Resources