My team is writing a service that leverages the retryable topics mechanism offered by Spring Kafka (version 2.8.2). Here is a subset of the configuration:
#Bean
public ConsumerFactory<String, UploadMessage> consumerFactory() {
return new DefaultKafkaConsumerFactory<>(
this.springProperties.buildConsumerProperties(),
new StringDeserializer(),
new ErrorHandlingDeserializer<>(new KafkaMessageDeserializer()));
}
#Bean
public RetryTopicConfiguration retryTopicConfiguration(KafkaTemplate<String, Object> kafkaTemplate) {
final var retry = this.applicationProperties.retry();
return RetryTopicConfigurationBuilder.newInstance()
.doNotAutoCreateRetryTopics()
.suffixTopicsWithIndexValues()
.maxAttempts(retry.attempts())
.exponentialBackoff(retry.initialDelay(), retry.multiplier(), retry.maxDelay())
.dltHandlerMethod(DeadLetterTopicProcessor.ENDPOINT_HANDLER_METHOD)
.create(kafkaTemplate);
}
KafkaMessageDeserializer is a custom deserialiser that decodes protobuf-encoded messages and may throw a SerializationException in case of a failure. This exception is correctly captured and transformed into a DeserializationException by Spring Kafka. What I find a bit confusing is that the intercepted poison pill message then hits all of the retry topics before eventually reaching the dead letter one. Obviously it fails with exactly the same error at every step.
I know that RetryTopicConfigurationBuilder::notRetryOn may be used to skip the retry attempts for particular exception types, but what if I want to use exactly the same list of exceptions as in ExceptionClassifier::configureDefaultClassifier? Is there a way to programmatically access this information without basically duplicating the code?
That is a good suggestion; it probably should be the default behavior (or at least optionally).
Please open a feature request on GitHub.
There is a, somewhat, related discussion here: https://github.com/spring-projects/spring-kafka/discussions/2101
I honestly have no idea where to begin. The repository aspect is relatively simple but I cannot seem to find any information on how to delete an aggregate root via the CommandGateway.
Any directions and/or documentation on how to achieve this would be greatly appreciated.
Putting this here for future reference for anyone else that might be as lost as I was initially.
When using the Event Sourcing Aggregate, one can make use of the markDeleted() static method on the Aggregate in question. I placed mine in the #EventSourcingHandler
import static org.axonframework.modelling.command.AggregateLifecycle.markDeleted;
#EventSourcingHandler
public void on(DeletedEvent event){
markDeleted();
}
Further information can be found at: https://docs.axoniq.io/reference-guide/implementing-domain-logic/command-handling/aggregate#aggregate-lifecycle-operations
To delete the view data associated with the aggregate I used an external #EventHandler:
#EventHandler
public void on(DeletedEvent event, ReplayStatus status){
entityRepo.deleteById(event.getId());
}
Thanks to Allard for engaging me in the comments section.
I read spring-kafka/kafka documentation back and forth, and still cannot find a way, how to do proper transactional behavior with error recovering. I believe this is not trivial question, so please read until end. I believe whole this question revolves around finding way how to reposition over failing record or how to ack in error handler. But mabye there are better ways, I don't know.
So records are flowing in, and some of them are invalid. What I would like to have as a minimal solution is(in which I will then fix sevaral problems you probably see as well):
1) we cannot afford the luxury of stopping the production in case of some trivial mishap, like one or few invalid records. Thus if there is invalid record in kafka topic, I would like to log it, or resend it to different queue, but then proceed with processing following records.
2) there are permanent and temporary failures. Permanent failure is record unable to deserialize, record failing data validation. In this case, I'd like to skip the invalid record, as discussed in 1). Temporary failure might be some specific exception or state, like for example database connection errors, network issues etc. In this case, we do not want to skip failing record, we want to retry, after some delay.
Subject of this question is ONLY implementing skip/don't skip behavior.
Lets say, that this is our starting point:
private Map<String, Object> createKafkaConsumerFactoryProperties(String bootstrapServers, String groupId, Class<?> valueDeserializerClass) {
Map<String, Object> props = new HashMap<>();
props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers);
props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, valueDeserializerClass);
props.put(ConsumerConfig.GROUP_ID_CONFIG, groupId);
return props;
}
#Bean(name="SomeFactory")
public ConcurrentKafkaListenerContainerFactory<String, String> kafkaListenerContainerFactory(
#Value("${…}") String bootstrapServers,
#Value("${…}") String groupId) {
ConcurrentKafkaListenerContainerFactory<String, String> factory =
new ConcurrentKafkaListenerContainerFactory<>();
ConsumerFactory<String, String> consumerFactory = new DefaultKafkaConsumerFactory<>(
createKafkaConsumerFactoryProperties(bootstrapServers, groupId, AvroDeserializer.class),
new StringDeserializer(),
new AvroDeserializer(SomeClass.class));
factory.setConsumerFactory(consumerFactory);
// factory.setConcurrency(2);
// factory.setBatchListener(true);
return factory;
}
and we have listener like:
#KafkaListener(topics = "${…}", containerFactory = "SomeFactory")
public void receive(#Valid List<SomeClass> messageList) {/*logic*/}
Now how this behave if I understand correctly:
when listener gets message, ~when we reach inside of receive method, the kafka message will be already acked, and if receive method throw an exception, the next poll will return following record. Because ack happened, and we do not have error handler defined, thus logging error handler will kick in. This is not necessarily what we want. We can use SeekToCurrentErrorHandler to reprocess the message. Or one can specify TransactionManager, and if exception 'leaks' from listener, repositioning will also happen. If someone know performance comparison of these two approaches, please tell me.
when message cannot be deserialized, deserializer will fail, message will not be acked and same record will be polled again. This is some sort of "poison packet" since kafka will spin on this message indefinitelly. We do have retry.backoff.ms to at least slow it down, but I can't see any max number retries or something. So the best thing we can do is to stop/pause container in this situation. Which is way to harsh. Btw. I'm new to kafka/spring-kafka, I did not see anywhere mention, how to manually reposition offset from outside of an application, meaning OK, listener is down, but now what? Another solution would be not to fail deserializer, and return something. But what?? KafkaNull, great, but then our listener will fail because SomeClass ClassCastException. We can send some artificial value of SomeClass, which is again horrible, because this is not a data what we actually get. Also this is architectonically incorrect.
or we can use repositioning error handler, which would be great, well if we know how to do that. I need to seek to next record. But while documentation says, that ErrorHandler should communicate which record caused the failure, it seems that it fails to do so. So even in non-batch listener I have list of records(1 failed + bunch of unprocessed), and have no idea where set offset to.
So what is the solution to this madness?
Well the best I can come up with right now is pretty ugly: do not fail in deserializer (bad), do not accept specific type in listener (bad), filter out KafkaNulls manually (bad) and finally trigger bean validation manually (bad). Is there a better way? Thanks for examplantion, I'd be grateful for every hint or direction given how to achieve this.
See the documentation for the upcoming 2.2 release (due tomorrow).
The DefaultAfterRollbackProcessor (when using transactions) and SeekToCurrentErrorHandler (when not using transactions) can now recover (skip) records that keep failing, and will do so after 10 failures, by default. They can be configured to publish failed records to a dead-letter topic.
Also see the Error Handling Deserializer which catches deserialization problems and passes them to the container so they can be sent to the error handler.
Does creating an activity using the .withIntent() not work in Robolectric 2? I'm doing the following
activity = Robolectric.buildActivity(MyActivity.class)
.create()
.withIntent(intent)
.get();
And i'm getting a NullPointerException when doing the following in the onCreate() of my activity.
Bundle bundle = getIntent().getExtras();
I can code a null check in my onCreate() and set the intent by doing the following but it seems redundant to set the intent and call the onCreate() method again when Robolectric already does that when creating the Activity instance. This seems like an unnecessary work around.
Robolectric.shadowOf(activity).setIntent(intent);
activity.onCreate(null);
This is a case where a fluent-style API kinds of leads you down the wrong path...
You want to:
activity = Robolectric.buildActivity(MyActivity.class)
.withIntent(intent)
.create()
.get();
so that the intent is provided to the builder before it calls onCreate().
For newer versions of Robolectric use Robolectric.buildActivity(Class, Intent).
I figured out my problem. I wasn't instantiating the Intent properly. I was instantiating it with the no-arg constructor when i needed to give a Context and the class of the Activity it was being sent to
EDIT: It was fixed in version 2.2.
I tackled with the same issue. It was reported but no fix has been provided yet. For now, I manage to hack it using Activity's setter before calling onCreate(), taking advantage from the fact that its lifecycle has not yet started:
Intent intent = new Intent();
MainActivity mainActivity = Robolectric.buildActivity(MainActivity.class)
.create()
.get();
mainActivity.setIntent(intent);
mainActivity.onCreate(null);
Is there any performance problem or something else about letting the exception to propagate, or it is better to write it like this
try
{
}
catch
{
throw;
}
If you're not going to handle the exception it's better to have nothing rather than what you propose. All that does is add the overhead of catching and then rethrowing the same exception.
If you can handle the exception do so, but then don't propagate it further up the call stack.
The only time I can think of when I'd have that kind of empty catch\rethrow logic is when I'd want to log the exception in some way, otherwise I'd just let it propagate.
EDIT: added the missing word empty