RetryingBatchErrorHandler - Offset commit handling - spring-kafka

Question

I'm using spring-kafka 2.3.8 and I'm trying to log the recovered records and commit the offsets using RetryingBatchErrorHandler. How would you commit the offset in the recoverer?
public class Customizer implements ContainerCustomizer{
private static ConsumerRecordRecoverer createConsumerRecordRecoverer() {
return (consumerRecord, e) -> {
log.info("Number of attempts exhausted. parition: " consumerRecord.partition() + ", offset: " + consumerRecord.offset());
# need to commit the offset
};
}
#Override
public void configure(AbstractMessageListenerContainer container) {
container.setBatchErrorHandler(new RetryingBatchErrorHandler(new FixedBackOff(5000L, 3L), createConsumerRecordRecoverer()));
}

Answer 1

The container will automatically commit the offsets if the error handler "handles" the exception, unless you set the ackAfterHandle property to false (it is true by default).
EDIT
This works as expected for me:
#SpringBootApplication
public class So69534923Application {
private static final Logger log = LoggerFactory.getLogger(So69534923Application.class);
public static void main(String[] args) {
SpringApplication.run(So69534923Application.class, args);
}
#KafkaListener(id = "so69534923", topics = "so69534923")
void listen(List<String> in) {
System.out.println(in);
throw new RuntimeException("test");
}
#Bean
RetryingBatchErrorHandler eh() {
return new RetryingBatchErrorHandler(new FixedBackOff(1000L, 2), (rec, ex) -> {
this.log.info("Retries exchausted for " + ListenerUtils.recordToString(rec, true));
});
}
#Bean
ApplicationRunner runner(ConcurrentKafkaListenerContainerFactory<?, ?> factory,
KafkaTemplate<String, String> template) {
factory.getContainerProperties().setCommitLogLevel(Level.INFO);
return args -> {
template.send("so69534923", "foo");
template.send("so69534923", "bar");
};
}
}
spring.kafka.consumer.auto-offset-reset=earliest
spring.kafka.listener.type=batch
so69534923: partitions assigned: [so69534923-0]
[foo, bar]
[foo, bar]
[foo, bar]
Retries exchausted for so69534923-0#2
Retries exchausted for so69534923-0#3
Committing: {so69534923-0=OffsetAndMetadata{offset=4, leaderEpoch=null, metadata=''}}
The log was from the second run.
EDIT2
It does not work with 2.3.x; you should upgrade to a supported version.
https://spring.io/projects/spring-kafka#learn