Reactor Kafka how to gracefully shutdown? - spring-kafka

Question

We use reactiveKafkaConsumerTemplate to receive messages,then process the message. We try to graceful shutdown the application.
Stop poll new messages
Wait for the current message finish process
Shutdown application
We tried reactiveKafkaConsumerTemplate.pause. It doesn’t stop consumer poll new messages when there is back pressure pause. We also tried dispose, it doesn't wait for message complete processing. How could we archive the graceful shutdown? Thanks.
log
r.k.r.internals.ConsumerEventLoop - Paused - back pressure
ReactiveKafkaConsumerTemplate pause topic=test-topic, partition=5
r.k.r.internals.ConsumerEventLoop - Paused - Consumer woken
r.k.r.internals.ConsumerEventLoop - Paused - Resume
reactiveKafkaConsumerTemplate.
receiveAutoAck()
.publishOn(Schedulers.boundedElastic())
.flatMap(x -> Mono.just(x)
.delayElement(Duration.ofMillis(300)),5)
.flatMap(message -> Mono.just(message)
.flatMap(processMessageImp::processMessage)
.onErrorResume(t -> Mono.empty())
);
public void pauseKafkaMessageConsumer() {
reactiveKafkaConsumerTemplate.assignment()
.doOnNext(tp -> log.info("ReactiveKafkaConsumerTemplate pause topic={}, partition={}",
tp.topic(),tp.partition()))
.flatMap(topicParts -> reactiveKafkaConsumerTemplate.pause(topicParts))
.subscribe();
}
#PreDestroy
public void onExit() {
pauseKafkaMessageConsumer();
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
log.error("onExit Error while PreDestroy ");
}
}
with 1.3.13 consumer paused in back pressure case, But Consumer resumed after rebalance, Would it be expected behavior?
12/8/22 10:53:42.559 PM DEBUG r.k.r.internals.ConsumerEventLoop Emitting 1 records, requested now 1
12/8/22 10:53:42.559 PM DEBUG r.k.r.internals.ConsumerEventLoop - onRequest.toAdd 1, paused false
12/8/22 10:53:45.966 PM DEBUG r.k.r.internals.ConsumerEventLoop Async committing: {test-topic-9=OffsetAndMetadata{offset=10934, leaderEpoch=null, metadata=''}}
12/8/22 10:53:46.258 PM reactiveKafkaConsumerTemplate pause
12/8/22 10:54:06.185 PM DEBUG r.k.r.internals.ConsumerEventLoop onPartitionsRevoked [test-topic-9, test-topic-8]
12/8/22 10:54:07.289 PM DEBUG r.k.r.internals.ConsumerEventLoop onPartitionsAssigned [test-topic-9, test-topic-8]
12/8/22 10:54:07.505 PM DEBUG r.k.r.internals.ConsumerEventLoop Emitting 12 records, requested now 1