spring-kafka kafkaStreamsBuilder.getKafkaStreams() is null - spring-kafka

Question

Here is my code
The first bean is watching the messages on Topic.TRANSACTION_RAW and split one message into two and send them to Topic.TRANSACTION_INTERNAL
And the second bean is doing group and reducing and materialize it to the state store "StateStore.BALANCE".
The last one is to get the ReadOnlyKeyValueStore to read state from "ReadOnlyKeyValueStore".
#Configuration(proxyBeanMethods = false)
#EnableKafkaStreams
public class MyKafkaStreamsConfiguration {
#Bean
public KStream<String, BankTransaction> alphaBankKStream(StreamsBuilder streamsBuilder) {
JsonSerde<BankTransaction> valueSerde = new JsonSerde<>(BankTransaction.class);
KStream<String, BankTransaction> stream = streamsBuilder.stream(Topic.TRANSACTION_RAW,
Consumed.with(Serdes.String(), valueSerde));
stream.flatMap((k, v) -> {
List<BankTransactionInternal> txInternals = BankTransactionInternal.splitBankTransaction(v);
List<KeyValue<String, BankTransactionInternal>> result = new LinkedList<>();
result.add(KeyValue.pair(v.getFromAccount(), txInternals.get(0)));
result.add(KeyValue.pair(v.getToAccount(), txInternals.get(1)));
return result;
}).filter((k, v) -> !Constants.EXTERNAL_ACCOUNT.equalsIgnoreCase(k))
.to(Topic.TRANSACTION_INTERNAL, Produced.with(Serdes.String(), new JsonSerde<>()));
return stream;
}
#Bean
public KStream<String, BankTransactionInternal> alphaBankInternalKStream(StreamsBuilder streamsBuilder) {
JsonSerde<BankTransactionInternal> valueSerde = new JsonSerde<>(BankTransactionInternal.class);
KStream<String, BankTransactionInternal> stream = streamsBuilder.stream(Topic.TRANSACTION_INTERNAL,
Consumed.with(Serdes.String(), valueSerde));
KGroupedStream<String, Double> groupedByAccount = stream
.map((k,v) -> KeyValue.pair(k, v.getAmount()))
.groupBy((account, amount) -> account, Grouped.with(Serdes.String(), Serdes.Double()));
groupedByAccount.reduce(Double::sum,
Materialized.<String, Double, KeyValueStore<Bytes, byte[]>>as(StateStore.BALANCE)
.withValueSerde(Serdes.Double()));
return stream;
}
#Bean
public ReadOnlyKeyValueStore<String, Double> balanceStateStore(StreamsBuilderFactoryBean defaultKafkaStreamsBuilder) {
if (defaultKafkaStreamsBuilder == null) {
System.out.println("... defaultKafkaStreamsBuilder is null ...");
}
if (defaultKafkaStreamsBuilder.getKafkaStreams() == null) {
System.out.println("... defaultKafkaStreamsBuilder.getKafkaStreams() is null ...");
// this one got printed
}
ReadOnlyKeyValueStore<String, Double> store = defaultKafkaStreamsBuilder.getKafkaStreams().store(
StateStore.BALANCE,
QueryableStoreTypes.keyValueStore());
return store;
}
}
I always got NullPointException on defaultKafkaStreamsBuilder.getKafkaStreams().
Any idea what is wrong here? Thanks!

Answer 1

if (defaultKafkaStreamsBuilder.getKafkaStreams() == null) {
System.out.println("... defaultKafkaStreamsBuilder.getKafkaStreams() is null ...");
// this one got printed
}
This operation is not good to do during bean definition phase.
See its JavaDocs:
/**
* Get a managed by this {#link StreamsBuilderFactoryBean} {#link KafkaStreams} instance.
* #return KafkaStreams managed instance;
* may be null if this {#link StreamsBuilderFactoryBean} hasn't been started.
* #since 1.1.4
*/
public synchronized KafkaStreams getKafkaStreams() {
since you call this method far too early before a lifecycle start phase, you end-up with that error.
You should reconsider your logic in favor of SmartLifecycle.start() in the target service where you'd like to use that ReadOnlyKeyValueStore. So, you autowire over there this StreamsBuilderFactoryBean and call its getKafkaStreams() from the start() implementation.