I want to update the properties of consumers in runtime.
I found this solution - Reload consumer properties with spring kafka.
But there are no containers at the start of the application. I am using starter from Spring Boot for spring-kafka configuration.
#Bean
fun refresh(
#Qualifier(KafkaListenerConfigUtils.KAFKA_LISTENER_ENDPOINT_REGISTRY_BEAN_NAME)
registry: KafkaListenerEndpointRegistry
): Any? {
registry.allListenerContainers.forEach {
it.stop()
it.containerProperties.authExceptionRetryInterval
}
return null
}
What am I doing wrong? And what is the best approach for runtime updating consumers and producers?
Using spring - kafka and starter Spring Boot
The logic is not correct. You have a #Bean and you call that registry.allListenerContainers just on a bean initialization phase. That's the time when there is no created listener containers in a registry yet.
According to the answer you are pointing to you are missing a #RefreshScope annotation on this bean definition.
Related
I'm creating an application using RabbitMQ.Client.Core.DependencyInjection as a consumer of messages in the BackgroundService. Receiving and deserializing the message works correctly, but when I'm sending command to the MediatR I got an error:
---> System.InvalidOperationException: Error while validating the service descriptor 'ServiceType: RabbitMQ.Client.Core.DependencyInjection.Services.IMessageHandlerContainerBuilder Lifetime: Singleton ImplementationType: RabbitMQ.Client.Core.DependencyInjection.Services.MessageHandlerContainerBuilder': Cannot consume scoped service 'Application.Common.Interfaces.IDbContext' from singleton 'RabbitMQ.Client.Core.DependencyInjection.Services.IMessageHandlerContainerBuilder'.
---> System.InvalidOperationException: Cannot consume scoped service 'Application.Common.Interfaces.IDbContext' from singleton 'RabbitMQ.Client.Core.DependencyInjection.Services.IMessageHandlerContainerBuilder'.
My Command's Handler's constructor is creating dbContext in the constructor, and outside the BackgroundService it works correctly.
I understand that for BackgroundService one needs to create scope and get service from the ServiceProvider:
using var scope = _sp.CreateScope();
var dbContext = scope.ServiceProvider.GetRequiredService<IDbContext>();
But how can I use the rest of the application, where I'm using injected services as singletons?
I believe the same issue would happen if I would call a Service class that constructs injected dbContext in the same way, it does not seem to be a problem with a MediatR.
ok, found a problem with RabbitMQ.Client.Core.DependencyInjection NuGet, instead used RabbitMQ.Client and all works as expected
I have a spring-boot Kafka project which is a web-service exposing API to get Kafka message in response.
What i want is whenever i call the rest end point the Kafka should start searching from beginning it does it as i used earliest in auto-reset config but i have to start server again and again to make it listen to Kafka from starting.
#KafkaListener(topics = {"topic"})
public void storeMessagesMessages(ConsumerRecord record) {
if (record.value().toString().contains(uuid) {
this.messageToBeReturnedByApi = record.value()
}
}
Or i can say i want this listener part to be invoked only when i call web service endpoint
Your listener should extend AbstractConsumerSeekAware; you can then perform arbitrary seek operations. See https://docs.spring.io/spring-kafka/docs/2.6.2/reference/html/#seek
In a spring-boot (version 2.1.4) application, there is requirement to migrate apache-kafka to spring-kafka.
Current kafka consumer does:
1) KafkaConsumer bean initialized at the time of application up
2) It has "0" topic partition set
3) poll the data using apache kafkaConsumer into ConsumerRecord
4) The application its own has Retry mechanism to wait and poll again till max_retry
The legacy code looks below:
while (!done.get()) {
ConsumerRecords<byte[], <byte[]> records = kafkaConsumer.poll(<MAX_VALUE>);
if (records.isEmpty()) {
retryCount++;
Thread.sleep(<some_time>);
} else {
// Process records;
}
if (retryCount > <max_retry_count>) {
done.set(true);
}
}
Tried below approaches:
1) Using spring kafka annotation (#KafkaListener), but it does not let us have control over polling.
2) Created "ConcurrentMessageListenerContainer" and setupMessageListener adds records into queue for polling. This gives us control on consumer.
I wanted to know, am I heading towards correct direction?
What would be better solution to achieve above requirement using spring-kafka?
It's not clear what you mean by "control on consumer". Creating a container is the same as using a #KafkaListener (a container is created under the covers).
Spring uses a "message-driven" approach.
You can set the idleEventInterval and the container will publish a ListenerContainerIdleEvent if no records are received during that time. You can listen for these events with an ApplicationListener bean, or an #EventListener method.
I have a spring boot (1.3.2) app in which I have implemented a HttpSessionListener. I registered the listener from a #Configration class
#Configuration
#EnableRedisHttpSession
public class ApplicationSessionConfiguration {
#Bean
public ServletListenerRegistrationBean<HttpSessionListener> sessionListener() {
return new ServletListenerRegistrationBean<HttpSessionListener>(new SessionListener());
}
}
I have debugged into ServletListenerRegistrationBean.onInitialize method and the listener is getting registered with the ServletContext. Problem is now when I make a dummy REST call to the app the session gets created properly and sent back as a SESSION cookie, but the HttpSessionListener.createSession method never gets called. I am not sure what I am missing here.
Looks like the feature you need is not yet released in a stable build. However as per this ticket this is fixed and is available in 1.1.0 M1 release for spring-session. You may want to try 1.1.0.RC1 release of spring-session to see if this helps what you want. Exact details on how to get this done can be found in this doc link
In case using 1.1.0.RC1 release is NOT an option (or if you prefer not use RC1 due to what ever reason), you can still intercept session creation and destroy events by extending default CookieHttpSessionStrategy with you own implementation (say MyCookieHttpSessionStrategy) and then overriding the onNewSession(..) and onInvalidateSession(..) to intercept these events. Register MyCookieHttpSessionStrategy as normal bean and you are all set (it'll be automatically picked up by Redis session repository). This works just fine with Redis sessions, I am using these events in my spring boot web app this way.
Hope this helps!!
I am new to the EJB Projects. And am trying to understand the usage of #Transactional annotation at top of my EJB methods. I have searched for the content and there is no clear explanation on this. Can anyone explain clearly about this.
#Transactional comes from the Spring world, but Oracle finally included it in Java EE 7 specification (docs). Previously, you could only annotate EJBs with #TransactionAttribute annotation, and similar is now possible for CDIs as well, with #Transactional. What's the purpose of these annotations? It is a signal to the application server that certain class or method is transactional, indicating also how it is gonna behave in certain conditions, e.g. what if it's called inside a transaction etc.
An example:
#Transactional(Transactional.TxType.MANDATORY)
public void methodThatRequiresTransaction()
{
..
}
The method above will throw an exception if it is not called within a transaction.
#Transactional(Transactional.TxType.REQUIRES_NEW)
public void methodThatWillStartNewTransaction()
{
..
}
Interceptor will begin a new JTA transaction for the execution of this method, regardless whether it is called inside a running transaction or not. However, if it is called inside a transaction, that transaction will be suspended during the execution of this method.
See also:
TransactionalTxType