I have a Flink Job reading events from a Kafka queue then calling another service if certain conditions are met.
I wanted to use Retrofit2 to call the REST endpoint of that service but I get a is not Serializable Exception. I have several Flat Maps connected to each other (in series) then calling the service happens in the last FlatMap. The exception I get:
Exception in thread "main"
org.apache.flink.api.common.InvalidProgramException: The
implementation of the RichFlatMapFunction is not serializable. The
object probably contains or references non serializable fields.
...
Caused by: java.io.NotSerializableException: retrofit2.Retrofit$1
...
The way I am initializing retrofit:
RetrofitClient.getClient(BASE_URL).create(NotificationService.class);
And the NotificationService interface
public interface NotificationService {
#PUT("/test")
Call<String> putNotification(#Body Notification notification);
}
The RetrofitClient class
public class RetrofitClient {
private static Retrofit retrofit = null;
public static Retrofit getClient(String baseUrl) {
if (retrofit == null) {
retrofit = new Retrofit.Builder().baseUrl(baseUrl).addConverterFactory(GsonConverterFactory.create())
.build();
}
return retrofit;
}
Put your Notification class code for more details, but looks like this answer helps
java.io.NotSerializableException with "$1" after class
Related
I'm setting up contract tests for Kafka messaging with Test Containers in a way described in spring-cloud-contract-samples/producer_kafka_middleware/. Works good with Embedded Kafka but not with TestContainers.
When I try to run the generated ContractVerifierTest:
public void validate_shouldProduceKafkaMessage() throws Exception {
// when:
triggerMessageSent();
// then:
ContractVerifierMessage response = contractVerifierMessaging.receive("kafka-messages",
contract(this, "shouldProduceKafkaMessage.yml"));
Cannot invoke "org.springframework.messaging.Message.getPayload()" because "receive" is null
is thrown
Kafka container is running, the topic is created. When debugging receive method I see the message is null in the message(destination);
Contract itself:
label("triggerMessage")
input {
triggeredBy("triggerMessageSent()")
}
outputMessage {
sentTo "kafka-messages"
body(file("kafkaMessage.json"))
Base test configuration:
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.NONE, classes = {TestConfig.class, ServiceApplication.class})
#Testcontainers
#AutoConfigureMessageVerifier
#ActiveProfiles("test")
public abstract class BaseClass {
What am I missing? Maybe a point of communication between the container and ContractVerifierMessage methods?
Resolved the issue by adding a specific topic name to listen() method in KafkaMessageVerifier implementation class.
So instead of #KafkaListener(id = "listener", topicPattern = ".*"), it works with:
#KafkaListener(topics = {"my-messages-topic"})
public void listen(ConsumerRecord payload, #Header(KafkaHeaders.RECEIVED_TOPIC)
Is there a way to use #Header inside the following kafka consumer code ? I am using Spring Cloud Stream (Kafka Stream binder implementation), and there after my implemention is using functional model for example.
#Bean
public Consumer<KStream<String, Pojo>> process() {
return messages -> messages.foreach((k, v) -> process(v));
}
If using Spring for apache kafka then this can be as simple as
#KafkaListener(topics = "${mytopicname}", clientIdPrefix = "${myprefix}", errorHandler = "customEventErrorHandler")
public void processEvent(#Header(KafkaHeaders.RECEIVED_MESSAGE_KEY) String key,
#Header(KafkaHeaders.RECEIVED_PARTITION_ID) int partition,
#Header(KafkaHeaders.RECEIVED_TOPIC) String topic,
#Header(KafkaHeaders.RECEIVED_TIMESTAMP) long ts
#Valid Pojo pojo) {
...
// use headers here
...
}
No; the Kafka Streams binder is not based on Spring Messaging.
You can access headers, topic, and such in a Transformer (via the ProcessorContext) added to your stream.
You can use the Kafka Message Channel binder with
#Bean
public Consumer<Message<Pojo>> process() {
return message -> ...
}
Is it possible to get a hold of underlying KafkaConsumer bean for a defined PollableMessageSource?
I have Binding defined as:
public interface TestBindings {
String TEST_SOURCE = "test";
#Input(TEST_SOURCE)
PollableMessageSource testTopic();
}
and config class:
#EnableBinding(TestBindings.class)
public class TestBindingsPoller {
#Bean
public ApplicationRunner testPoller(PollableMessageSource testTopic) {
// Get kafka consumer for PollableMessageSource
KafkaConsumer kafkaConsumer = getConsumer(testTopic);
return args -> {
while (true) {
if (!testTopic.poll(...) {
Thread.sleep(500);
}
}
};
}
}
The question is, how can I get KafkaConsumer that corresponds to testTopic? Is there any way to get it from beans that are wired in spring cloud stream?
The KafkaMessageSource populates a KafkaConsumer into headers, so it is available in the place you receive messages: https://github.com/spring-projects/spring-kafka/blob/master/spring-kafka/src/main/java/org/springframework/kafka/support/converter/MessageConverter.java#L57.
If you are going to do stuff like poll yourself, I would suggest to inject a ConsumerFactory and use a consumer from there already.
I have few custom annotations defined on fields of an object like:
public class Person{
#Accountname
String email;
}
Implementation class of #Accountname:
#Autowired ValidationService service;
#Autowired ClientSession clientSession;
#Override
public boolean isValid(final String email, final ConstraintValidatorContext ctx) {
if(!service.isAccountValid(email, clientSession)){
return false;
}
}
I am trying to write junits for these annotations.
#Test
public void validEmailTest()
{
person.setEmail("abc#xyz.com");
Set<ConstraintViolation<Person>> violations = validatorInstance.getValidator().validateProperty(person, "email");
Assert.assertEquals(1, violations.size());
}
But its throwing this error when I execute the test:
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'scopedTarget.clientSession': Scope 'session' is not active for the current thread; consider defining a scoped proxy for this bean if you intend to refer to it from a singleton; nested exception is java.lang.IllegalStateException: No thread-bound request found: Are you referring to request attributes outside of an actual web request, or processing a request outside of the originally receiving thread? If you are actually operating within a web request and still receive this message, your code is probably running outside of DispatcherServlet/DispatcherPortlet: In this case, use RequestContextListener or RequestContextFilter to expose the current request.
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:343)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
at org.springframework.aop.target.SimpleBeanTargetSource.getTarget(SimpleBeanTargetSource.java:34)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.getTarget(CglibAopProxy.java:663)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:614)
at org.hibernate.validator.internal.engine.ConstraintTree.validateSingleConstraint(ConstraintTree.java:308)
... 45 more
Caused by: java.lang.IllegalStateException: No thread-bound request found: Are you referring to request attributes outside of an actual web request, or processing a request outside of the originally receiving thread? If you are actually operating within a web request and still receive this message, your code is probably running outside of DispatcherServlet/DispatcherPortlet: In this case, use RequestContextListener or RequestContextFilter to expose the current request.
at org.springframework.web.context.request.RequestContextHolder.currentRequestAttributes(RequestContextHolder.java:131)
at org.springframework.web.context.request.SessionScope.get(SessionScope.java:90)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:329)
... 54 more
Any idea on how to write junits if a validator class has a dependency on services like session etc.?
This should be tested separately (units).
The real logic that validates is in your ValidationService, so test it there, in AccountnameValidator test only the logic that is in there, injecting your dependencies:
#Mock ValidationService service;
#Mock ClientSession clientSession;
#InjectMocks AccountnameValidator av = new AccountnameValidator()
//initialize mocks
//example test
when(service.isAccountValid(email, clientSession)).thenReturn(true);
boolean result = av.isValid(email, ctx);
assertTrue(result);
And finally if you want you can validate presence of the annotation in Person class on email field using reflection.
using ejb 3.1, servlet 3.0 (glassfish server v3)
Scenario:
I have MDB that listen to jms messages and give processing to some other session bean (Stateless).
Servelet injecting jms resource.
Question 1: Why servlet can`t inject jms resources when they use static declaration ?
#Resource(mappedName = "jms/Tarturus")
private static ConnectionFactory connectionFactory;
#Resource(mappedName = "jms/StyxMDB")
private static Queue queue;
private Connection connection;
and
#PostConstruct
public void postConstruct() {
try {
connection = connectionFactory.createConnection();
} catch (JMSException e) {
e.printStackTrace();
}
}
#PreDestroy
public void preDestroy() {
try {
connection.close();
} catch (JMSException e) {
e.printStackTrace();
}
}
The error that I get is :
[#|2010-05-03T15:18:17.118+0300|WARNING|glassfish3.0|javax.enterprise.system.container.web.com.sun.enterprise.web|_ThreadID=35;_ThreadName=Thread-1;|StandardWrapperValve[WorkerServlet]:
PWC1382: Allocate exception for
servlet WorkerServlet
com.sun.enterprise.container.common.spi.util.InjectionException:
Error creating managed object for
class
ua.co.rufous.server.services.WorkerServiceImpl
at
com.sun.enterprise.container.common.impl.util.InjectionManagerImpl.createManagedObject(InjectionManagerImpl.java:312)
at
com.sun.enterprise.web.WebContainer.createServletInstance(WebContainer.java:709)
at
com.sun.enterprise.web.WebModule.createServletInstance(WebModule.java:1937)
at
org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1252)
Caused by:
com.sun.enterprise.container.common.spi.util.InjectionException:
Exception attempting to inject
Unresolved Message-Destination-Ref
ua.co.rufous.server.services.WorkerServiceImpl/queue#java.lang.String#null
into class
ua.co.rufous.server.services.WorkerServiceImpl
at
com.sun.enterprise.container.common.impl.util.InjectionManagerImpl._inject(InjectionManagerImpl.java:614) at
com.sun.enterprise.container.common.impl.util.InjectionManagerImpl.inject(InjectionManagerImpl.java:384)
at
com.sun.enterprise.container.common.impl.util.InjectionManagerImpl.injectInstance(InjectionManagerImpl.java:141)
at
com.sun.enterprise.container.common.impl.util.InjectionManagerImpl.injectInstance(InjectionManagerImpl.java:127)
at
com.sun.enterprise.container.common.impl.util.InjectionManagerImpl.createManagedObject(InjectionManagerImpl.java:306)
... 27 more Caused by:
com.sun.enterprise.container.common.spi.util.InjectionException:
Illegal use of static field private
static javax.jms.Queue
ua.co.rufous.server.services.WorkerServiceImpl.queue
on class that only supports
instance-based injection at
com.sun.enterprise.container.common.impl.util.InjectionManagerImpl._inject(InjectionManagerImpl.java:532) ... 31 more |#]
my MDB :
/**
* asadmin commands
* asadmin create-jms-resource --restype javax.jms.ConnectionFactory jms/Tarturus
* asadmin create-jms-resource --restype javax.jms.Queue jms/StyxMDB
* asadmin list-jms-resources
*/
#MessageDriven(mappedName = "jms/StyxMDB", activationConfig =
{
#ActivationConfigProperty(propertyName = "connectionFactoryJndiName", propertyValue = "jms/Tarturus"),
#ActivationConfigProperty(propertyName = "acknowledgeMode", propertyValue = "Auto-acknowledge"),
#ActivationConfigProperty(propertyName = "destinationType", propertyValue = "javax.jms.Queue")
})
public class StyxMDB implements MessageListener {
#EJB
private ActivationProcessingLocal aProcessing;
public StyxMDB() {
}
public void onMessage(Message message) {
try {
TextMessage msg = (TextMessage) message;
String hash = msg.getText();
GluttonyLogger.getInstance().writeInfoLog("geted jms message hash = " + hash);
} catch (JMSException e) {
}
}
}
everything work good without static declaration:
#Resource(mappedName = "jms/Tarturus")
private ConnectionFactory connectionFactory;
#Resource(mappedName = "jms/StyxMDB")
private Queue queue;
private Connection connection;
Question 2:
what is the best practice for working with MDB : processing full request in onMessage() or calling another bean(Stateless bean in my case) in onMessage() method that would process it.
Processing including few calls to soap services, so the full processing time could be for a 3 seconds.
Thank you.
Answers:
1. You cannot inject a resource into a static field. Injection into member fields occurs during object construction, static fields are not part of the object (only part of the class). In addition EJBs and servlets are threaded objects so this could possibly be dangerous to do.
2. If splitting the processing up into multiple EJB(s) makes sense do it that way, otherwise processing in onMessage() is perfectly valid.
One additional suggestion that I can give, is that you should take a look at CDI which is a new addition to the EE 6 specification and provides rich dependency injection.
Are you using an MDB to perform asynchronous operations, Servlet 3.0 has some neat asynchronous capabilities. I suggest you watch the entire presentation if you are not familiar with Servlet 3.0.