LightAdmin - Customise parsing DateTime with app timezone - spring-mvc

I am using LightAdmin 1.1.0.Snapshot with Spring Boot. I am using Joda DateTime to represent time with zone.
I can see LightAdmin captures date-time in UTC and Default Deserialization context used for parsing data is by UTC in LightAdmin. From debugging, I see LightAdmin uses its own ObjectMapper and MessageConverters using LightAdminRestMvcConfiguration, so it is not using the Spring Boot global overriders for customising the Jackson2ObjectMapperBuilder like the one below.
#Bean
public Jackson2ObjectMapperBuilder jacksonBuilder() {
Jackson2ObjectMapperBuilder builder = new Jackson2ObjectMapperBuilder();
builder.timeZone(coreProperties.appTimeZone());
return builder;
}
Any help on how to 1) override settings for Jackson in LightAdmin to parse with default app timezone or 2) Handle Json serialization / converter outside LightAdmin to solve this problem differently. Any help would be awesome.
Thanks,
Alex

One way I solved the problem is to reconfigure the LightAdmin beans after the context is loaded using the below.
#Component
public class AppContextListener implements ApplicationListener<ContextRefreshedEvent>{
#Inject
CoreProperties coreProperties;
#Override
public void onApplicationEvent(ContextRefreshedEvent event) {
GenericWebApplicationContext appContext = getRootApplicationContext(event);
WebApplicationContext lightAdminWebContext = getWebApplicationContext(appContext.getServletContext(), LightAdminWebApplicationInitializer.SERVLET_CONTEXT_ATTRIBUTE_NAME);
lightAdminWebContext.getBeansOfType(ObjectMapper.class).values().stream()
.forEach(objectMapper -> objectMapper.setTimeZone(coreProperties.appTimeZone()));
}
private GenericWebApplicationContext getRootApplicationContext(ContextRefreshedEvent event) {
return (GenericWebApplicationContext) (event.getApplicationContext().getParent() != null ? event.getApplicationContext().getParent() : event.getApplicationContext());
}
}

Related

Combining blocking and non-blocking retries in Spring Kafka

I am trying to implement non blocking retries with single topic fixed back-off.
I am able to do so, thanks to documentation https://docs.spring.io/spring-kafka/reference/html/#single-topic-fixed-delay-retries.
Now I also need to perform a few blocked/local retries on main topic. I have been trying to implement this using DefaultErrorHandler as below:
#Bean
public DefaultErrorHandler retryErrorHandler() {
return new DefaultErrorHandler(new FixedBackOff(2000, 3));
}
This does not seem to work with RetryableTopic.
I have also tried the following approach retry-topic-combine-blocking https://docs.spring.io/spring-kafka/reference/html/#retry-topic-combine-blocking using ListenerContainerFactoryConfigurer
but issue I am facing here is creating beans KafkaConsumerBackoffManager, DeadLetterPublishingRecovererFactory and especially KafkaConsumerBackoffManager.
I need to know if this another way to achieve this using spring kafka framework or is there a way to construct above beans ?
We're currently working on improving configuration for the non-blocking retries components.
For now, as documented here, you should inject these beans such as:
#Bean(name = RetryTopicInternalBeanNames.LISTENER_CONTAINER_FACTORY_CONFIGURER_NAME)
public ListenerContainerFactoryConfigurer lcfc(KafkaConsumerBackoffManager kafkaConsumerBackoffManager,
DeadLetterPublishingRecovererFactory deadLetterPublishingRecovererFactory,
#Qualifier(RetryTopicInternalBeanNames
.INTERNAL_BACKOFF_CLOCK_BEAN_NAME) Clock clock) {
ListenerContainerFactoryConfigurer lcfc = new ListenerContainerFactoryConfigurer(kafkaConsumerBackoffManager, deadLetterPublishingRecovererFactory, clock);
lcfc.setBlockingRetryableExceptions(MyBlockingRetryException.class, MyOtherBlockingRetryException.class);
lcfc.setBlockingRetriesBackOff(new FixedBackOff(500, 5)); // Optional
return lcfc;
}}
Also, there's a known issue where if you try to inject the beans before the first #KafkaListener bean with retryable topic is processed, the feature's component's beans won't be present in the context yet and will throw an error.
Does that happen to you?
We're currently working on a fix for this, but we should be able to work around that if that's your problem.
EDIT: Since the problem is that components are not instantiated yet, the most guaranteed workaround is to provide the components yourself.
Here's a sample on how to do that. Of course, adjust it accordingly if you need any further customization.
#Configuration
public static class SO71705876Configuration {
#Bean(name = RetryTopicInternalBeanNames.LISTENER_CONTAINER_FACTORY_CONFIGURER_NAME)
public ListenerContainerFactoryConfigurer lcfc(KafkaConsumerBackoffManager kafkaConsumerBackoffManager,
DeadLetterPublishingRecovererFactory deadLetterPublishingRecovererFactory) {
ListenerContainerFactoryConfigurer lcfc = new ListenerContainerFactoryConfigurer(kafkaConsumerBackoffManager, deadLetterPublishingRecovererFactory, Clock.systemUTC());
lcfc.setBlockingRetryableExceptions(IllegalArgumentException.class, IllegalStateException.class);
lcfc.setBlockingRetriesBackOff(new FixedBackOff(500, 5)); // Optional
return lcfc;
}
#Bean(name = RetryTopicInternalBeanNames.KAFKA_CONSUMER_BACKOFF_MANAGER)
public KafkaConsumerBackoffManager backOffManager(ApplicationContext context) {
PartitionPausingBackOffManagerFactory managerFactory =
new PartitionPausingBackOffManagerFactory();
managerFactory.setApplicationContext(context);
return managerFactory.create();
}
#Bean(name = RetryTopicInternalBeanNames.DEAD_LETTER_PUBLISHING_RECOVERER_FACTORY_BEAN_NAME)
public DeadLetterPublishingRecovererFactory dlprFactory(DestinationTopicResolver resolver) {
return new DeadLetterPublishingRecovererFactory(resolver);
}
#Bean(name = RetryTopicInternalBeanNames.DESTINATION_TOPIC_CONTAINER_NAME)
public DestinationTopicResolver destinationTopicResolver(ApplicationContext context) {
return new DefaultDestinationTopicResolver(Clock.systemUTC(), context);
}
In the next release this should not be a problem anymore. Please let me know if that works for you, or if any further adjustment to this workaround is necessary.
Thanks.

Kafka Streams Serdes having nested generic doesn't work

I have following code which uses functional style to define two functions for kafka topics
#Bean
public Function<KStream<String, CloudEvent<ClassA>>, KStream<String, CloudEvent<ClassB>>> method1() {
....... //lambda
}
#Bean
public Function<KStream<String, CloudEvent<ClassB>>, KStream<String, CloudEvent<ClassC>>> method2() {
...... //lambda
}
For these two functions I define serdes so
#Bean
public Serde<CloudEventMessage<ClassA>> classASerde(ObjectMapper mapper, Validator validator) {
return StreamsSerdes.classASerde(mapper,validator);
}
#Bean
public Serde<CloudEventMessage<ClassB>> classBSerde(ObjectMapper mapper, Validator validator) {
return StreamsSerdes.classBSerde(mapper,validator);
}
This construction doesn't work as at runtime spring tries to deserialize CloudEvent<ClassB> with Serde of CloutEvent<ClassA>. Is there someway to give hint to use the correct serde for method1 and method2 ?
Secondly I could bypass the above issues by mentioning Serdes in application.properties
spring.application.cloud.stream.kafka.streams.bindings.method1-in-0.consumer.valueSerde=package.serde.StreamsSerdes$ClassASerde
spring.application.cloud.stream.kafka.streams.bindings.method2-in-0.consumer.valueSerde=package.serde.StreamsSerdes$ClassBSerde
However now I get other issues as these Serde classes don't have default constructor. I do need ObjectMapper, Validator from Spring to inject beans (#Service) to perfrom converstions/validations during deserialization.
Has anyone come across similar issues or perhaps have ideas how to resolve them ?
Thanks
I think it is a gap that the nested generics are not working right now in the binder. Do you mind creating an issue in the repository and linking this thread?
As to the second issue that you are running into when providing properties in application.properties, you can try using a workaround. The Serde interface has a configure method that takes a map.
default void configure(Map<String, ?> configs, boolean isKey) {
// intentionally left blank
}
Override this method in your Serde implementation and set those bean objects under some keys.
ObjectMapper mapper;
Validator validator;
#Override
public void configure(Map<String, ?> configs, boolean isKey) {
this.mapper = (ObjectMapper) configs.get("mapper.key");
this.validator = (Validator) configs.get("validator.key");
}
You need to remove accessing them from the constructor and use those fields directly for deserialization and serialization.
Then you provide this bean in your application to populate the map:
#Bean
public StreamsBuilderFactoryBeanCustomizer streamsBuilderFactoryBeanCustomizer(ObjectMapper mapper, Validator validator) {
return factoryBean -> {
factoryBean.getStreamsConfiguration().put("mappeer.key", mapper);
factoryBean.getStreamsConfiguration().put("validator.key", validator);
};
}
I haven't tried this code in an application, but it is something that you can try and see if it works with your code.

Integration Spring Reactive with Spring MVC + MySQL

Trying to figure out if I can use Spring Reactive (Flux/Mono) with Spring MVC ?
The structure of microservices using Spring MVC + Feign Client, Eureka Server (Netflix OSS), Hystrix, MySQL database.
My first microservice addDistanceClient adds data to the database.
Here is an example controller:
#RequestMapping("/")
#RestController
public class RemoteMvcController {
#Autowired
EmployeeService service;
#GetMapping(path = "/show")
public List<EmployeeEntity> getAllEmployeesList() {
return service.getAllEmployees();
}
}
Here I can use Mono/Flux, I think there will be no problems.
My second microservice is showDistanceClient - it is not directly connected to the database.
He has a method that calls the method (as described above) on the first microservice to retrieve data from the database.
It uses the Feign Client.
Second microservice controller:
#Controller
#RequestMapping("/")
public class EmployeeMvcController {
private ServiceFeignClient serviceFeignClient;
#RequestMapping(path = "/getAllDataFromAddService")
public String getData2(Model model) {
List<EmployeeEntity> list = ServiceFeignClient.FeignHolder.create().getAllEmployeesList();
model.addAttribute("employees", list);
return "resultlist-employees";
}
}
and ServiceFeignClient itself, with which we call the method on the first microservice, looks like this:
#FeignClient(name = "add-client", url = "http://localhost:8081/", fallback = Fallback.class)
public interface ServiceFeignClient {
class FeignHolder {
public static ServiceFeignClient create() {
return HystrixFeign.builder().encoder(new GsonEncoder()).decoder(new GsonDecoder()).target(ServiceFeignClient.class, "http://localhost:8081/", new FallbackFactory<ServiceFeignClient>() {
#Override
public ServiceFeignClient create(Throwable throwable) {
return new ServiceFeignClient() {
#Override
public List<EmployeeEntity> getAllEmployeesList() {
System.out.println(throwable.getMessage());
return null;
}
};
}
});
}
}
#RequestLine("GET /show")
List<EmployeeEntity> getAllEmployeesList();
}
It is working properly now. Those, if both microservices are OK, I get data from the database.
If the first microservice (addDistanceClient) is dead, then when I call the method on second microservice (showDistanceClient) to get data from the database through the first microservice (using Feign Client on second microservice), I get a page on which the spinner is spinning and the text that the service is unavailable, try again later. All perfectly.
My goal:
To do this using Spring Reactive (not sure if this will help me, but I think I'm thinking in the right direction) to make the message that the service is currently unavailable and the spinning spinner on the second microservice will automatically disappear and the data from the database will be displayed as soon as the first microservice (addDistanceClient) will come to life again (without re-sending the request, i.e. without reloading the page).
Will I be able to do this through Spring WebFlux ?
I know that a stream is used through Spring WebFlux, which itself will notify us if data appears in it, we do not need to resubmit the request here.
I started thinking about this and cannot figure out how to do this:
1) using Spring Reactive
In this case, I need to implement Flux/Mono into the MVC model in the second showDistanceClient microservice, which returns HTML. I don't understand how. I know how to do this with REST.
2) If the first item is incorrect, maybe I need to use a WebSocket for this ?
If so, please share useful links with examples. I will be very grateful.
Indeed, this topic is very interesting to me and I want to understand it.
I will be very grateful for your help. Thanks everyone!
UPDATED POST:
I updated both controllers with REST + WebFlux. Everything works for me.
The first addDistanceClient service and its controller:
#RestController
#RequestMapping("/")
public class BucketController {
#Autowired
private BucketRepository bucketRepository;
// Get all Bucket from the database (every 1 second you will receive 1 record from the DB)
#GetMapping(value = "/stream/buckets/delay", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<Bucket> streamAllBucketsDelay() {
return bucketRepository.findAll().delayElements(Duration.ofSeconds(5));
}
}
He pulls out all the records from the database with an interval of 5 seconds each record. I added an interval for an example to test.
The second service is showDistanceClient and its controller.
Here I used WebClient instead of Feign Client.
#RestController
#RequestMapping("/")
public class UserController {
#Autowired
private WebClient webClient;
#Autowired
private WebClientService webClientService;
// Using WebClient
#GetMapping(value = "/getDataByWebClient",produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<Bucket> getDataByWebClient() {
return webClientService.getDataByWebClient();
}
}
and its Service layer (WebClientService):
#Service
public class WebClientService {
private static final String API_MIME_TYPE = "application/json";
private static final String API_BASE_URL = "http://localhost:8081";
private static final String USER_AGENT = "User Service";
private static final Logger logger = LoggerFactory.getLogger(WebClientService.class);
private WebClient webClient;
public WebClientService() {
this.webClient = WebClient.builder()
.baseUrl(API_BASE_URL)
.defaultHeader(HttpHeaders.CONTENT_TYPE, API_MIME_TYPE)
.defaultHeader(HttpHeaders.USER_AGENT, USER_AGENT)
.build();
}
public Flux<Bucket> getDataByWebClient() {
return webClient.get()
.uri("/stream/buckets/delay")
.exchange()
.flatMapMany(clientResponse -> clientResponse.bodyToFlux(Bucket.class));
}
}
Now everything works in a reactive environment. Fine.
But my problem remained unresolved.
My goal: everything works, everything is fine, and if I suddenly called on the second service a method that using WebClient called the first service to get the data, and at that moment my first service died, I received a message that the service is temporarily unavailable and then my first service My request for data was revived and I received all the data and instead of reporting that the service was temporarily unavailable I would get all the data (important: without reloading the page).
How do I achieve this ?

spring boot Joda DateTime Serialisation

I'm trying to serialize Joda DateTime properties as ISO-8601 using Spring Boot v1.2.0.BUILD-SNAPSHOT
Here is my very simple REST Application.
#RestController
#Configuration
#ComponentScan
#EnableAutoConfiguration
public class Application {
class Info{
private DateTime dateTime;
public Info(){
dateTime = new DateTime();
}
public DateTime getDateTime() {
return dateTime;
}
public void setDateTime(DateTime dateTime) {
this.dateTime = dateTime;
}
}
#RequestMapping("/info")
Info info() {
return new Info();
}
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
#Bean
public Module getModule(){
return new JodaModule();
}
}
The dateTime is being serialized as a timestamp e.g. {"dateTime":1415954873412}
I've tried adding
#Bean
#Primary
public ObjectMapper getObjectMapper() {
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.configure(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS,
false);
return objectMapper;
}
but that didn't help either. So how do I configure Jackson in Spring Boot to serialize using the ISO-8601 format?
BTW: I only added the following dependencies to my Gradle build
compile("joda-time:joda-time:2.4")
compile("org.jadira.usertype:usertype.jodatime:2.0.1")
compile("com.fasterxml.jackson.datatype:jackson-datatype-joda:2.4.2");
Since you're using Spring Boot 1.2 you should be able to simply add the following to your application.properties file:
spring.jackson.serialization.write_dates_as_timestamps=false
This will give output in the form:
{
"dateTime": "2014-11-18T19:01:38.352Z"
}
If you need a custom format you can configure the JodaModule directly, for example to drop the time part:
#Bean
public JodaModule jacksonJodaModule() {
JodaModule module = new JodaModule();
DateTimeFormatterFactory formatterFactory = new DateTimeFormatterFactory();
formatterFactory.setIso(ISO.DATE);
module.addSerializer(DateTime.class, new DateTimeSerializer(
new JacksonJodaFormat(formatterFactory.createDateTimeFormatter()
.withZoneUTC())));
return module;
}
With Spring Boot 1.2 you can use a fluent builder for building ObjectMapper instance:
#Bean
public ObjectMapper objectMapper(Jackson2ObjectMapperBuilder builder) {
return builder
.featuresToDisable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS)
.build();
}
As for JodaModule, it will be autoconfigured when com.fasterxml.jackson.datatype:jackson-datatype-joda is on classpath.
Pass a new JodaModule() to the constructor of your object mapper.
Annotate your Info methods with the ISO pattern
#JsonFormat(shape = JsonFormat.Shape.STRING, pattern = "yyyy-MM-dd'T'HH:mm:ss.SSSZ")
or I think you can use this if you're using spring
#JsonFormat(shape = JsonFormat.Shape.STRING, pattern = DateTimeFormat.ISO.DATE_TIME)
Add this to your application.* in your resources. (I use yamel so it's .yml for me, but should be .properties by default)
spring.jackson.date-format: yyyy-MM-dd'T'HH:mm:ssZ
Or whatever format you want.
There is also a joda-date-time-format property (I think this property appeared for the first time in Spring boot 1.3.x versions) that you can set in your application.properties which will work for jackson serialization/deserialization:
From: https://docs.spring.io/spring-boot/docs/current/reference/html/common-application-properties.html
spring.jackson.joda-date-time-format= # Joda date time format string. If not configured, "date-format" will be used as a fallback if it is configured with a format string.
So if you want to use the ISO format you can set it like this:
spring.jackson.joda-date-time-format=yyyy-MM-dd'T'HH:mm:ss.SSSZ
You can have different number of 'Z' which changes the way the time zone id or offset hours are shown, from the joda time documentation (http://joda-time.sourceforge.net/apidocs/org/joda/time/format/DateTimeFormat.html):
Zone: 'Z' outputs offset without a colon, 'ZZ' outputs the offset with a colon, 'ZZZ' or more outputs the zone id.

Unit Testing I18N RESTful Web Services with Spring, RestTemplate and Java Config

Trying to get Unit Tests to work when using Spring RestTemplate and I18N. Everything in the setup works fine for all the other test cases.
Based upon what I read, this is what I put into the Java Config:
#Bean
public LocaleChangeInterceptor localeChangeInterceptor() {
return new LocaleChangeInterceptor();
}
#Bean
public DefaultAnnotationHandlerMapping handlerMapping() {
DefaultAnnotationHandlerMapping mapping = new DefaultAnnotationHandlerMapping();
Object[] interceptors = new Object[1];
interceptors[0] = new LocaleChangeInterceptor();
mapping.setInterceptors(interceptors);
return mapping;
}
#Bean
public AnnotationMethodHandlerAdapter handlerAdapter() {
return new AnnotationMethodHandlerAdapter();
}
Then in my usage with RestTemplate I have:
public MyEntity createMyEntity(MyEntity bean) {
Locale locale = LocaleContextHolder.getLocale();
String localeString = "";
if (locale != Locale.getDefault()) {
localeString = "?locale=" + locale.getLanguage();
}
HttpEntity<MyEntity> req = new HttpEntity<MyEntity>(bean);
ResponseEntity<MyEntity> response = restTemplate.exchange(restEndpoint + "/url_path" + localeString, HttpMethod.POST, req, MyEntity.class);
return response.getBody();
}
While this could be cleaned up a bit, it should work - but the LocalChangeInterceptor never gets invoked. I am debugging this now and will post again as soon as I figure it out - but in the hope this is a race condition that I lose - does anyone know why?
Was lucky and stumbled upon this thread. One of the notes clued me into the right direction. You don't need all those beans in the Java Config. But if you are using #EnableWebMvc as I am, but I didn't know it was important enough to even mention, all you need to do in your Java Config is:
#Bean
public LocaleChangeInterceptor localeChangeInterceptor() {
return new LocaleChangeInterceptor();
}
#Override
public void addInterceptors(InterceptorRegistry registry) {
registry.addInterceptor(new LocaleChangeInterceptor());
super.addInterceptors(registry);
}
Add the one bean for the Interceptor and then override the method to add the interceptor. Here my configuration class (annotated with #Configuration and #EnableWebMvc) also extends WebMvcConfigurerAdapter, which should be common usage.
This, at least, worked for me. Hope it may help someone else.

Resources