On https://docs.spring.io/spring-kafka/reference/html/#overview-2 the following is stated:
Starting with version 2.5.8, you can now configure the maxAge property on the producer factory. This is useful when using transactional producers that might lay idle for the broker’s transactional.id.expiration.ms. With current kafka-clients, this can cause a ProducerFencedException without a rebalance. By setting the maxAge to less than transactional.id.expiration.ms, the factory will refresh the producer if it is past its max age.
How ie. where can maxAge be configured for the default producer factory?
I used the DefaultKafkaProducerFactoryCustomizer which seems to work:
#Bean
public DefaultKafkaProducerFactoryCustomizer producerFactoryCustomizer() {
return (producerFactory) -> producerFactory.setMaxAge(Duration.ofDays(1));
}
This should be in a #Configuration class.
Related
We need to access a header in our ConsumeContext when adding a transient service.
We have been using IHttpContextAccessor previously to get the headers for a normal http request, and we now need to do similarly for our event consumers.
How would we go about accessing the headers for a consumed event when using MassTransit, when setting up our dependencies/services?
services.TryAddTransient<ISapService>(provider =>
{
var httpContextAccessor = provider.GetService<IHttpContextAccessor>();
httpContextAccessor.HttpContext.Request.Headers.TryGetValue(
"x-plant-id",
out var plantHeader
);
return new SapService(plantHeader);
});
I'm not sure if it works with transient services, but MassTransit does support scoped filters. They're resolved within the consumer scope.
In a SpringBoot application, I have the following configuration:
axon:
axonserver:
servers: "${AXON_SERVER:localhost}"
serializer:
general: jackson
messages: jackson
events: jackson
logging.level:
org.axonframework.modelling.saga: debug
Downsizing the scenario to bare minimum, the relevant portion of Saga class:
#Slf4j
#Saga
#ProcessingGroup("AuctionEventManager")
public class AuctionEventManagerSaga {
#Autowired
private transient EventScheduler eventScheduler;
private ScheduleToken scheduleToken;
private Instant auctionTimerStart;
#StartSaga
#SagaEventHandler(associationProperty = "auctionEventId")
protected void on(final AuctionEventScheduled event) {
this.auctionTimerStart = event.getTimerStart();
// Cancel any pre-existing previous job, since the scheduling thread might be lost upon a crash/restart of JVM.
if (this.scheduleToken != null) {
this.eventScheduler.cancelSchedule(this.scheduleToken);
}
this.scheduleToken = this.eventScheduler.schedule(
this.auctionTimerStart,
AuctionEventStarted.builder()
.auctionEventId(event.getAuctionEventId())
.build()
);
}
#EndSaga
#SagaEventHandler(associationProperty = "auctionEventId")
protected void on(final AuctionEventStarted event) {
log.info(
"[AuctionEventManagerSaga] Current state: {scheduleToken={}, auctionTimerStart={}}",
this.scheduleToken,
this.auctionTimerStart
);
}
}
In the final compiled class, we will end up having 4 properties: log (from #Slf4j), eventScheduler (transient, #Autowired), scheduleToken and auctionTimerStart.
For reference information, here is a sample of the general approach I've been using for both Command and Event classes:
#Value
#Builder
#JsonDeserialize(builder = AuctionEventStarted.AuctionEventStartedBuilder.class)
public class AuctionEventStarted {
AuctionEventId auctionEventId;
#JsonPOJOBuilder(withPrefix = "")
public static final class AuctionEventStartedBuilder {}
}
When executing the code, you get the following output:
2020-05-12 15:40:01.180 DEBUG 1 --- [mandProcessor-4] o.a.m.saga.repository.jpa.JpaSagaStore : Updating saga id c8aff7f7-d47f-4616-8a96-a40044cb7e3b as {}
As soon as the general serializer is changed to xstream, the content is serialized properly, but I face another issue during deserialization, since I have private static final class Builder classes using Lombok.
So is there a way for Axon to handle these scenarios:
1- Axon to safely manage Jackson to ignore #Autowired, transient and static properties from #Saga classes? I've attempted to manually define #JsonIgnore at non-state properties and it still didn't work.
2- Axon to safely configure XStream to ignore inner classes (mostly Builder classes implemented as private static final)?
Thanks in advance,
EDIT: I'm pursuing a resolution using my preferred serializer: JSON. I attempted to modify the saga class and extend JsonSerializer<AuctionEventManagerSaga>. For that I implemented the methods:
#Override
public Class<AuctionEventManagerSaga> handledType() {
return AuctionEventManagerSaga.class;
}
#Override
public void serialize(
final AuctionEventManagerSaga value,
final JsonGenerator gen,
final SerializerProvider serializers
) throws IOException {
gen.writeStartObject();
gen.writeObjectField("scheduleToken", value.eventScheduler);
gen.writeObjectField("auctionTimerStart", value.auctionTimerStart);
gen.writeEndObject();
}
Right now, I have something being serialized, but it has nothing to do with the properties I've defined:
2020-05-12 16:20:01.322 DEBUG 1 --- [mandProcessor-0] o.a.m.saga.repository.jpa.JpaSagaStore : Storing saga id c4b5d94c-7251-40a5-accf-332768b1cacd as {"delegatee":null,"unwrappingSerializer":false}
EDIT 2 Decided to add more insight into the issue I experience when I switch general to use XStream (even though it's somewhat unrelated to the main issue described in the title).
Here is the issue it complains to me:
2020-05-12 17:08:06.495 DEBUG 1 --- [ault-executor-0] o.a.a.c.command.AxonServerCommandBus : Received command response [message_identifier: "79631ffb-9a87-4224-bed3-a957730dced7"
error_code: "AXONIQ-4002"
error_message {
message: "No converter available\n---- Debugging information ----\nmessage : No converter available\ntype : jdk.internal.misc.InnocuousThread\nconverter : com.thoughtworks.xstream.converters.reflection.ReflectionConverter\nmessage[1] : Unable to make field private static final jdk.internal.misc.Unsafe jdk.internal.misc.InnocuousThread.UNSAFE accessible: module java.base does not \"opens jdk.internal.misc\" to unnamed module #7728643a\n-------------------------------"
location: "1#600b5b87a922"
details: "No converter available\n---- Debugging information ----\nmessage : No converter available\ntype : jdk.internal.misc.InnocuousThread\nconverter : com.thoughtworks.xstream.converters.reflection.ReflectionConverter\nmessage[1] : Unable to make field private static final jdk.internal.misc.Unsafe jdk.internal.misc.InnocuousThread.UNSAFE accessible: module java.base does not \"opens jdk.internal.misc\" to unnamed module #7728643a\n-------------------------------"
}
request_identifier: "2f7020b1-f655-4649-bbe0-d6f458b3c2f3"
]
2020-05-12 17:08:06.505 WARN 1 --- [ault-executor-0] o.a.c.gateway.DefaultCommandGateway : Command 'ACommandClassDispatchedFromSaga' resulted in org.axonframework.commandhandling.CommandExecutionException(No converter available
---- Debugging information ----
message : No converter available
type : jdk.internal.misc.InnocuousThread
converter : com.thoughtworks.xstream.converters.reflection.ReflectionConverter
message[1] : Unable to make field private static final jdk.internal.misc.Unsafe jdk.internal.misc.InnocuousThread.UNSAFE accessible: module java.base does not "opens jdk.internal.misc" to unnamed module #7728643a
-------------------------------)
Still no luck on resolving this...
I've worked on Axon systems where the only used Serializer implementation was the JacksonSerializer too. Mind you though, this is not what the Axon team recommends. For messages (i.e. commands, events and queries) it makes perfect sense to use JSON as the serialized format. But switching the general Serializer to jackson means you have to litter your domain logic (e.g. your Saga) with Jackson specifics "to make it work".
Regardless, backtracking to my successful use case of jackson-serialized-sagas. In this case we used the correct match of JSON annotations on the fields we desired to take into account (the actual state) and to ignore the one's we didn't want deserialized (with either transient or #JsonIgnore). Why both do not seem to work in your scenario is not entirely clear at this stage.
What I do recall is that the referenced project's team very clearly decided against Lombok due to "overall weirdnes" when it comes to de-/serialization. As a trial it thus might be worth to not use any Lombok annotations/logic in the Saga class and see if you can de-/serialize it correctly in such a state.
If it does work at that moment, I think you have found your culprit for diving in further search.
I know this isn't an exact answer, but I hope it helps you regardless!
Might be worthwhile to share the repository where this problems occurs in; might make the problem clearer for others too.
I was able to resolve the issue #2 when using XStream as general serializer.
One of the Sagas had an #Autowired dependency property that was not transient.
XStream was throwing some cryptic message, but we managed to track the problem and address it.
As for JSON support, we had no luck. We ended up switched everything to XStream for now, as the company only uses Java and it would be ok to decode the events using XStream.
Not the greatest solution, as we really wanted (and hoped) JSON would be supported properly out of the box. Mind you, this is in conjunction with using Lombok which caused for the nuisance in this case.
As of Corda 2.0, constructors of classes annotated with #CordaService are automatically invoked during node start. Corda Service original class implementations may, for example, initialise network connections in their constructors, which may not be desirable in unit tests.
My current workaround is to use a boolean flag (set from an environment variable) that marks whether to run or skip Corda Service initialisation code. Is there a better way to do this (e.g. inject a mock Corda Service class instance into MockNetwork.MockNode without using PowerMock and the like)?
Your workaround is currently the best way to handle this. See https://r3-cev.atlassian.net/browse/CORDA-989.
In your approach, you might define your service as follows:
#CordaService
class MyService(services: ServiceHub) : SingletonSerializeAsToken() {
private companion object {
val logger = loggerFor<MyService>()
val load = System.getenv("enable-my-service")
}
init {
if (load == "false") {
logger.warn("Not loading service")
} else {
logger.info("Loading service")
}
}
}
And then you can toggle your service off in tests using an environment variable.
I have a requirement to reset the expire time if the record is accessed before its initial expire time. I am using Spring data redis API to use Redis as Cache. I am using RediscacheManager's setDefaultExpiration(5000) to set default expiration. Unable to find any solutions or documentation about resetting the expiry time. Any guidance is appreciated.
Also, wondering, why couldn't this be a natural feature of Redis Cache, after all, it should get the most used records from cache.
Wrote this method and called from appropriate places. Worked like a charm for me.
public void resetExpire(String keyPattern) {
LOG.debug("Getting Multiple keys from cache with pattern: " + keyPattern);
Set<String> keylist = redisTemplate.keys(keyPattern);
redisTemplate.executePipelined(new RedisCallback<Object>() {
public Object doInRedis(RedisConnection connection) throws DataAccessException {
keylist.forEach(key->
redisTemplate.expire(key, 5000, TimeUnit.SECONDS));
return null;
}
});
}
As described in OpenEJB docs, we can configure JMS connection factory and queues, and they will appear in JNDI as:
openejb:Resource/MyJmsConnectionFactory,
openejb:Resource/MyQueue
Given those JNDI entries, how can I tell to MDB to use them?
Is it possible to change JNDI name, for example ConnectionFactory to appear as java:/ConnectionFactory
or ConnectionFactory
Things work differently than you may be imagining. Specifying that an MDB is tied to a javax.jms.Queue and the name of that queue is part of the EJB specification and done via the ActivationConfig, like so:
#MessageDriven(activationConfig = {
#ActivationConfigProperty(
propertyName = "destinationType",
propertyValue = "javax.jms.Queue"),
#ActivationConfigProperty(
propertyName = "destination",
propertyValue = "FooQueue")})
public static class JmsBean implements MessageListener {
public void onMessage(Message message) {
}
}
The MDB container itself is not actually JMS-aware at all. It simply understands that it should hook the bean up to a specific Resource Adapter.
<openejb>
<Resource id="MyJmsResourceAdapter" type="ActiveMQResourceAdapter">
ServerUrl tcp://someHostName:61616
</Resource>
<Container id="MyJmsMdbContainer" ctype="MESSAGE">
ResourceAdapter MyJmsResourceAdapter
</Container>
</openejb>
The above shows an MDB Container hooked up to a Resource Adapter that uses JMS via ActiveMQ.
Here is an example that shows an MDB Container hooked up to a Quartz Resource Adapter
It isn't possible to tell the MDB Container about JMS specific things as per specification, the relationship is much more generic than that. This blog post gives some insight as to how things work.