Axon framework- storing a list of events - axon

I have a list of entities List<Entity> entitiesList. I need to publish and store the list of events for each of entity.
I have an aggregate for Entity , all necessary handlers, CreateEntityCommand and EntityCreatedEvent.
Currently what I do:
1. Create commands in the loop and send these commands via command gateway for each entity from entitiesList.
for (Entity entity : entitiesList) {
CreateEntityCommand createEntityCommand = new CreateEntityCommand();
… here I set command’s fields …
commandGateway.send(createEntityCommand);
}
Inside the aggregate I have
#CommandHandler
public EntityAggregate(CreateEntityCommand createAlertCommand) {
EntityCreatedEvent entityCreatedEvent = new EntityCreatedEvent();
…. here I set event’s fields
AggregateLifecycle.apply(entityCreatedEvent);
}
As the result, the events are created published and saved into the DomainEventEntry table inside the loop one by one.
If I have 10000 of entities – this process takes a lot of time …
My question is – how can I improve this process of creating, publishing and saving a list of entities ?
I use this version of axon:
<dependency>
<groupId>org.axonframework</groupId>
<artifactId>axon-spring-boot-starter</artifactId>
<version>4.3</version>
<exclusions>
<exclusion>
<groupId>org.axonframework</groupId>
<artifactId>axon-server-connector</artifactId>
</exclusion>
</exclusions>
</dependency>
SpringBoot configuration with annotation #SpringBootApplication.
I haven't configured anything specific around Axon.

What I believe you need is to parallelize the processing of the commands to speed up the work. There are two ways to achieve this:
Change the local CommandBus
Distributed your application
I am assuming you are on pointer 1, hence my answer will be tailored towards this.
When you have a single instance of an Axon application using Spring Boot a SimpleCommandBus will be auto configured for you. This does not provide any possibility for concurrent work. Thus configuring a different CommandBus bean should be the way to go.
I'd suggest to first start using the AsynchronousCommandBus. This implementation uses an Executor (which you can further configure if you so desire) to spin up threads to dispatch (and handle) each command going through.
If this is still to slow to your liking, I'd try out the DisruptorCommandBus (for specifics on what "Disruptor" is, you can look here). This CommandBus implementation will use two threads pools; one pool for handling commands and another for storing the events.
Lastly, if you are already working with a distributed version of the CommandBus (like Axon Server, or with the DistributedCommandBus), you will have to provide a CommandBus bean with the qualifier "localSegment" attached to it. For a quick overview of the command buses provided by Axon, I'd have a look at there Reference Guide (up here).

Related

Adds a state store to the underlying Topology before StreamsBuilder started

Is there a way in StreamsBuilderFactoryBeanCustomizer to add state store to the underlying Topology before StreamsBuilder started, in order to use it in the processor or transformer.
when using the Kafka Streams DSL there is such a way
StreamsBuilder builder = new StreamsBuilder();
builder.addStateStore(getStoreBuilder());
We have an open pull request to add a topology customizer to the factory bean but it hasn't had any attention for a while. The author doesn't appear to have responded to the last review comments.
Maybe it can be enhanced to allow customizing the builder itself.
I can take a look at it next week and try to get it into next Wednesday's releases.

Crda contracts and states upgrades questions

I am going through this documentation and I have several uncertainties.
Performing explicit contract and state upgrades
Preserve the existing state and contract definitions
Write the new state and contract definitions
Create the new CorDapp JAR
Distribute the new CorDapp JAR
Stop the nodes
Re-run the network bootstrapper (only if you want to whitelist the
new contract)
Restart the nodes
Authorise the upgrade
Perform the upgrade
Migrate the new upgraded state to the Signature Constraint from the
zone constraint
Questions:
1. Preserve the existing state and contract definitions
2. Write the new state and contract definitions
3. Create the new CorDapp JAR
How do I do that? is it meant only to preserve jars with contracts and states on nodes, not preserving them in source code? If I do not preserve them in source code then how can I create the upgrade method?
interface UpgradedContract<in OldState : ContractState, out NewState : ContractState> : Contract {
val legacyContract: ContractClassName
fun upgrade(state: OldState): NewState
}
If I do not preserve old state in source code, then shoud I name the jar differently each time I need to do an upgrade?
Can old jars be reoved from the node when the upgrade was completed?
6. Re-run the network bootstrapper (only if you want to whitelist the
new contract)
8. Authorise the upgrade
Am I right that only those 2 steps are related to Explicit contact upgrades? And If I use implicit flow with signature, then I need to skip only those two steps, while the others are still aplicable and must be performed?
9. Perform the upgrade
Should this be done for each state separately by the owner of the state? In that case should I run it on each node for specific contrcats where the node is the participant of the state? (In doc it is mentioned to be run on single node - but what id=f a single node is not participant of some state)
Other questions
This section describes explicit contracts and states update.
https://docs.corda.net/upgrading-cordapps.html#performing-explicit-contract-and-state-upgrades while signature constraint section (https://docs.corda.net/api-contract-constraints.html#signature-constraints) does not describe an update process for states.
is it the same as for explicit upgrades with the difference only in steps 6,8 or it is somewhat completely different?
Do I need to create the function transforming old states to new states in that case? if not , then how the old states will be handled by new flows?
I see you have many some great questions about contract upgrades. Here is an article that is written by one of our dev-relation engineers. https://medium.com/corda/contract-upgrades-and-constraints-in-corda-425055a9a47f
Feel free to follow up any additional questions that you have.
If you are new to Corda, feel free to join the Corda community channels #http://slack.corda.net/
While performing legacy contract upgrades, you need both the old and new contract jars installed on your node. (present in the cordapps folder).
You can create a new Gradle module say v2-contract and write the new contract in this. This is where you will write your UpgradedContract. You will need to refer to the old v1-conract jar as well as it needs to know what the old state was. To do this add a gradle dependency in v2-contract like below.
dependencies {
// Corda dependencies.
cordapp project(v1_contract)
}
The old jar can be removed from the cordapps folder, once all the states have been upgraded to new v2-contract.
a. For HashConstraints there is no need to run the bootstrapper again. You will write the new contract by implementing UpgradedContractWithLegacyConstraint,run the jar task to build this new jar, add it to the cordapps folder, run the Authorise Flow from all nodes, run the Initiate flow from one of he node's terminal. This is the explicit way of upgrading.
b. However if you are using Whitelistzoneconstraint, you want to make sure to add the new v2-contract jar's hash to whitelist param in network param. You will need to run the network Bootstrapper to whitelist this new v2-contracts jar hash. https://docs.corda.net/network-bootstrapper.html#whitelisting-contracts.
Once you do that you can either go for an explicit upgrade by implementing UpgradedContract, or you can use implicit upgrade.
c. If you are using Signature Constraints, no need to run the network Bootstrapper for the new jar, write the new v2-contract, build it using gradle jar task, replace old jar with new jar. This is the implicit way of upgrading.
You should run the Authorise Flow for all the participants only.
Other questions
There is no explicit upgrade in Signature constraints. You need to make sure you write your state in a backward compatible way, build new jar, replace old jar with new jar. The states will refer to the new contract then.
Hope that helps. Feel free to post more questions on the above answer or ping on Slack.

Cloudera Post deployment config updates

In cloudera is there a way to update list of configurations at a time using CM-API or CURL?
Currently I am updating one by one one using below CM API.
services_api_instance.update_service_config()
How can we update all configurations stored in json/config file at a time.
The CM API endpoint you're looking for is PUT /cm/deployment. From the CM API documentation:
Apply the supplied deployment description to the system. This will create the clusters, services, hosts and other objects specified in the argument. This call does not allow for any merge conflicts. If an entity already exists in the system, this call will fail. You can request, however, that all entities in the system are deleted before instantiating the new ones.
This basically allows you to configure all your services with one call rather than doing them one at a time.
If you are using services that require a database (Hive, Hue, Oozie ...) then make sure you set them up before you call the API. It expects all the parameters you pass in to work so external dependencies must be resolved first.

AWS X-Ray AmazonDynamoDBv2 segment not found

I have a web application (spring) which I want to instrument using AWS-XRay. I have added "AWSXRayServletFilter" in my web.xml, and the below snippet in my spring configuration class, as per documentation.
static {
AWSXRayRecorderBuilder builder = AWSXRayRecorderBuilder.standard()
.withPlugin(new EC2Plugin()).withPlugin(new ECSPlugin());
builder.withSamplingStrategy(new DefaultSamplingStrategy());
AWSXRay.setGlobalRecorder(builder.build());
}
The below dependency is also added in pom.xml
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-xray-recorder-sdk-aws-sdk-instrumentor</artifactId>
<version>1.2.0</version>
</dependency>
During the application start up, I am getting the below exception.
com.amazonaws.xray.exceptions.SegmentNotFoundException: Failed to begin subsegment named 'AmazonDynamoDBv2': segment cannot be found
Any pointers to solve this will be helpful
When you init the global recorder, you should also start the parent segment. You're trying to create a SubSegment, without a Segment.
AWSXRay.setGlobalRecorder(AWSXRayRecorderBuilder.defaultRecorder());
AWSXRay.beginSegment("MySeg");
I only got this in junit-tests where dynamodb was mocked. To fix I put
AWSXRay.beginSegment("AmazonDynamoDBv2");
in test setup (#Before)
I did not touch implementation since I believe this is something Amazon already does in its SDK?

Oracle coherence with weblogic server?

Hi i am new to oracle coherence,
Question 1 : my scenario is, i have to implement the oracle coherence replicated cache in my webapplication.(with weblogic server).the coherence should be the part of the weblogic server means when i start the weblogic server the coherence should start
(both should run in the single JVM).please help me how to do it ?
Question 2 : whether i need a database to maintain the records or oracle coherence it self maintain in file system ? if yes means how and what will happen for the cached data when i shutdown the server?
Q1:
I would describe it in couple of steps:
Place coherence.jar in the classpath. Depending on specific case it can be WLS classpath or application's classpath. Unless you want to share coherence node between many applications it is often a better idea to put it to application's classpath. It also has other advantages like easier maintenance.
Prepare your own cache configuration for the replicated topology. You can skip this step if you want to use coherence default cache configuration coherence-cache-config.xml which includes replicated topology, but keep in mind that your cache name must start with repl- and this is in general not recommended for production. Otherwise put the following to your custom-cache-config.xml file and add it to your application's classpath.
<?xml version="1.0"?>
<cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config
coherence-cache-config.xsd">
<caching-scheme-mapping>
<cache-mapping>
<cache-name>my-repl-cache</cache-name>
<scheme-name>replicated</scheme-name>
</cache-mapping>
</caching-scheme-mapping>
<caching-schemes>
<replicated-scheme>
<scheme-name>replicated</scheme-name>
<backing-map-scheme>
<local-scheme/>
</backing-map-scheme>
<autostart>true</autostart>
</replicated-scheme>
</caching-schemes>
</cache-config>
Create a ContextListener for your application and place the following code into contextInitialized method:
// join existing cluster or form a new one
CacheFactory.ensureCluster();
Start your WLS with the following option:
-Dtangosol.coherence.cacheconfig=custom-cache-config.xml
Deploy and start your application (possibly on many servers)
Q2:
In general coherence is in memory solution and doesn't persist data by default. If you need to manage data in persistent store you can look into CacheStore interface. This is described here in the documentation.
Keep in mind that often you have more than one coherence node in the cluster so you will not lose your data when you shutdown one of them because data is always stored also in other JVM(s). When you restart your node it will join the cluster and your data will be there.
Starting with WebLogic 12.1.2, there is excellent Coherence integration via the "Coherence Containers" functionality of WebLogic, in addition to the ActiveCache feature of WebLogic. Here is a URL for the container feature: http://docs.oracle.com/middleware/1212/wls/WLCOH/deploy-wls-coherence.htm
For the sake of full disclosure, I work at Oracle. The opinions and views expressed in this post are my own, and do not necessarily reflect the opinions or views of my employer.

Resources