The problem is that I need a standard way to serialize and deselialize domain events between different microservices (either by a unique identifier for each type event), so the contract type of these messages must be agnostic to the programming language.
Is there any protocol or standard of communication between the passage of events between microsrevices in order to identify them with queues? What is the best way for you? Or some standard framework for communicating these events on net.Core?
We copy the events as external events into the other services. We also use a shared event model to communicate upcoming changes to everyone. https://eventmodeling.org/posts/what-is-event-modeling/
This is a case of the Publisher/Subscriber model/pattern of events. Usually, an organization may define its own format of an event message, which must be followed by each publisher (domain in this case, such as Customer, Order, Product, Payment, etc.), and then the subscriber app/domain will consume the published event [if Subscribed to].
While there are many options available for implementing this, such as RabitMQ, Azure Service Bus, Azure EventHUb, and open-source Kafka or Azure Event Hubs for Apache Kafka. You may want to 1st align the pub/sub and what contract will work best for your organization (you can always evolve).
Please refer to https://learn.microsoft.com/en-us/dotnet/architecture/microservices/multi-container-microservice-net-applications/integration-event-based-microservice-communications for some samples of using RabitMQ if that will interest you.
Related
I'm using Axon version (3.3) which seamlessly supports Kafka with annotation in the SpringBoot Main class using
#SpringBootApplication(exclude = KafkaAutoConfiguration.class)
In our use case, the command side microservice need to pick message from kafka topic rather than we expose it as Rest api. It will store the event in event store and then move it to another kafka topic for query side microservice to consume.
Since KafkaAutoCOnfiguration is disabled, I cannot use spring-kafka configuration to write a consumer. How can I consume a normal message in Axon?
I tried writing a normal Kafka spring Consumer but since Kafka Auto COnfiguration is disabled, initial trigger for the command is not picked up from the Kafka topic
I think I can help you out with this.
The Axon Kafka Extension is solely meant for Events.
Thus, it is not intended to dispatch Commands or Queries from one node to another.
This is very intentionally, as Event messages have different routing needs apposed to Command and Query messages.
Axon views Kafka a fine fit as an Event Bus and as such this is supported through the framework.
It is however not ideal for Command messages (should be routed to a single handler, always) or Query messages (can be routed to a single handler, several handlers or have a subscription model).
Thus, I you'd want to "abuse" Kafka for different types of messages in conjunction with Axon, you will have to write your own component/service for it.
I would however stick to the messaging paradigm and separate these concerns.
For far increasing simplicity when routing messages between Axon applications, I'd highly recommend trying out Axon Server.
Additionally, here you can hear/see Allard Buijze point out the different routing needs per message type (thus the reason why Axon's Kafka Extension only deals with Event messages).
Is it possible using Rebus to set priority on messages?
The scenario is that we have a queueworker singing documents, for different services, some of them witch the user awaits the result, and some that the signed document is just stored for later use.
It would then be nice to prioritize the messages of the users that awaits a response. Is this possible ?
We are using Rebus2 with Azure ServiceBus
Unfortunately, since Azure Service Bus does not natively support message prioritization, and implementing it with the priority queue pattern would be cumbersome, it cannot readily be done by setting a priority on a message.
A simple approach that works in the general case is to have separate Rebus instances for different priorities, where one particular instance could then be used to "fast-track" messages that need to overtake all the other messages.
The instances could have the exact same configuration, except they use different input queues. This way, the routing configuration (endpoint mappings) get to determine which priority a message gets.
Why we need to use BizTalk to process HL7 Messages? Is there any other alternative rather than BizTalk?
I need exact reason why we need to opt out for BizTalk only...Is there anyother way to Process HL7 Messages?
Q: 'How many ways are there to skin a cat?' A: 'Lots'
There are lots of ways to implement HL7 messaging, which at the end of the day is just a messaging standard (see http://www.hl7.org/ or http://www.hl7.org.uk/). Oracle's WebLogic product has a HL7 adapter, IBM's WebSphere product has a HL7 adapter, iWay has a HL7 adapter etc. etc. etc. or you could 'roll your own' using your favourite language.
Companies usually go with an integration product (BizTalk, WebLogic, Websphere etc.) rather than rolling their own given that these products usually have done the hard-work of implementing HL7 (add whatever other message standard you want here), allowing you to easily interface with that message standard. You also benefit from the additional features of the integration product (in the case of BizTalk, this is redundant messaging, mapping, a WIDE variety of transport adapters, Orchestration, message validation, enterprise grade load-balancing etc.)
Alternatively, you could 'roll your own' for the particular messages you are concerned with in the language of your choice AND then add any of the enterprisey features you need, however this will be a much bigger undertaking that buying an off-the-shelf middleware product that already implements the standard and is likely to be dearer once you have factored in Project Management, Testing etc.
At the end of the day, it comes down to which technology platform you use within your business - are you a Java / .Net / or Other shop? Once you clearly have that defined, look for the middleware product that suits your platform knowledge and evaluate, or 'roll your own'.
I have created a simple order manager wf service (state machine) in WF4.
Order (EF entity) properties: Id, IsExport, NumOfProduct, ProductName, Status (waiting, approved, rejected).
State machine states:
1. OrderReceived (validation -> response activity)
2. Waiting (empty)
- Transitions:
update(update order activity) -> waiting state
approve(assign status field, update order and response activities) -> final state
3. Final state.
Correlation key: Order.Id
The implementation rised a few questions.
WF can manage one flow of the order instance, the order flow and the order entity is in one-one relation.
Question is that where and how should I implement the listing of entityes according to a state filter (eg. approved orders or waiting orders). The list should be accessible via WCF service method.
What is the best practise to manage the batch data processing. (eg: Multiple order approval. "Foreach" in the client is not the required sln.)
The state of the order is symbolized by the "state activity persisted instances" and the entity's status field in the db as well.
What is the best practise to decide the state of the entity, listing the active persisted activity instances in the defined state or select the entities from the db (by an activity) according to a state filter parameter?
Any help would be appreciated.
Good questions!
Taking your first and third questions, there are several possible approaches to this. All require that you write a custom WCF service to enumerate the required orders. This would probably not be a WF service; it might be a REST or OData service. How would you implement the service?
You could do it entirely by querying your database through EF. This would have no dependency on WF at all, and is probably the easiest way. Your workflow would update the database record on each state change, and the service would only need to read that value.
You could rely on the tracking mechanism provided by WF, and the extensions that Ron Jacobs refers to in his answer to your question. The tracking infrastructure is described here on MSDN. It is possible to use the tracking object in memory to get the state of active workflows. However, this probably won't work well with IIS/WF services, which are automatically persisted and unloaded when dormant. You would be better off using the tracking facilities to write state records to a database. Your custom service would then just query this tracking database.
Unless you want comprehensive information about the state changes and updates that have occurred through your WF service, suggestion number one should suffice.
As for your second question, that is a little more complicated. Let's say you write a REST service that lists the orders awaiting approval. You write a Web page that displays those orders, and the user can check the orders he wants to approve. Now, the number of workflows that you need to update is the same as the number of orders he approves.
You could, as you mention, call the Web service multiple times—but for a large number of orders that would be an unnecessary overhead.
What's the alternative? You would need to write a custom service method on your non-WF service that takes an array of order ids. That service would have to call your WF service multiple times to update each one. Since the WF service is being called from another service on the same machine, you can use the .Net Named Pipe binding instead of one of the HTTP bindings so that the overhead is much less.
It's worth noting that Entity Framework doesn't support batched updates either. You'd need to write a stored procedure or custom SQL if you wanted the database update to be batched too.
Is all of this worth the effort? Probably! Using WCF and the named pipes binding is pretty standard with WF. You'll need to configure Windows Activation Service for named pipes. Also, if you're not already using AppFabric for Windows Server, have a look into it, because it adds some very good management tools for WF services.
I recently published some new samples to show how you can access the current state of the StateMachine and possible transitions. These might help you.
Windows Workflow Foundation (WF4) - Tracking State Machine Workflow Service
Windows Workflow Foundation (WF4) - Tracking State Machine
I am designing a Biztalk solution which requires client applications to subscribe and receive only a certain subset of event messages depending on their user permissions. Subscription will be done through topic or content based routing. The client will subscribe once and receive many messages until they choose to unsubscribe.
Client applications will number in the 100s and subscribed topics could change on a regular basis, so defining an individual send port from Biztalk for each reciever isn't a viable solution.
I have thought I could build an additional message broker service which holds the individual client subscriptions and distributes messages sent from a biztalk port.
I have also seen that a recipient list pattern can be build using orchestrations. This appears to me to still follow a request-response pattern though and I am after 1 way subscribe message to many returned event messages.
My message broker solution seems to me to be doubling up on what Biztalk should be good at so I imagine I am missing some important functionality somewhere. Has anyone tried such an application before and can give some pointers? Should I be investingating the ESB toolkit as a solution? I have had a look on the net but nothing makes it very clear for this type of topic-subscription model.
Thanks,
Phil
Do take a look at the ESB Toolkit. You can use the itinerary functionality that it adds to BizTalk, either with one of the built-in resolvers (e.g., UDDI) or with your own custom resolver. This allows you to route messages based on configuration (stored in Business Rules or elsewhere).
You will find a developer-oriented overview video of the ESB Toolkit on MSDN, which is a decent introduction to the design process and tooling. There are several other helpful videos there as well.
Your specific scenario can accomplished with a single itinerary, as described here. Use a receive pipeline with the ESB Dispatch Disassembler component, configure multiple resolvers, and for each resolver a new message is produced.
There are also two samples to look at:
The Itinerary On-Ramp Sample - builds a set of SOAP headers that contain the itinerary that you create in the test client, loads the specific message file from disk, appends the itinerary headers to the message, and submits it to the ESB through an Itinerary on-ramp for processing.
The Scatter-Gather Sample - Also appends SOAP headers containing the itinerary to the message, which is submitted to the ESB through an on-ramp for processing. A Broker orchestration analyzes the settings for its itinerary step, retrieves a collection of resolvers associated with the itinerary step, and for each of those resolvers resolves the service endpoint. After that, the orchestration activates the proper ServiceDispatcher orchestration instances to dispatch the outbound request messages.
You should also look at "How to: Route a Single Message to Multiple Recipients Using an Itinerary Routing Slip" or perhaps look into creating a custom itinerary message service (documentation is here).