I have been trying out axon recently and read lots of stuff. From what I understand, the concept of event sourcing says that system state is rebuilt from the event store, while CQRS updates a view model that can be queried with the command side side is not queryable.
I will like to implement rebuilding of state myself each time the UI requests for some information. I've implemented the event processor and seen its replay capabilities. However I can't seem to find any evidence that axon allows for replays triggered on user demand. Therefore I'm asking here if its possible to personally trigger a replay to build a DAO required by the UI or axon only supports the CQRS way of doing it.
When axon does a replay (when token is deleted), does it read from the snapshot table (if implemented), then from the event table or does it always start from the beginning of time?
Dropping the token manually is currently the way to go if you want your query side to be rebuilt.
So, if you'd like to trigger a replay, you'd have to add a component which does that for you.
This might be a tricky process as well, since you'd probably not want any other parties trying to access your view(s) while they're rebuilding.
Take note that for this you do need to put your event handling components in a TrackingEventProcessor, otherwise it will have no recollection of how much of the events it has handled or not in the form of a TrackingToken.
The snapshot table is meant for your aggregates to ease the loading process. When you've set up axon to use the (Caching)EventSourcingRepository, it will first load an aggregate based on a snapshot and then source the remaining events.
So, to answer your second question: when you drop the token, it will read from the domain event entry table from the beginning of time.
Hope this helps!
Update
For those looking for replay support in Axon Framework, it is recommended to look at this page of the Reference Guide. It describes the replay support constructed for Streaming Event Processsors (like the TrackingEventProcessor).
Related
we're looking into using the new feature of Axon, dead letter queue(DLQ).
In our previous application (axon 4.5x) we have a lot of eventhandlers updating projections. Our default is to rethrow exceptions when they occur, which will trigger a rollback for the database updates. Perhaps not the best practice to rely on this behaviour (because it can not rollback everything, eg sending an email from event can not be reverted of course)
This behaviour seems to get lost when introducing DLQ in our applications, which has big impact on our current application flow (projections are updated when they previously weren't). This makes upgrading not that easy.
Is it possible to still get the old behaviour(transaction rolled back in case of exceptions) together with DLQ processing?
What we tried was building a test application to test the new DLQ features. While playing around all looks fine in case of exceptions (they were moved to dlq) but the projections still got updated (not rolled back as before)
We throw an exception after the .save() of the projection simulating a database failure to see if events involved (we have multiple eventhandlers for an event updating projections) got rolled back.
You need to choose here, #davince. Storing a dead-letter in the dead-letter queue similarly requires a database transaction.
To ensure the token still progresses and the dead letter is entered, the framework uses the existing transaction.
Furthermore, in practical terms event handling was successful.
Hence, a rollback for some of the parts wouldn't be feasible.
The best way to deal with this, as is mentioned in the Reference Guide, is to make your event handlers idempotent:
Before configuring a SequencedDeadLetterQueue it is vital to validate whether your event handling functions are idempotent.
As a processing group consists of several Event Handling Components
(as explained in the intro of this chapter), some handlers may succeed
in event handling while others will not. As a configured dead-letter
queue does not stall event handling, a failure in one Event Handling
Component does not cause a rollback for other event handlers.
Furthermore, as the dead-letter support is on the processing group
level, dead-letter processing will invoke all event handlers for that
event within the processing group.
Thus, if your event handlers are not idempotent, processing letters may result in undesired side effects.
Hence, we strongly recommend making your event handlers idempotent when using the dead-letter queue.
The principle of exactly-once delivery is no longer guaranteed; at-least-once delivery is the reality to cope with.
I understand this requires some rework on your end, #davince. Although the feature is powerful, it comes with a certain level of responsibility before you use it.
I hope the above clarifies this for you.
Added, I'd like to point out that the version upgrade in itself does not require you to use the dead-letter queue. Hence, this change shouldn't impose any strains for updating to the latest release.
Update 1
Sometimes you need to think about an issue a bit longer.. I was just wondering the following things about your setup. Perhaps I can help out on that front:
What storage mechanism do you use to store projections in?
Where are you storing your tokens?
Where are you planning to store your dead-letters?
How are you invoking the storage layer from your event handlers?
Update 2
Thanks for sharing that you're using PostgreSQL for your projections, tokens, and dead letters. And that you're using JPA entities for storage.
It gives more certainty about your setup, as it may impact how your system would react in case of a rollback.
However, as it's rather vanilla/regular, the shared comment from the Reference Guide still applies.
This, sadly enough, means some work on your end, #davince. I hope the road forward to start using the SequencedDeadLetterQueue from Axon Framework is clear.
By the way, if you ever have recommendations on how the framework or the documentation may be improved, be sure to file issues in GitHub here and here, respectively.
Is there a way to replay a Saga instance in Axon Framework in such a manner that the transactions that already happened are not replayed?
The simple answer to your question is no #2011ashwini. The Saga's you create in Axon Framework are backed by a dedicated type of Event Handling Component, which enforces that the reset API (provided by the TrackingEventProcessor) has zero impact.
It is this exact API which is the only means to tell Axon Framework you are doing triggering a replay, thus the only means to use things like Axon's #ResetHandler, #DisallowReplay and ReplayStatus parameter (as described further here). Only through these handles would you be able to know whether the Saga is handling events a second time (as you are replaying) yes or no.
The only way to thus forcefully replay a Saga, is by dropping the TrackingToken which is stored for it.
However, Saga's typically introduce side effects. Especially side effects are things you normally would not want to be replayed, ever. Hence why this is closed of by Axon's Saga logic to begin with.
On top of that, you've asked the following:
Is there a way to replay a Saga instance in Axon Framework in such a manner that the transactions that already happened are not replayed?
To be honest with you, that doesn't sound like a replay at all. The transaction which have been performed by the Saga are done so by the events it has handled. Replaying means re-handling those exact same events. Thus you are somewhat contradicting yourself with the question you are asking (hence why I asked for clarification).
Regardless, I hope this provides the additional background you need.
It seems restricting that I can only get a snapshot from my database to read data from on data change. Is there something I'm missing about OnDataChange?
What if I want to populate a page with data read dynamically from my database, yet no data is changing in the database? I still need to call OnDataChange?
Firebase's onDataChange fires immediately with a snapshot of the current value in the database and subsequently whenever the data changes.
In fact, the Firebase documentation says this:
This method is triggered once when the listener is attached and again every time the data, including children, changes.
The simple answer is you can't block the User Interface with a long running task such as a database query or network request. At best the user will have an unresponsive application at worst an "application not responding exception" (ANR)crash will happen. That's why it is designed with a listener pattern. I assume this is android we are talking about, however the answer is valid for other platforms. If you are doing this on a background thread yes you could do what you are saying, in theory. I don't think firebase is designed that way.
I am building a project from scratch using event-sourcing with Java and Cassandra.
My apps we be based on microservices and in some use cases information will be processed asynchronously. I was wondering what part a Message Queue (such as Rabbit, Active MQ Artemis, Kafka, etc) would play to improve the technology stack in this environment and if I understand the scenarios if I won't use it.
I would start with separating messaging infrastructure like RabbitMQ from event streaming/storing/processing like Kafka. These are two different things made for two (or more) different purposes.
Concerning the event sourcing, you have to have a place where you must store events. This storage must be append-only and support fast reads of unstructured data based on an identity. One example of such persistence is the EventStore.
Event sourcing goes together with CQRS, which means you have to project your changes (event) to another store, which you can query. This is done by projecting events to that store, this is where events get processed to change the domain object state. It is important to understand that using message infrastructure for projections is generally a bad idea. This is due to the nature of messaging and two-phase commit issue.
If you look at how events get persisted, you can see that they get saved to the store as one transaction. If you then need to publish events, this will be another transaction. Since you are dealing with two different pieces of infrastructure, things can get broken.
The messaging issue as such is that messages are usually guaranteed to be delivered "at least once" and the order of messages is usually not guaranteed. Also, when your message consumer fails and NACKs the message, it will be redelivered but usually a bit later, again breaking the sequence.
The ordering and duplication concerns, whoever, do not apply to event streaming servers like Kafka. Also, the EventStore will guarantee once only event delivery in order if you use catch-up subscription.
In my experience, messages are used to send commands and to implement event-driven architecture to connect independent services in a reactive way. Event stores, at the other hand, are used to persist events and only events that get there are then projected to the query store and also get published to the message bus.
Make sure you are clear on the distinction between send(command) and publish(event). Udi Dahan touches on that topic in his essay on busses and brokers.
In most cases where you are event sourcing, you do not want to be reconstructing state from published events. If you need state, then query the technical authority/book of record for the history, and reconstruct the state from the history.
On the other hand, event driven activity off of a message queue should be fine. When a single event (plus the subscriber's state) has everything you need, then running off of the bus is fine.
In some cases, you might do both. For example, if you were updating cached views, you'd subscribe to various BobChanged events to know when your cached data was stale; to rebuild a stale view, you would reload a representation of the history and transform it into an updated view.
In the world of event-sourcing applications, message queues usually allow you to implement publish-subscribe pattern style of communication between producers and consumers. Also, they usually help you with delivery guarantees: which messages were delivered to which subscribers and which ones were not.
But they don't store all messages indefinitely. You need to have an event store to do any kind of event sourcing.
The question is not 'to queue or not to queue', but it is more like:
can this thing store huge volume of events indefinitely?
does it have publish-subscribe capabilities?
does it provide at-least-once delivery guarantees?
So, you should use something like Kafka or EventStore to have all that out-of-the-box. Alternatively, you can combine event store with message queue manually, but this is going to be more involved.
I want to make a form where people can sign up for a course. Number of people for a course is limited. I want to make a page where user can see how many places are still available and that number is dynamically updated, so if another user signs for a course the other one sees change. When number of available places reaches 0 the signup button should be disabled. Such task should be easy to implement but I am afraid it is not. I suppose some Ajax will be involved but how to handle server side counting? WebServices? I have a problem to design a logic behind all of this.
The technology/technique you're looking for is called Server Push.
Basic idea: Client should respond to some events happening on Server.
Possible solutions:
Polling some server action via AJAX in a timely fashion;
Keeping long-running AJAX request open on server-side until timeout occurs or event happens, then process acquired result on client (determine if it was server action or just timeout), reestablish connection from client if necessary.
and a couple of other solutions which are basically variations of the above two. Also solution will much depend on server-side technology you're using.
Google has a short yet very informative article on what this technique is and how it can be implemented here. It's (almost) technology agnostic so it should help you to understand concepts and possible solutions.
I'd use a database on the server. For the "courses" table, have an associated table containing the "bookings". Add them up in a SQL query.