I need clarification on using transaction scopes in Rebus.
In Configuration options I have o.HandleMessagesInsideTransactionScope();
Now Do I need to use the statement using scope=new RebusTransactionScope() in my Handlers or the Setup I have made will suffice.
In my Handler I do a database transaction and later send messages to other handlers for further processing.
I need all the processing in the hander to be atomic and hence be wrapped in a transaction scope.
I just need to know the right approach to achieving this.
Regards
Amour Rashid.
You're confusing two types of transaction scopes here: Rebus has its own transaction scope, which you can use like this:
using (var scope = new RebusTransactionScope())
{
await bus.Send(oneMessage);
await bus.Send(anotherMessage);
// no messages have been sent at this point
//
//
// but when we do this 👇 they will be sent
await scope.CompleteAsync();
}
When you're in a Rebus handler, there's no need to use RebusTransactionScope, because the handler will have its own transaction scope, which gets completed after your code has finished executing without any exceptions.
The other transaction scope is a part of .NET. That's what you enable when you call
Configure.With(...)
.(...)
.Options(o => o.HandleMessagesInsideTransactionScope())
.(...)
which will cause Rebus handlers to execute inside a System.Transactions.TransactionScope.
(...) I need all the processing in the hander to be atomic (...)
Unfortunately, it can never be atomic*, because the queueing system and your database cannot be enlisted in the same transaction. I mean, they can PRETEND to be, by elevating your transaction to a DISTRIBUTED TRANSACTION, using the two-phase commit protocol to orchestrate commits, but unfortunately distributed transactions are slow and come with a bunch of problems.
Generally, since Rebus ensures "at least once delivery" of messages, you're better off ensuring that your code is idempotent.
(*) at least in general, it can't
Related
I am new to Rebus and am trying to get up to speed with some patterns we currently use in Azure Logic Apps. The current target implementation would use Azure Service Bus with Saga storage preferably in Cosmos DB (still investigating that sample implementation). Maybe even use Rebus Mongo DB with Cosmos DB using the Mongo DB API (not sure if that is possible though).
One major use case we have is an event/timeout pattern, and after doing some reading of samples/forums/Stack Overflow this is not uncommon. The tricky part is that our Sagas would behave more as a Finite State Machine vs. a Directed Acyclic Graph. This mainly happens because dates are externally changed and therefore timeouts for events change.
The Defer() method does not return a timeout identifier, which we assume is an implementation restriction (Azure Service Bus returns a long). Since we must ignore timeouts that had been scheduled for an event which has now shifted in time, we see a way of having those timeouts "ignored" (since they cannot be cancelled) as follows:
Use a Dictionary<string, Guid> in our own SagaData-derived base class, where the key is some derivative of the timeout message type, and the Guid is the identifier given to the timeout message when it was created. I don't believe this needs to be a concurrent dictionary but that is why I am here...
On receipt of the event message, remove the corresponding timeout message type key from the above dictionary;
On receipt of the timeout message:
Ignore if it's timeout message type key is not present or the Guid does not match the dictionary key/value; else
Process. We could also remove the dictionary key at this point as well.
When event rescheduling occurs, simply add the timeout message type/Guid dictionary entry, or update the Guid with the new timeout message Guid.
Is this on the right track, or is there a more 'correct' way of handling defunct timeout (deferred) messages?
You are on the right track 🙂
I don't believe this needs to be a concurrent dictionary but that is why I am here...
Rebus lets your saga handler work on its own copy of the saga data (using optimistic concurrency), so you're free to model the saga data as if it's being only being accessed by one at a time.
I have a CRUD app in Blazor that simply fetches the assignment lists from a table and has an AssignmentReminderService for data access layer that has a method (async version)
public async Task<AssignmentReminder> AddAssignment(AssignmentReminder assignment)
{
_context.assignments.Add(assignment);
await _context.SaveChangesAsync();
return assignment;
}
I can also call the method with synchromus code as :
public AssignmentReminder AddAssignment(AssignmentReminder assignment)
{
_context.assignments.Add(assignment);
_context.SaveChanges();
return assignment;
}
Now it is just one database being accessed from a local server(could be hosted on cloud as well) with just one assignment table and the default authentication/authorization tables generated when one uses Individual User Account (aspnetusers, aspnetroles etc)
Can someone let me know which of the two ways I should use (between async or sync) method declaration?
In the general case, you should use asynchronous APIs if they are available. This will result in greater scalability on the server side, since asynchrony will allow the calling request thread to be used for other requests while the asynchronous operation is in progress.
There are specific scenarios where this guideline doesn't apply. E.g., if you're calling a generic asynchronous API (like Stream.ReadAsync) but you know that the implementation is actually synchronous (like MemoryStream). But in general, if there's an asynchronous API, then that's the one you should use.
You should be clear about the version of blazor you're talking about, because using async methods in the client is different from using them in the server version.
which of the two ways I should use (between async or sync) method declaration?
The first one.
The scarce resource here are the Threads. You want to keep their number down, and the first approach enables that by releasing the Thread to do other work.
In the second approach the Thread is suspended for the duration of the I/O operation. You would need more Threads to handle the same number of requests.
So using async I/O lets the same hardware handle more requests at the same time.
We are capturing a new committed state in the vault through vaultTrack method on Corda RPC proxy for using in the logs recording. Although it’s working properly, we thinks it might have cause some overhead for network connection. So, we decided to try using ServiceHub in the CorDapp for capturing the new event instead. Unfortunately, the event keep occurring every time when the flow is called (based on observable concept?). Maybe we did not set up properly?. Based on your experience and expertise, could you
Suggest what went wrong; and
The corresponding solutions?
More details here:
As we are using the logs of CorDapp for a performance benchmark. Therefore, we are focusing only new committed state event. In API endpoint where we had started, we are using VaultTrack in RPC to record each new committed state event as shown in the example below:
Although the API seems to work properly but we think it might consume RPC connection in the overall performance since the observable is called every time a new state is committed. Please correct us if we're wrong. As such we decided to change to logging the events in the flow instead.
In CorDapp, we are using VaultService in ServiceHub to record each new committed state event in the ‘call function’ of flow initiator as shown in the example below:
We found that the logs recording in CorDapp i.e. in the flow (from the serviceHub mentioned above) keep gaining duplicated log every time the flow is called. From our initial investigation, we found that the problem is "vaultService" keep getting subscribed every time the flow is initiated. Therefore, we switched back to use the API endpoint method. Please could you advise us the right way to capture the event in CorDapp. To log the event of a newly committed state during our performance testing.
The approach of subscribing to a vault observable within a flow will not work. Once the flow ends, the subscription will not be terminated. Every time you run the flow, an additional subscriber will be added. This will degrade performance (although the RPC overhead is generally quite low as long as the states serialise quickly enough).
You should observe updates to the vault using an RPC client instead. Here is an example:
val client = CordaRPCClient(nodeAddress)
val proxy = client.start(rpcUserUsername, rpcUserPassword).proxy
// Track IOUState updates in the vault
val (snapshot, updates) = proxy.vaultTrack(IOUState::class.java)
// Log the existing IOUStates and listen for new ones.
snapshot.states.forEach { logState(it) }
updates.toBlocking().subscribe { update ->
update.produced.forEach { logState(it) }
}
When you call start on the CordaRPCClient, you will connect to the node's Artemis message queue. This message queue will be used to stream updates from the vault back to the client over time.
In the example above, the vault updates are simply logged. You can change this behaviour as required (e.g. to call an API whenever an update is produced).
Meteor's documentation states:
In Meteor, your server code runs in a single thread per request, not in the asynchronous callback style typical of Node
Do they actually mean?
A) the server is running multiple threads in parallel (which seems unusual within the Node.js ecosystem)
or
B) There is still only a single thread within an evented server and each request is processed sequentially, at least until it makes calls to resources outside the server - like the datastore, at which point the server itself is handling the callbacks while it processes with other requests, so you don't have to write/administer the callbacks yourself.
Brad, your B is correct.
Meteor uses fibers internally. As you said, there's only one thread inside an evented server, but when you do (eg) a database read, Fibers yields and control quickly gets back to the event loop. So your code looks like:
doc = MyCollection.findOne(id);
(with a hidden "yield to the event loop, come back when the doc is here") rather than
MyCollection.findOne(id, function (err, doc) {
if (err)
handle(err);
process(doc);
});
Error handling in the fiber version also just uses standard JavaScript exceptions instead of needing to check an argument every time.
I think this leads to an easier style of code to read for business logic which wants to take a bunch of actions which depend on each other in series. However, most of Meteor's synchronous APIs optionally take callbacks and become asynchronous, if you'd like to use the async style.
I have a web service that receives requests from users and returns some json. I need to save the json string in the database so for the moment, the write query occurs before the response is sent back.
Is there a way to send the response first and then do the write query, after the response left the web service?
Thanks.
There's a couple of different options here - they all have tradeoffs, though, and would be pretty esoteric. You don't mention why you want to do this, so I'm guessing performance. If that's the case, I think you're barking up the wrong tree - a simple write is almost certainly not your performance problem.
So, off the top of my head:
Queuing, as Ragesh mentions, would be a nice approach. This gets you similar semantics of a transaction, while off loading the write. You still have to write to the queue, though, which may be about the same overhead as writing to the DB.
You could spawn a new thread (using either the ThreadPool or System.Threading.Thread - there's some debates about which is preferable in ASP.NET) to handle the write. This can generally work, but you may have issues with unhandled exceptions, app domain restarts, etc.
You could store the JSON data into a static or Application variable, then use a Timer to periodically write them to the DB. This will be multithreaded code, so you will need to synchronize read/writes to the collection.
Similar to #3, store the JSON data into Cache and use the invalidation callback to write to the DB.
Lots of variations on store somewhere (memory, disk, flat DB table, etc.), process later (ASP.NET, scheduled task, Windows Service, Sql Agent, etc.).
#frenchie says: a response starts by reading the json string from the db and ends with writing it back. In other words, if the user sends a request, the json string that's going to be read must be the one that was written in the previous response.
That complicates things, since inherent in async work is not knowing when something is done. If you require the async portion (writing back to the DB) to be done before handling the next request, you'll have to execute a wait to make sure it actually completed. In order to do that, you'll need to keep server side state on the client - not exactly a best practice as far as services go (though, it sounds like you're already doing that with these JSON request/response pairs).
Given the complications, I would make sure that you've done your profiling and determined it is indeed a performance problem.
You can do schedule a query work like
ThreadPool.QueueUserWorkItem(state =>
this.AsynchronousExecuteReference());
// and run
static void AsynchronousExecuteReference()
{
// run here your sql update
}
One other example using Thread inside an class and you can pass parameters to it.
public class RunThreadProcess
{
// Some parametres
public int cProductID;
// my thread
private Thread t = null;
// start it
public Thread Start()
{
t = new Thread(new ThreadStart(this.work));
t.IsBackground = true;
t.SetApartmentState(ApartmentState.MTA);
t.Start();
return t;
}
// actually work
private void work()
{
// do thread work
all parametres are available here
}
}
And here is how I run it
var OneAction = new RunThreadProcess();
OneAction.cProductID = 100;
OneAction.Start();
Do not worry about memory, CG knows that this process is used until the thread ends, so I have check it and CG not delete it and wait the thread to ends.
You should look at using message queues like MSMQ, ActiveMQ or RabbitMQ to do this. When you receive your request, you'll put the relevant data in to the queue, and send your response to the client. At the other end of the queue, you'll have some process that reads from the queue and inserts data in to your database.
this is missing the point of a request/response. unless you want to get into async commands like a service bus, but that's pub/sub, not request/response. the point of request/response is to do the work on the server after receiving the request and before sending the response. even if the work is sending an async message to a service bus.
You could try moving your web service URL to an ASPX page where the lifecycles come in to play.
In the code-behind, call your routine that does the main portion of the work in Page_Load or Page_Prerender (or whenever is appropriate prior to the response being sent) and then do your DB work in the Page_Unload event which occurs after the response has been sent (http://msdn.microsoft.com/en-us/library/ie/ms178472.aspx).