I have a workflow that manages contacting people over the phone. It is designed to call a series of numbers until answered or an expiration time is reached. I receive the error below when the workflow is completed following the expiration period, but is in the middle of a phone call. As a result a hangup message is sent to the completed workflow and fails.
How can I detect the workflow has been completed when receiving the hangup message?
System.ServiceModel.FaultException: The execution of an InstancePersistenceCommand was interrupted because the instance 'a9ecca9f-9edd-4dd0-939c-2a7091a6b59c' is already complete.
Server stack trace:
at System.ServiceModel.Channels.ServiceChannel.HandleReply(ProxyOperationRuntime operation, ProxyRpc& rpc)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)
This is a pretty fundamental problem with Workflow Services - there are no guarantees that your workflow is in a proper state to receive the incoming message, and if it's not you get an obscure error. There are too many conditions where the workflow won't be ready, such as a workflow faulted on some transient error that can be retried. To make your code robust you pretty much always need to validate the incoming call to check that the workflow is waiting on that event, and possibly record the event so it can be acted on when the workflow gets into that state. I've found it best to manage the bookmarks myself rather than relying on the WCF send and receive activities, and have the service operations to wake a workflow just be normal WCF operations that record the event, with a separate polling process to attempt to wake the workflow, with retry if the workflow isn't in the right state yet.
Related
I am considering using deferred messages as a delay/timeout alerting mechanism within my Saga. I would do this by sending a deferred message at the same time as sending a message to a handler to do some time-consuming work.
If that handler fails to publish a response within the deferred timespan the Saga is alerted to the timeout/delay via the deferred message arrival. This is different to a handler failure as its still running, just slower than expected.
The issue comes if everything runs as expected, its possible that the Saga will complete all of its steps and you'd find many deferred messages waiting to be delivered to a Saga that no longer exists. Is there a way to clean up the deferred messages you know are no longer required?
Perhaps there is a nicer way of implementing this functionality in Rebus?
Once sent, deferred messages cannot be cancelled.
But Rebus happens to ignore messages that cannot be correlated with a saga instance, and the saga handler does not allow for that particular message type to initiate a new saga, so if the saga instance is gone, the message will simply be swallowed.
That's the difference between using IHandleMessages<CanBeIgnored> and IAmInitiatedBy<CannotBeIgnored> on your saga. 🙂
I know that is a common question, but I've read a kiloton of articles and feel confused. And now I think that it would be better not to read them at all )).
So, how ASP.NET works (only about threads):
http request is served by thread from the thread pool.
while request is processing this thread is busy, because request is processing inside exactly this thread.
when request processing finishes, the thread returns to thread pool, server sends a response.
Is this described behaviour right ?
What really happens when I start new task inside ASP.NET MVC controller?
public ActionResult Index()
{
var task1 = Task.Factory.StartNew(() => DoSomeHeavyWork());
return View();
}
private static async Task DoSomeHeavyWork()
{
await Task.Delay(5000);
}
controller action starts to execute inside thread that processes current request - T1.
the thread pool allocates another one thread (T2) for task1.
task1 starts "immediately" inside T2.
the View result returns "immediately".
ASP.NET do some work, server sends a response, T1 returns to the thread pool, T2 is still alive.
after some time when the DoSomeHeavyWork will be finished, the T2 thread will be returned to the thread pool.
Is it correct ?
Now lets look to async action
public async Task<ActionResult> Index()
{
await DoSomeHeavyWork();
return View();
}
, I understand the difference with previous code sample, but not the process, in this example the behaviour is the following:
action starts to execute inside thread that processes current request - T1.
DoSomeHeavyWork "immediately" returns a task, let's call it "task1" too.
T1 returns to the thread pool.
after the DoSomeHeavyWork finishes, Index action continue to execute.
after Index action execution the server will send a response.
Please explain what happening between points 2 and 5, the questions are:
is the DoSomeHeavyWork processed inside task1 or where (where it is "awaited") ? I think this a key question.
which thread will continue to process the request after await - any new one from the thread pool, right ?
request produces thread allocating from the thread pool, but response will not be sent until the DoSomeHeavyWorkAsync finished and it doesn't matter in which thread this method executes. In other words, according to single request and single concrete task (DoSomeHeavyWork) there is no benefits of using async. Is it correct ?
if the previous statement is correct, then I don't understand how the async can improve performance for multiple requests with the same single task. I'll try to explain. Let's assume that the thread pool has 50 threads available to handle requests. Single request should be processed at least by one single thread from the thread pool, if the request starts another threads, then all of them will be taken from the thread pool, e.g. request takes one thread to process himself, starts 5 different tasks in parallel and wait all of them, thread pool will have 50 - 1 - 5 = 44 free threads to handle incoming requests - so this is a parallelism, we can improve performance for single request, but we reduce number of requests which can be processed. So according request processing in ASP.NET I suppose that only task that somehow starts IO completion thread can achieve a goal of async (TAP). But how IO completion thread calls back thread pool thread in this case ?
Is this described behaviour right ?
Yes.
Is it correct ?
Yes.
is the DoSomeHeavyWork processed inside task1 or where (where it is
"awaited") ? I think this a key question.
From the current code, DoSomeHeavyWork will asynchronously wait for Task.Delay to complete. Yes, this will happen on the same thread allocated by the thread-pool, it won't spin any new threads. But there isn't a guarantee that it will be the same thread, though.
which thread will continue to process the request after await?
Because we're talking about ASP.NET, that will be an arbitrary thread-pool thread, with the HttpContext marshaled onto it. If this was WinForms or WPF app, you'd be hitting the UI thread again right after the await, given that you don't use ConfigureAwait(false).
request produces thread allocating from the thread pool, but response
will not be sent until the DoSomeHeavyWorkAsync finished and it
doesn't matter in which thread this method executes. In other words,
according to single request and single concrete task (DoSomeHeavyWork)
there is no benefits of using async. Is it correct ?
In this particular case, you won't see the benefits of async. async shines when you have concurrent requests hitting the server, and alot of them are doing IO bound work. When using async while hitting the database, for example, you gain freeing the thread-pool thread for the amount of time the query executes, allowing the same thread to process more requests in the meanwhile.
But how IO completion thread calls back thread pool thread in this
case ?
You have to separate parallelism and concurrency. If you need computation power for doing CPU bound work in parallel, async isn't the tool that will make it happen. On the other hand, if you have lots of concurrent IO bound operations, like hitting a database for CRUD operations, you can benefit from the usage of async by freeing the thread while to IO operation is executing. That's the major key point for async.
The thread-pool has dedicated pool of IO completion threads, as well as worker threads, which you can view by invoking ThreadPool.GetAvailableThreads. When you use IO bound operations, the thread that retrieves the callbacks is usually an IO completion thread, and not a worker thread. They are both have different pools.
We encounter a bunch of suspended service instances (like 100). Also we notice that there are more than 100 (related) suspended messages (mostly with NACKs).
What is the difference then to resume from a suspended service instance or to resume from a suspended message ?
Service instances process messages.
BizTalk breaks up services into Service Classes, such as Routing Failure, Isolated Adapter, and Messaging. Those services are distinct from the messages, though the messages are associated with a service. When something fails in BizTalk, typically both a message and a service instance show up in the BizTalk Administration Console as being suspended. If you view the details of the service, then you'll see that it contains a tab with the message(s).
In this context, the message is a property of the service instance. The service was trying to do something with the message and failed. Thus is makes sense to resume the action (service instance), which will make use of the data (message). It doesn't make sense to try and do something like resume a NACK (a message); instead, you should resume the service instance. The NACK can help you find out what went wrong, but if it doesn't go away after resolving the problem and resuming the service instance, then it typically can be safely cleared out.
Is Javamail asynchronous or synchronous? That is, if I send off an email, do I continue processing immediately afterwards, or do I wait until it's complete?
Furthermore, are there any ways that I could catch that an email failed to be delivered for any reason?
I'd also like to know these answers for Spring's MailSender abstraction.
Thanks.
It is synchronous, since it transfers the message to the server and processes the server's response before returning. The send docs explain in further detail. The message will throw a SendFailedException, or another MessagingException,
if the send fails immediately. But "success does not imply that the message was delivered to the ultimate recipient, as failures may occur in later stages of delivery."
Can any one explain what the difference is between the following methods of WorkflowApplication:
Abort
Cancel
Terminate
After further investigating this issue, I want to summarize the differences:
Terminate :
the Completed event of the workflow application will be triggered
the CompletionState (WorkflowApplicationCompletedEventArgs) is Faulted
the Unloaded event of the workflow application will be triggered
the workflow completes
OnBodyCompleted on the activity will be called
Cancel:
the Completed event of the workflow application will be triggered
the CompletionState (WorkflowApplicationCompletedEventArgs) is Cancelled
the Unloaded event of the workflow application will be triggered
the workflow completes
OnBodyCompleted on the activity will be called
Abort:
the Aborted event of the workflow application will be triggered
the workflow does not complete
An unhandled exception
triggers OnUnhandledException
in this eventhandler the return value (of type UnhandledExceptionAction) determines what will happen next:
UnhandledExceptionAction.Terminate will terminate the workflow instance
UnhandledExceptionAction.Cancel will cancel the workflow instance
UnhandledExceptionAction.Abort will abort the workflow instance
Each will trigger the corresponding events explained above
Update: Abort does not seem to trigger unloading of the instance in the SQL persistence store. So it seems to me, you better use Cancel or Terminate and if you have to perform some action based upon the completion status, you can check CompletionState in the Complete event.
First, hats off to Steffen Opel (and his comments below). I failed to catch that my original post linked documentation that was WF 3.5 specific. Did a little more digging around.
For posterity's sake, I have left my previous response below, labelled as WF3.5. Please see WF4.0 for a few notes regarding Canceling, Abort, and Terminate in WF4.0.
WF4.0
Unfortunately, there is little explicit documentation discussing differences in Cancel, Abort, and Terminate in WF4.0. However, from member method documentation,
On Abort, a) activity is immediately halted, b) Aborted handler is invoked, and c) Completed handler is not invoked.
On Cancel, a) activity is given a grace period to stop gracefully after which a TimeoutException is thrown, b) Completed handler is invoked.
On Terminate, a) activity is given a grace period to stop gracefully after which a TimeoutException is thrown, b) Completed handler is invoked.
The differences between Abort and Cancel/Terminate is quite striking. Simply call Abort to kill a Workflow outright. The difference between Cancel and Terminate is more nuanced. Cancel does not require any sort of reason (it is a void parameterless method), whereas Terminate requires a reason (in either string or Exception format).
In all cases, Workflow runtime will not perform any implicit action on your behalf (ie Workflows will not auto-self destruct a la WF3.5 Terminate). However, with the highly customizable exception\event handling exposed by the runtime, any such features may be implemented with relative ease.
WF3.5
Canceling
According to Msdn documentation
An activity is put into the Canceling state by a parent activity explicitly, or because an exception was thrown during the execution of that activity.
While Canceling may be used to stop an entire Workflow (ie invoked on root Activity), it is typically used to stop discrete portions of a Workflow (ie either as error-recovery or an explicit action on part of parent). In short, Canceling is a means of Workflow control-flow.
Abort and Terminate
Again, according to Msdn documentation
Abort is different from Terminate in that while Abort simply clears the in-memory workflow instance and can be restarted from the last persistence point, Terminate clears the in-memory workflow instance and informs the persistence service that the instance has been cleared from memory. For the SqlWorkflowPersistenceService, this means that all state information for that workflow instance is deleted from the database upon termination. You will not be able to reload the workflow instance from a previously stored persistence point.
Which is pretty clear in itself. Abort merely stops in-memory execution whereas Terminate stops in-memory execution and destroys any persisted state.