Restoring messages from database - rebus

I am thinking of a way to manage failed messages in Rebus.
In my second level retry strategy I want to save the message and exception details into the database so that I can later review the error details and decide whether to resend the message to the be reprocessed or ignore and delete.
In the handler I am capturing details as follows:
public async Task Handle(IFailed<StudentCreated> failedMessage)
{
//Logic to Defer Message with rebus_defer_count not shown
DictionarySerializer dictionarySerializer = new
DictionarySerializer();
ObjectSerializer objectSerializer = new ObjectSerializer();
string headers =
dictionarySerializer.SerializeToString(failedMessage.Headers);
string message =
objectSerializer.SerializeToString(failedMessage.Message);
Exception lastException= failedMessage.Exceptions.Last();
string exception = objectSerializer.SerializeToString(lastException);
//Logic to save the message and error details in the database not shown
}
This will enable me to save the message and error details into the database where I can create a dashboard to view the messages and resolve them as I wish rather than in the broker queue such as RabbitMQ.
Now my question is how can I return them to the handler where the error was raised using the information provided in the headers?
What is the best way to do it with REBUS provided I have all the details from the Failed Message as shown in my code snippet?
Regards

What you're trying to achieve will be much easier if you make a small change to your application. You see, Rebus already has a built-in service in place for handling failed messages called IErrorHandler.
You can register your own error handler like this:
Configure.With(...)
.(...)
.Options(o => o.Register<IErrorHandler>(c => new MyCustomErrorHandler()))
.Start();
thus replacing the default error handler (which btw. is PoisonQueueErrorHandler)
The error handler gets to handle the message in the form of the raw TransportMessage (i.e. simply headers and a byte[]) when all retries have failed, so this is the perfect place to save the message to your database.
If you then look here, you can see how Rebus' default error handler adds its own queue name as the rbs2-source-queue header, meaning that the message can later be sent back to that queue.
With this information, it should be fairly easy to write some code that inspects the message for its source queue and sends a RabbitMQ message to that queue.
This will only work if the re-delivery service has access to the RabbitMQ instance where all of your Rebus endpoints are running, of course. It's less straightforward, if you want to implement this in a general way: E.g. if you were using Fleet Manager, each Rebus instance would use a long-polling protocol to query the server for commands, which enables Fleet Manager to tell any Rebus instance to e.g. send a previously failed message to any queue it has access to.

Related

Timing issue a C++/winRT BLE connection attempt?

I am using C++/winRT UWP to discover and connect to Bluetooth Low Energy devices. I am using the advertisment watcher to look for advertisements from devices I can support. This works.
Then I pick one to connect to. The connection procedure is a little weird by my way of thinking but according to the microsoft docs one Calls this FromBluetoothAddressAsync() with the BluetoothAddress and two things happen; one gets the BluetoothLEDevice AND a connection attempt is made. One needs to register a handler for the connection status changed event BUT you can't do that until you get the BluetoothLEDevice.
Is there a timing issue causing the exception? Has the connection already happened BEFORE I get the BluetoothLEDevice object? Below is the code and below that is the log:
void BtleHandler::connectToDevice(BluetoothLEAdvertisementReceivedEventArgs eventArgs)
{
OutputDebugStringA("Connect to device called\n");
// My God this async stuff took me a while to figure out! See https://msdn.microsoft.com/en-us/magazine/mt846728.aspx
IAsyncOperation<Windows::Devices::Bluetooth::BluetoothLEDevice> async = // assuming the address type is how I am to behave ..
BluetoothLEDevice::FromBluetoothAddressAsync(eventArgs.BluetoothAddress(), BluetoothAddressType::Random);
bluetoothLEDevice = async.get();
OutputDebugStringA("BluetoothLEDevice returned\n");
bluetoothLEDevice.ConnectionStatusChanged({ this, &BtleHandler::onConnectionStatusChanged });
// This method not only gives you the device but it also initiates a connection
}
The above code generates the following log:
New advertisment/scanResponse with UUID 00001809-0000-1000-8000-00805F9B34FB
New ad/scanResponse with name Philips ear thermometer and UUID 00001809-0000-1000-8000-00805F9B34FB
Connect to device called
ERROR here--> onecoreuap\drivers\wdm\bluetooth\user\winrt\common\bluetoothutilities.cpp(509)\Windows.Devices.Bluetooth.dll!03BEFDD6: (caller: 03BFB977) ReturnHr(1) tid(144) 80070490 Element not found.
ERROR here--> onecoreuap\drivers\wdm\bluetooth\user\winrt\device\bluetoothledevice.cpp(428)\Windows.Devices.Bluetooth.dll!03BFB9B7: (caller: 03BFAF01) ReturnHr(2) tid(144) 80070490 Element not found.
BluetoothLEDevice returned
Exception thrown at 0x0F5CDF2F (WindowsBluetoothAdapter.dll) in BtleScannerTest.exe: 0xC0000005: Access violation reading location 0x00000000.
It sure looks like there is a timing issue. But if it is, I have no idea how to resolve it. I cannot register for the event if I don't have a BluetoothLEDevice object! I cannot figure out a way to get the BluetoothLEDevice object without invoking a connection.
================================ UPDATE =============================
Changed the methods to IAsyncAction and used co_await as suggested by #IInspectable. No difference. The problem is clearly that the registered handler is out of scope or something is wrong with it. I tried a get_strong() instead of a 'this' in the registration, but the compiler would not accept it (said identifier 'get_strong()' is undefined). However, if I commented out the registration, no exception is thrown but I still get these log messages
onecoreuap\drivers\wdm\bluetooth\user\winrt\common\bluetoothutilities.cpp(509)\Windows.Devices.Bluetooth.dll!0F27FDD6: (caller: 0F28B977) ReturnHr(3) tid(253c) 80070490 Element not found.
onecoreuap\drivers\wdm\bluetooth\user\winrt\device\bluetoothledevice.cpp(428)\Windows.Devices.Bluetooth.dll!0F28B9B7: (caller: 0F28AF01) ReturnHr(4) tid(253c) 80070490 Element not found.
But the program continues to run an I continue to discover and connect. But since I can't get the connection event it is kind of useless at this stage.
I hate my answer. But after asynching and co-routining everything under the sun, the problem is unsolvable by me:
This method
bluetoothLEDevice = co_await BluetoothLEDevice::FromBluetoothAddressAsync(eventArgs.BluetoothAddress(), BluetoothAddressType::Random);
returns NULL. That should not happen and there is not much I can do about it. I read that as a broken BLE API.
A BTLE Central should be able to do as follows
Discover a device if new then:
If user selects connect, connect to
the device
perform service discovery
read/write/enable
characteristics as needed
handle indications/notifications
If at any time the peripheral sends a security request or insufficient authentication error, start pairing
repeat the action that caused the insufficient authentication.
On disconnect, save the paired and bonded state if the device is pairable.
On rediscovery of the device, if unpaired (not a pairable device)
repeat above
If paired and bonded
start encryption
work with the device; no need to re-enable or do service discovery
========================= MORE INFO ===================================
This is what the log shows when the method is called
Connect to device called
onecoreuap\drivers\wdm\bluetooth\user\winrt\common\bluetoothutilities.cpp(509)\Windows.Devices.Bluetooth.dll!0496FDD6: (caller: 0497B977) ReturnHr(1) tid(3b1c) 80070490 Element not found.
onecoreuap\drivers\wdm\bluetooth\user\winrt\device\bluetoothledevice.cpp(428)\Windows.Devices.Bluetooth.dll!0497B9B7: (caller: 0497AF01) ReturnHr(2) tid(3b1c) 80070490 Element not found.
BluetoothLEDevice returned is NULL. Can't register
Since the BluetoothLEDevice is NULL, I do not attempt to register.
================= MORE INFO ===================
I should also add that taking an over-the-air sniff reveals that there is never a connection event. Though the method is supposed to initiate a connection as well as return the BluetoothLEDevice object, it ends up doing neither. My guess is that the method requires more pre-use setup of the system that only the DeviceWatcher does. The AdvertisementWatcher probably does not.
In BLE you always have to wait for every operation to complete.
I am not an expert in C++, but in C# the async connection procedure returns a bool if it was successful.
In C++ the IAsyncOperation does not have a return type, so there is no way to know if the connection procedure was successful or completed.
You will have to await the IAsyncOperation and make sure that you have a BluetoothLEDevice object, before you attach the event handler.
To await an IAsyncOperation there is a question/answer on how to await anIAsyncOperation:
How to wait for an IAsyncAction? How to wait for an IAsyncAction?

How to set Rebus transaction context for a web service

Given that I have a web/SOAP service, how do I setup and teardown a proper transaction context for Rebus (the messaging bus)? When Rebus is calling into a message handler this is not a problem since Rebus will setup the transaction context before calling the handler - but what about the opposite where a web service handler needs to send/publish a message via Rebus?
I am not interested in how to implement an HTTP module or similar - only the basics around Rebus: what is needed to prepare Rebus for sending a message?
The web service code has its own transaction going on when talking to the application database. I need to be able to setup Rebus when setting up the database transaction and comit/rollback Rebus when doing the same with the database.
I have a similar problem with standalone command line programs that needs to both interaction with a database and sending Rebus messages.
Rebus will automatically enlist send and publish operations in its own "ambient transaction context", which is accessed via the static(*) AmbientTransactionContext.Current property.
You could implement ITransactionContext yourself if you wanted to, but Rebus comes with DefaultTransactionContext in the box.
You use it like this:
using(var context = new DefaultTransactionContext())
{
AmbientTransactionContext.Current = context;
// send and publish things in here
// complete the transaction
await context.Complete();
}
which could easily be put e.g. in an OWIN middleware or something similar.
(*) The property is static, but the underlying value is bound to the current execution context (by using CallContext.LogicalGet/SetData), which means that you can think of it as thread-bound, with the nice property that it flows as expected to continuations.
In Rebus 2.0.2 it is possible to customize the accessors used to get/set the context by calling AmbientTransactionContext.SetAccessors(...) with an Action<ITransactionContext> and a Func<ITransactionContext>, e.g. like this:
AmbientTransactionContext.SetAccessors(
context => {
if (HttpContext.Current == null) {
throw new InvalidOperationException("Can't set the transaction context when there is no HTTP context");
}
HttpContext.Current.Items["current-rbs-context"] = context
},
() => HttpContext.Current?.Items["current-rbs-context"] as ITransactionContext
);
which in this case makes it work in a way that flows properly even when using old school HTTP modules ;)

Restore a 1:1 conversation in Skype Web SDK

Is it possible to restore a 1:1 conversation?
The Conversation object in the Skype SDK seems to have such functionality. You should be able to restore a conversation by passing a href to it. But when I pass a href string as parameter to createConversation it throws the following error:
Error: ResourceNotFound
at Error (native)
at Exception (http://.../SkypeSDK.js:3346:31)
at UCWA.get (http://.../SkypeSDK.js:15141:31)
at init (http://.../SkypeSDK.js:40672:50)
at new Conversation (http://.../SkypeSDK.js:41826:25)
at createConversationModel (http://.../SkypeSDK.js:41963:36)
at BaseModel.createConversation (http://.../SkypeSDK.js:42037:48)
The lines can be a little bit off. I modified the createConveration method to pass the href to Conversation.
The href string has this format:
/ucwa/oauth/v1/applications/xxxxxxxxxxxx/communication/conversations/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
We have the following situation:
One site has the normal SDK and waits for incoming calls. If you accept the call you should be redirected to a site with the SDK+CC and answer the call. Now we are stuck at how to pass the call. We also tried with it getConversation, but it doesn't return the last incoming conversation.
Once you accept a call on one endpoint you can transfer it to another endpoint. However you cannot accept a call and then re-answer it on a different endpoint. Also, answering the call starts the process of connecting media so that endpoint has effectively picked up the call.
The href of each conversation is unique per application, and in your scenario you will have one for each site. These cannot be shared between applications.

Skype returning empty CHATMESSAGES results through the DBus API

I am trying to use Skype's DBus API in order to retrieve the list of messages (message IDs) I've exchanged with a contact. However, both the SEARCH CHATMESSAGES <target> (protocol >= 3) and the SEARCH MESSAGES <target> (protocol < 3) commands return unexpectedly empty results.
Here is the trace of a few exchanges I had with the API. I used d-feet to send my requests, but the result is exactly the same when I send the request from my own program.
Bus name: com.Skype.API
Object: /com/Skype
Interface: com.Skype.API
Method used: Invoke(String request)
Trace:
-> NAME dfeet
<- OK
-> PROTOCOL 8
<- PROTOCOL 8
-> SEARCH CHATMESSAGES mycontact
<-
The same thing happens with two other SEARCH commands:
SEARCH MESSAGES <target> (with PROTOCOL 2).
SEARCH CHATS
Additionally, I also get an empty result when I try to request a message list based on a chat ID: GET CHAT <chat_id> GETMESSAGES.
However, commands such as SEARCH FRIENDS, SEARCH CALLS, or SEARCH ACTIVECHATS work just fine, and return their lists of IDs (contacts IDs, calls IDs, or chat IDs) as expected.
It might also be worth noting that this happens for all contacts, regardless of how many messages I've exchanged with them (I thought at first that there might be too many messages involved, but the result is the same, whether I've sent 3, or thousands of messages to the contact).
Is there anything that would explain why I get these empty responses through DBus, for these requests?
Skype will not use Invoke's return value when its reply is too heavy. As it so happens, when Skype has too much data to prepare and transfer after a request, it automatically returns an empty string to the Invoke call. The true, heavy reply is then prepared asynchrously by Skype, and the client program must be ready to receive it when it eventually arrives.
Whenever you are communicating with Skype over DBus, your application must act as both a client (calling Invoke), and a server (providing a DBus object for Skype to reach). This design was a little unexpected (I guess we could argue on its quality), but here is what it requires you to do:
Make your program a DBus "server" (providing objects to reach). Through your bus name to Skype, register an object path called /com/Skype/Client implementing the com.Skype.API.Client interface.
Prepare a message handler for the only method of this interface: Notify(s). This is the method Skype will try to call to send you the heavy reply to one of your previous requests.
Program your own mechanism to match your Invoke request with the asynchronous Notify message coming in as an answer later on.
The creation of an object can be done through dbus_connection_register_object_path, the parameters for which are:
The DBusConnection structure representing your bus name.
The object path you are registering, here /com/Skype/Client.
A table of message handlers (DBusObjectPathVTable) used to process all incoming requests.
Data to be sent to these handlers when they are called. This is additional data, not the actual message being received since you're just setting up the handler here.
For instance...
DBusHandlerResult notify_handler(DBusConnection *connection,
DBusMessage *message,
void *user_data){
return DBUS_HANDLER_RESULT_HANDLED;
}
void unregister_handler(DBusConnection *connection,
void *user_data){}
DBusObjectPathVTable vtable = {
unregister_handler,
message_handler,
NULL
};
if(!dbus_connection_register_object_path(connection,
"/com/Skype/Client",
&vtable, NULL)){
// Error...
}
Note that this is just an object's definition. In order to actually hook on the Notify calls, you'll have to select() on a DBusWatch file descriptor, and dispatch the incoming DBusMessage in order to have your message handler called.
If you are working with other bindings, you'll probably find much faster ways to setup objects and start working as a client application. See:
GLib's g_dbus_connection_register_object
Exporting objects with dbus-python
QtDBus's QDBusConnection::registerObject
... (other bindings)

How to log Error Message Contents in Rebus?

Is there anyway to log message contents when an exception occurs?
I looked at various logging extensions but they are just logging CorrelationId. And message contents are not available.
There is a CurrentMessge property in MessageContext, but that is not available at the time logger writes the exception.
I tried to handle PoisonMessage Event, which allows me to log the message contents.
public static void OnPoisonMessage(IBus bus, ReceivedTransportMessage receivedTransportMessage, Rebus.Bus.PoisonMessageInfo poisonMessageInfo) {
var message = new JsonMessageSerializer().Deserialize(receivedTransportMessage);
Log.Error("{#messageType} failed {#message}", message.Messages[0].GetType(), message);
}
This works great, but now I have two errors in the log one coming from my handler and the other coming from logger.
I am wondering if there is a better way to handle this requirement.
If your requirement is to simply log the message contents as JSON, I think you've found the right way to do it - at least that's the way I would have done it.
I'm curious though as to what problem you're solving by logging the message contents - you are aware of the fact that the failing message will end up in an error queue where you can inspect it?
If you're using MSMQ, you can inspect JSON-serialized messages using Rebus' Snoop-tool, which is a simple MSMQ inspector. It will also allow you to move the message back into the input queue where it failed ("return to source queue")
A good way to monitor your Rebus installation is to set up some kind of alerts when something arrives in an error queue, and then you can look at the message (which event includes the caught exceptions in a special header) and then resolve the situation from there.

Resources