In my setup I have two RabbitMQ servers that are used by different applications employing Rebus ESB. What I would like to know is if I can map a message to a queue on a different Host the way I can with MassTransit.
I also would like to know if I can send messages in a batch mode the same way with MassTransit.
Thanks In Advance.
In my setup I have two RabbitMQ servers that are used by different applications employing Rebus ESB. What I would like to know is if I can map a message to a queue on a different Host the way I can with MassTransit.
I am not sure how this works with MassTransit, but I'm pretty sure it's not readily possible with Rebus.
With Rebus, you're encouraged to treat this as you would any other integration scenario, where you'd put a ICanSendToOtherSystem in your IoC container, which just happens to be implemented by CanSendToOtherSystemUsingRebus. Your CanSendToOtherSystemUsingRebus class would probably look somewhat like this:
public class CanSendToOtherSystemUsingRebus : ICanSendToOtherSystem, IDisposable
{
readonly IBus _bus;
public CanSendToOtherSystemUsingRebus(string connectionString)
{
_bus = Configure.With(new BuiltinHandlerActivator())
.Transport(t => t.UseRabbitMqAsOneWayClient(connectionString))
.Start();
}
public Task Send(object message) => _bus.Send(message);
public void Dispose() => _bus.Dispose();
}
(i.e. just something that wraps a one-way client that can connect to that other RabbitMQ host, registered as a SINGLETON in the container)
I also would like to know if I can send messages in a batch mode the same way with MassTransit. Thanks In Advance.
Don't know how this works with MassTransit, but with Rebus, you can give the transport more convenient circumstances for optimizing the send operation(s) by using scopes:
using var scope = new RebusTransactionScope();
foreach (var message in lotsOfMessages)
{
// scope is automagically detected
await bus.Send(message);
}
await scope.CompleteAsync();
which will improve the rate with which you can send/publish with most transports. Just remember that the scope results in queuing up messages in memory before actually sending them, so you'll probably not want to send millions of messages in each batch.
I hope that answered your questions 🙂
Related
Consider this extremely simple .NET Core 3.1 (and .NET 5) application with no special config or hosted services:
using System.Threading.Tasks;
using Microsoft.Extensions.Hosting;
internal class Program
{
public static async Task Main(string[] args)
{
var builder = Host.CreateDefaultBuilder(args);
builder.UseWindowsService();
var host = builder.Build();
var fireAndforget = Task.Run(async () => await host.RunAsync());
await Task.Delay(5000);
await host.StopAsync();
await Task.Delay(5000);
await host.RunAsync();
}
The first Run (sent as a background fire and forget task only for the purpose of this test) and Stop complete successfully. Upon calling Run a second time, I receive this exception:
System.AggregateException : 'Object name: 'EventLogInternal'.Cannot access a disposed object. Object name: 'EventLogInternal'.)'
If I do the same but using StartAsync instead of RunAsync (this time no need for a fireAndForget), I receive a System.OperationCanceledException upon called StartAsync the second time.
Am I right to deduce that .NET Generic Host aren't meant to be stopped and restarted?
Why do I need this?
My goal is to have a single application running as a Windows Service that would host two different .NET Generic Host. This is based on recommendation from here in order to have separate configuration and dependency injection rules and message queues.
One would stay active for all application lifetime (until the service is stopped in the Windows services) and would serve as a entry point to receive message events that would start/stop the other one which would be the main processing host with full services. This way the main services could be in "idle" state until they receive a message triggering their process, and another message could return them to idle state.
The host returned by CreateDefaultBuilder(...).Build() is meant to represent the whole application. From docs:
The main reason for including all of the app's interdependent resources in one object is lifetime management: control over app startup and graceful shutdown.
The default builder registers many services in singleton scope and when the host is stopped all of these services are disposed or switched to some "stopped" state. For example before calling StopAsync you can resolve IHostApplicationLifetime:
var appLifetime = host.Services.GetService<IHostApplicationLifetime>();
It has cancellation tokens representing application states. When you call StartAsync or RunAsync after stopping, all tokens still have IsCancellationRequested set to true. That's why the OperactionCancelledException is thrown in Host.StartAsync.
You can list other services during configuration:
For me it sounds like you just need some background jobs to process messages but I've never used NServiceBus so I don't know how it will work with something like Hangfire. You can also implement IHostedService and use it in the generic host builder.
I'm doing something like:
do
{
using IHost host = BuildHost();
await host.RunAsync();
} while (MainService.Restart);
with MainService constructor:
public MainService(IHostApplicationLifetime HostApplicationLifetime)
MainService.Restart is a static bool set by the MainService itself in response to some event which also calls HostApplicationLifetime.StopApplication().
I have been playing around with Rebus and RabbitMQ, and came across a scenario I cannot seem to get working.
I have a couple of queues; queue1 & queue2 and they take the same class/message type. Now, Rebus seems to prefer different message types per queue, this is not an option for me right now, so i use the advanced routing bus.Advanced.Routing.Send("queue1", Message)
I would like to utilise the bus.defer functionality but am unsure how to combine them both. I know I might need to introduce a waiting queue as an external timeout manager (which I have yet to get working too, but thats for another day)
Has anyone done anything similar?
How to send the message
As you have probably discovered, when you bus.Defer, Rebus will use the endpoint mappings to look up the destination queue from the type of the message being deferred (which is analogous to bus.Send/bus.SendLocal, in that it has an accompanying bus.DeferLocal too, which always sends to the sender's own input queue).
What is missing, is something analogous to bus.Advanced.Routing.Send, but fortunately it is pretty easy to emulate a combination of bus.Defer and an explicitly routed message but setting the rbs2-deferred-recipient header on a message:
var headers = new Dictionary<string, string> {
{Headers.DeferredRecipient, "destination-queue"}
};
var delay = TimeSpan.FromMinutes(5);
await bus.DeferLocal(delay, yourMessage, headers);
How to configure the timeout manager
You can use Rebus' internal timeout manager by configuring some kind of timeout persistence – e.g. by pulling in Rebus.SqlServer and using SQL Server to store timeouts like so:
Configure.With(...)
.(...)
.Timeouts(t => t.StoreInSqlServer(...))
.Start();
Another option is to install a Rebus endpoint as a dedicated timeout manager, which simply uses the same configuration as can be seen above, and then all other endpoints do this:
Configure.With(...)
.(...)
.Timeouts(t => t.UseExternalTimeoutManager("timeouts"))
.Start();
assuming that your timeout manager uses the timeouts queue.
Update relevant from Rebus 5
Rebus 5 (which is currently available as a prerelease package on Nuget.org) has builtin support for deferring messages to an explicitly specified destination queue.
It can be done like this:
var delay = TimeSpan.FromMinutes(2);
await bus.Advanced.Routing.Defer("dest-queue", delay, message);
which will simply carry out the steps mentioned above underneath the covers.
Is it possible to batch Rebus messages (using Azure Servicebus) ?
The reason is that we are going to send a lot of message to save log events and want to batch up.
While old versions of Rebus did have a batch API for wrapping up multiple logical messages inside one single transport message, that functionality turned out to bring very little advantage at the expense of increased complexity in many places.
If you want to send batches of messages, I suggest you simply code your own message batch message, something like
public class BatchOfLogEvents
{
public BatchOfLogEvents(IEnumerable<LogEvent> logEvents)
{
LogEvents = logEvents.ToArray();
}
public IReadOnlyCollection<LogEvent> LogEvents { get; }
}
and then you send that and create a handler for it in the other end.
Update regarding Azure Service Bus: Please remember that Azure Service Bus has a 256 kB maximum message size (or 1MB if you're on Premium).
Also: If you have not done so already, you can probably benefit from enabling GZip compression of messages by going
.Options(o => o.EnableCompression())
in your Rebus configurations.
Given that I have a web/SOAP service, how do I setup and teardown a proper transaction context for Rebus (the messaging bus)? When Rebus is calling into a message handler this is not a problem since Rebus will setup the transaction context before calling the handler - but what about the opposite where a web service handler needs to send/publish a message via Rebus?
I am not interested in how to implement an HTTP module or similar - only the basics around Rebus: what is needed to prepare Rebus for sending a message?
The web service code has its own transaction going on when talking to the application database. I need to be able to setup Rebus when setting up the database transaction and comit/rollback Rebus when doing the same with the database.
I have a similar problem with standalone command line programs that needs to both interaction with a database and sending Rebus messages.
Rebus will automatically enlist send and publish operations in its own "ambient transaction context", which is accessed via the static(*) AmbientTransactionContext.Current property.
You could implement ITransactionContext yourself if you wanted to, but Rebus comes with DefaultTransactionContext in the box.
You use it like this:
using(var context = new DefaultTransactionContext())
{
AmbientTransactionContext.Current = context;
// send and publish things in here
// complete the transaction
await context.Complete();
}
which could easily be put e.g. in an OWIN middleware or something similar.
(*) The property is static, but the underlying value is bound to the current execution context (by using CallContext.LogicalGet/SetData), which means that you can think of it as thread-bound, with the nice property that it flows as expected to continuations.
In Rebus 2.0.2 it is possible to customize the accessors used to get/set the context by calling AmbientTransactionContext.SetAccessors(...) with an Action<ITransactionContext> and a Func<ITransactionContext>, e.g. like this:
AmbientTransactionContext.SetAccessors(
context => {
if (HttpContext.Current == null) {
throw new InvalidOperationException("Can't set the transaction context when there is no HTTP context");
}
HttpContext.Current.Items["current-rbs-context"] = context
},
() => HttpContext.Current?.Items["current-rbs-context"] as ITransactionContext
);
which in this case makes it work in a way that flows properly even when using old school HTTP modules ;)
I am trying to implement delayed queue with overriding of messages using Active MQ.
Each message is scheduled to be delivered with delay of x (say 60 seconds)
In between if same message is received again it should override previous message.
So even if I receive 10 messages say in x seconds. Only one message should be processed.
Is there clean way to accomplish this?
The question has two parts that need to be addressed separately:
Can a message be delayed in ActiveMQ?
Yes - see Delay and Schedule Message Delivery. You need to set <broker ... schedulerSupport="true"> in your ActiveMQ config, as well as setting the AMQ_SCHEDULED_DELAY property of the JMS message saying how long you want the message to be delayed (10000 in your case).
Is there any way to prevent the same message being consumed more than once?
Yes, but that's an application concern rather than an ActiveMQ one. It's often referred to as de-duplication or idempotent consumption. The simplest way if you only have one consumer is to keep track of messages received in a map, and check that map whether you receive a message. It it has been seen, discard.
For more complex use cases where you have multiple consumers on different machines, or you want that state to survive application restart, you will need to keep a table of messages seen in a database, and query it each time.
Please vote this answer up if it helps, as it encourages people to help you out.
Also according to method from ActiveMQ BrokerService class you should configure persistence to have ability to use scheduler functionality.
public boolean isSchedulerSupport() {
return this.schedulerSupport && (isPersistent() || jobSchedulerStore != null);
}
you can configure activemq broker to enable "schedulerSupport" with the following entry in your activemq.xml file located in conf directory of your activemq home directory.
<broker xmlns="http://activemq.apache.org/schema/core" brokerName="localhost" dataDirectory="${activemq.data}" schedulerSupport="true">
You can Override the BrokerService in your configuration
#Configuration
#EnableJms
public class JMSConfiguration {
#Bean
public BrokerService brokerService() throws Exception {
BrokerService brokerService = new BrokerService();
brokerService.setSchedulerSupport(true);
return brokerService;
}
}