Pub sub pattern created Queues and Topics on Azure Service Bus with Rebus - rebus

The app below is tweaked to work with Azure Service Bus.
https://github.com/rebus-org/RebusSamples/tree/master/PubSub
However, below are created.
Queues
error
publisher
subscriber
Topics
messages_datetimemessage__messages: subscriber
messages_stringmessage__messages: subscriber
messages_timespanmessage__messages: subscriber
My question
is that is this correct?
And is it possible to reduce the number of artifacts that are created? For example, reduce to only one topic, because topic is used for pub sub.
Update
I need to use Pub sub pattern with one Topic and one or two subscriptions if possible.
However, I have got an error below:
System.AggregateException HResult=0x80131500 Message=One or more
errors occurred. (Could not publish to topic 'order')
Source=System.Private.CoreLib StackTrace: at
System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout,
CancellationToken cancellationToken) at
System.Threading.Tasks.Task.Wait() at Publisher.Program.Main() in
C:_MyLab\ReBus\PubSub\Publisher\Program.cs:line 43
Inner Exception 1: RebusApplicationException: Could not publish to
topic 'order'
Inner Exception 2: InvalidOperationException: Cannot open a Topic
client for entity type Queue.
TrackingId:5c380af2-ad8f-4788-85b8-5427dd7873e4_B4,
SystemTracker:myapp:Queue:order, Timestamp:2019-04-29T22:31:57
TrackingId:9c3e0c40-4410-4102-a705-86a6528cd030_B4,
SystemTracker:myapp:Queue:order, Timestamp:2019-04-29T22:31:57
TrackingId:401a15d284ad44989f5e451c963d81e5_G16,
SystemTracker:gateway7, Timestamp:2019-04-29T22:31:57
UseAzureServiceBus seems wrong, because it is using queue
class Publisher
{
static void Main()
{
using (var activator = new BuiltinHandlerActivator())
{
Configure.With(activator)
.Logging(l => l.ColoredConsole(minLevel: LogLevel.Warn))
.Transport(t => t.UseAzureServiceBus(Consts.ServiceBusConnectionString, Consts.Order))
.Start();
activator.Bus.Advanced.Topics.Publish(Consts.Order, new StringMessage("Hello there, I'm a publisher!")).Wait();
}
}
UseAzureServiceBus seems wrong, because it is using queue. Is the Handler able to handle message?
class Subscriber
{
static void Main()
{
using (var activator = new BuiltinHandlerActivator())
{
activator.Register(() => new Handler());
Configure.With(activator)
.Logging(l => l.ColoredConsole(minLevel: LogLevel.Warn))
.Transport(t => t.UseAzureServiceBus(Consts.ServiceBusConnectionString, Consts.Order))
.Routing(r => r.TypeBased().MapAssemblyOf<StringMessage>(Consts.Order))
.Start();
activator.Bus.Advanced.Topics.Subscribe(Consts.Order);
Console.WriteLine("This is Subscriber 1");
Console.WriteLine("Press ENTER to quit");
Console.ReadLine();
Console.WriteLine("Quitting...");
}
}
}
class Handler : IHandleMessages<StringMessage>, IHandleMessages<DateTimeMessage>, IHandleMessages<TimeSpanMessage>
{
public async Task Handle(StringMessage message)
{
Console.WriteLine("Got string: {0}", message.Text);
}
public async Task Handle(DateTimeMessage message)
{
Console.WriteLine("Got DateTime: {0}", message.DateTime);
}
public async Task Handle(TimeSpanMessage message)
{
Console.WriteLine("Got TimeSpan: {0}", message.TimeSpan);
}
}
The code above create an Order queue, which is not what I want.
I want a topic, and one or two subscriptions.

Rebus pretty much encourages you to use .NET types as topics, which in this case means that the topics
messages_datetimemessage__messages
messages_stringmessage__messages
messages_timespanmessage__messages
get created, because you publish events of types DateTimeMessage, StringMessage, and TimeSpanMessage (all residing in the Messages namespace in the Messages assembly).
If that is not what you want, Rebus allows you to publish to custom topics like so:
await bus.Advanced.Publish("my-topic", new DateTimeMessage(...))`;
which in this case will result in the creation of a single topic: my-topic.
The subscriber(s) will need to subscribe accordingly:
await bus.Advanced.Subscribe("my-topic");
Keep in mind that there's no filtering of the types of events published to a topic, so the subscriber above will receive anything published to the my-topic topic, regardless of whether it is capable of handling it.

Related

MassTransit exchange creation

I'm using MassTransit with RabbitMQ on a .net core 6 web application. My goal is to keep in sync several instances of an application, running on different plants. The application needs to be able to publish / consume messages.
When a site publishes something, this is broadcasted to all the sites queues (also itself, it will simply discard the message).
In order to do it, I configured MassTransit queue names with the suffix of the plant: eg norm-queue-CV, norm-queue-MB. I configured also the Consumer to bind to a generic fanout exchange name (norm-exchange).
Here my configuration extract:
public void ConfigureServices(IServiceCollection services)
{
services.AddMassTransit(x =>
{
x.AddBus(provider => Bus.Factory.CreateUsingRabbitMq(cfg =>
{
cfg.Host(new Uri(_configuration["RabbitMQ:URI"] + _configuration["RabbitMQ:VirtualHost"]), $"ENG {_configuration["Application:PlantID"]} Producer", h =>
{
h.Username(_configuration["RabbitMQ:UserName"]);
h.Password(_configuration["RabbitMQ:Password"]);
});
cfg.Publish<NormCreated>(x =>
{
x.Durable = true;
x.AutoDelete = false;
x.ExchangeType = "fanout"; // default, allows any valid exchange type
});
}));
});
// consumer
var busControl = Bus.Factory.CreateUsingRabbitMq(cfg =>
{
cfg.Host(new Uri(_configuration["RabbitMQ:URI"] + _configuration["RabbitMQ:VirtualHost"]), $"ENG {_configuration["Application:PlantID"]} Consumer", h =>
{
h.Username(_configuration["RabbitMQ:UserName"]);
h.Password(_configuration["RabbitMQ:Password"]);
});
cfg.ReceiveEndpoint($"norm-queue-{_configuration["Application:PlantID"]}", e =>
{
e.Consumer<NormConsumer>();
e.UseConcurrencyLimit(1);
e.UseMessageRetry(r => r.Intervals(100, 200, 500, 800, 1000));
e.Bind<NormCreated>();
e.Bind("norm-exchange");
});
});
busControl.Start();
And here how NormConsumer is defined
public class NormConsumer : IConsumer<NormCreated>
{
private readonly ILogger<NormConsumer>? logger;
public NormConsumer()
{
}
public NormConsumer(ILogger<NormConsumer> logger)
{
this.logger = logger;
}
public async Task Consume(ConsumeContext<NormCreated> context)
{
logger.LogInformation("Norm Submitted: {NormID}", context.Message.NormID);
//await context.Publish<NormCreated>(new
//{
// context.Message.OrderId
//});
}
}
Here the queues automatically created. To me they look fine
And here the exchange created. I was trying to get only one exchange (norm-exchange), but also the other 2 are created.
My problem is first of all understand if my layout makes sense (I'm quite new to Rabbit/Masstransit).
Moreover I'd like to override how exchanges are named, forcing to have for this queues only one exchange: "norm-exchange". I tried to override it in "producer" part, but not able to do it
RabbitMQ broker topology is covered extensively in RabbitMQ - The Details, and also in the documentation.
You do not need to call Bind in the receive endpoint, consumer message types are already bound for you. Remove both Bind statements, and any published messages will be routed by type to the receive endpoints.

.NET Framework SignalR Clients.Caller Not working as expected

I am currently using SignalR in my .NET framework project to send updates to the client for a long running process. There can be many processes running simultaneously and the client will subscribe to any one of the process using an unique ID. I am using Groups to identify the clients who are subscribed to a particular process. If a client subscribes to a process in middle, I must send all the previous messages to that client. The code goes something like this
public class ProgressHub : Hub
{
public async Task SubscribeToProgress(string id)
{
foreach (var message in GetPreviousMessages(id)) // Getting all the previous messages
{
await Clients.Caller.SendMessage(message); // Sending Messages to the current caller alone
}
await Groups.Add(Context.ConnectionId, id); // Added the current client to a group to be used further
}
}
The client listens to Send Message
The above code snippet is not working (No messages in the network tab).
I tried many things
await Clients.Client(Context.ConnectionId).SendMessage(message);
await Clients.All.SendMessage(message); // Just to check if it works
all the above without await, but nothing seems to work.
After fiddling around a bit, I was able to come up with this
public class ProgressHub : Hub
{
public async Task SubscribeToProgress(string id)
{
await Groups.Add(Context.ConnectionId, id); // Adding client to the group first
foreach (var message in GetPreviousMessages(id))
{
await Clients.Group(id).SendMessage(message); // Sending messages to the group all together
}
}
}
But this has an undesirable side effect of sending the older messages to client who are already connected. Sure, I can exclude the other connectionIDs and send out the message, but this seems like an hack. Logically speaking, the first snippet should have worked just fine.
are you add configuration in Program.cs ?
using SignalRChat.Hubs;
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddRazorPages();
builder.Services.AddSignalR();
var app = builder.Build();
if (!app.Environment.IsDevelopment())
{
app.UseExceptionHandler("/Error");
app.UseHsts();
}
app.UseHttpsRedirection();
app.UseStaticFiles();
app.UseRouting();
app.UseAuthorization();
app.MapRazorPages();
app.MapHub<ChatHub>("/chatHub");
app.Run();
and you can read this reference :
Microsoft

SignalR client does not receive message through Masstransit

I had a problem sending messages to clients via MassTransit and SignalR
Startup:
//SignalR
services.AddSignalR().AddMassTransitBackplane();
#region MassTransit RabbitMq
services.AddScoped<SendCosistListToScaleConsumer>();
services.AddScoped<CreateConsistListConsumer>();
services.AddMassTransit(x =>
{
x.AddSignalRHubConsumers<NotifyHub>();
x.AddBus(provider => Bus.Factory.CreateUsingRabbitMq(conf =>
{
conf.Host(Configuration["Rabbit:Host"], host => {
host.Username(Configuration["Rabbit:Username"]);
host.Password(Configuration["Rabbit:Password"]);
});
conf.ReceiveEndpoint(Configuration["Rabbit:ReceiveEndpoint"], e => {
e.PrefetchCount = 16;
e.UseMessageRetry(n => n.Interval(3, 100));
#region Consumers
e.Consumer<SendCosistListToScaleConsumer>();
e.Consumer<CreateConsistListConsumer>();
#endregion
});
conf.AddSignalRHubEndpoints<NotifyHub>(provider);
}));
});
services.AddMassTransitHostedService();
#endregion
....
app.UseSignalR(endpoints =>
{
endpoints.MapHub<NotifyHub>("/notify");
});
Consumer:
public class CreateConsistListConsumer : IConsumer<ICreateConsistList>
{
IReadOnlyList<IHubProtocol> protocols = new IHubProtocol[] { new JsonHubProtocol() };
public Task Consume(ConsumeContext<ICreateConsistList> context)
{
context.Publish<All<NotifyHub>>(
new
{
Message = protocols.ToProtocolDictionary("SendMessageToAllUsers", new object[] { "CompanyId", context.Message.CompanyId })
});
return Task.CompletedTask;
}
}
Console App (SignalR Client):
hubConnection.On<Object>("SendMessageToAllUsers", param => {
Console.WriteLine(param);
});
If I understand correctly how MassTransii and SignalR work, then this code is enough to send messages to clients.
With the help of debugging, I looked that CreateConsistListConsumer is working, but clients do not receive reporting.
  At the same time, the client connects to the hub and correctly receives messages from other sources, but not from MassTransit.
What am I doing wrong?
I have been facing the same issue last week.
It seem SignalR is doing some special work with handling hubs, and couldn't make Masstransit SignalR service to work.
I ended up using a static hub reference as described here.
Basically, I am just calling Core DI to get my hub context, then store it into a static property (as in the sample in the Github issue listed above).
When needed, I call the reference from within my MassTransit Consumer, and I am done.

Handle Publish/Subscribe in Rebus with Azure Service Bus

I am looking into Rebus and to use it with Azure Service Bus. Using it with regalure Queues was easy, but when I want to use Topic instead I can't get it to work.
Is there any here that have a done a setup and use it with Topic/Subscription. This is what I have so far.
static void Main(string[] args)
{
_bus1 = InitializeBus(System.Environment.MachineName);
_bus2 = InitializeBus(System.Environment.MachineName + "_2");
_bus3 = InitializeBus();
Run();
Console.WriteLine("Press Enter to exit!");
Console.ReadLine();
}
private static void Run()
{
try
{
_bus1.Handle<string>((b, c, m) => { Console.WriteLine(m); return null; });
_bus2.Handle<string>((b, c, m) => { Console.WriteLine(m); return null; });
_bus1.Bus.Subscribe<string>();
_bus2.Bus.Subscribe<string>();
_bus3.Bus.Publish("Publish test message");
}
catch (Exception ex)
{
throw;
}
}
private static BuiltinHandlerActivator InitializeBus(string queueName = null)
{
var activator = new BuiltinHandlerActivator();
if(string.IsNullOrEmpty(queueName))
Configure.With(activator)
.Transport(t => t.UseAzureServiceBusAsOneWayClient(connectionString))
.Options(o => { o.SetNumberOfWorkers(10); o.SetMaxParallelism(10); })
.Start();
else
Configure.With(activator)
.Transport(t => t.UseAzureServiceBus(connectionString, queueName).EnablePartitioning().DoNotCreateQueues())
.Options(o => { o.SetNumberOfWorkers(10); o.SetMaxParallelism(10); })
.Start();
return activator;
}
First I create all the buses. I am using DontCreateQueues() since I don't want the queues to be duplicated created in my root but only under the Topic as Subscription.
Then I set up the buses and the Publish works fine, there is one Topic created and 2 subscriptions created under this Topic, and there is 1 message in each of this subscriptions. But the messages is never collected.
If I remove the DontCreateQueues() method in the Configuration the code work, but then 2 queues are created in the root togheter with the topic and it's 2 subscriptions, but I can't have it like that.
Best Regards
Magnus
Rebus uses topics by creating a subscription for each topic you subscribe to, and then configure the subscription to forward received messages to the input queue of the bus.
If the bus does not have an input queue with the expected name (either one created by Rebus, or one you created manually), things will not work.
The reason DontCreateQueues() exists is to allow for expert users to configure their queue settings beyond what Rebus is capable of (and willing) to do. It requires a pretty detailed knowledge about how Rebus expects your queue entities to be laid out though, so I would recommend almost anyone to NOT create anything manually, and then simply let Rebus set things up.

Rebus / Content based routing

As you see there are a Headquarter as root node and some branches as child nodes. There is a message of Data type, and I want to publish a message based on content of Data object, for example :
if (data.value == xxxx) publish(data, Br1, Br2)
else if (data.value == yyyy) publish(data, Br3, Br4)
else if (data.value == zzzz) publis(data, Br5, Br6)
This is somehow customized version of pub/sub pattern. But I want publish message of type Data to just some special subscribers based on content of message.
Is there a solution in Rebus?
There's several solutions in Rebus :)
For your scenario, I can see two ways of solving it: 1) Use custom topics, or 2) Implement a real content-based router.
If it makes sense, you can model this pub/sub scenario using topics by using Rebus' topics API to take care of the routing. This makes sense if you can say that your data messages each belong to some category, which your subscribers can then subscribe to.
Compared to "real" topics-based queueing systems, like e.g. RabbitMQ, the topics API in Rebus is very crude. It doesn't allow for wildcards(*) or anything advanced like that – topics are just simple strings that you can subscribe to and then use as a pub/sub channel to have an event routed to multiple subscribers.
You can use it like this in the subscriber's end:
await bus.Advanced.Topics.Subscribe("department_a");
and then in the publisher's end:
var data = new Data(...);
await bus.Advanced.Topics.Publish("department_a", data);
If that doesn't cut it, you can insert a "real" content-based router which is simply an endpoint to which you await bus.Send(eachDataMessage), which in turn forwards the message to the relevant subscribers.
It can be done at two levels with Rebus, depending on your requirements. If it is enough to look at the message's headers, you should implement it as a "transport message forwarder", because that skips deserialization and provides a nice API for simply forwarding messages:
Configure.With(...)
.Transport(t => t.UseMsmq("router"))
.Routing(r => {
r.AddTransportMessageForwarder(async transportMessage => {
var headers = transportMessage.Headers;
var subscribers = Decide(headers);
return ForwardAction.ForwardTo(subscribers);
});
})
.Start();
If you need to look at the actual message, you should just implement an ordinary message handler and then use the bus to forward the message:
public class Router : IHandleMessages<Data>
{
readonly IBus _bus;
public Router(IBus bus)
{
_bus = bus;
}
public async Task Handle(Data message)
{
var subscribers = Decide(message);
foreach(var subscriber in subscribers)
{
await _bus.Advanced.TransportMessage.ForwardTo(subscriber);
}
}
}
The custom-implemented router is the most flexible solution, as you can implement any logic you like, but as you can see it is slightly more involved.
(*) Rebus doesn't allow for using wildcards in general, although it does pass the topics directly to RabbitMQ if you happen to be using that as the transport, which means that you can actually take full advantage of RabbitMQ (see this issue for some more info about that)
static void Main()
{
using (var activator = new BuiltinHandlerActivator())
{
activator.Handle<Packet>(async (bus, packet) =>
{
string subscriber = "subscriberA";
await bus.Advanced.TransportMessage.Forward(subscriber);
});
Configure.With(activator)
.Logging(l => l.ColoredConsole(minLevel: LogLevel.Warn))
.Transport(t => t.UseMsmq("router"))
.Start();
for (int i = 0; i < 10; i++)
{
activator.Bus.SendLocal(
new Packet()
{
ID = i,
Content = "content" + i.ToString(),
Sent = false,
}).Wait();
}
}
Console.ReadLine();
}
using (var trScope = new TransactionScope(TransactionScopeAsyncFlowOption.Enabled))
{
scope.EnlistRebus();
Packet packet = ReadFromDB()
activator.Bus.SendLocal(packet).Wait()
scope.Complete()
}
activator.Handle<Packet>(async (bus, packet) =>
{
string subscriber = "subscriberA";
await bus.Advanced.TransportMessage.Forward(subscriber);
});
using (var activator = new BuiltinHandlerActivator())
{
activator.Handle<Packet>(async message =>
{
string connectionString =
"Data Source=.;Initial Catalog=Rebus;User ID=sa;Password=123456";
using (SqlConnection connection = new SqlConnection(connectionString))
{
string queryString = #"INSERT INTO CLIENTPACKET(ID, CONTENT, SENT) VALUES(#id, #content, #sent)";
connection.Open();
using (SqlCommand command = new SqlCommand(queryString, connection))
{
command.Parameters.Add(new SqlParameter("#id", message.ID));
command.Parameters.Add(new SqlParameter("#content", message.Content));
command.Parameters.Add(new SqlParameter("#sent", message.Sent));
await command.ExecuteNonQueryAsync();
}
}
});
Configure.With(activator)
.Logging(l => l.ColoredConsole(minLevel: LogLevel.Warn))
.Transport(t => t.UseMsmq(#"subscriberA"))
.Routing(r => r.TypeBased().MapAssemblyOf<Packet>("router"))
.Options(o =>
{
TransactionOptions tranOp = new TransactionOptions();
tranOp.IsolationLevel = IsolationLevel.ReadCommitted;
o.HandleMessagesInsideTransactionScope(tranOp);
o.SetNumberOfWorkers(2);
o.SetMaxParallelism(2);
})
.Start();
activator.Bus.Subscribe<Packet>().Wait();
Console.WriteLine("Press ENTER to quit");
Console.ReadLine();
}

Resources