In RabbitMQ official website, they suggest to create a single callback queue per client to implement RPC pattern.
The code snippet for client(producer) looks like:
public RPCClient() throws Exception {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("localhost");
connection = factory.newConnection();
channel = connection.createChannel();
replyQueueName = channel.queueDeclare().getQueue();
consumer = new QueueingConsumer(channel);
channel.basicConsume(replyQueueName, true, consumer);
But I can't find any article to implement this by Spring Boot - AMQP, all the examples just send and receive sequentially (or say synchronously).
I wonder know how to achieve asynchronous message communication with just one request queue and reply queue.
p.s. I also tried AsyncRabbitTemplete but it didn't work.
Related
In my setup I have two RabbitMQ servers that are used by different applications employing Rebus ESB. What I would like to know is if I can map a message to a queue on a different Host the way I can with MassTransit.
I also would like to know if I can send messages in a batch mode the same way with MassTransit.
Thanks In Advance.
In my setup I have two RabbitMQ servers that are used by different applications employing Rebus ESB. What I would like to know is if I can map a message to a queue on a different Host the way I can with MassTransit.
I am not sure how this works with MassTransit, but I'm pretty sure it's not readily possible with Rebus.
With Rebus, you're encouraged to treat this as you would any other integration scenario, where you'd put a ICanSendToOtherSystem in your IoC container, which just happens to be implemented by CanSendToOtherSystemUsingRebus. Your CanSendToOtherSystemUsingRebus class would probably look somewhat like this:
public class CanSendToOtherSystemUsingRebus : ICanSendToOtherSystem, IDisposable
{
readonly IBus _bus;
public CanSendToOtherSystemUsingRebus(string connectionString)
{
_bus = Configure.With(new BuiltinHandlerActivator())
.Transport(t => t.UseRabbitMqAsOneWayClient(connectionString))
.Start();
}
public Task Send(object message) => _bus.Send(message);
public void Dispose() => _bus.Dispose();
}
(i.e. just something that wraps a one-way client that can connect to that other RabbitMQ host, registered as a SINGLETON in the container)
I also would like to know if I can send messages in a batch mode the same way with MassTransit. Thanks In Advance.
Don't know how this works with MassTransit, but with Rebus, you can give the transport more convenient circumstances for optimizing the send operation(s) by using scopes:
using var scope = new RebusTransactionScope();
foreach (var message in lotsOfMessages)
{
// scope is automagically detected
await bus.Send(message);
}
await scope.CompleteAsync();
which will improve the rate with which you can send/publish with most transports. Just remember that the scope results in queuing up messages in memory before actually sending them, so you'll probably not want to send millions of messages in each batch.
I hope that answered your questions 🙂
I am working on a microservice based application in azure. My requirement is I had a service bus and I need to consume that service bus message in web api. Currently I implemented through azure functions, but my company asked to use api. Is it possible?, If possible please show me how to do it
You can create Background service to listen to message from service bus queue.
Below are few key points that needs to be noted:
Background task that runs on a timer.
Hosted service that activates a scoped service. The scoped service can use dependency injection (DI).
Queued background tasks that run sequentially.
App Settings:
1. {
2. "AppSettings": {
3. "QueueConnectionString": "<replace your RootManageSharedAccessKey here>",
4. "QueueName": "order-queue"
5. }
6. }
You can refer to c-sharpcorner blog for step by step process.
There is a simple way to get a simple message.
ServiceBusClient client = new ServiceBusClient("Endpoint=sb://yourservicesbusnamespace.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=Your_SharedAccess");
var receiver = client.CreateReceiver("Your Queue");
var message = await receiver.ReceiveMessagesAsync(1);
string ascii = Encoding.ASCII.GetString(message.FirstOrDefault().Body);
Console.WriteLine("Received Single Message: " + ascii);
await receiver.CompleteMessageAsync(message.FirstOrDefault());
I did some modifications from this post
https://ciaranodonnell.dev/posts/receiving-from-azure-servicebus/
I have a spring-boot Kafka project which is a web-service exposing API to get Kafka message in response.
What i want is whenever i call the rest end point the Kafka should start searching from beginning it does it as i used earliest in auto-reset config but i have to start server again and again to make it listen to Kafka from starting.
#KafkaListener(topics = {"topic"})
public void storeMessagesMessages(ConsumerRecord record) {
if (record.value().toString().contains(uuid) {
this.messageToBeReturnedByApi = record.value()
}
}
Or i can say i want this listener part to be invoked only when i call web service endpoint
Your listener should extend AbstractConsumerSeekAware; you can then perform arbitrary seek operations. See https://docs.spring.io/spring-kafka/docs/2.6.2/reference/html/#seek
I am thinking of a way to manage failed messages in Rebus.
In my second level retry strategy I want to save the message and exception details into the database so that I can later review the error details and decide whether to resend the message to the be reprocessed or ignore and delete.
In the handler I am capturing details as follows:
public async Task Handle(IFailed<StudentCreated> failedMessage)
{
//Logic to Defer Message with rebus_defer_count not shown
DictionarySerializer dictionarySerializer = new
DictionarySerializer();
ObjectSerializer objectSerializer = new ObjectSerializer();
string headers =
dictionarySerializer.SerializeToString(failedMessage.Headers);
string message =
objectSerializer.SerializeToString(failedMessage.Message);
Exception lastException= failedMessage.Exceptions.Last();
string exception = objectSerializer.SerializeToString(lastException);
//Logic to save the message and error details in the database not shown
}
This will enable me to save the message and error details into the database where I can create a dashboard to view the messages and resolve them as I wish rather than in the broker queue such as RabbitMQ.
Now my question is how can I return them to the handler where the error was raised using the information provided in the headers?
What is the best way to do it with REBUS provided I have all the details from the Failed Message as shown in my code snippet?
Regards
What you're trying to achieve will be much easier if you make a small change to your application. You see, Rebus already has a built-in service in place for handling failed messages called IErrorHandler.
You can register your own error handler like this:
Configure.With(...)
.(...)
.Options(o => o.Register<IErrorHandler>(c => new MyCustomErrorHandler()))
.Start();
thus replacing the default error handler (which btw. is PoisonQueueErrorHandler)
The error handler gets to handle the message in the form of the raw TransportMessage (i.e. simply headers and a byte[]) when all retries have failed, so this is the perfect place to save the message to your database.
If you then look here, you can see how Rebus' default error handler adds its own queue name as the rbs2-source-queue header, meaning that the message can later be sent back to that queue.
With this information, it should be fairly easy to write some code that inspects the message for its source queue and sends a RabbitMQ message to that queue.
This will only work if the re-delivery service has access to the RabbitMQ instance where all of your Rebus endpoints are running, of course. It's less straightforward, if you want to implement this in a general way: E.g. if you were using Fleet Manager, each Rebus instance would use a long-polling protocol to query the server for commands, which enables Fleet Manager to tell any Rebus instance to e.g. send a previously failed message to any queue it has access to.
I have been reading This Book on page 58 to understand how to do asynchronous event integration between microservices.
Using RabbitMQ and publish/subscribe patterns facilitates pushing events out to subscribers. However, given microservice architectures and docker usage I expect to have more than once instance of a microservice 'type' running. From what I understand all instances will subscribe to the event and therefore would all receive it.
The book doesn't clearly explain how to ensure only one of the instances handle the request.
I have looked into the duplication section, but that describes a pattern that explains how to deduplicate within a service instance but not necessarily against them...
Each microservice instance would subscribe using something similar to:
public void Subscribe<T, TH>()
where T : IntegrationEvent
where TH : IIntegrationEventHandler<T>
{
var eventName = _subsManager.GetEventKey<T>();
var containsKey = _subsManager.HasSubscriptionsForEvent(eventName);
if (!containsKey)
{
if (!_persistentConnection.IsConnected)
{
_persistentConnection.TryConnect();
}
using (var channel = _persistentConnection.CreateModel())
{
channel.QueueBind(queue: _queueName,
exchange: BROKER_NAME,
routingKey: eventName);
}
}
_subsManager.AddSubscription<T, TH>();
}
I need to understand how a multiple microservice instances of the same 'type' of microservice can deduplicate without loosing the message if the service goes down while processing.
From what I understand all instances will subscribe to the event and
therefore would all receive it.
Only one instance of subscriber will process the message/event. When you have multiple instances of a service running and subscribed to same subscription the first one to pick the message will set the message invisible from the subscription (called visibility timeout). If the service instance is able to process the message in given time it will tell the queue to delete the message and if it's not able to process the message in time , the message will re-appear in queue for any instance to pick it up again.
All standard service bus (rabbitMQ, SQS, Azure Serivce bus etc) provide this feature out of box.
By the way i have read this book and used the above code from eShotContainers and it works the way i described.
You should look into following pattern as well
Competing Consumers pattern
Hope that helps!