I have a signal r application. I have an issue of concurrent connection maxing out on iis. To avoid that i want my hub to send notification to groups created by in pack of 25 with delay of suppose 500ms. I add users to group as they connect. After count of 25 i add them to next group.Need help with how shall i send notification to each group with delay of 500ms.
I have looped through the number of groups and used
Thread.Sleep(500);
but all connected users do not receive the notification
Your question almost describes the solution - buffer the notifications fro 500ms then send an array of all of them.
You could do that by pushing the notification in a queue or list and and use a timer to send the notifications every 500ms.
A far better solution though would be to use Reactive Extensions and the Observable.Buffer(Timespan) method, thus avoiding queues and timers.
Subject<string> _subject;
.....
public void InitSubject()
{
_subject=new Subject<string>();
_subject.Buffer(Timespan.FromMilliseconds(500))
.Subscribe(notifications=>SendBatch(notifications));
}
....
public void SendBatch(IList<string> notifications)
{
//Get hold of the SignalR context and send the notifications here
}
public void QueueNotification(string notification)
{
_subject.OnNext(notification);
}
Buffer also allows you to batch a specific number of notifications, or use both a timespan and item limit to send a batch whenever one of the limits is reached.
Your code should call QueueNotification instead of sending the notification directly. Reactive Extensions also provide LINQ extensions, so you could apply filtering with the Where() method, eg to use different delays for different user groups, bypass buffering completely, or use any other combination.
You should note that since a Hub is transient, you can't put your subject inside the hub, although you can put QueueNotification in there, provided you provide a reference to the subject somewhere, eg by using Dependency injection to initialize the hub
Related
Do event listeners guarantee that all data ever written to a path will be delivered to the client eventually?
For instance if I have a game client that pushes moves to the same path one after the other will the listening client receive all updates?
What would happen in this situation: client A pushes move 1 to game1/user1/move_data which client B is listening on; client A then immediately pushes another move updating the value at game1/user1/move_data.
Will the listening client be guaranteed to receive all moves pushed?
Currently I have a system that creates a new path per move and then I am calling single listeners on each move as each client reaches that move in it's state. It doesn't seem efficient as if the client A receives the most recent move that client B has made then client A begins listening on a path that doesn't exist yet.
The below quotes are from this link: https://firebase.google.com/docs/database/admin/retrieve-data
"The value event is used to read a static snapshot of the contents at a given database path, as they existed at the time of the read event. It is triggered once with the initial data and again every time the data changes. The event callback is passed a snapshot containing all data at that location, including child data. In the code example above, value returned all of the blog posts in your app. Everytime a new blog post is added, the callback function will return all of the posts."
The part about as they existed at the time of the read event causes me to think that if a listener is on a path then the client will receive all values ever on that path eventually.
There is also this line from the guarantees section which I am struggling to decipher:
"Value events are always triggered last and are guaranteed to contain updates from any other events which occurred before that snapshot was taken."
I am working with a language that does not have a Google based sdk and am asking this question, so I can further assess Firebases' suitability for my uses.
Firebase Realtime Database performs state synchronization. If a client is listening to data in a location, it will receive the state of that data. If there are changes in the data, it will receive the latest state of that data.
...if I have a game client that pushes moves to the same path one after the other will the listening client receive all updates?
If there are multiple updates before the Firebase server has a chance to send the state to a listener, it may skip some intermediate values. So there is no guarantee that your client will see every state change, there is just a guarantee that it will eventually see the latest state.
If you want to ensure that all clients (can) see all state changes, you should store the state changes themselves in the database.
try to this code to get update value from firebase database:-
mFirebaseInstance = FirebaseDatabase.getInstance();
mFirebaseDatabase = mFirebaseInstance.getReference();
mFirebaseDatabase.child("new_title").setValue("Realtime Database");
mFirebaseDatabase.child("new_title").addValueEventListener(new ValueEventListener() {
#Override
public void onDataChange(DataSnapshot dataSnapshot) {
String appTitle = dataSnapshot.getValue().toString();
Log.e("Hey", appTitle);
title.setText(appTitle);
}
#Override
public void onCancelled(DatabaseError error) {
// Failed to read value
Log.e("Hey", "Failed to read app title value.", error.toException());
}
});
I have been reading This Book on page 58 to understand how to do asynchronous event integration between microservices.
Using RabbitMQ and publish/subscribe patterns facilitates pushing events out to subscribers. However, given microservice architectures and docker usage I expect to have more than once instance of a microservice 'type' running. From what I understand all instances will subscribe to the event and therefore would all receive it.
The book doesn't clearly explain how to ensure only one of the instances handle the request.
I have looked into the duplication section, but that describes a pattern that explains how to deduplicate within a service instance but not necessarily against them...
Each microservice instance would subscribe using something similar to:
public void Subscribe<T, TH>()
where T : IntegrationEvent
where TH : IIntegrationEventHandler<T>
{
var eventName = _subsManager.GetEventKey<T>();
var containsKey = _subsManager.HasSubscriptionsForEvent(eventName);
if (!containsKey)
{
if (!_persistentConnection.IsConnected)
{
_persistentConnection.TryConnect();
}
using (var channel = _persistentConnection.CreateModel())
{
channel.QueueBind(queue: _queueName,
exchange: BROKER_NAME,
routingKey: eventName);
}
}
_subsManager.AddSubscription<T, TH>();
}
I need to understand how a multiple microservice instances of the same 'type' of microservice can deduplicate without loosing the message if the service goes down while processing.
From what I understand all instances will subscribe to the event and
therefore would all receive it.
Only one instance of subscriber will process the message/event. When you have multiple instances of a service running and subscribed to same subscription the first one to pick the message will set the message invisible from the subscription (called visibility timeout). If the service instance is able to process the message in given time it will tell the queue to delete the message and if it's not able to process the message in time , the message will re-appear in queue for any instance to pick it up again.
All standard service bus (rabbitMQ, SQS, Azure Serivce bus etc) provide this feature out of box.
By the way i have read this book and used the above code from eShotContainers and it works the way i described.
You should look into following pattern as well
Competing Consumers pattern
Hope that helps!
Do event listeners guarantee that all data ever written to a path will be delivered to the client eventually?
For instance if I have a game client that pushes moves to the same path one after the other will the listening client receive all updates?
What would happen in this situation: client A pushes move 1 to game1/user1/move_data which client B is listening on; client A then immediately pushes another move updating the value at game1/user1/move_data.
Will the listening client be guaranteed to receive all moves pushed?
Currently I have a system that creates a new path per move and then I am calling single listeners on each move as each client reaches that move in it's state. It doesn't seem efficient as if the client A receives the most recent move that client B has made then client A begins listening on a path that doesn't exist yet.
The below quotes are from this link: https://firebase.google.com/docs/database/admin/retrieve-data
"The value event is used to read a static snapshot of the contents at a given database path, as they existed at the time of the read event. It is triggered once with the initial data and again every time the data changes. The event callback is passed a snapshot containing all data at that location, including child data. In the code example above, value returned all of the blog posts in your app. Everytime a new blog post is added, the callback function will return all of the posts."
The part about as they existed at the time of the read event causes me to think that if a listener is on a path then the client will receive all values ever on that path eventually.
There is also this line from the guarantees section which I am struggling to decipher:
"Value events are always triggered last and are guaranteed to contain updates from any other events which occurred before that snapshot was taken."
I am working with a language that does not have a Google based sdk and am asking this question, so I can further assess Firebases' suitability for my uses.
Firebase Realtime Database performs state synchronization. If a client is listening to data in a location, it will receive the state of that data. If there are changes in the data, it will receive the latest state of that data.
...if I have a game client that pushes moves to the same path one after the other will the listening client receive all updates?
If there are multiple updates before the Firebase server has a chance to send the state to a listener, it may skip some intermediate values. So there is no guarantee that your client will see every state change, there is just a guarantee that it will eventually see the latest state.
If you want to ensure that all clients (can) see all state changes, you should store the state changes themselves in the database.
try to this code to get update value from firebase database:-
mFirebaseInstance = FirebaseDatabase.getInstance();
mFirebaseDatabase = mFirebaseInstance.getReference();
mFirebaseDatabase.child("new_title").setValue("Realtime Database");
mFirebaseDatabase.child("new_title").addValueEventListener(new ValueEventListener() {
#Override
public void onDataChange(DataSnapshot dataSnapshot) {
String appTitle = dataSnapshot.getValue().toString();
Log.e("Hey", appTitle);
title.setText(appTitle);
}
#Override
public void onCancelled(DatabaseError error) {
// Failed to read value
Log.e("Hey", "Failed to read app title value.", error.toException());
}
});
I am developing a Qt5 server application and I am using the QAMQP library.
What I want to do is the following:
Another server should send a message whenever something about a user
should change
My server, which is distributed among multiple machines and has multiple processes per machine needs to be notified about these updates
The thing is, I am not sure about the architecture that I should build. I just know that whenever something about some user changes, the server needs to send a message to the RabbitMQ broker and all my processes that are interested in updates for that particular user should get the message. But should I create one queue per process, and bind it with a separate exchange for each user? Or maybe create in each process a separate queue for each user and bind that somehow to some exchange. Fanout exchanges come to mind, and one queue per process, I am just not sure about the queue-exchange relations even though I've spent quiet some time trying to figure it out.
Update, in order to clarify things and write about the progress
I have a distributed application that needs to be notified for product changes. Those changes happen often and are tracked by another platform. I want to get those updates in my application.
In order to achieve that, each one of my application instances creates it's own queue. Then, whenever an instance is interested in updates for a particular product it creates an exchange for that product and binds it to the queue, like this:
Exchange type : 'direct'
Exchange name : 'product_update'
Routing key : 'PRODUCT_CODE'
Where PRODUCT_CODE is a string that represents the code of the product. In the platform that track the changes, I just publish messages with the corresponding exchanges.
The problem comes when i need to unsubscribe for a product update. I am using the QAMQP library, and in the destructor of the QAMQP::Exchange there's an unconditional remove() call.
When that function is called I am getting error in the RabbitMQ log, which looks like this:
=ERROR REPORT==== 28-Jan-2014::08:41:35 ===
connection <0.937.0>, channel 7 - soft error:
{amqp_error,precondition_failed,
"exchange 'product_update' in vhost 'test-app' in use",
'exchange.delete'}
I am not sure how to properly unsubscribe. I know from the RabbitMQ web interface that I have only one exchange ('product_update') which has bindings to multiple queues with difference routing keys.
I can see that the call to remove() in QAMQP tries to delete the exchange, but since it's used by my other processes, it's still in use and cannot be removed, which I beleive is ok.
But what should I do to delete the exchange object that I created? Should I first unbind it from the queue? I believe that i should be able to delete the object without calling remove(), but I may be mistaken or I may doing it wrong.
Also, if there's a better pattern for what I am trying to accomplish, please advice.
Here's some sample code, per request.
ProductUpdater::ProductUpdater(QObject* parent) : QObject(parent)
{
mClient = new QAMQP::Client(this);
mClient->setAutoReconnect(true);
mClient->open(mConnStr);
connect(mClient, SIGNAL(connected()), this, SLOT(amqp_connected()));
}
void ProductUpdater::amqp_connected()
{
mQueue = mClient->createQueue();
connect(mQueue, SIGNAL(declared()), this, SLOT(amqp_queue_declared()));
connect(mQueue, SIGNAL(messageReceived(QAMQP::Queue*)),
this, SLOT(message_received(QAMQP::Queue*)));
mQueue->setNoAck(false);
mQueue->declare(QString(), QAMQP::Queue::QueueOptions(QAMQP::Queue::AutoDelete));
}
void ProductUpdater::amqp_queue_declared()
{
mQueue->consume();
}
void ProductUpdater::amqp_exchange_declared()
{
QAMQP::Exchange* exchange = qobject_cast<QAMQP::Exchange*>(sender());
if (mKeys.contains(exchange))
mQueue->bind(exchange, mKeys.value(exchange));
}
void ProductUpdater::message_received(QAMQP::Queue* queue)
{
while (queue->hasMessage())
{
const QAMQP::MessagePtr message = queue->getMessage();
processMessage(message);
if (!queue->noAck())
queue->ack(message);
}
}
bool ProductUpdater::subscribe(const QString& productId)
{
if (!mClient)
return false;
foreach (const QString& id, mSubscriptions) {
if (id == productId)
return true; // already subscribed
}
QAMQP::Exchange* exchange = mClient->createExchange("product_update");
mSubscriptions.insert(productId, exchange);
connect(exchange, SIGNAL(declared()), this, SLOT(amqp_exchange_declared()));
exchange->declare(QStringLiteral("direct"));
return true;
}
void ProductUpdater::unsubscribe(const QString& productId)
{
if (!mSubscriptions.contains(productId))
return;
QAMQP::Exchange* exchange = mSubscriptions.take(productId);
if (exchange) {
// This may even be unnecessary...?
mQueue->unbind(exchange, productId);
// This will produce an error in the RabbitMQ log
// But if exchange isn't destroyed, we have a memory leak
// if we do exchange->deleteLater(); it'll also produce an error...
// exchange->remove();
}
}
Amy,
I think your doubt is related to the message distribution style (or patterns) and the exchange types available for RabbitMQ. So, I'll try to cover them all with a short explanation and you can decide which will fit best for your scenario (RabbitMQ tutorials explained in another way).
Work Queue
Using the default exchange and a binding key you can post messages directly yo a queue. Once a message arrives for a queue, the consumers "compete" to grab the message, it means a message is not delivered to more than one consumer. If there are multiple consumers listening to a single queue, the messages will be delivered in a round-robin fashion.
Use this approach when you have work to do and you want to scale across multiple servers/processes easily.
Publish/Subscribe
In this model, one single sent message may reach many consumers listening on their queues. For this scenario, where you must unselectively dispatch messages to all consumers, you can use a fanout exchange. These exchanges are "dumb" and acts just like their names imply: like a fan. One thing enters and is replicated without any intelligence to all queues that are bound to the exchange. You could as well use direct exchanges, but only if you need to do any filtering or routing on the messages.
Use this scenario when you have something like an event and you may need multiple servers, processes and consumers to handle that event, each one doing a task of different nature to handle the event. If you do not need any filter/routing, use fanout exchange for this scenario.
Routing / Topic
A particular case of the Publish/Subscribe model, where you can have queues "listen" on the exchange using filters, that may have pattern matching (topics) or not (just route).
If you need pattern matching, use topic exchange type. If you don't, use direct.
When a queue "listens" to an exchange, a binding is used. In this binding, you may specify a binding key.
To deliver the message to the correct queues, the exchange examines the message's routing key. If it matches the binding key, the message is forwarded to that queue. The match strategy depends on wether you are using topic or direct exchange, as said before.
TL;DR:
For your scenario, if each process do something different with the User change event, use a single exchange with fanout type. Each class of handler declares the same queue name bound to that exchange. This relates to the Publish/Subscribe model above. You can distribute work to among consumers of the same class listening on the same queue name, even if they don't reside on the same process.
However, if all the consumers that are interested in the event perform the same task when handling, use the work queue model.
Hope this helps,
I am trying to implement delayed queue with overriding of messages using Active MQ.
Each message is scheduled to be delivered with delay of x (say 60 seconds)
In between if same message is received again it should override previous message.
So even if I receive 10 messages say in x seconds. Only one message should be processed.
Is there clean way to accomplish this?
The question has two parts that need to be addressed separately:
Can a message be delayed in ActiveMQ?
Yes - see Delay and Schedule Message Delivery. You need to set <broker ... schedulerSupport="true"> in your ActiveMQ config, as well as setting the AMQ_SCHEDULED_DELAY property of the JMS message saying how long you want the message to be delayed (10000 in your case).
Is there any way to prevent the same message being consumed more than once?
Yes, but that's an application concern rather than an ActiveMQ one. It's often referred to as de-duplication or idempotent consumption. The simplest way if you only have one consumer is to keep track of messages received in a map, and check that map whether you receive a message. It it has been seen, discard.
For more complex use cases where you have multiple consumers on different machines, or you want that state to survive application restart, you will need to keep a table of messages seen in a database, and query it each time.
Please vote this answer up if it helps, as it encourages people to help you out.
Also according to method from ActiveMQ BrokerService class you should configure persistence to have ability to use scheduler functionality.
public boolean isSchedulerSupport() {
return this.schedulerSupport && (isPersistent() || jobSchedulerStore != null);
}
you can configure activemq broker to enable "schedulerSupport" with the following entry in your activemq.xml file located in conf directory of your activemq home directory.
<broker xmlns="http://activemq.apache.org/schema/core" brokerName="localhost" dataDirectory="${activemq.data}" schedulerSupport="true">
You can Override the BrokerService in your configuration
#Configuration
#EnableJms
public class JMSConfiguration {
#Bean
public BrokerService brokerService() throws Exception {
BrokerService brokerService = new BrokerService();
brokerService.setSchedulerSupport(true);
return brokerService;
}
}