We are using signal r to fetch data for 20 tiles.
We call FetchData() on all 20 tiles at the same time, it then fires of a message on signal r to request that data. (each tile has subscribed to get the answers)
We find that each tile will populate its data one at a time, as if the signal r only fetches the next tiles response after the first tile has been completed?
I know this is super high level, but in my mind, it worked like an AJAX request. Where if I fired off 20 requests in a row, they would all randomly return out of order?
By default, SignalR only allows a single execution from a client at a time. So if you fire of 20 concurrent calls to invoke a hub method, it'll run those in sequence (there are exceptions to this rule but that's the idea). This article talks about some of the configuration https://learn.microsoft.com/en-us/aspnet/core/signalr/configuration?view=aspnetcore-6.0&tabs=dotnet#configure-server-options. You'd want to increase MaximumParallelInvocationsPerClient so something other than 1.
Related
I've encountered this problem a few times and now I wonder what the industry best practice is, the context is, we have a data store which aggregates pieces of information taken from multiple micro-services, the way the data comes to us is through messages broadcasted by every source when there is a change
The problem is how to guarantee that our data will be eventually consistent and that the updates were applied in the order they were meant to be received. For example, Let's say we have an entity User
User {
display_name : String,
email: String,
bio: String
}
And we are listening changes on those users to keep "display_name" updated in our data store, the messages come in a format such as
{
event: "UserCreated",
id: 1000,
display_name: "MyNewUser"
}
{
event: "UserChanged",
id: 1000,
display_name: "MyNewUser2"
}
There is a scenario where "UserChanged" reaches our listeners before "UserCreated" therefore our code won't be able to find user with id 1000 and fail both transactions. This is where a mechanism to sort those two is desired, we have considered:
Timestamps: The problem with timestamps is that although we know the last time we read an update we don't know how many events happened between the last event seen and the one we are currently processing
Sequence numbers: This is slightly better but if a sequence is lost then we won't update our storage unless we relax the rules a little bit, we could say that after some time if a sequence hasn't been seen then proceed with the rest of operations
If anyone knows common design patterns that tackle this sort of issue would be great to know, also open to suggestions on perhaps data modeling, etc. Bottomline, I'm pretty sure this is a common software problem that has been solved many times before
Thanks a lot for the help!
My first thought here would be to jump directly to a sequence numbers-based approach, but this works when you got 1 to 1 communication, like in TCP orientated communications. In your case, there is many to one, so without a coordination between the senders, it would be challenging to implement this approach correctly (ex. 2 senders can use the same sequence number).
Yes, losing the messages would be problematic, but I don't think that's the case of SQS or other cloud-based message queues (of course, it depends on the scale you're working on), because they're known for data duplication instead of data loss (AFAIK).
One idea I can think of right now is to add a new layer between the senders and the consumer, which will orchestrate the events. It can be the consumer itself, but it can be another service in front of it, let's call it orchestrator.
The orchestrator is connected with each senders (individually) via 2 queues:
The first queue is used to get the actual events from the sender
The second queue is used to signal back to the sender an ACK-like event (the message has been received, validated and successfully passed downstream to the consumer (or consumed directly)).
The way it works is the following:
The orchestrator gets the event from the sender A
It tries to execute a validation-like operation specific to the message (update on an inexisting user), the operation fails, so it sends a N-ACK message back to sender A, signaling that its message was not able to be processed successfully. Sender A will try to resend the message after some time.
In the mean time, it gets the "create user" message from sender B, the message get passed downstream to the consumer
Finally, it will get the message from sender A (after some retries).
This solution ensures message ordering in a pretty basic way, without keeping the events in memory, but rather in the queues. It may work, but it depends on a lot of factors, like number of events, number of senders, etc.
I was confused by vertx instance. The first time seeing the docs, i think the instance means the the number of event-looping threads.
As i dig into the source code(vertx 2.1.2), i found the verticle instance means a task in the event-loop thread group. The event-loop thread always waits on the selector and run tasks.
Then the first Question comes:
Is it necessary to have verticle instances in the vertx? Since the vertcle run only once by one event loop. To be more precise, the event-loop thread run the Verticle start method and throw it away, it works like an entry and that is all.
My second Question is:
How to collect the results of multiple events?
Scenario
send multiple queries on the event bus with the same handler instance
the handler waits for every callback and modify flags
if the flags reach the threadhold, do some jobs
problem
when multiple events callback, it has a chance that multiple event-loop threads will execute the handler, thus there is a race condition that the jobs will be run multiple times. How can i avoid it?
Any solutions will be apperciated.
Is it necessary to have verticle instances in the vertx?
No. This would not be required. You do not have to create any instances of the Verticle class.
How to collect the results of multiple events?
problem
when multiple events callback, it has a chance that multiple event-loop threads will execute the handler, thus there is a race condition that the jobs will be run multiple times. How can i avoid it?
Each query which is sent over the event bus will have a corresponding Handler object. They will not share the same Handler instance. For every response to a query, it's corresponding Handler object's handle() method is called. Hence, there would be no place for a race condition over a specific Handler object.
I am working on an application which requires video frame capture from different frame-grabber cards. I am using directshow ISampleGrabberCB::SampleCB callback to receive pointer to the new frame. Now I want to know when exactly this callback gets called ? Is it guaranteed that every time frame-grabber receives a new frame it will automatically get called ?
I was trying for 120Hz signals with various resolutions but I this callback is only getting called 50-55 times. So there is a possibility that my frame-grabber is not able to capture at that rate (although theoretically its capable). I want to find out whether this callback is bottleneck or framegrabber card.
Thank You
SampleCB is immediately called called from streaming thread, you have one call for every frame. While in callback, you block further streaming, that is you need to return control from your callback in order to resume (in particular, if your callback is "slow", it can reduce effective FPS).
I want to animate a node receiving messages from three different nodes in OMNET++. Right now the nodes send in a sequential manner. But, I want the nodes to send messages to the root node simultaneously. Root node is occupied with an array of input gates. Is it possible in Omnet?
You can configure the 3 nodes to send the messages at the same time by scheduling messages to be sent at the same time using the scheduleAt() function. The simulation will always show that they are transmitted sequentially, but check the T (event time) value in the simulation window. If the T value is the same every time any of the 3 messages are being sent, it means they are sent simultaneously.
The messages arriving to the root node can't be processed at the same time. Every node, including the root node, implements the handleMessage() function which will analyze each incoming message individually.
I hope this is the answer you were looking for.
You have to define parameter id in ned file. In initialize set the condition if(getIndex==id), send the message. Same message will be send to the node simultaneously.
I'm on Flash Builder 4.5 and I'm using remote object with amfphp and when I call two method (method1 and method2) at the same time the response of method2 always arrives after method1's response even though method2 is much more faster to return the result.
Here's the scenario:
I set a remote object which refers to a remote php class "Newsletter" which contains the sendNewsletter and getProgress methods.
Here's the code:
-sendNewsletter() reads the email archive and send the newsletter. After each email has sent it writes a log into the database.
-getProgress() reads the log wrote by sendNewsletter, counts how many email have been sent, compares it with the total number of the email that have to be sent and return the progress percentage
From the flex interface the users select a Newsletter to be sent and click on a "send" button which calls a function that calls the sendNewsletter() and then instantiate a loop of calls to getProgress (as you can see when getProgress returns something it calls the setProgress which updates a progress bar and calls getProgress again until the progress percentage reach 100%.
So right after I call sendNewsletter() I call getProgress() on the same remoteClass().
sendNewsletter() can take several minutes to complete (in my tests for sending 4 email it takes about 4 seconds so I think that sending thousands of email will take much more!!) and the trouble I'm encountering here is that getProgress() result arrives only after sendNewsletter() concludes its execution while what I would like to achieve is:
-call sendNewsletter()
-while sendNewsletter() does its stuff() call getProgress several time in order to get the progress percentage:
What I've got now:
call to sendNewsletter()----------------------->response
call to getProgress()------------------------------------->response after sendNewsletter()
What I want to achieve:
sendNewsletter()------------------------------------------------------------------>response()
getProgress()--->response, getProgress() again--->response-->getProgress()-->respone-->etc...
I read many post on how to work around this problem but no solution worked for me.
I tried to to "emulate" to different channel by creating two remote object with endpoint set once to gateway.php?parallel=0 and once gateway.php?parallel=1, but flash builder still send everything in one big request and get the response in one big http packet (I need tow different packet since sendNewsletter takse ages to complete compared to getProgress)
I also tried to delay the call of getProgress() after sendNewsletter() with a Timer of 500ms and flash builder makes two different calls (I can see them in firebug) but the call of getProgress gives response after sendNewsletter() anyway.
I alse tried to call sendNewsletter this way
this.myNewsletter.getOperation("sendNewsletter").send(idNewsletter)
this.myNewsletter.getOperation("sendNewsletter").cancel()
in order to let flash builder forget about the response but no way!!!
So far the only way to work around I found is creating a common httpservice which refers to a php which instantiate the Newsletter class and calls the getProgress method.
By using two different channel I can call the getProgress httpservice while sendNewsletter is being execute. It works but I don't like it and I don't want to create an httpservice for each method I need to call in background, so I want to achieve this with remote object only.
Anyone has addressed the same problem?
You Flash builder guru, I know you're around, please help me!!!!!
Thanks in advance!!!
Bye,
Luke
P.S.
sorry if this post is a little bit long but the situation it's quite complicated.
I don't know exactly, what you want to..
But when working with Remote Object, there is a best practices to use Responder to handle the responses that were arrived parallel from the single Remote Object.
So try to add responder to your service calls like
remoteObject.methodCall().addResponder(new YourResponder(resultEvent, faultEvent));
So when specific response will be come, it will be handled by your different custom responders.
And by that you will be able to handle your response separately.