I am developing a Live DirectShow filter.
I have an H264 stream source which i can get streams via SDK API.
In my filter I have a Queue which i Enqueue (push) incoming stream from a thread.
Then I consume (Dequeue,pop) these streams inside the filter FillBuffer...
So I make a thread safe Queue...But this causes some problem....
At FillBuffer if i check is there any incoming packets and if there is , process
logic is like this:
...
bool hasElement = SynchronizedQueue.pop(element);
if(!hasElement)
{
return S_OK
}
...
...this consume to much CPU...
Howewer using boost lib to implement lock with condition variable
...
SynchronizedQueue.waitAndPop(element) ;// which wait until we have some
Which have a lover CPU ...But sometime when there is no data in Queue, this block FillBuffer function and filter may not be stopped...
So any design idea alternatives for a live source filter which takes input streams from a remote mechine and pass it to decoder?
Or how can I make my design better....lower CPU and can be stopped?
The source filter owns a pushing thread, so you need to wait there using a synchronization object (event, mutex) to yield control until a new frame is available for pushing off the output pin.
Whenever you receive a frame from SDK and you put it onto queue, you will indicate this availability using synchronization object, e.g. you will set an event. The worker thread wlil see the event and start processing the frame.
The worker thread needs to be able to respond to two events, at least: a new frame, and filter/graph stop. So you will need WaitForMultipleObjects to wait for multiple evetns and wake up on the first occurred.
Related
i use grpc::CompletionQueue in my programm, you can also find the example in
"grpc/examples/cpp/helloword/greeter_async_clients.cc".
The problem code as follow!
// stub_->PrepareAsyncSayHello() creates an RPC object, returning
// an instance to store in "call" but does not actually start the RPC
// Because we are using the asynchronous API, we need to hold on to
// the "call" instance in order to get updates on the ongoing RPC.
call->response_reader =
stub_->PrepareAsyncSayHello(&call->context, request, &cq_);
// StartCall initiates the RPC call
call->response_reader->StartCall();
// Request that, upon completion of the RPC, "reply" be updated with the
// server's response; "status" with the indication of whether the operation
// was successful. Tag the request with the memory address of the call object.
call->response_reader->Finish(&call->reply, &call->status, (void*)call);
The Client send 1,2,3...100 to Server, but the Server get the number list is "100,99,98...2,1". Why? i could not find any source code about this... thank you very much
and does is Nagle Algorithm for gRPC?
CompletionQueue is somewhat of a misnomer. They will return events in the order they finish (rather than the order they are issued).
gRPC C++ will disable Nagle Algorithm by default.
I'm building a simple talker/listener app that receives OSC data through UDP. I'm using OSCKit pod which itself uses CocoaAsyncSocket library for the internal UDP communication.
When I'm listening to a particular port to receive data from another OSC capable software, I log the received commands to a NSTextView. The problem is that sometimes, I receive thousands of messages in a very short period of time (EDIT: I just added a counter to see how many messages I'm receiving. I got over 14000 in just a few seconds and that is only a single moving object in my software). There is no way to predict when this is gonna happen so I cannot lock the textStorage object of the NSTextView to keep it from sending all its notifications to update the UI. The data is processed through a delegate callback function.
So how would you go around that limitation?
///Handle incoming OSC messages
func handle(_ message: OSCMessage!) {
print("OSC Message: \(message)")
let targetPath = message.address
let args = message.arguments
let msgAsString = "Path: \"\(targetPath)\"\nArguments: \n\(args)\n\n"
print(msgAsString)
oscLogView.string?.append(msgAsString)
oscLogView.scrollToEndOfDocument(self)
}
As you can see here (this is the callback function) I'm updating the TextView directly from the callback (both adding data and scrolling to the end), every time a message is received. This is where Instruments tell me the slow down happens and the append is the slowest one. I didn't go further than that in the analysis, but it certainly is due to the fact that it tries to do a visual update, which takes a lot more time than parsing 32bits of data, and when it's finished it receives another update right away from the server's buffer.
Could I send that call to the background thread? I don't feel like filling up the background thread with visual updates is such a great idea. Maybe growing my own string buffer and flushing it to the TextView every now and then with a timer?
I want to give this a console feel, but a console that freezes is not a console.
Here is a link to the project on github. the pods are all there and configured with cocoapods, so just open the workspace. You guys might not have anything to generate that much OSC traffic, but if you really feel like digging in, you can get IanniX, which is an open-source sequencer/trajectory automator that can generate OSC and a lot of it. I've just downloaded it and I'll build a quick project that should send enough data to freeze the app and I'll add it to the repo if anybody want to give it a shot.
I append the incoming data to a buffer variable and I use a timer that flushes that buffer to the textview every 0.2 seconds. The update cycle of the textview is way too slow to handle the amount of incoming data so unloding the network callback to a timer let the server process the data instead of being stopped every 32bits.
If anybody come up with a more elegant method, I'm open minded.
I am trying to use Skype's DBus API in order to retrieve the list of messages (message IDs) I've exchanged with a contact. However, both the SEARCH CHATMESSAGES <target> (protocol >= 3) and the SEARCH MESSAGES <target> (protocol < 3) commands return unexpectedly empty results.
Here is the trace of a few exchanges I had with the API. I used d-feet to send my requests, but the result is exactly the same when I send the request from my own program.
Bus name: com.Skype.API
Object: /com/Skype
Interface: com.Skype.API
Method used: Invoke(String request)
Trace:
-> NAME dfeet
<- OK
-> PROTOCOL 8
<- PROTOCOL 8
-> SEARCH CHATMESSAGES mycontact
<-
The same thing happens with two other SEARCH commands:
SEARCH MESSAGES <target> (with PROTOCOL 2).
SEARCH CHATS
Additionally, I also get an empty result when I try to request a message list based on a chat ID: GET CHAT <chat_id> GETMESSAGES.
However, commands such as SEARCH FRIENDS, SEARCH CALLS, or SEARCH ACTIVECHATS work just fine, and return their lists of IDs (contacts IDs, calls IDs, or chat IDs) as expected.
It might also be worth noting that this happens for all contacts, regardless of how many messages I've exchanged with them (I thought at first that there might be too many messages involved, but the result is the same, whether I've sent 3, or thousands of messages to the contact).
Is there anything that would explain why I get these empty responses through DBus, for these requests?
Skype will not use Invoke's return value when its reply is too heavy. As it so happens, when Skype has too much data to prepare and transfer after a request, it automatically returns an empty string to the Invoke call. The true, heavy reply is then prepared asynchrously by Skype, and the client program must be ready to receive it when it eventually arrives.
Whenever you are communicating with Skype over DBus, your application must act as both a client (calling Invoke), and a server (providing a DBus object for Skype to reach). This design was a little unexpected (I guess we could argue on its quality), but here is what it requires you to do:
Make your program a DBus "server" (providing objects to reach). Through your bus name to Skype, register an object path called /com/Skype/Client implementing the com.Skype.API.Client interface.
Prepare a message handler for the only method of this interface: Notify(s). This is the method Skype will try to call to send you the heavy reply to one of your previous requests.
Program your own mechanism to match your Invoke request with the asynchronous Notify message coming in as an answer later on.
The creation of an object can be done through dbus_connection_register_object_path, the parameters for which are:
The DBusConnection structure representing your bus name.
The object path you are registering, here /com/Skype/Client.
A table of message handlers (DBusObjectPathVTable) used to process all incoming requests.
Data to be sent to these handlers when they are called. This is additional data, not the actual message being received since you're just setting up the handler here.
For instance...
DBusHandlerResult notify_handler(DBusConnection *connection,
DBusMessage *message,
void *user_data){
return DBUS_HANDLER_RESULT_HANDLED;
}
void unregister_handler(DBusConnection *connection,
void *user_data){}
DBusObjectPathVTable vtable = {
unregister_handler,
message_handler,
NULL
};
if(!dbus_connection_register_object_path(connection,
"/com/Skype/Client",
&vtable, NULL)){
// Error...
}
Note that this is just an object's definition. In order to actually hook on the Notify calls, you'll have to select() on a DBusWatch file descriptor, and dispatch the incoming DBusMessage in order to have your message handler called.
If you are working with other bindings, you'll probably find much faster ways to setup objects and start working as a client application. See:
GLib's g_dbus_connection_register_object
Exporting objects with dbus-python
QtDBus's QDBusConnection::registerObject
... (other bindings)
I have a signal r application. I have an issue of concurrent connection maxing out on iis. To avoid that i want my hub to send notification to groups created by in pack of 25 with delay of suppose 500ms. I add users to group as they connect. After count of 25 i add them to next group.Need help with how shall i send notification to each group with delay of 500ms.
I have looped through the number of groups and used
Thread.Sleep(500);
but all connected users do not receive the notification
Your question almost describes the solution - buffer the notifications fro 500ms then send an array of all of them.
You could do that by pushing the notification in a queue or list and and use a timer to send the notifications every 500ms.
A far better solution though would be to use Reactive Extensions and the Observable.Buffer(Timespan) method, thus avoiding queues and timers.
Subject<string> _subject;
.....
public void InitSubject()
{
_subject=new Subject<string>();
_subject.Buffer(Timespan.FromMilliseconds(500))
.Subscribe(notifications=>SendBatch(notifications));
}
....
public void SendBatch(IList<string> notifications)
{
//Get hold of the SignalR context and send the notifications here
}
public void QueueNotification(string notification)
{
_subject.OnNext(notification);
}
Buffer also allows you to batch a specific number of notifications, or use both a timespan and item limit to send a batch whenever one of the limits is reached.
Your code should call QueueNotification instead of sending the notification directly. Reactive Extensions also provide LINQ extensions, so you could apply filtering with the Where() method, eg to use different delays for different user groups, bypass buffering completely, or use any other combination.
You should note that since a Hub is transient, you can't put your subject inside the hub, although you can put QueueNotification in there, provided you provide a reference to the subject somewhere, eg by using Dependency injection to initialize the hub
I am developing a Qt5 server application and I am using the QAMQP library.
What I want to do is the following:
Another server should send a message whenever something about a user
should change
My server, which is distributed among multiple machines and has multiple processes per machine needs to be notified about these updates
The thing is, I am not sure about the architecture that I should build. I just know that whenever something about some user changes, the server needs to send a message to the RabbitMQ broker and all my processes that are interested in updates for that particular user should get the message. But should I create one queue per process, and bind it with a separate exchange for each user? Or maybe create in each process a separate queue for each user and bind that somehow to some exchange. Fanout exchanges come to mind, and one queue per process, I am just not sure about the queue-exchange relations even though I've spent quiet some time trying to figure it out.
Update, in order to clarify things and write about the progress
I have a distributed application that needs to be notified for product changes. Those changes happen often and are tracked by another platform. I want to get those updates in my application.
In order to achieve that, each one of my application instances creates it's own queue. Then, whenever an instance is interested in updates for a particular product it creates an exchange for that product and binds it to the queue, like this:
Exchange type : 'direct'
Exchange name : 'product_update'
Routing key : 'PRODUCT_CODE'
Where PRODUCT_CODE is a string that represents the code of the product. In the platform that track the changes, I just publish messages with the corresponding exchanges.
The problem comes when i need to unsubscribe for a product update. I am using the QAMQP library, and in the destructor of the QAMQP::Exchange there's an unconditional remove() call.
When that function is called I am getting error in the RabbitMQ log, which looks like this:
=ERROR REPORT==== 28-Jan-2014::08:41:35 ===
connection <0.937.0>, channel 7 - soft error:
{amqp_error,precondition_failed,
"exchange 'product_update' in vhost 'test-app' in use",
'exchange.delete'}
I am not sure how to properly unsubscribe. I know from the RabbitMQ web interface that I have only one exchange ('product_update') which has bindings to multiple queues with difference routing keys.
I can see that the call to remove() in QAMQP tries to delete the exchange, but since it's used by my other processes, it's still in use and cannot be removed, which I beleive is ok.
But what should I do to delete the exchange object that I created? Should I first unbind it from the queue? I believe that i should be able to delete the object without calling remove(), but I may be mistaken or I may doing it wrong.
Also, if there's a better pattern for what I am trying to accomplish, please advice.
Here's some sample code, per request.
ProductUpdater::ProductUpdater(QObject* parent) : QObject(parent)
{
mClient = new QAMQP::Client(this);
mClient->setAutoReconnect(true);
mClient->open(mConnStr);
connect(mClient, SIGNAL(connected()), this, SLOT(amqp_connected()));
}
void ProductUpdater::amqp_connected()
{
mQueue = mClient->createQueue();
connect(mQueue, SIGNAL(declared()), this, SLOT(amqp_queue_declared()));
connect(mQueue, SIGNAL(messageReceived(QAMQP::Queue*)),
this, SLOT(message_received(QAMQP::Queue*)));
mQueue->setNoAck(false);
mQueue->declare(QString(), QAMQP::Queue::QueueOptions(QAMQP::Queue::AutoDelete));
}
void ProductUpdater::amqp_queue_declared()
{
mQueue->consume();
}
void ProductUpdater::amqp_exchange_declared()
{
QAMQP::Exchange* exchange = qobject_cast<QAMQP::Exchange*>(sender());
if (mKeys.contains(exchange))
mQueue->bind(exchange, mKeys.value(exchange));
}
void ProductUpdater::message_received(QAMQP::Queue* queue)
{
while (queue->hasMessage())
{
const QAMQP::MessagePtr message = queue->getMessage();
processMessage(message);
if (!queue->noAck())
queue->ack(message);
}
}
bool ProductUpdater::subscribe(const QString& productId)
{
if (!mClient)
return false;
foreach (const QString& id, mSubscriptions) {
if (id == productId)
return true; // already subscribed
}
QAMQP::Exchange* exchange = mClient->createExchange("product_update");
mSubscriptions.insert(productId, exchange);
connect(exchange, SIGNAL(declared()), this, SLOT(amqp_exchange_declared()));
exchange->declare(QStringLiteral("direct"));
return true;
}
void ProductUpdater::unsubscribe(const QString& productId)
{
if (!mSubscriptions.contains(productId))
return;
QAMQP::Exchange* exchange = mSubscriptions.take(productId);
if (exchange) {
// This may even be unnecessary...?
mQueue->unbind(exchange, productId);
// This will produce an error in the RabbitMQ log
// But if exchange isn't destroyed, we have a memory leak
// if we do exchange->deleteLater(); it'll also produce an error...
// exchange->remove();
}
}
Amy,
I think your doubt is related to the message distribution style (or patterns) and the exchange types available for RabbitMQ. So, I'll try to cover them all with a short explanation and you can decide which will fit best for your scenario (RabbitMQ tutorials explained in another way).
Work Queue
Using the default exchange and a binding key you can post messages directly yo a queue. Once a message arrives for a queue, the consumers "compete" to grab the message, it means a message is not delivered to more than one consumer. If there are multiple consumers listening to a single queue, the messages will be delivered in a round-robin fashion.
Use this approach when you have work to do and you want to scale across multiple servers/processes easily.
Publish/Subscribe
In this model, one single sent message may reach many consumers listening on their queues. For this scenario, where you must unselectively dispatch messages to all consumers, you can use a fanout exchange. These exchanges are "dumb" and acts just like their names imply: like a fan. One thing enters and is replicated without any intelligence to all queues that are bound to the exchange. You could as well use direct exchanges, but only if you need to do any filtering or routing on the messages.
Use this scenario when you have something like an event and you may need multiple servers, processes and consumers to handle that event, each one doing a task of different nature to handle the event. If you do not need any filter/routing, use fanout exchange for this scenario.
Routing / Topic
A particular case of the Publish/Subscribe model, where you can have queues "listen" on the exchange using filters, that may have pattern matching (topics) or not (just route).
If you need pattern matching, use topic exchange type. If you don't, use direct.
When a queue "listens" to an exchange, a binding is used. In this binding, you may specify a binding key.
To deliver the message to the correct queues, the exchange examines the message's routing key. If it matches the binding key, the message is forwarded to that queue. The match strategy depends on wether you are using topic or direct exchange, as said before.
TL;DR:
For your scenario, if each process do something different with the User change event, use a single exchange with fanout type. Each class of handler declares the same queue name bound to that exchange. This relates to the Publish/Subscribe model above. You can distribute work to among consumers of the same class listening on the same queue name, even if they don't reside on the same process.
However, if all the consumers that are interested in the event perform the same task when handling, use the work queue model.
Hope this helps,