I am using a complex state engine system build with Qt 5.4 (using custom state engine classes).
Part of that code is logging of events, transitions, etc. It is very important for me to log all events the engine/state objects are receiving so I can completely track what is happening in the state engines.
For most event types logging is easy. However I failed to log queued connections (i.e. meta call events). QMetaCallEvent is private so there is not much I can do. However it is hard to believe that such an integral part of Qt can not be inspected properly.
Is there some way I missed that allows to log queued connections (including signal name, slot name, sender name, receiver name and arguments if possible)?
Install an event filter and intercept events with ev->type() == QEvent::MetaCall. All members visible in the debugger.
Need access to private headers? Use QT += core-private in your .pro file.
(tone mode="original poster")It's hard to believe that nobody reads documentation(/tone)
There is no official API that allows to do what I intend.
Inspecting QMetaCall events (using private framework headers) is a bad idea. First they are private (and may break your code any time) second the QMetaCall event sender() pointer may be invalid if sender was deleted immediately and I could not find a clean way to inspect events in such cases.
The way I am using now is totally different. Instead of inspecting the arriving event objects I am using a modified variant of QSignalSpy that allows to do more than the original class and helps logging the signal emissions using secondary connections.
In my situation this seems feasible even if it is pretty complicated and not a universal solution. At least no private headers are involved.
Related
I'm maintaining an event-sourced application that went far off the road I'm afraid.
In one case a command is received by an aggregate root that publishes an event that is handled by an event handler that needs to do 2 things:
send a command (cmd1) to another aggregate root that will publish an event that will create a number of sagas each firing of some commands that are eventually handled by a number of aggregates
send a second command (cmd2) that will also lead to all sorts of command/event/command sequences.
In schematic form:
cmd0 -> AR0 -> evt0 -> evtHandler -> cmd1 -> AR1 -> evt1 -> saga stuff and more cmds and evts
|-> cmd2 -> AR2 -> evt2 -> more saga stuff, cmds and evts
Everything happens in the same thread and everything happens in 1 transaction started at the first command handling.
Now the goal: all events, saga's, aggregate calls originated from the first command (cmd1) should happen first and then all events, saga's and aggregate calls originated from the second command (cmd2) should happen.
Here's the observation: cmd1 calls AR1 that published evt1 but after that cmd2 calls AR2 publishing evt2. All other events and commands originating from cmd1 are mingled with those from cmd2.
First I thought I could get away with it using the UnitOfWork but even explicitly creating a separate unit of work for handling cmd1 didn't solve the problem. Looking at the implementation in AbstractEventBus I see that the events are simply merged in the parent unit of work and thus end up being merged with the ones originating from cmd2.
Here's the question: Is there a way to first call cmd1 and wait until all effects originating from that command are handled before calling cmd2 while still preserving the transactional atomicity that I currently have?
To be completely honest with you Jan, the best would be if the components within your application don't rely to much on that order.
It essentially means you have distinct message handling components, which in essence could be different micro service, but they are all tied together as the order is important.
Ideally, you'd set up your components to work on their own.
So, aggregates handle a command and publish the result, done.
Sagas react to events, regardless of where they come from, and react on them with actions (e.g. command dispatching).
Embracing the eventuality would help here, as it will drop the entire requirement of waiting for one process to complete.
From a theoretical stance, that would be my response.
From a more pragmatic corner looking at your question, I'd like to point out that it sounds like a rabbit hole you are going in to. You don't only want cmd1 handling to be done, you want event handling on all sagas to be resolved, including commands coming out of that too, correct? Who's here to tell what the number of Sagas is? Or what the number of commands those saga dispatch need to be taken into account? These criteria will likely change over time, adding more an more stuff which needs to happen "in a single transaction".
Well, yes there are way to wait for processing from some parts, to pull them all in a single transaction. But to be honest with you, I wouldn't recommend taking that route, as it will only make using such a message based system more and more complex.
The crux is what all effects are. From the point of dispatching that command, you should only care if that exact command handles successfully yes or no, and that's where the concerns should end.
I know this does not give you a simple programmatic solution, as you need to adjust the design. But I think decoupling is the only right way to go hear.
That's my two cents to the situation, hope this helps you further in any way Jan.
Message Anticipation explanation update
In essence, the messages you'd use in an Axon application form a boundary. A boundary after which the components essentially don't have a clue what is going to handle those messages. The behaviour per message differs a little, but might clarify what opens you have too:
Commands - Commands are consistently routed to a single handle, on a single instance. Furthermore, you can anticipate a response, in the form of an OK or NOK. OK's mean the handler is void or the identifier of a created entity (like the aggregate itself). NOK's typically are the exceptions you throw from your command handling methods, which signal something went wrong or the command simply couldn't be executed and it should be let know to the dispatching end.
Events - Events will be broadcast to any component which has subscribed itself to the EventBus as being capable to handle a given event. Note that event handling is segregated in time from the actual publication point of the event. This means there is no way there are results from event handling which could (or should) be returned to the dispatcher of an event.
Queries - Query messages can be routed in several forms. Either a single component is best suited to answer the query (called Point-to-Point queries). You can also dispatch a query to several handlers and aggregate the results (called Scatter-Gather queries). Lastly, you can subscribe to query models by doing a "Subscription query", which is essentially a combination of a point-to-point followed up by a Flux of updates. Clearly, query dispatching would mean you are receiving a result from some component. It's just that you have freedom in the type of query you do. If any assurance is required about the "up-to-date"-ness of a query response should be part of the implementation of the query being sent and how it is handled by a #QueryHandler annotated method.
Hope this provides some additional clarity at what each of the messages do in an Axon application!
With Qt, I can filter all the QEvent events my application receives or produces with:
qApp.installEventFilter(my_filter_object)
Is there any way to filter Qt signals (of signals and slots) in a similar manner I can filter QEvent events?
With QtCore.QStateMachine.SignalEvent extending QEvent, and StateMachineSignal QEvent.Type being there, everything seems to be in place, but my event filter can't seem to catch one of these.
IOW, is there any way to get the signal's name (index), the signal emitting object, and the passed arguments, for each signal of every QObject in my application without explicitly connecting to it?
Thanks!
No. The only generic API for monitoring signals is QSignalSpy, but that requires an explicit connection to a specific object and signal. The only other thing is the TestLib Framework, which has a command-line option (-vs) that can output all the emitted signals during a test run. But that seems pretty far removed from what you're asking for.
Of course, you can use QMetaObject to list all the signals of a QObject and try to explicitly monitor everything. But that presupposes you can get a reference to all the objects you might be interested in (and that you can actually know in advance what they will be).
The only exception to the above, is for signals that are emitted via a queued connection. These signals, which are most commonly emitted across threads, are wrapped in an event and posted to the event-queue of the receiver's thread. It is possible to detect these kinds of signals by filtering on events of type QEvent.MetaCall.
However, the associated event objects are of type QMetaCallEvent, which is an internal Qt class which is not wrapped by PyQt. Normally, these objects would have id(), sender(), signalId() and args() methods. But in PyQt, all you'll get is a plain QEvent, and thus no information about the emitted signal will be available.
Even if QMetaCallEvent was available, though, the default connection type is not queued - and there is no way to globally reset the default for all signals.
My guess is that the signal events you described are only used for queued connections. In this case events are sent to ensure slots are called after control is passed to the eventloop. Direct connections (wich you would use normally) are essentially just function calls and I doubt there is a way to intercept them without having access to the specific objects.
I have a Java program which connects to the internet and send files (emails with attachments, SSL, javamail).
It sends only one email at a time.
Is there a way that my program could track the network traffic it itself is generating?
That way I could track progress of emails being sent...
It would also be nice if it was cross-platform solution...
Here's another approach that only works for sending messages...
The data for a message to be sent is produced by the Message.writeTo method and filtered through various streams that send it directly out the socket. You could subclass MimeMessage, override the writeTo method, wrap the OutputStream with your own OutputStream that counts the data flowing through it (similar to my other suggestion), and reports that to your program. In code...
public class MyMessage extends MimeMessage {
...
public void writeTo(OutputStream os, String[] ignoreList) throws IOException, MessagingException {
super.writeTo(new MyCountingStream(os), ignoreList);
}
}
If you want percent completion you could first use Message.writeTo to write the message to a stream that does nothing but count the amount of data being written, while throwing away the data. Then you know how big the message really is, so when the message is being sent you can tell what percent of the message that is.
Hope that helps...
Another user's approach is here:
Using JProgressBar with Java Mail ( knowing the progress after transport.send() )
At a lower level, if you want to monitor how many bytes are being sent, you should be able to write your own SocketFactory that produces Sockets that produce wrapped InputStreams and OutputStreams that monitor the amount of data passing through them. It's a bit of work, and perhaps lower level than you really want, but it's another approach.
I've been meaning to do this myself for some time, but I'm still waiting for that round tuit... :-)
Anyway, here's just a bit more detail. There might be gotchas I'm not aware of once you get into it...
You need to create your own SocketFactory class. There's a trivial example in the JavaMail SSLNOTES.txt file that delegates to another factory to do the work. Instead of factory.createSocket(...), you need to use "new MySocket(factory.createSocket(...))", where MySocket is a class you write that overrides all the methods to delegate to the Socket that's passed in the constructor. Except the getInputStream and getOutputStream methods, which have to use a similar approach to wrap the returned streams with stream classes you create yourself. Those stream classes then have to override all the read and write methods to keep track of how much data if being transferred, and make that information available however you want to your code that wants to monitor progress. Before you do an operation that you want to monitor, you reset the count. Then as the operation progresses, the count will be updated. What it won't give you is a "percent completion" measure, since you have no idea how much low level data needs to be sent to complete the operation.
It is hard for me to understand the difference between signals and events in Qt, could someone explain?
An event is a message encapsulated in a class (QEvent) which is processed in an event loop and dispatched to a recipient that can either accept the message or pass it along to others to process. They are usually created in response to external system events like mouse clicks.
Signals and Slots are a convenient way for QObjects to communicate with one another and are more similar to callback functions. In most circumstances, when a "signal" is emitted, any slot function connected to it is called directly. The exception is when signals and slots cross thread boundaries. In this case, the signal will essentially be converted into an event.
Events are something that happened to or within an object. In general, you would treat them within the object's own class code.
Signals are emitted by an object. The object is basically notifying other objects that something happened. Other objects might do something as a result or not, but this is not the emitter's job to deal with it.
My impression of the difference is as follows:
Say you have a server device, running an infinite loop, listening to some external client Events and reacting to them by executing some code.
(It can be a CPU, listening to interrupts from devices, or Client-side Javascript browser code, litsening for user clicks or Server-side website code, listening for users requesting web-pages or data).
Or it can be your Qt application, running its main loop.
I'll be explaining with the assumption that you're running Qt on Linux with an X-server used for drawing.
I can distinguish 2 main differences, although the second one is somewhat disputable:
Events represent your hardware and are a small finite set. Signals represent your Widgets-layer logic and can be arbitrarily complex and numerous.
Events are low-level messages, coming to you from the client. The set of Events is a strictly limited set (~20 different Event types), determined by hardware (e.g. mouse click/doubleclick/press/release, mouse move, keyboard key pressed/released/held etc.), and specified in the protocol of interaction (e.g. X protocol) between application and user.
E.g. at the time X protocol was created there were no multitouch gestures, there were only mouse and keyboard so X protocol won't understand your gestures and send them to application, it will just interpret them as mouse clicks. Thus, extensions to X protocol are introduced over time.
X events know nothing about widgets, widgets exist only in Qt. X events know only about X windows, which are very basic rectangles that your widgets consist of. Your Qt events are just a thin wrapper around X events/Windows events/Mac events, providing a compatibility layer between different Operating Systems native events for convenience of Widget-level logic layer authors.
Widget-level logic deals with Signals, cause they include the Widget-level meaning of your actions. Moreover, one Signal can be fired due to different events, e.g. either mouse click on "Save" menu button or a keyboard shortcut such as Ctrl-S.
Abstractly speaking (this is not exactly about Qt!), Events are asynchronous in their nature, while Signals (or hooks in other terms) are synchronous.
Say, you have a function foo(), that can fire Signal OR emit Event.
If it fires signal, Signal is executed in the same thread of code as the function, which caused it, right after the function.
On the other hand, if it emits Event, Event is sent to the main loop and it depends on the main loop, when it delivers that event to the receiving side and what happens next.
Thus 2 consecutive events may even get delivered in reversed order, while 2 consecutively fired signals remain consecutive.
Though, terminology is not strict. "Singals" in Unix as a means of Interprocess Communication should be better called Events, cause they are asynchronous: you call a signal in one process and never know, when the event loop is going to switch to the receiving process and execute the signal handler.
P.S. Please forgive me, if some of my examples are not absolutely correct in terms of letter. They are still good in terms of spirit.
An event is passed directly to an event handler method of a class. They are available for you to overload in your subclasses and choose how to handle the event differently. Events also pass up the chain from child to parent until someone handles it or it falls off the end.
Signals on the other hand are openly emitted and any other entity can opt to connect and listen to them. They pass through the event loops and are processed in a queue (they can also be handled directly if they are in the same thread).
There is an object of class QNetworkReply. There is a slot (in some other object) connected to its finished() signal. Signals are synchronous (the default ones). There is only one thread.
At some moment of time I want to get rid of both of the objects. No more signals or anything from them. I want them gone.
Well, I thought, I'll use
delete obj1; delete obj2;
But can I really?
The specs for ~QObject say:
Deleting a QObject while pending events are waiting to be delivered can cause a crash.
What are the 'pending events'?
Could that mean that while I'm calling my delete, there are already some 'pending events' to be delivered and that they may cause a crash and I cannot really check if there are any?
So let's say I call:
obj1->deleteLater(); obj2->deleteLater();
To be safe.
But, am I really safe? The deleteLater adds an event that will be handled in the main loop when control gets there. Can there be some pending events (signals) for obj1 or obj2 already there, waiting to be handled in the main loop before deleteLater will be handled? That would be very unfortunate. I don't want to write code checking for 'somewhat deleted' status and ignoring the incoming signal in all of my slots.
Deleting QObjects is usually safe (i.e. in normal practice; there might be pathological cases I am not aware of atm), if you follow two basic rules:
Never delete an object in a slot or method that is called directly or indirectly by a (synchronous, connection type "direct") signal from the object to be deleted.
E.g. if you have a class Operation with a signal Operation::finished() and a slot Manager::operationFinished(), you don't want delete the operation object that emitted the signal in that slot. The method emitting the finished() signal might continue accessing "this" after the emit (e.g. accessing a member), and then operate on an invalid "this" pointer.
Likewise, never delete an object in code that is called synchronously from the object's event handler. E.g. don't delete a SomeWidget in its SomeWidget::fooEvent() or in methods/slots you call from there. The event system will continue operating on the already deleted object -> Crash.
Both can be tricky to track down, as the backtraces usually look strange (Like crash while accessing a POD member variable), especially when you have complicated signal/slot chains where a deletion might occur several steps down originally initiated by a signal or event from the object that is deleted.
Such cases are the most common use case for deleteLater(). It makes sure that the current event can be completed before the control returns to the event loop, which then deletes the object. Another, I find often better way is defer the whole action by using a queued connection/QMetaObject::invokeMethod( ..., Qt::QueuedConnection ).
The next two lines of your referred docs says the answer.
From ~QObject,
Deleting a QObject while pending events are waiting to be delivered can cause a crash. You must not delete the QObject directly if it exists in a different thread than the one currently executing. Use deleteLater() instead, which will cause the event loop to delete the object after all pending events have been delivered to it.
It specifically says us to not to delete from other threads. Since you have a single threaded application, it is safe to delete QObject.
Else, if you have to delete it in a multi-threaded environment, use deleteLater() which will delete your QObject once the processing of all the events have been done.
You can find answer to your question reading about one of the Delta Object Rules which states this:
Signal Safe (SS).
It must be safe to
call methods on the object, including
the destructor, from within a slot
being called by one of its signals.
Fragment:
At its core, QObject supports being
deleted while signaling. In order to
take advantage of it you just have to
be sure your object does not try to
access any of its own members after
being deleted. However, most Qt
objects are not written this way, and
there is no requirement for them to be
either. For this reason, it is
recommended that you always call
deleteLater() if you need to delete an
object during one of its signals,
because odds are that ‘delete’ will
just crash the application.
Unfortunately, it is not always clear
when you should use ‘delete’ vs
deleteLater(). That is, it is not
always obvious that a code path has a
signal source. Often, you might have a
block of code that uses ‘delete’ on
some objects that is safe today, but
at some point in the future this same
block of code ends up getting invoked
from a signal source and now suddenly
your application is crashing. The only
general solution to this problem is to
use deleteLater() all the time, even
if at a glance it seems unnecessary.
Generally I regard Delta Object Rules as obligatory read for every Qt developer. It's excellent reading material.
As far as I know, this is mainly an issue if the objects exist in different threads. Or maybe while you are actually processing the signals.
Otherwise deleting a QObject will first disconnect all signals and slots and remove all pending events. As a call to disconnect() would do.