In tornado, we can use ThreadPoolExecutor to execute calls asynchronously, but the ThreadPoolExecutor or #run_on_executor is thread-safe?
If the answer is 'no', how to solve the problem of resource sharing?
Is ThreadPoolExecutor thread safe?
ThreadPoolExecutor is safe as long as your code is safe. All it does is it runs your code in multiple threads. Ultimately, it's all up to you to make your code thread-safe.
How to solve the problem of resource sharing?
By not sharing the resource. Anytime you need to do anything with the resource you call the main thread and do it there.
In Tornado, you can submit control back to the main thread using ioloop.IOLoop.add_callback. So, you can run your code in ThreadPoolExecutor, but anytime you want to do anything with a resource, you submit control back to the main thread.
Example:
x = 1
def print_x():
global x
if x == 1:
print x
x += 1
As you can see, the above code is not thread safe because every thread can modify x.
To run it in a thread-safe manner, you'll have to perform the condition check and modify x in the main thread. That is, you'll have to run the print_x function only in the main thread, like this:
x = 1
def print_x():
# run it only in the main thread for thread-safety
global x
if x == 1:
print x
x += 1
def call_print_x():
# this can be run in any thread
# it asks the main thread to run `print_x`
loop = ioloop.IOLoop.current()
loop.add_callback(print_x) # submit control to main thread
Related
I have a GUI in PyQt5, which starts a QThread that reads from a serial port. The thread does quit, when it read all the data, but I want to be able to stop it when I click on a stop button. How do I do that? Here is the basic code:
# ...
class Worker(QObject):
finished = pyqtSignal()
progress = pyqtSignal(list)
def __init__(self):
QObject.__init__(self)
self._Reader = Reader()
self._Reader.progress = self.progress
self._Reader.finished = self.finished
def run(self):
self._Reader.read()
class Ui(QtWidgets.QMainWindow):
# ...
def startClicked(self):
self.thread = QThread()
self.worker = Worker()
self.worker.moveToThread(self.thread)
self.thread.started.connect(self.worker.run)
self.worker.finished.connect(self.thread.quit)
self.worker.finished.connect(self.worker.deleteLater)
self.worker.finished.connect(self.workerFinished)
self.thread.finished.connect(self.thread.deleteLater)
self.worker.progress.connect(self.reportProgress)
self.thread.start()
def stopClicked(self):
# How do I stop the thread?
pass
when managing threads you can do, as states in the doc here: https://doc.qt.io/qt-5/qthread.html
You can stop the thread by calling exit() or quit().
https://doc.qt.io/qt-5/qthread.html#exit
exit:
Tells the thread's event loop to exit with a return code.
After calling this function, the thread leaves the event loop and returns from the call to QEventLoop::exec(). The QEventLoop::exec() function returns returnCode.
By convention, a returnCode of 0 means success, any non-zero value indicates an error.
https://doc.qt.io/qt-5/qthread.html#quit
quit:
Tells the thread's event loop to exit with return code 0 (success). Equivalent to calling QThread::exit(0).
This function does nothing if the thread does not have an event loop.
I assume that you read data in some data processing loop. If this assumption is wrong, then the following is not valid, of course.
You cannot call secondary thread's quit() directly from the main thread and expect that the secondary thread will process it immediately and quit the thread. The reason is that the thread is busy reading the data in the data processing loop. So you need to break the data processing loop in the secondary thread to make the event loop idle.
(Btw. do not confuse the data processing loop with the event loop. Data processing loop is the one which you have written yourself to read data from the port. The event loop is the loop created by Qt automatically after you called QThread::start() and which is processing events, signals and slots in the secondary thread. This event loop is blocked while your data processing loop is running.)
In order to break the data processing loop, you need to do two things:
call QThread::requestInterruption() from the main thread as response to some "Abort" button having been pressed (do not worry about thread safety, requesting interruption is thread safe/atomic)
within the loop in the secondary thread you need to periodically check QThread::isInterruptionRequested(), and if this returns true, then break the loop and emit worker's finished() signal
Once you broke from the data processing loop in the secondary thread, the event loop in the secondary thread becomes available for processing signals sent from the main thread.
I can see in your code that worker's finished() signal is connected to QThread::quit(). So emitting finished() from the secondary thread (after you broke from the data processing loop) will call thread's quit() which will be processed by the secondary thread's event loop (which is now idle) and it will quit the event loop and subsequently the thread and if you have connected everything correctly it will delete the worker and the thread. (though I have not checked this part of your code)
using QtCreator to make a loftier interface to a sofware.
There is basically a set of buttons to tune and inputs, a start and stop job
buttons, my problem comes from an infinite loop that freezes the display so I came up with using fork() so that the loop have to compete with the main program instead of eating up the whole resources (no multithreading), but the program crashes spiting:
[xcb] Unknown sequence number while processing queue
[xcb] Most likely this is a multi-threaded client and XInitThreads has not
been called
[xcb] Aborting, sorry about that.
a.out: ../../src/xcb_io.c:274: poll_for_event: Assertion
`!xcb_xlib_threads_sequence_lost' failed.
the fonction calling the loop is called 'ON', 'OFF' is supposed to exit the forked process.
//in button 'ON' func
ps = fork();
if(getpid() == ps)
{
while(1)
{
strcpy(word, charset(minlength, maxlength, N));
ui->pass->setText(word);//operation on the display
....SNIP
}
}
//In button 'OFF' func
if(getpid() == ps)
exit(0);
I'm really asking what is the correct way of starting a while(1) and be able to break, exit, or return from it while not freezing the window using QT, thanks.
You crash probably here:
ui->pass->setText(word);//operation on the display
as in Qt, you can not change UI from non UI threads directly. Only from signals and slots mechanism.
The proper way to not freeze UI is obviously to compute lengthy operations in another thread.
You can achieve this in several ways:
One is by sub-classing QObject class to create 'worker object' which would perform all heavy operations. You create new QThread object that should live as long as you need your object. And use QObject::moveToThread method to move created object to new thread. To control your worker object you should send signals from object and call it's slots also via signal-slot mechanism. If you call them directly - they will be executed in caller thread (so do not perform stuff like worker->startHeavyJob(); in UI thread). Instead emit signal in UI (emit sigStartHeavyStuff();) and connect it to slot of your worker object (slotDoHeavyStuff();)
if you do not want to bother with this (if operation is pretty small)
- you can use QApplication::processEvents() to process events in UI event loop while going in your infinite while loop.
Another way is to use QtConcurrentRun framework to run function in separate thread which manages itself. Threads are taken from thread pool and are managed by Qt. This approach looks like something you want to do. Though you still will be able to access UI objects only through signals and slots.
I see one big issue in the presented code that is causing your freeze: You never let Qt process anything while you are in the loop. You need to allow Qt to run it's event loop. The easiest way is to use QApplication::processEvents() inside the loop.
I'm also not a fan of a while(1) loop for a couple of reasons. The first of which is that it can eat machine cycles waiting for things to happen. I doubt you really need to run the code as fast as possible, you could probably get away with some sleeping in your loop.
The other issue is that it is hard to break out. A cleaner approach would be something like this
void MyClass::on_pushButton_ON_clicked()
{
MyClass::done = false; // this is a class attribute
while (!MyClass::done) {
QApplication::processEvents();
//...
}
}
void MyClass::on_pushButton_OFF_clicked()
{
MyClass::done = true;
}
I have been trying in various ways to make my program sleep for 10 seconds before running the next line of code.
this.SetContentView (Resource_Layout.Main)
let timer = new System.Timers.Timer(10000.0)
async{do timer.Start()}
this.SetContentView (Resource_Layout.next)
I can't get any solution to work.
If you want to use async rather than the more direct way (of creating a timer and setting the content view in the event handler of the timer), then you need something like this:
this.SetContentView (Resource_Layout.Main)
async{
do! Async.Sleep(10000.0)
this.SetContentView (Resource_Layout.next) }
|> Async.StartImmediate
The key points:
Using do! Async.Sleep you can block the execution of asynchronous computation
By moving the SetContentView call inside the async, it will happen after the sleep
Using Async.StartImmediate, you start the workflow - and the sleeping ensures that the rest of the computation runs in the same threading context (meaning that it will run on the UI thread and the code will be able to access UI elements).
I'm a pure Qt beginner and currently want to understand its basic concepts and how to use them in the right way. My question might appear as somehow "ragged" and I want to apologize for that in advance. This having said, the question is:
I want to implement a method, which "blocks" until the handling is finished (for what reason ever...). In order to do the internals, I have to use an asynchronous class, emmiting a signal when it has finished its job in the background. I want to use that in a blocking way:
QEventLoop myLoop;
workerClass x;
QObject::connect(x, SIGNAL(finished()), &myLoop, SLOT(quit()));
/* x is asnchrounous and will emit "finished" when ist job is done */
x.doTheJob();
myLoop.exec();
Will that work? According to some other postings around - it should. But what is going on exactly?
When a new instance of QEventLoop is just created, does it automatically mean, that signals emitted from myLoop are automatically handled within that new handler? Moreover, the loop starts some while after having started the job. What happens, when it emits already "finished", before myLopp is started?
I have been told by a collegue, that I could use
QEventLoop myLoop;
workerClass x;
QObject::connect(x, SIGNAL(finished()), &myLoop, SLOT(quit()));
/* doTheJob is a slot now */
QMetaObject::invokeMethod(x, "doTheJob", Qt::QueuedConnection);
myLoop.exec();
I understand, that in this case doTheJob is called from the event loop. But from which event loop? invokeMethod() is not told to invoke the method in a particular event loop (myLoop), so why should the event posted to myLoop instead into the "outer" loop? In fact, myLoop is not even running at that time...
I hope I was able to transport my question to be understandable by experts.
Many Thanks,
Michael
The key is QObject::connect:
QObject::connect(x, SIGNAL(finished()), &myLoop, SLOT(quit()));
This function says when finished() is emitted, myLoop.quit() should be called just after the event occurs. So in your example,
<--------thread A--------> | <--------thread B-------->
QEventLoop myLoop;
workerClass x;
x.doTheJob(); starts thread B
sleep in exec() workerClass::JobA();
sleeping..... workerClass::JobB();
sleeping..... workerClass::JobC();
sleeping..... workerClass::JobD();
sleeping..... workerClass::finished();
sleeping.....
wake up by quit();
myLoop.quit must be called after thread A is sleeping, otherwise thread A may sleep forever because quit may be called before sleeping. You have to find a way to specify this. But how? Take a look at QObject::connect, the last argument is
Qt::ConnectionType type = Qt::AutoConnection
And about Qt::AutoConnection,
(Default) If the receiver lives in the thread that emits the signal, Qt::DirectConnection is used. Otherwise, Qt::QueuedConnection is used.
The signal is emitted in thread B and the receiver myLoop is in thread A, that means you are using Qt::QueuedConnection:
The slot is invoked when control returns to the event loop of the receiver's thread. The slot is executed in the receiver's thread.
The slot myLoop.quit is invoked when control returns to the event loop, myLoop.exec. In other words, quit is invoked only when exec is running. This is exactly what we want and everything works fine. Therefore, your first example always runs correctly because the signal and slot are connecting together using Qt::QueuedConnection (from default value Qt::AutoConnection).
Your second example works fine because QObject::connect is the same, too.
If you change it to Qt::DirectConnection, the story is different. quit is called in thread B, it is possible that quit is called before exec and thread A is going to sleep forever. So in this scenario you should never use Qt::DirectConnection.
The story is about QObject::connect and threads. QMetaObject::invokeMethod is not related to this, it simply calls a function when the thread executes an exec function.
on your first part of the question: yes that will work.
There's actually a class for it in Qxt.
in order to understand how that works, you need to understand how event handling in general works, and this particular hack, while being a bad idea in generall, is a great way to learn it.
let's imaging a simplified call stack with just one mainloop:
6 myslot()
5 magic
4 someEvent.emit
3 eventloop.exec
2 qapplication.exec
1 main
as long as you don't return from myslot(), the "magic" cannot call any other slots that are connected to the same signal, and no other signal can be processed.
Now, with the nested eventloop "signalwaiter" hack:
10 myslot2()
9 magic
8 someEvent2.emit
7 eventloop.exec
6 myslot()
5 magic
4 someEvent.emit
3 eventloop.exec
2 qapplication.exec
1 main
we still cannot continue processing the event that we're blocking, but we can process new events, such as that signal you're connecting to quit.
This is not re-entrant, and generally a very bad idea. You may well end up with very hard to debug code, because you might be calling slots from a stack you don't expect.
The way to avoid the fire-before-exec problem, is to connect your signal to a slot that sets a flag. if the flag is set before exec, just don't exec.
However, for the sake of explanation, lets's see how your method works (it does work):
when you use Qt::QueuedConnection, a signal emission is not a stacked call anymore. It looks more like this:
5 eventloop.putQueue
4 someSignal.emit
3 eventloop.exec
2 qapplication.exec
1 main
then emit returns
3 eventloop.exec
2 qapplication.exec
1 main
and eventloop puts your slot on the stack
5 myslot()
4 eventloop.pop
3 eventloop.exec
2 qapplication.exec
1 main
So at the time your slot is called, your signal has already been removed from the stack and all local variables are gone.
this is vastly different than DirectConnection, and important to understand for other stuff like QObject::deleteLater()
To answer the question which eventloop executes your queue: the one on top of the stack.
I am doing this inside my RubyMotion application:
Dispatch::Queue.concurrent('google').async {
BubbleWrap::HTTP.get("http://google.com") do |response|
p response.body.to_str
end
}
This call does not complete.
However if I take the BubbleWrap code outside of the Dispatch queue, it completes fine.
BubbleWrap::HTTP is an abstraction over NSURLConnection, which depends on a RunLoop for its asynchronous processing. Unfortunately RunLoops aren't set up on GCD queues, and I haven’t figured out how to start a runloop on a GCD queue. When I've needed the above, I've resorted to instantiating a new NSThread, and started the run loop manually instead:
action = lambda do
runLoop = NSRunLoop.currentRunLoop
BW::HTTP.get("http://www.google.com") do |response|
NSLog("Fetched Google!")
end
runLoop.run
end
thread = NSThread.alloc.initWithTarget action, selector:"call", object:nil
thread.start
This answer is a rehash of my blogpost on the same topic and Cocoa In The Shell