I have a Qt application that launches two threads from the main thread at start up. Both these threads make network requests using distinct instances of the QNetworkAccessManager object. My program keeps crashing about 50% of the times and I'm not sure which thread is crashing.
There is no data sharing or signalling occurring directly between the two threads. When a certain event occurs, one the threads signals the main thread, which may in turn signal the second thread. However, by printing logs, I am pretty certain the crash doesn't occur during the signalling.
The structure of both threads is as follows. There's hardly any difference between the threads except for the URL etc.
MyThread() : QThread() {
moveToThread(this);
}
MyThread()::~MyThread() {
delete m_manager;
delete m_request;
}
MyThread::run() {
m_manager = new QNetworkAccessManager();
m_request = new QNetworkRequest(QUrl("..."));
makeRequest();
exec();
}
MyThread::makeRequest() {
m_reply = m_manager->get(*m_request);
connect(m_reply, SIGNAL(finished()), this, SLOT(processReply()));
// my log line
}
MyThread::processReply() {
if (!m_reply->error()) {
QString data = QString(m_reply->readAll());
emit signalToMainThread(data);
}
m_reply->deleteLater();
exit(0);
}
Now the weird thing is that if I don't start one of the threads, the program runs fine, or at least doesn't crash in around 20 invocations. If both threads run one after the other, the program doesn't crash. The program only crashes about half the times if I start and run both the threads concurrently.
Another interesting thing I gathered from logs is that whenever the program crashes, the line labelled with the comment my log line is the last to be executed by both the threads. So I don't know which thread causes the crash. But it leads me to suspect that QNetworkAccessManager is somehow to blame.
I'm pretty blank about what's causing the crash. I will appreciate any suggestions or pointers. Thanks in advance.
First of all you're doing it wrong! Fix your threading first
// EDIT
From my own experience with this pattern i know that it may lead to many unclear crashes. I would start from clearing this thing out, as it may straighten some things and make finding problem clear. Also I don't know how do you invoke makeRequest. Also about QNetworkRequest. It is only a data structure so you don't need to make it on heap. Stack construction would be enough. Also you should remember (or protect somehow) from overwriting m_reply pointer. Do you call makeRequest more than once? If you do, then it may lead to deleting currently processed request after previous request finished.
What does happen if you call makeRequest twice:
First call of makeRequest assigns m_reply pointer.
Second call of makeRequest assigns m_reply pointer second time (replacing assigned pointer but not deleting pointed object)
Second request finishes before first, so processReply is called. deleteLater is queued at second
Somewhere in eventloop second reply is deleted, so from now m_reply pointer is pointing at some random (deleted) memory.
First reply finishes, so another processReply is called, but it operates on m_reply that is pointing a garbage, so every call at m_reply produces crash.
It is one of possible scenarios. That's why you don't get crash every time.
I'm not sure why do you call exit(0) at reply finish. It's also incorrect here if you use more then one call of makeRequest. Remember that QThread is interface to a single thread, not thread pool. So you can't call start() second time on thread instance when it is still running. Also if you're creating network access manager in entry point run() you should delete it in same place after exec(). Remember that exec() is blocking, so your objects won't be deleted before your thread exits.
Related
I have a question regarding how the init process in UNIX works. As i understand it the init process is the first to start and then other processes fork off it.
Say we start the init process then fork a child process which we call exec on with a new program which happens to cause the child to wait for some I/O input. Now the parent init process could wait on the child but if it did that then there are no other processes to be run. Conversely if the init process does not wait and instead falls into a waiting loop or something then when the child is resumed the parent is now taking up processor time doing nothing.
What is the best way to manage this problem? Should the init process simply always run an infinite loop and we not worry about the wasted resources? Or is there a better way.
Any help would be much appreciated,
Ben
Process 1 must never exit; many (all?) implementations of Unix will force a system crash if it does.
However, process 1 doesn't need to do anything more than this (I'm assuming the kernel opens fds 0, 1, and 2 on the console before transferring control to user space - check your kernel's documentation for that and other details of the bootstrap environment, if you're actually gonna write init yourself):
#include <sys/types.h>
#include <sys/wait.h>
#include <unistd.h>
#include <stdio.h>
int main(void)
{
pid_t child = fork();
if (child == -1) {
perror("init: fork");
return 1;
}
if (child == 0) {
execl("/etc/rc", "/etc/rc", (char*)0);
perror("/etc/rc");
return 1;
}
for (;;)
wait(0);
}
After starting /etc/rc it does go into an infinite loop, calling wait over and over again, and throwing away the results. But wait is a blocking system call. Each time it's called the kernel will take the CPU away from process 1 and give it to a process that has useful work to do; wait will only return when there is an exited child to report. (If there are no processes with useful work to do, the CPU will be put into a low-power "sleep" state until some external event, e.g. a human typing on the keyboard or a network packet arriving, gives a running process some work to do.)
With this minimal init, it is entirely /etc/rc's responsibility to start up all of the programs needed to make the computer do something useful, and those programs' responsibility to keep running as long as needed; if it should come to pass that every process other than this one exits, it'll just sleep in wait forever. More sophisticated implementations will do more, e.g. restarting network servers if they crash.
There is a solution for this: SIGCHLD. It's a signal that can be delivered to parent when a child changes its status (stops or exits). So the parent can goes sleeping (sigpause, sigsuspend for example) and will be interrupted when a child terminates, then the parent runs an appropriate signal handler to call one of the wait-family functions.
I would not worry about resources during init start up. Your server is booting up and not being used for it's intended purpose, therefore there is not a performance demand on it during that time.
I have never seen a process written to take standard input during the boot up process, although this is possible if you wanted to write one. I know the init scripts can be written with dependencies, depending on which distro you use and what exactly is the boot up process(upstart, system V init, etc). But by default, they run in a sync fashion on a order that the init uses. I am not sure how blocking that sync process...waiting for input would effect the system. Most likely, it would do just that....stop and wait for input before continuing.
The init process does indeed run an infinite loop but this doesn't use any significant resource as it is interrupt driven. It simply waits for processes to die or for other signals to be sent to it. During the wait intervals, zero CPU cycles are used by init.
I always use QObject::connect() in all my applications but it is not clear to me its effect when my program is currently inside a function. Suppose I have the following code:
void main() {
//other stuffs here
QObject::connect(xxx,SIGNAL(yyy()),this,SLOT(zzz());
}
void aFunction()
{
//a bunch of codes here
//i am here when suddenly signal is emitted from QObject::connect();
//another bunch of codes here
}
I assume that when the signal is emitted, QObject::connect leaves the function "aFunction()" to execute "zzz()". What will happen to the remaining codes in the "aFunction()"
Thanks.
I can understand the confusion, coming from procedural to event based programming gives me same experience like you do now.
Short answer:
in non multi threaded environment, slot zzz() will be executed after aFunction() finishes. In fact, the signal probably gets emitted after aFunction finishes.
in multi threaded env., same thing but it is "after some time", not after.
Key to understanding this is Event Loop. QApplication::exec() runs a forever loop polling for event. New event is then handled, signals get emitted and depending on the fifth argument of QObject::connect which is a Qt::ConnectionType, ultimately runs the connected slot. You can read QObject documentation for more detail..
So your aFunction probably gets called after some signal, so after it is finished, it's back to event loop again, then your 'suddenly emitted' signal actually gets emitted and zzz is executed.
Even in multi threading environment, inter thread signals and slots work with Qt::QueuedConnection which basically just posts the emitted signal to corresponding thread so that when that thread's event loop come around to process it - it will be executed sequentially as well.
Ultimately what you will have to remember is that this Turing Machine called computers is executing sequences of codes, whether it's semi paralel (e.g time sharing, pipelining) or truly paralel (e.g multi cores, multi cpu) the part where codes get sent (or distributed? or fetched?) to its execution will always run in sequences or in one or more ways have to be simulated to be sequential so that no code is executed twice in multiple execution node.
There is no "suddenly"
I've written a Qt console application to try out QSemaphores and noticed some strange behavior. Consider a semaphore with 1 resource and two threads getting and releasing a single resource. Pseudocode:
QSemaphore sem(1); // init with 1 resource available
thread1()
{
while(1)
{
if ( !sem.tryAquire(1 resource, 1 second timeout) )
{
print "thread1 couldn't get a resource";
}
else
{
sem.release(1);
}
}
}
// basically the same thing
thread2()
{
while(1)
{
if ( !sem.tryAquire(1 resource, 1 second timeout) )
{
print "thread2 couldn't get a resource";
}
else
{
sem.release(1);
}
}
}
Seems straightforward, but the threads will often fail to get a resource. A way to fix this is to put the thread to sleep for a bit after sem.release(1). What this tells me is that the release() member does not allow other threads waiting in tryAquire() access to the semaphore before the current thread loops around to the top of while(1) and grabs the resource again.
This surprises me because similar testing with QMutex showed proper behavior... i.e. another thread hanging out in QMutex::tryLock(timeout) gets notified properly when QMutex::unlock() is called.
Any ideas?
I'm not able to fully test this out or find all of the suppporting links at the moment, but here are a few observations...
First, the documentation for QSemaphore.tryAcquire indicates that the timeout value is in milliseconds, not seconds. So your threads are only waiting 1 millisecond for the resource to become free.
Secondly, I recall reading somewhere (I unfortunately can't remember where) a discussion about what happens when multiple threads are trying to acquire the same resource simultaneously. Although the behavior may vary by OS and situation, it seemed that the typical result is that it is a free-for-all with no one thread being given any more priority than another. As such, a thread waiting to acquire a resource would have just as much chance of getting it as would a thread that had just released it and is attempting to immediately reacquire it. I'm unsure if the priority setting of a thread would affect this.
So, why might you get different results for a QSemaphore versus a QMutex? Well, I think a semaphore may be a more complicated system resource that would take more time to acquire and release than a mutex. I did some simple timing recently for mutexes and found that on average it was taking around 15-25 microseconds to lock or unlock one. In the 1 millisecond your threads are waiting, this would be at least 20 cycles of locking and unlocking, and the odds of the same thread always reacquiring the lock in that time are small. The waiting thread is likely to get at least one bite at the apple in the time that it is waiting, so you won't likely see any acquisition failures when using mutexes in your example.
If, however, releasing and acquiring a semaphore takes much longer (I haven't timed them but I'm guessing they might), then it's more likely that you could just by chance get a situation where one thread is able to keep reacquiring the resource repeatedly until the wait condition for the waiting thread runs out.
A while ago I wrote a little RAII class to wrap the setOverrideCursor() and restoreOverrideCursor() methods on QApplication. Constructing this class would set the cursor and the destructor would restore it. Since the override cursor is a stack, this worked quite well, as in:
{
CursorSentry sentry;
// code that takes some time to process
}
Later on, I found that in some cases, the processing code would sometimes take a perceptible time to process (say more than half a second) and other times it would be near instantaneous (because of caching). It is difficult to determine before hand which case will happen, so it still always sets the wait cursor by making a CursorSentry object. But this could cause an unpleasant "flicker" where the cursor would quickly turn from the wait cursor to the normal cursor.
So I thought I'd be smart and I added a separate thread to manage the cursor override. Now, when a CursorSentry is made, it puts in a request to the cursor thread to go to the wait state. When it is destroyed it tells the thread to return to the normal state. If the CursorSentry lives longer than some amount of time (50 milliseconds), then the cursor change is processed and the override cursor is set. Otherwise, the change request is discarded.
The problem is, the cursor thread can't technically change the cursor because it's not the GUI thread. In most cases, it does happen to work, but sometimes, if I'm really unlucky, the call to change the cursor happens when the GUI thread gets mixed in with some other X11 calls, and the whole application gets deadlocked. This usually only happens if the GUI thread finishes processing at nearly the exact moment the cursor thread decides to set the override cursor.
So, does anyone know of a safe way to set the override cursor from a non-GUI thread. Keep in mind that most of the time, the GUI thread is going to be busy processing stuff (that's why the wait cursor is needed after all), so I can't just put an event into the GUI thread queue, because it won't be processed until its too late. Also, it is impractical to move the processing I'm talking about to a separate thread, because this is happening during a paint event and it needs to do GUI work when its done (figuring out what to draw).
Any other ideas for adding a delay to setting the override cursor would be good, too.
I don't think there is any other way besides a Signal-Slot connection going to the GUI thread followed by a qApp->processEvents() call, but like you said, this would probably not work well when the GUI thread is tied up.
The documentation for QCoreApplication::processEvents also has some recommended usages with long event processing:
This function overloads processEvents(). Processes pending events for
the calling thread for maxtime milliseconds or until there are no more
events to process, whichever is shorter.
You can call this function
occasionally when you program is busy doing a long operation (e.g.
copying a file).
Calling this function processes events only for the
calling thread.
If possible break up the long calls in the paint event and have it periodically check to see how long it has been taking. And in any of those checks, have it set the override cursor then from in the GUI Thread.
Often a QProgressBar can go a long way to convey the same information to the user.
Another option that could help quite a bit would be to render outside of the GUI thread onto a QImage buffer and then post it to the GUI when it is done.
I am new to QT. I have created object class QNetworkAccessManager main window as parent. Also registered to SIGNAL finished. It is working fine.
But I want to know in which thread it will run. Will it block the main thread. If i need to perform sequence of get operation how should I need to write the code.
Please give me some sample to understand concept properly.
It certainly does not run in the main thread, the calls to get() are asynchronous.
For example this would keep firing get requests:
while (condition) {
QNetworkRequest request;
request.setUrl(QUrl(m_ServerURL);
m_httpGetUpdatedFile->get(request);
}
You then have the slot for the finished signal which is processing the QNetworkReply. That slot basically should be getting called for each get request you make (even if it fails). If you need to keep track of when all your get requests have finished, you need to keep a track of how many you posted and then have your own finished flag or signal.
QNAM does use threads in the background, but this is completely invisible for your application code. Everything you see will run in the main thread.
QNAM works in the usual Qt way, it will emit signals when things happen, and you connect these signals to slots in your own code, which do things as much as they can. If they don't for example have enough data, then your slots must not block to wait for new data, instead they must return. And then they will be called again when/if there's more data (or you'll get another signal if for example connection was terminated).
Some links, in case you have not read these:
Qt signal & slot documentation
Qt networking documentation