When dealing with QNetworkReply, it is prescribed to use timers to abort the connection.
Here is my current code:
void ImageDownloader::download(QString imgUrl){
this->timeoutTimer = new QTimer(this);
this->timeoutTimer->setSingleShot(true);
this->timeoutTimer->setInterval(15000);
connect(this->timeoutTimer, SIGNAL(timeout()), this, SLOT(timeout()));
QUrl requestUrl(imgUrl);
QNetworkRequest nwRequest(requestUrl);
this->imageNwReply = this->nam->get(nwRequest);
connect(imageNwReply,SIGNAL(finished()),this,SLOT(imgDownloaded()));
connect(imageNwReply, SIGNAL(downloadProgress(qint64,qint64)), this->timeoutTimer, SLOT(start()));
this->timeoutTimer->start();
}
void ImageDownloader::timeout(){
qDebug()<<__FUNCTION__<<" Forced timeout!";
this->imageNwReply->abort();
}
The confusion I am facing is when should I start the timer? At times I have to make around 50 concurrent Get requests from QNetworkAccessManager but since there is throttling for maximum concurrent connections, at times it happens that some of the requests get timed out even before they have been processed.
Is there a signal to know exactly when the processing for a request is started by QNeworkAccessManager so that I can start the corresponding timer only then?
One possible solution might be to implement a Queue of requests and have only the maximum possible connections to process but I am looking for a cleaner solution
There is an open bug/enhancement request for the issue. I got to know about this bug from Qt forum
Related
I'm trying to maintain a persistent connection between client and Remote server using Qt. My sever side is fine. I'm doing my client side in Qt. Here I will be using QNetworkAccessManager for requesting server with get method(Part of QNetworkRequest method). I will be able to send and receive requests.
But after sometime(approx ~ 2 min)the client is intimating server, the connection has closed by posting a request automatically. I think QNetworkAccessManager is setting a timeout for this connection. I want to maintain a persistent connection between the ends.
Is my approach correct, if not, can someone guide me in correct path?
This question is interesting, so let's do some research. I set up a nginx server with big keep alive timeout and wrote the simpliest Qt application:
QApplication a(argc, argv);
QNetworkAccessManager manager;
QNetworkRequest r(QUrl("http://myserver/"));
manager.get(r);
return a.exec();
Also I used the following command (in Linux console) to monitor connections and check if the problem reproduces at all:
watch -n 1 netstat -n -A inet
I took quick look at Qt sources and found that it uses QTcpSocket and closes it in QHttpNetworkConnectionChannel::close. So I opened the debugger console (Window → Views → Debugger log in Qt Creator) and added a breakpoint while the process was paused:
bp QAbstractSocket::close
Note: this is for cdb (MS debugger), other debuggers require other commands. Another note: I use Qt with debug info, and this approach may not work without it.
After two minutes of waiting I got the backtrace of the close() call!
QAbstractSocket::close qabstractsocket.cpp 2587 0x13fe12600
QHttpNetworkConnectionPrivate::~QHttpNetworkConnectionPrivate qhttpnetworkconnection.cpp 110 0x13fe368c4
QHttpNetworkConnectionPrivate::`scalar deleting destructor' untitled 0x13fe3db27
QScopedPointerDeleter<QObjectData>::cleanup qscopedpointer.h 62 0x140356759
QScopedPointer<QObjectData,QScopedPointerDeleter<QObjectData>>::~QScopedPointer<QObjectData,QScopedPointerDeleter<QObjectData>> qscopedpointer.h 99 0x140355700
QObject::~QObject qobject.cpp 863 0x14034b04f
QHttpNetworkConnection::~QHttpNetworkConnection qhttpnetworkconnection.cpp 1148 0x13fe35fa2
QNetworkAccessCachedHttpConnection::~QNetworkAccessCachedHttpConnection untitled 0x13fe1e644
QNetworkAccessCachedHttpConnection::`scalar deleting destructor' untitled 0x13fe1e6e7
QNetworkAccessCachedHttpConnection::dispose qhttpthreaddelegate.cpp 170 0x13fe1e89e
QNetworkAccessCache::timerEvent qnetworkaccesscache.cpp 233 0x13fd99d07
(next lines are not interesting)
The class responsible for this action is QNetworkAccessCache. It sets up timers and makes sure that its objects are deleted when QNetworkAccessCache::Node::timestamp is in the past. And these objects are HTTP connections, FTP connections, and credentials.
Next, what is timestamp? When the object is released, its timestamp is calculated in the following way:
node->timestamp = QDateTime::currentDateTime().addSecs(ExpiryTime);
And ExpiryTime = 120 is hardcoded.
All involved classes are private, and I found no way to prevent this from happening. So it's way simpler to send keep-alive requests each minute (at least now you know that 1 minute is safe enough), as the alternative is to rewrite Qt code and compile custom version.
I'd say by definition, a 2 minute timeout connection qualifies for persistent. I mean if it wasn't persistent, you'd have to reconnect on every request. 2 minutes is quite generous comparing to some other software out there. But it is set to eventually timeout after a period of inactivity and that's a good thing which should not surprise. Some software allows the timeout period to be changed, but from Pavel's investigation it would appear in the case of Qt the timeout is hardcoded.
Luckily the solution is simple, just rig a timer to send a heartbeat (just a dummy request, do not confuse with "heartbeat network") every 1 minute or so to keep the connection alive. Before you use your connection deactivate the timer, and after you are done with the connection, restart the timer.
Please advise how can I setup default connection timeout without using blocking waitForConnected() method? I noticed that socket emit error signal (QAbstractSocket::SocketTimeoutError) after about 60 seconds that I can handle as a timeout, but can this timeout be adjusted?
You could use a QTimer:
Start it after you have called connectToHost.
You may want to reset the timer when socket state changes from QAbstractSocket::HostLookupState to QAbstractSocket::ConnectingState, perhaps with different timeout, if you want more fine-grained control on when exactly will the timeout happen.
If you get connection, stop the timer, or reset it for similar send/receive timeout use.
If you get timer timeout, do disconnectFromHost() and possibly do something like retry when you get disconnect signal.
When developing, make sure to connect the stateChanged(...) and error(...) signals at least to debug slots, which just print the arguments. That way you will see when something happens in a way you did not expect.
I'm trying to understand the idea of non-blocking web server and it seems like there is something I miss.
I can understand there are several reasons for "block" web request(psuedocode):
CPU bound
string on_request(arg)
{
DO_SOME_HEAVY_CPU_CALC
return "done";
}
IO bound
string on_request(arg)
{
DO_A_CALL_TO_EXTERNAL_RESOURCE_SUCH_AS_WEB_IO
return "done";
}
sleep
string on_request(arg)
{
sleep(VERY_VERY_LONG_TIME);
return "done";
}
are all the three can benefit from non-blocking server?
how the situation that do benefit from the non-blocking web server really do that?
I mean, when looking at the Tornado server documentation, it seems
like it "free" the thread. I know that a thread can be put to sleep
and wait for a signal from the operation system (at least in Linux),
is this the meaning of "freeing" the thread? is this some higher
level implementation? something that actually create a new thread
that is waiting for new request instead of the "sleeping" one?
Am I missing something here?
Thanks
Basically the way the non-blocking sockets I/O work is by using polling and the state machine. So your scheme for many connections would be something like that:
Create many sockets and make them nonblocking
Switch the state of them to "connect"
Initiate the connect operation on each of them
Poll all of them until some events fire up
Process the fired up events (connection established or connection failed)
Switch the state those established to "sending"
Prepare the Web request in a buffer
Poll "sending" sockets for WRITE operation
send the data for those who got the WRITE event set
For those which have all the data sent, switch the state to "receiving"
Poll "receiving" sockets for READ operation
For those which have the READ event set, perform read and process the read data according to the protocol
Repeat if the protocol is bidirectional, or close the socket if it is not
Of course, at each stage you need to handle errors, and that the state of each socket is different (one may be connecting while another may be already reading).
Regarding polling I have posted an article about how different polling methods work here: http://www.ulduzsoft.com/2014/01/select-poll-epoll-practical-difference-for-system-architects/ - I suggest you check it.
To benefit from a non-blocking server, your code must also be non-blocking - you can't just run blocking code on a non-blocking server and expect better performance. For example, you must remove all calls to sleep() and replace them with non-blocking equivalents like IOLoop.add_timeout (which in turn involves restructuring your code to use callbacks or coroutines).
How To Use Linux epoll with Python http://scotdoyle.com/python-epoll-howto.html may give you some points about this topic.
I use qt4.8.5 and windows.
I use QTcpSocket to connect a server.
If the server doesn't open, then I will try to connect it.
My way is:
sock = new QTcpSocket(this);
sock->connectToHost("127.0.0.1", 1234);
connect(sock, SIGNAL(disconnected()), this, SLOT(disconnected()));
connect(sock, SIGNAL(error(QAbstractSocket::SocketError)),
this, SLOT(error(QAbstractSocket::SocketError)));
I will get the error() signal.
sock->close();
sock->deletelater();
I use a timer to try to connect, and it did call the error() to delete the sock.
But I found that the handles increase in the Task Manager. Does it have memory leak? but I have close the sock and use deletelater().
What's more, the socket object not run in the main thread.
If any one know the reason cause the handles increase, please tell me.
Thanks!
As far as I understand from the documentation the QUdpSocket are async but, still, reading from the socket is performed on the UI thread when the signal readyRead() is emitted. Is that correct? Is that safe from the performance perspective? Should I read the socket's content in another thread?
Everything depends if you will have a lot of data to process. I've used the TCP socket with avarage transfer ~20mbps without blocking gui. Best thing would be check it yourself. I think you won't have to move socket to other thread