How in Qt5 to check if url is available? - qt

Using Qt5, how to simply check if given url is available?
Not using special functions for signal slots, but simply using something like bool isUrlAvailable(QString url), are there any function like this?
Update QUrl.isValid() is incorrect answer, it is just checks if url is correctly formed.
Update 2 QUrl.host() is incorrect answer too, it is just returns host part of given url, it does not check for its availability.
Update 3 pinging host is also incorrect, because url may be available, but does not accept icmp echo (=ping)

Yes, you can do a HEAD request to a given URL.
bool urlExists (QString url_string) {
QUrl url(url_string);
QTcpSocket socket;
socket.connectToHost(url.host(), 80);
if (socket.waitForConnected()) {
socket.write("HEAD " + url.path().toUtf8() + " HTTP/1.1\r\n"
"Host: " + url.host().toUtf8() + "\r\n\r\n");
if (socket.waitForReadyRead()) {
QByteArray bytes = socket.readAll();
if (bytes.contains("200 OK")) {
return true;
}
}
}
return false;
}
This is just an example for 200 OK and you might also want to check if the status code is some other in 2XX or in 3XX (redirection) class.

So taking from pajaja + a few other SO answers + a tutorial I found
(http://www.blikoon.com/networking/http-potocol-writting-a-simple-client-using-qt-qtcpsocket-and-troubleshooting-using-telnet)
I came up with this tweaked version because the one above didn't work
bool urlExists(QUrl theurl){
QTextStream out(stdout);
QTcpSocket socket;
QByteArray buffer;
socket.connectToHost(theurl.host(), 80);
if (socket.waitForConnected()) {
//Standard http request
socket.write("GET / HTTP/1.1\r\n"
"host: " + theurl.host().toUtf8() + "\r\n\r\n");
if (socket.waitForReadyRead()) {
while(socket.bytesAvailable()){
buffer.append(socket.readAll());
int packetSize=buffer.size();
while(packetSize>0)
{
//Output server response for debugging
out << "[" << buffer.data() << "]" <<endl;
//set Url if 200, 301, or 302 response given assuming that server will redirect
if (buffer.contains("200 OK") ||
buffer.contains("302 Found") ||
buffer.contains("301 Moved")) {
return true;
}
buffer.remove(0,packetSize);
//packetSize=getPacketSize(buffer);
packetSize=buffer.size();
} //while packet size >0
} //while socket.bytesavail
} //socket wait for ready read
}//socket write
return false;
}
The QTextStream prints what is being read from the socket so you can know what conditions to add and why your http request didn't work (I used it to figure out that I needed 301 and 302). The while loops are a modified version of ratchetfreak's answer here
How to read complete data in QTcpSocket?
to make sure you get everything out of the socket. I'm testing if I need to change the "/index.html" part of the socket write but so far it seems fine.
Edit: Should just be "GET /" not "GET /index.html"

Related

Web2Py GET requests from Arduino (ESP8266WiFi)

I'm using a NodeMCU and want to log data to my local Web2Py server.
The request: "http://minion.local:8000/ardulog/default/add/6476366/45643" works fine from the browser and returns a record id.
My Arduino can connect to my server but don't get any return data error or otherwise and nothing appears in my database.
// This will send the request to the server
samptime = millis();
rpm = (samptime + 333) % 96789;
String request = "10.0.0.244:8000/ardulog/default/add/"+String(samptime)+"/"+String(rpm)+" HTTP/1.1";
Serial.println("\ntrying: ");
Serial.println("GET " + request);
Serial.println("host: minion.local");
client.println("GET " + request);
client.println("host: minion.local");
// if there are incoming bytes available
// from the server, read them and print them:
while (client.available()) {
char c = client.read();
Serial.print(c);
}
Serial.println("closing connection");
client.stop();
I've tried every variant I can think of but get only the following:
connecting to minion.local
[hostByName] request IP for: minion.local
[hostByName] Host: minion.local IP: 10.0.0.244
Requesting:
GET 10.0.0.244:8000/ardulog/default/add/112725/16269 HTTP/1.1
host: minion.local
closing connection
wait 5 sec...
Why am I not reading anything retuned from the server?
SOLVED! Though I was unsuccessful POSTing to Google Sheets, simply changing the word from GET to POST worked with Web2Py without sending any body data:
if(client.connect(host,port))
client.println("POST /ardulog/default/add/" + String(samptime)+ "/" + String(rpm) + " HTTP/1.1");
(still not receiving a result page from the server though)

server.args() ESP8266 Arduino

I am using the ESP8266 with Arduino IDE and have set up a server at port 200 . The IP is also defined as the same as 192.168.1.100.
ESP8266WebServer server(200);
IPAddress ip(192, 168, 1, 100); //Node static IP
IPAddress gateway(192, 168, 1, 1);
IPAddress subnet(255, 255, 255, 0);
server.on("/parseIFTTT", parseIFTTT);
void parseIFTTT() {
String message;
message += server.args();
message += "\n";
for (uint8_t i = 0; i < server.args(); i++) {
message += " " + server.argName(i) + ": " + server.arg(i) + "\n";
}
Serial.println(message);
Serial.println(server.argName(0));
Serial.println(server.arg(0));
server.send(200, "text/plain", "Success " + message);
}
is done to route the parseIFTTT request to this parseIFTTT() method.
I have done port forwarding and using the duckdns to access this server from outside.
This is my duckdns address
http://xxxxxx.duckdns.org:200/parseIFTTT
When I make a POST using a POSTMAN tool with content type as text/plain , the body contents are shown in the serial monitor as
plain
--body contents--
But when the same request is made from IFTTT the serial monitor shows nothing but plain as empty .Initially I felt the issue is with IFTTT.
But that is not the issue as when I use the WiFiWebServer example in the arduino , using the following code
String req = client.readString();
Serial.println(req);
client.flush();
I see the data from IFTTT as :
POST /parseIFTTT HTTP/1.1
Content-type: text/plain
host: xxxxxx.duckdns.org:200
content-length: 27
x-newrelic-id: XAMGV15QGwQJVllRDgQ=
x-newrelic-transaction: PxQFA1NbAQQJVwJWA1dSB0YdUFIOFQZOEgEPVA5ZBFYGXAwECFgFAFcUG0MHUwoLBAcDAxVs
Connection: close
{"value":"test data from IFTTT"}
So I believe I am doing something wrong with the server.args(). I am under the impression that server.args() should give the body contents used in the POST whether contentType is text/plain or x-www-form-urlencoded.
Am I doing something wrong or with the server.args() can't we get the body data from the POST request ?
There are a few 'gotchas' in the ESP8266WebServer's implementation. I have found the body shows up in the
server.arg("plain")
but only if the class cannot find any key value pairs. The ESP8266WebServer will look for an '=' and only if it cannot find one will it put the body in the "plain" arg.
This arg will contain the full body so you will have to parse the JSON yourself. I have found ArduinoJson to be a very easy to use library to do so.
Short example:
void handleRequest() {
StaticJsonBuffer<200> jsonBuffer;
JsonObject& root = jsonBuffer.parseObject(server.arg("plain"));
root.printTo(Serial);
}
On a sidenote. If you are testing with a POSTMAN like tool, do not forget to set the 'Content-length'. ESP8266WebServer will treat your body as empty (or of a different length) if it does not correspond with this header value.
Hope this answers your question.
Thanks the code solution for " how to get the IP client using ESP8266WebServer" is :
// HTTP Request Path, IRIA Entry Form
server.on("/", [](){
// As sending the form as a response to the client
server.send(200, "text/html",login_CASA);
String addy = server.client().remoteIP().toString();
Serial.println(addy);
});

Connecting Qt with SSL to a jetty server

I have some problems with connecting a qt client to an embedded jetty server.
At first, I use the following components:
Qt 4.4.3 (compiled with active openssl support)
jetty 8.8.1
java 6
I know, the versions are not most recent, but because of licencing issues and customer wishes I can not use newer one.
So, the scenario is that a qt client has to send http GET and POST requests to the jetty server. As long I use simple http with the QHttp object it works fine, the problems start when I switch to SSL.
My first try was to use the QSslSocket object for the GET request:
// Load certs + private key to socket
_pSocket = new QSslSocket(this);
_pSocket->setLocalCertificate(_certificate);
_pSocket->setPrivateKey(_privatekey);
_pSocket->addDefaultCaCertificate(_cacertificate);
connect (_pSocket, SIGNAL(encrypted()), this, SLOT(_encrypted()));
_pSocket->connectToHostEncrypted("localhost", 8000);
with the following slot function for the encrypted state:
void TestClient::_encrypted() {
QString _path("/testpath/list");
QByteArray buffer("GET ");
buffer.append(_path).append(" HTTP/1.1\r\n");
_pSocket->write(buffer);
}
Here I have my first problem:
This results in the following string, which is as far as I see compliant to RFC 2616:
"GET /testpath/list HTTP/1.1\r\n"
For some reason, the jetty server has a problem with that, keeping in a loop till the client close the connection because of a time out.
But if I use the following string, it works perfect:
"GET /testpath/list\r\n"
Here is my first question: Do you now an explanation for this behaviour ? I can live with it, but I want to know the reason
My second problem is the POST request, this fails always.
These examples I already tried:
"POST /testpath/receive/\r\n{"data":"hello world ?!"}\r\n"
"POST /testpath/receive/ HTTP/1.1\r\n{"data":"hello world ?!"}\r\n"
"POST /testpath/receive/\r\n\r\n{"data":"hello world ?!"}\r\n"
"POST /testpath/receive/ HTTP/1.1\r\n\r\n{"data":"hello world ?!"}\r\n"
I have the feeling, that the body is every time empty, so my server crashes because he tries to parse an empty string as json.
At least, the following log shows that:
2013-11-19 17:11:51.671, INFO, foo.bar.RepositoryHandler, qtp11155366-16 - /testpath/receive request type : /receive
2013-11-19 17:11:51.811, ERROR, foo.bar.RepositoryHandler, qtp11155366-16 - /testpath/receive missing or unknown elements in JSON request. Check JSON against documentation
2013-11-19 17:11:51.874, WARN, org.eclipse.jetty.server.AbstractHttpConnection, qtp11155366-16 - /testpath/receive /testpath/receive
java.lang.NullPointerException: null
at foo.bar.RepositoryHandler.decodeViewingRequest(RepositoryHandler.java:366) ~[MyServer.jar:na]
at foo.bar.RepositoryHandler.handle(RepositoryHandler.java:182) ~[MyServer.jar:na]
So, all together, I think I have several major errors in my requests. But which ?
My second try was to use the QHttp object and change the QSocket it uses with a QSslSocket I already initiated.
Here's the code of the main function:
QSslSocket* _pSocket;
QHttp* _pHttp;
int _id;
QBuffer* _pBuffer;
QByteArray _data;
_pSocket = new QSslSocket(this);
_pSocket->setLocalCertificate(_certificate);
_pSocket->setPrivateKey(_privatekey);
_pSocket->addDefaultCaCertificate(_cacertificate);
QUrl url;
url.setScheme("https");
url.setHost("localhost");
url.setPort(8001);
url.setPath("/testpath/receive");
connect (_pSocket, SIGNAL(encrypted()), this, SLOT(_encrypted()));
connect(_pHttp,SIGNAL(requestFinished(int,bool)),this,SLOT(_requestFinished(int,bool)));
connect(_pHttp,SIGNAL(done(bool)),this,SLOT(_done(bool)));
_pBuffer = new QBuffer(&_data);
_pHttp->setSocket(_pSocket);
_pSocket->connectToHostEncrypted(strHost, strPort.toInt());
_id = _pHttp->get(url.toString(),_pBuffer);
And the callbacks:
void _requestFinished(int id, bool error) {
if(id = _id)
qDebug() << "data=" << _data;
}
void _encrypted() {
qDebug() << "encrypted";
}
void _done(bool error) {
logInfo() << "_done";
if(_pHttp) {
_pHttp->abort();
delete _pHttp;
_pHttp = 0;
}
if(_pBuffer) {
delete _pBuffer;
_pBuffer = 0;
}
if(_pSocket) {
_pSocket->disconnectFromHost();
delete _pSocket;
_pSocket = 0;
}
}
I think, I only have to change the position of the _pHttp->get call, perhaps in the _encrypted callback, but I'm not sure.
Some good advise ?
Thanks,
Robert
Your HTTP request is incomplete, per RFC2616.
"GET /testpath/list HTTP/1.1\r\n"
That is invalid.
Try this instead.
"GET /testpath/list HTTP/1.1\r\n" + /* request line (required) */
"Host: localhost\r\n" + /* host header (required minimum) */
"\r\n" /* terminating CR + LF (required) */
As outlined in Section 5.1.2
The most common form of Request-URI is that used to identify a
resource on an origin server or gateway. In this case the absolute
path of the URI MUST be transmitted (see section 3.2.1, abs_path) as
the Request-URI, and the network location of the URI (authority) MUST
be transmitted in a Host header field. For example, a client wishing
to retrieve the resource above directly from the origin server would
create a TCP connection to port 80 of the host "www.w3.org" and send
the lines:
GET /pub/WWW/TheProject.html HTTP/1.1
Host: www.w3.org
The Request-URI line and Host header Header are mandated.

Logging into a remote website with QT5

I am attempting to create an application that can login to a website. The specific website is:
http://adfast.biz
Here is the code I am currently using:
void MainWindow::http_finish(QNetworkReply *reply)
{
int code = reply->attribute(QNetworkRequest::HttpStatusCodeAttribute).toInt();
if (code >= 300 && code < 400)
{ //HTTP 3XX codes are redirections
QUrl redirectTo = reply->attribute(QNetworkRequest::RedirectionTargetAttribute).toUrl();
reply->manager()->get(QNetworkRequest(redirectTo));
return;
}
if (reply->error() == QNetworkReply::NoError)
{
QString Msg = QString::fromUtf8(reply->readAll());
if (cdone == 0)
{//Runs only once, causing a reload of the main page
++cdone;
QUrl URL("http://adfast.biz");
QNetworkRequest QNR(URL);
reply->manager()->get(QNR);
QMessageBox::information(0,"1)" + QString::number(code),Msg);
return;
}
QMessageBox::information(0,"2)" + QString::number(code),Msg);
}
else
{
QMessageBox::information(0,"Error:",reply->errorString());
}
reply->deleteLater();
}
void MainWindow::on_Send_clicked()
{
QNetworkAccessManager* MNAM = new QNetworkAccessManager(this); //Stored within QNetworkReply->manager()
connect(MNAM,SIGNAL(finished(QNetworkReply*)),this,SLOT(http_finish(QNetworkReply*)));
QUrlQuery postData;
postData.addQueryItem("email","mail#mail.net");
postData.addQueryItem("senha","Password");
postData.addQueryItem("logar","ok");
QUrl URL(ui->TXT_Input->toPlainText());
URL.setQuery(postData);
QNetworkRequest QNR(URL);
QNR.setHeader(QNetworkRequest::ContentTypeHeader,"application/x-www-form-urlencoded");
MNAM->post(QNR,URL.toEncoded());
}
I am guessing that I either am sending the information the wrong way or that I need to possibly manage cookies? Both responses come back with HTTP status code: 200. The first comes with no source the second comes with the full web-source but is NOT logged in. I am positive that the user-data being sent is correct, but not that it is being sent correctly.
Edit:
I've changed a little, with no luck. First I have added a cookie-jar using:
QNetworkAccessManager* MNAM = new QNetworkAccessManager(this); //Stored within QNetworkReply->manager()
QNetworkCookieJar* cJar = new QNetworkCookieJar;
MNAM->setCookieJar(cJar);
connect(MNAM,SIGNAL(finished(QNetworkReply*)),this,SLOT(http_finish(QNetworkReply*)));
Then, I tested to see if any cookies are being received using the following code at the top of MainWindow::http_finish:
QList<QNetworkCookie> cookies = reply->manager()->cookieJar()->cookiesForUrl(QUrl("http://adfast.biz/"));
QMessageBox::information(0,"Cookies",QString::number(cookies.count()));
I want to add that the post is being sent to: http://adfast.biz/login (That is the value of: ui->TXT_Input->toPlainText()) But it seems that I can't get this to login at all. And I am not sure what I'm missing.
So, I did manage to login into this site with testguest#yahoo.com and mypassword12. But I can only offer very quick and dirty code, which explains how. The 'polishing' you must do yourself. :-)
QFile f("/tmp/cookie.txt");
f.open(QIODevice::ReadOnly);
QDataStream s(&f);
while(!s.atEnd()){
QByteArray c;
s >> c;
QList<QNetworkCookie> list = QNetworkCookie::parseCookies(c);
qDebug() << "eee" << list;
jar->insertCookie(list.at(0));
}
connect(MNAM,SIGNAL(finished(QNetworkReply*)),
this,SLOT(http_finish(QNetworkReply*)));
QUrlQuery postData;
postData.addQueryItem("email","testguest#yahoo.com");
postData.addQueryItem("senha","mypassword12");
postData.addQueryItem("logar","ok");
//QUrl URL("http://adfast.biz/");
//QUrl URL("http://adfast.biz/anuncios_telexfree");
URL.setQuery(postData);
QNetworkRequest QNR(URL);
QNR.setHeader(QNetworkRequest::ContentTypeHeader,
"application/x-www-form-urlencoded");
MNAM->get(QNR);
Above code reads the cookies, which are written in the http_finish slot below into /tmp/cookie.txt.
The first time you run this code, you must uncomment the first URL, the second time, when you have your cookies, the second URL. Ignore that I made your post into a get. I did it just for debugging reasons.
void MainWindow::http_finish(QNetworkReply *reply){
qDebug() << reply->readAll();
QList<QNetworkCookie> list =
MNAM->cookieJar()->cookiesForUrl(QUrl("http://adfast.biz/"));
QFile f("/tmp/cookie.txt");
f.open(QIODevice::ReadWrite);
for(int i = 0; i < list.size(); ++i){
QDataStream s(&f);
s << list.at(i).toRawForm();
}
f.close();
}
The code above writes the cookies from http://adfast.biz/ into /tmp/cookie.txt. Below an example of a cookie, which I received:
2-Sep-2013 16:41:39 GMT; domain=adfast.biz; path=/3 16:41:39 GMT;
domain=adfast.biz; path=/n=adfast.biz; path=/
Summary: You must connect to http://adfast.biz/ to get your cookie. When you have it, you must again connect, but this time to http://adfast.biz/anuncios_telexfree.
You may need to add a QNetworkCookieJar to your QNetworkAccessManager for keeping and sending cookies between requests and responses. You can set your cookie jar by using QNetworkAccessManager::setCookieJar(QNetworkCookieJar* cookieJar). Have a look at the requests being sent using HttpFox in Firefox and you can see if there are any cookies being sent back and forth.
Try to connect:
connect(&MNAM,SIGNAL(authenticationRequired(QNetworkReply*,QAuthenticator*)),
this,SLOT(provideAuthentication(QNetworkReply*,QAuthenticator*)));
And create a slot:
void TrackerClient::provideAuthentication(QNetworkReply *reply,
QAuthenticator *auth){
Q_UNUSED(reply);
auth->setUser(<your username>);
auth->setPassword(<your password>);
}
Ok, second try. The idea to add a cookiejar is correct, but just adding the jar is not enough. I tried the following with another site, which requires login. I did work. For your site I don't have login and password, and since it is not English, it is a little bit too hard for me for a quick help. :-)
How the login procedure works on "my" site.
You go to the main url, e.g. www.mysite.something.
The site asks you for a cookie. You have none.
Your are redirected (status 302 temporary moved) to a
page www.mysite.something/takelogin.php <--- example.
You enter your credentials. You already do this in your postdata.addQuery calls.
You send your post. If your creds are ok, the site sends you a cookie.
So far so good. What did you wrong? Now that you have the cookie you must again go to www.mysite.something. You are not automatically logged in and redirected to the main page of your site. And keep in mind, that cookies in QNetworkCookieJar are not stored on disk. They are only kept in memory. So, if your jar/MNAM is deleted, everything is gone.
You can easily store your received cookies to disk using the toRawForm() method of QNetworkCookie. To restore the cookie use the static QNetworkCooky::parseCookies(const QByteArray &cookieString) method.
Oh, almost forgot: The main url www.mysite.something did not send any cookies. I had to follow the redirect www.mysite.something/takelogin.php.
Disclaimer: It worked on my site. I don't know, if this is general for all sites, which require a login.
Checked with wrong creds: I was redirected to adfast.biz/login. This seems to be your redirect login page. More I cannot do without real creds.

Reading HTTP headers

I am trying to connect my application with a web service and here ,a user suggested to send custom headers back to my application.
I am using this code
void Coonnec::serviceRequestFinished(QNetworkReply *reply)
{
QByteArray bytes = reply->readAll();
if (reply->error() != QNetworkReply::NoError) {
qDebug() << "Reply error: " + reply->errorString();
}
else
{
qDebug() << "Uploaded: " + QDateTime::currentDateTime().toString();
qDebug() << reply->rawHeaderList();
}
reply->close();
bytes.clear();
reply->deleteLater();
}
from php i send this header
header('XAppRequest-Status: complete');
When running the application i can see that i get this header but i can't take the value of it cause
reply->rawHeader(bytes);
returns nothing.
How can i take the value 'complete'?
I suggest to connect a slot to the void QNetworkReply::metaDataChanged () signal of your reply.
The Qt doc says
This signal is emitted whenever the metadata in this reply changes.
metadata is any information that is not the content (data) itself,
including the network headers. In the majority of cases, the metadata
will be known fully by the time the first byte of data is received.
However, it is possible to receive updates of headers or other
metadata during the processing of the data.
I do use web-services/client with Qt and I noticed that some header's information are not available when I expected it to be ! I had to 'wait' for this signal to check the header content.

Resources