I am trying to download 1.4 mb file using tinygsm library, and for that i am using FileDownload example. However my device downloads 96,000 bytes and then stops.
After debugging the program i found out that client.connected() condition is turning false and that is why i am not be able to download the file further.
Hardware being used is esp32 and sim800c module.
I am sharing the code snippet below, Please guide me resolving this issue.
TinyGsmClientSecure client(modemGSM);
if (client.connect(host, port)) {
SerialDebug.println("connected");
// Make a HTTP GET request:
SerialDebug.println("Performing HTTP GET request...");
client.print(String("GET ") + path + " HTTP/1.1\r\n");
client.print(String("Host: ") + host + "\r\n");
client.print("Connection: close\r\n\r\n");
client.println();
}else {
SerialDebug.println("connection failed");
return;
}
while (client.connected() && millis() - timeout < 10000L) {
while (client.available()) {
char c = client.read();
readBytes++;
if(readBytes % 1000 == 0)
SerialDebug.println(readBytes);
timeout = millis();
}
}
The connection can break. You should monitor it and try to reconnect.
You also should use HTTP Get request with Range header in order to resume the download.
https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests#single_part_ranges
Related
So I've been struggling to have this working.
I have an ESP32, but I can't use the wifi on the module (I simply can't, due to the implementation of the ESP).
Now, I have an W5500 shield for arduino, but I figure, that this should work just fine on an ESP, since it communicates via SPI.
I have had code that works perfectly with the Wifi of the ESP, but now it needs to run on the Ethernet thing.
HTTPClient http;
void sendHeartBeat(void *pvParameters) {
// Infinite loop (while true is more risky for some reason)
unsigned long previousMillis = 0;
const long interval = 300000; // 5 minutes
for (;;) {
unsigned long currentMillis = millis();
if (currentMillis - previousMillis >= interval) {
previousMillis = currentMillis; // reset timer
int httpResponseCode;
http.begin(DIGITAL_HQ_HEARTBEAT_ENDPOINT); // test environment
http.addHeader("Content-Type", "application/json");
http.setUserAgent(DIGITAL_HQ_USER_AGENT);
StaticJsonDocument<128> doc;
doc["mac"] = deviceMacAddress;
doc["key"] = DEVICE_SECRET;
doc["type"] = DIGITAL_HQ_SOFTWARE_TYPE;
String output;
serializeJson(doc, output);
httpResponseCode = http.POST(output);
// If the device was deleted from HQ, re-register it.
if (httpResponseCode == 202) {
registerDevice();
}
http.end();
}
}
}
The above code worked perfectly on the ESP when using wifi. Now it doesn't anymore and just crashes the whole device.
This is the stacktrace;
16:20:52.874 -> assertion "Invalid mbox" failed: file "/home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/api/tcpip.c", line 374, function: tcpip_send_msg_wait_sem
16:20:53.110 -> abort() was called at PC 0x400dced7 on core 1
16:20:53.143 ->
16:20:53.143 -> ELF file SHA256: 0000000000000000
16:20:53.176 ->
16:20:53.176 -> Backtrace: 0x400891f4:0x3ffb1a00 0x4008946d:0x3ffb1a20 0x400dced7:0x3ffb1a40 0x401122bb:0x3ffb1a70 0x40111d55:0x3ffb1aa0 0x40111f08:0x3ffb1ac0 0x40108f10:0x3ffb1b00 0x400d4fbd:0x3ffb1b20 0x400d4cee:0x3ffb1da0 0x400d4e09:0x3ffb1dd0 0x40126337:0x3ffb1df0 0x400d783e:0x3ffb1e10 0x400d78d1:0x3ffb1e40 0x400d7a63:0x3ffb1e90 0x400d7a83:0x3ffb1eb0 0x400d284e:0x3ffb1ed0 0x400d9fd1:0x3ffb1fb0 0x4008a472:0x3ffb1fd0
What am I doiçng wrong? How can I resolve this issue without too much hassle? I also have this on any other used library (e.g. MQTTClient.h, WiFiClientSecure.h). Those I use and I really don't want to re-write all of the code just for the sake of 'I use an rj45 connector instead of wifi'....
Edit:
The issue was with https, but I need to be able to use https urls also... When I use http, the request just doesn't hapen either.
I'm using a NodeMCU and want to log data to my local Web2Py server.
The request: "http://minion.local:8000/ardulog/default/add/6476366/45643" works fine from the browser and returns a record id.
My Arduino can connect to my server but don't get any return data error or otherwise and nothing appears in my database.
// This will send the request to the server
samptime = millis();
rpm = (samptime + 333) % 96789;
String request = "10.0.0.244:8000/ardulog/default/add/"+String(samptime)+"/"+String(rpm)+" HTTP/1.1";
Serial.println("\ntrying: ");
Serial.println("GET " + request);
Serial.println("host: minion.local");
client.println("GET " + request);
client.println("host: minion.local");
// if there are incoming bytes available
// from the server, read them and print them:
while (client.available()) {
char c = client.read();
Serial.print(c);
}
Serial.println("closing connection");
client.stop();
I've tried every variant I can think of but get only the following:
connecting to minion.local
[hostByName] request IP for: minion.local
[hostByName] Host: minion.local IP: 10.0.0.244
Requesting:
GET 10.0.0.244:8000/ardulog/default/add/112725/16269 HTTP/1.1
host: minion.local
closing connection
wait 5 sec...
Why am I not reading anything retuned from the server?
SOLVED! Though I was unsuccessful POSTing to Google Sheets, simply changing the word from GET to POST worked with Web2Py without sending any body data:
if(client.connect(host,port))
client.println("POST /ardulog/default/add/" + String(samptime)+ "/" + String(rpm) + " HTTP/1.1");
(still not receiving a result page from the server though)
So I am using the Arduino Uno and the ESP8266 module. I am trying to get the module to send an HTTP get request using the AT serial firmware however I am not quite able to do so. The project gets data from a sensor and sends it in real time using the Pubnub Service by posting the data to the pubnub rest API.
I have the function:
void sendSerialCommand(String cmd, int t) {
int i=0;
while(1) {
Serial.println(cmd);
client.println(cmd);
while(client.available()) {
if(client.find("OK"))
i=8;
}
delay(t);
if(i>5)
break;
i++;
}
if(i==8)
Serial.println("OK");
else
Serial.println("Error");
}
Using this function, I set up a connection with the server as :
sendSerialCommand("AT+CIPMUX=1",100);
sendSerialCommand("AT+CIPSTART=0,\"TCP\",\"" + pubnubIP + "\",80",1000);
Serial.println("Established connection with DashBoard Servers.....");
Then I try:
sendSerialCommand("AT+CIPSEND=0," + String(getRequestLength) + "," + "http://" + pubnubIP + "/publish/<publish-key>/<sub-key>/0/water/0/<data>", 1000);
(I have put in the values for the publish and sub key in the actual code.)
The getRequestLength is the length of the data to be sent.
When I run the code, the output says wait..... and the the connection just closes. Could anyone please tell me the correct Syntax or method of sending the data?
I am using the ESP8266 with Arduino IDE and have set up a server at port 200 . The IP is also defined as the same as 192.168.1.100.
ESP8266WebServer server(200);
IPAddress ip(192, 168, 1, 100); //Node static IP
IPAddress gateway(192, 168, 1, 1);
IPAddress subnet(255, 255, 255, 0);
server.on("/parseIFTTT", parseIFTTT);
void parseIFTTT() {
String message;
message += server.args();
message += "\n";
for (uint8_t i = 0; i < server.args(); i++) {
message += " " + server.argName(i) + ": " + server.arg(i) + "\n";
}
Serial.println(message);
Serial.println(server.argName(0));
Serial.println(server.arg(0));
server.send(200, "text/plain", "Success " + message);
}
is done to route the parseIFTTT request to this parseIFTTT() method.
I have done port forwarding and using the duckdns to access this server from outside.
This is my duckdns address
http://xxxxxx.duckdns.org:200/parseIFTTT
When I make a POST using a POSTMAN tool with content type as text/plain , the body contents are shown in the serial monitor as
plain
--body contents--
But when the same request is made from IFTTT the serial monitor shows nothing but plain as empty .Initially I felt the issue is with IFTTT.
But that is not the issue as when I use the WiFiWebServer example in the arduino , using the following code
String req = client.readString();
Serial.println(req);
client.flush();
I see the data from IFTTT as :
POST /parseIFTTT HTTP/1.1
Content-type: text/plain
host: xxxxxx.duckdns.org:200
content-length: 27
x-newrelic-id: XAMGV15QGwQJVllRDgQ=
x-newrelic-transaction: PxQFA1NbAQQJVwJWA1dSB0YdUFIOFQZOEgEPVA5ZBFYGXAwECFgFAFcUG0MHUwoLBAcDAxVs
Connection: close
{"value":"test data from IFTTT"}
So I believe I am doing something wrong with the server.args(). I am under the impression that server.args() should give the body contents used in the POST whether contentType is text/plain or x-www-form-urlencoded.
Am I doing something wrong or with the server.args() can't we get the body data from the POST request ?
There are a few 'gotchas' in the ESP8266WebServer's implementation. I have found the body shows up in the
server.arg("plain")
but only if the class cannot find any key value pairs. The ESP8266WebServer will look for an '=' and only if it cannot find one will it put the body in the "plain" arg.
This arg will contain the full body so you will have to parse the JSON yourself. I have found ArduinoJson to be a very easy to use library to do so.
Short example:
void handleRequest() {
StaticJsonBuffer<200> jsonBuffer;
JsonObject& root = jsonBuffer.parseObject(server.arg("plain"));
root.printTo(Serial);
}
On a sidenote. If you are testing with a POSTMAN like tool, do not forget to set the 'Content-length'. ESP8266WebServer will treat your body as empty (or of a different length) if it does not correspond with this header value.
Hope this answers your question.
Thanks the code solution for " how to get the IP client using ESP8266WebServer" is :
// HTTP Request Path, IRIA Entry Form
server.on("/", [](){
// As sending the form as a response to the client
server.send(200, "text/html",login_CASA);
String addy = server.client().remoteIP().toString();
Serial.println(addy);
});
Using Qt5, how to simply check if given url is available?
Not using special functions for signal slots, but simply using something like bool isUrlAvailable(QString url), are there any function like this?
Update QUrl.isValid() is incorrect answer, it is just checks if url is correctly formed.
Update 2 QUrl.host() is incorrect answer too, it is just returns host part of given url, it does not check for its availability.
Update 3 pinging host is also incorrect, because url may be available, but does not accept icmp echo (=ping)
Yes, you can do a HEAD request to a given URL.
bool urlExists (QString url_string) {
QUrl url(url_string);
QTcpSocket socket;
socket.connectToHost(url.host(), 80);
if (socket.waitForConnected()) {
socket.write("HEAD " + url.path().toUtf8() + " HTTP/1.1\r\n"
"Host: " + url.host().toUtf8() + "\r\n\r\n");
if (socket.waitForReadyRead()) {
QByteArray bytes = socket.readAll();
if (bytes.contains("200 OK")) {
return true;
}
}
}
return false;
}
This is just an example for 200 OK and you might also want to check if the status code is some other in 2XX or in 3XX (redirection) class.
So taking from pajaja + a few other SO answers + a tutorial I found
(http://www.blikoon.com/networking/http-potocol-writting-a-simple-client-using-qt-qtcpsocket-and-troubleshooting-using-telnet)
I came up with this tweaked version because the one above didn't work
bool urlExists(QUrl theurl){
QTextStream out(stdout);
QTcpSocket socket;
QByteArray buffer;
socket.connectToHost(theurl.host(), 80);
if (socket.waitForConnected()) {
//Standard http request
socket.write("GET / HTTP/1.1\r\n"
"host: " + theurl.host().toUtf8() + "\r\n\r\n");
if (socket.waitForReadyRead()) {
while(socket.bytesAvailable()){
buffer.append(socket.readAll());
int packetSize=buffer.size();
while(packetSize>0)
{
//Output server response for debugging
out << "[" << buffer.data() << "]" <<endl;
//set Url if 200, 301, or 302 response given assuming that server will redirect
if (buffer.contains("200 OK") ||
buffer.contains("302 Found") ||
buffer.contains("301 Moved")) {
return true;
}
buffer.remove(0,packetSize);
//packetSize=getPacketSize(buffer);
packetSize=buffer.size();
} //while packet size >0
} //while socket.bytesavail
} //socket wait for ready read
}//socket write
return false;
}
The QTextStream prints what is being read from the socket so you can know what conditions to add and why your http request didn't work (I used it to figure out that I needed 301 and 302). The while loops are a modified version of ratchetfreak's answer here
How to read complete data in QTcpSocket?
to make sure you get everything out of the socket. I'm testing if I need to change the "/index.html" part of the socket write but so far it seems fine.
Edit: Should just be "GET /" not "GET /index.html"