WiFiClientSecure readString function slow - arduino

I have a sketch that requests a device token from Facebook in order to authenticate a Wemos D1 Mini.
All in all, I want to reproduce the following, which gives me a device code in less than a second:
curl https://graph.facebook.com/v2.7/device/login -d
"type=device_code&access_token=MYTOKEN"
I set up the following sketch which works albeit very slowly. It fetches the device token in about 17 seconds.
It seems that String response = client.readString(); is the culprit.
Can any offer advice on why that could be, and how to possibly remedy it?
Thank you kindly,
Nate
#include <ESP8266WiFi.h>
#include <WiFiClientSecure.h>
#include <ArduinoHttpClient.h>
#include <ArduinoJson.h>
WiFiClientSecure client; //edited
void setup() {
connect(WIFI_SSID, WIFI_PWD); //edited
Serial.println(getFBDeviceToken());
}
void connect(const char* WIFI_SSID, const char* WIFI_PWD) {
delay(1000);
WiFi.persistent(false);
delay(1000);
WiFi.mode(WIFI_STA);
if (WiFi.status() == WL_DISCONNECTED) {
WiFi.begin(WIFI_SSID, WIFI_PWD);
while(WiFi.status() != WL_CONNECTED){
delay(1000);
Serial.println("connecting...");
}
Serial.println("\r\n"+WiFi.localIP());
}
}
String getFBDeviceToken(){
//Connect to FB SSL
if(!client.connect(host, httpPort)){
return "** Failed to connect **";
}
client.println("POST " + path + " HTTP/1.1");
client.println("Host: " + String(host));
client.println("User-Agent: ESP8266/1.0");
client.println("Connection: close");
client.println("Content-Type: application/x-www-form-urlencoded;");
client.print("Content-Length: ");
client.println(data.length());
client.println();
client.println(data);
String response = client.readString();
int bodypos = response.indexOf("\r\n\r\n") + 4;
DynamicJsonBuffer jsonBuffer;
JsonObject& root = jsonBuffer.parseObject(response.substring(bodypos));
String device_token = root[String("user_code")];
return device_token;
}

I am not sure if this is still relevant for you but I faced the same situation and I believe I found a solution. If it is not relevant to you, perhaps it will help some other, especially given that the default WiFiClientSecure library will soon be switched to BearSSL and no support is currently being given for the existing implementation.
Although I did not manage to speed up readString function, I used the WiFiClientSecure::read(uint8_t *buf, size_t size) function to get the data from the server:
// Buffer size, 128 bytes in this case
#define RESP_BUFFER_LENGTH 128
// Pointer to actual buffer
uint8_t * _buffer = new uint8_t[RESP_BUFFER_LENGTH];
// String to hold the final response
String _responseString = "";
// If info is still available
while (wifiClient.available())
{
// Fill the buffer and make a note of the total read length
int actualLength = wifiClient.read(_buffer, RESP_BUFFER_LENGTH);
// If it fails for whatever reason
if(actualLength <= 0)
{
// Handle as you see fit
return -1;
}
// Concatenate the buffer content to the final response string
// I used an arduino String for convenience
// but you can use strcat or whatever you see fit
_responseString += String((char*)_buffer).substring(0, actualLength);
}
// Clear buffer memory
delete[] _buffer;
I don't know why the regular readString() is so slow, but this method is considerably faster, with my relatively small messages (~50 bytes) the response is being read almost instantly.

If look under hood, readString() always reads all data and when it reads the last symbol it stucks with server communication.
To avoid it you should not read last one symbol somehow.
I read headers by using readStringUntil('\n') and searching Content-Length header.
When headers has been read, just use readBytes with content-length.
HTTP request now took around 150ms.

Related

Need help getting a SparkFun ESP8266 connecting to a URL

I've been trying to get a piece of code that sends a text message to a phone when an IFTTT applet site is visited, I've been following this tutorial regarding the text message itself and this one for the WiFi shield for the ability to connect to a webpage and an HTTP request.
Basically, my problem is that it will connect to any "simple" site like google.com but it can't for "longer/complex" links. I was wondering if you would have any idea how would I solve this problem and get this to work. I've tried just using the addition symbol to combine the "simple" link and the rest of my desired link but that doesn't work either.
#include <SoftwareSerial.h> // Include software serial library, ESP8266 library dependency
#include <SparkFunESP8266WiFi.h> // Include the ESP8266 AT library
void setup() {
Serial.begin(9600);
String url = "/trigger/ESP/with/key/dwSukgpyQsyampQMkXXXX";
Serial.print (url);
// put your setup code here, to run once:
if (esp8266.begin()) // Initialize the ESP8266 and check it's return status
Serial.println("ESP8266 ready to go!"); // Communication and setup successful
else
Serial.println("Unable to communicate with the ESP8266 :(");
int retVal;
retVal = esp8266.connect("network", "networkpassword");
if (retVal < 0)
{
Serial.print(F("Error connecting: "));
Serial.println(retVal);
}
IPAddress myIP = esp8266.localIP(); // Get the ESP8266's local IP
Serial.print(F("My IP is: ")); Serial.println(myIP);
ESP8266Client client; // Create a client object
retVal = client.connect("maker.ifttt.com" + url, 80); // Connect to sparkfun (HTTP port)
if (retVal > 0)
Serial.println("Successfully connected!");
client.print("GET / HTTP/1.1\nHost: maker.ifttt.com" + url + "\nConnection: close\n\n");
while (client.available()) // While there's data available
Serial.write(client.read()); // Read it and print to serial
}
void loop() {
// put your main code here, to run repeatedly:
}
Thanks, any help would be very much appreciated!
First, the connect function requires a server(name) to connect to. In your case: maker.ifttt.com. Anything after the .com will make the connection fail (because it's not a correct servername).
Second: this function needs an IP address (like 54.175.81.255) or an array of characters. You cannot concatenate.
After you've established the connection, you can send and receive data to a specific part of this website, using the print function.
Also, in this function you can't concatenate.
Luckily, there is a String class where we easily can concatenate.
So, after you've created the client object (ESP8266Client client;), this could be the code:
String url;
char host[] = "maker.ifttt.com";
retVal = client.connect(host, 80);
if (retVal > 0) {
Serial.println("Successfully connected!");
}
url = "GET / HTTP/1.1\r\nHost: ";
url += host;
url += "/trigger/ESP/with/key/dwSukgpyQsyampQMkXXXX";
url += "\nConnection: close\n\n";
client.print(url);
while (client.connected() && !client.available());
while (client.available()) {
Serial.write(client.read());
}

BME680 gas sensor's value keeps increasing

I've been trying to get the BME680 to work and for the most part it seems to be working great. I do have one issue and that is with the gas sensor.
I write all the contents of the BME680 out to a webpage and all of the other values remain consistent.
Temperature: 77.29 *F
Humidity: 59.12 %
Pressure: 1010.45 millibars
Air Quality: 3.24 KOhms
On every refresh of the page the values for Temperature, Humidity, and Pressure all remain close to their values. They correct for a little while and show the minor fluctuations correctly. When it starts to rain the pressure goes down, the humidity goes up, etc... The issue is the Gas Sensor. On ever refresh the value keeps increasing. Regardless of whether I refresh it once per minute or per hour it keeps increasing. I'm clearly doing something wrong.
#include <Wire.h>
#include <SPI.h>
#include <Adafruit_Sensor.h>
#include "Adafruit_BME680.h"
#include <WiFi101.h>
#include <WiFiUdp.h>
#include "arduino_secrets.h"
///////please enter your sensitive data in the Secret tab/arduino_secrets.h
char ssid[] = SECRET_SSID; // your network SSID (name)
char pass[] = SECRET_PASS; // your network password (use for WPA, or use as key for WEP)
int keyIndex = 0; // your network key Index number (needed only for WEP)
int status = WL_IDLE_STATUS;
WiFiServer server(80);
#define SEALEVELPRESSURE_HPA (1023.03)
Adafruit_BME680 bme; // I2C
void setup() {
//Serial.begin(9600);
pinMode(LED_BUILTIN, OUTPUT); // set the LED pin mode
bme.begin();
// Set up oversampling and filter initialization
bme.setTemperatureOversampling(BME680_OS_8X);
bme.setHumidityOversampling(BME680_OS_2X);
bme.setPressureOversampling(BME680_OS_4X);
bme.setIIRFilterSize(BME680_FILTER_SIZE_3);
bme.setGasHeater(320, 150); // 320*C for 150 ms
// check for the presence of the shield:
if (WiFi.status() == WL_NO_SHIELD) {
//Serial.println("WiFi shield not present");
while (true); // don't continue
}
// attempt to connect to WiFi network:
while ( status != WL_CONNECTED) {
// Connect to WPA/WPA2 network. Change this line if using open or WEP network:
status = WiFi.begin(ssid, pass);
// wait 10 seconds for connection:
delay(10000);
}
server.begin(); // start the web server on port 80
}
void loop() {
WiFiClient client = server.available(); // listen for incoming clients
if (client) { // if you get a client,
//Serial.println("new client"); // print a message out the serial port
String currentLine = ""; // make a String to hold incoming data from the client
while (client.connected()) { // loop while the client's connected
if (client.available()) { // if there's bytes to read from the client,
char c = client.read(); // read a byte, then
//Serial.write(c); // print it out the serial monitor
if (c == '\n') { // if the byte is a newline character
// if the current line is blank, you got two newline characters in a row.
// that's the end of the client HTTP request, so send a response:
if (currentLine.length() == 0) {
// HTTP headers always start with a response code (e.g. HTTP/1.1 200 OK)
// and a content-type so the client knows what's coming, then a blank line:
client.println("HTTP/1.1 200 OK");
client.println("Content-type:text/html");
client.println();
if (! bme.performReading()) {
client.print("Failed to perform reading :(<br>");
return;
}
// the current weather condidtions
client.print("Temperature: ");
client.print((bme.temperature * 9/5) + 32);
client.print(" *F<br>");
client.print("Humidity: ");
client.print(bme.humidity);
client.print(" %<br>");
client.print("Pressure: ");
client.print(bme.pressure / 100.0);
client.print(" millibars<br>");
client.print("Air Quality: ");
client.print(bme.gas_resistance / 1000.0);
client.print(" KOhms<br>");
delay(2000);
// The HTTP response ends with another blank line:
client.println();
// break out of the while loop:
break;
}
else { // if you got a newline, then clear currentLine:
currentLine = "";
}
}
else if (c != '\r') { // if you got anything else but a carriage return character,
currentLine += c; // add it to the end of the currentLine
}
}
}
// close the connection:
client.stop();
//Serial.println("client disonnected");
}
}
I see two important points. Two for the beginning. Missing the ambient temperature and the sea-level.
Default ambient is: 25deg.C
Default sea-level is: 500 meter
Add this with correct values to your code. Then you have todo some automatic calibrations after burn in time. My own experience is that 2 weeks 24/7 is needed to get the sensor much old as needed. Next step look in the BOSCH original library and example code. Then start over again. Currently i'm rewriting the driver for the BME and BMP series for Tasmota (github). Lot's of work believe me. And i'm from the physics and chemical side. So some research helps always.

arduino and esp8266 interface

I want to send some AT commands to esp8266 using arduino and get the reply from serial monitor. this is the code:(the purpose of this code is to update a thingspeak channel)
#include<SoftwareSerial.h>
SoftwareSerial esp8266(3,2);
#define ID "user"
#define PASS "pass"
String apiKey = "apikey";
void setup() {
Serial.setTimeout(5000);
Serial.begin(9600);
esp8266.begin(9600);
// delay(1000);
String command6="AT+RST";
esp8266.println(command6);
if(esp8266.available())
{
while(esp8266.available())
{
char c=esp8266.read();
Serial.write(c);
}
}
delay(2000);
}
void loop() {
delay(2000);
String command="\nAT";
esp8266.println(command);
if(esp8266.available())
{
while(esp8266.available())
{
char c=esp8266.read();
Serial.write(c);
}
}
String cmd = "\nAT+CIPSTART=\"TCP\",\"";
cmd += "144.212.80.11"; // api.thingspeak.com
cmd += "\",80";
esp8266.println(cmd);
if(esp8266.available())
{
while(esp8266.available())
{
char c=esp8266.read();
Serial.write(c);
}
}
delay(3000);
String command3="\nAT+CIPSEND=200";
esp8266.println(command3);
if(esp8266.available())
{
while(esp8266.available())
{
char c=esp8266.read();
Serial.write(c);
}
}
delay(1000);
String getStr = "GET /update?api_key=";
getStr += apiKey;
getStr += "&field1=10";
esp8266.println(getStr);
esp8266.println("\r\r\r\r\r\r\r\r");
if(esp8266.available())
{
while(esp8266.available())
{
char c=esp8266.read();
Serial.write(c);
}
}
delay(15000);
}
user and pass are my wifi username and password. the problem is, the esp8266 responds "ok" to at commands but when it gets to the last parts, it gives me this:
A))-R¤%%JHÕ¨TUPZ="TCP","144.212.80.11",80
CONNECT
OK
ERROR
AT+CIPSEND=200
OK
> GET /update?api_key=apikey&field1=10
CAT
AT+CIPSTART="TCP","144.212.80.11",80
AT+CIPSEND=200
GET /update?api_key=apikey&field1=10
AT
AT+CIPSTART="TCP","144.212.80.11",80
busy s...
i have put a few delays inside the code but after it inserts the GET it gets back to the loop runs the program again with no delays and then esp8266 resets itself.
Besides waiting for the OK, you also need to make sure that you are using the right IP address for ThingSpeak. The offical static IP for ThingSpeak is 184.106.153.149 found here (http://www.mathworks.com/help/thingspeak/channel-settings.html#endpoints).
Try using /n after the At command not before and also check the correct format for AT+CIPSEND
GET http://api.thingspeak.com/update?api_key=KTQXXXXXXXXXXXXX&field1=10 HTTP/1.0 \r\n\r\n
try this format
There are a few things to keep in mind when working with ESP8266 communicate over a network.
1 The response might not be received in a constant time i.e 100ms
or 1ms etc. there will always be random delay.
2 Check if the ESP is not running out of current while making a
GET/POST request.
3 Check for every character/escape sequence ('\r' '\n' etc.) and
place them into right place into your "Request" string.
This might help you: Arduino ESP8266 AT GET Request
Thank you. :)

how to acquire arduino ethercard udp receive data?

I have a problem in accessing received udp string, I can get it by serial though. I just need to get my incoming udp data into a variable using the ethercard library in loop() function so I can use them in my program.Here's the code I'm working with:
#include <EtherCard.h>
#include <IPAddress.h>
#define STATIC 1 // set to 1 to disable DHCP (adjust myip/gwip values below)
#if STATIC
// ethernet interface ip address
static byte myip[] = { 192,168,1,200 };
// gateway ip address
static byte gwip[] = { 192,168,1,1 };
#endif
// ethernet mac address - must be unique on your network
static byte mymac[] = { 0x70,0x69,0x69,0x2D,0x30,0x31 };
byte Ethernet::buffer[500]; // tcp/ip send and receive buffer
//callback that prints received packets to the serial port
void udpSerialPrint(word port, byte ip[4], const char *data, word len) {
Serial.println(data);
}
void setup(){
Serial.begin(57600);
Serial.println("\n[backSoon]");
if (ether.begin(sizeof Ethernet::buffer, mymac, 10) == 0)
Serial.println( "Failed to access Ethernet controller");
#if STATIC
ether.staticSetup(myip, gwip);
#else
if (!ether.dhcpSetup())
Serial.println("DHCP failed");
#endif
ether.printIp("IP: ", ether.myip);
ether.printIp("GW: ", ether.gwip);
ether.printIp("DNS: ", ether.dnsip);
//register udpSerialPrint() to port 1337
ether.udpServerListenOnPort(&udpSerialPrint, 1337);
//register udpSerialPrint() to port 42.
ether.udpServerListenOnPort(&udpSerialPrint, 42);
}
void loop(){
//this must be called for ethercard functions to work.
ether.packetLoop(ether.packetReceive());
//? incoming = data; <--- this is my problem
//Serial.println(incoming);
}
It's just a slightly modified version of the UDPListener example that comes with the ethercard library.
Thank you
I'm still on a steep learning curve myself but have managed to get UDP talking between units so hope following helps. Suspect the quickest way would be to create a global variable such as:
char gUDPdata[30] = "";
then in your udpSerialPrint routine add the following for a quick and dirty result. This copies 'data' to a global variable that you can see in your main loop.
Serial.println(data);
data[0] = 0;
strcpy(data, gUDPdata);
then in your main loop following should produce same as Serial.print in the udpSerialPrint routine.
Serial.println(gUDPdata);

Should the xivelyclient.get API call take 1 minute to return?

I am using the Xively Arduino API. All the API calls I have used so far are working as expected, except the xivelyclient.get() call takes 1 minute to return with the data.
Is this the expected behaviour?
Below is my code. As you can see its basically one of the examples that come with the Arduino API for Xively. All I did to get it going is update the xivelyKey and the feedID.
#include <SPI.h>
#include <Ethernet.h>
#include <HttpClient.h>
#include <Xively.h>
// MAC address for your Ethernet shield
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
// Your Xively key to let you upload data
char xivelyKey[] = "abcdefghijklmnopqrstuvwxyz";
// Define the string for our datastream ID
char temperatureId[] = "temperature";
XivelyDatastream datastreams[] = {
XivelyDatastream(temperatureId, strlen(temperatureId), DATASTREAM_FLOAT),
};
// Finally, wrap the datastreams into a feed
XivelyFeed feed(123456789, datastreams, 1 /* number of datastreams */);
EthernetClient client;
XivelyClient xivelyclient(client);
void setup() {
// put your setup code here, to run once:
Serial.begin(9600);
Serial.println("Reading from Xively example");
Serial.println();
while (Ethernet.begin(mac) != 1)
{
Serial.println("Error getting IP address via DHCP, trying again...");
delay(15000);
}
}
void loop() {
int ret = xivelyclient.get(feed, xivelyKey);
Serial.print("xivelyclient.get returned ");
Serial.println(ret);
if (ret > 0)
{
Serial.println("Datastream is...");
Serial.println(feed[0]);
Serial.print("Temperature is: ");
Serial.println(feed[0].getFloat());
}
Serial.println();
delay(15000UL);
}
The output on the serial monitor is as expected:
Reading from Xively example
xivelyclient.get returned 200
Datastream is...
{ "id" : "temperature", "current_value" : "23.00" }
Temperature is: 23.00
xivelyclient.get returned 200
Datastream is...
{ "id" : "temperature", "current_value" : "23.00" }
Temperature is: 23.00
The responses come at approximately 1 minute 10 seconds.
I did some debugging and found that the implementation of xivelyclient.get() in XivelyClient.cpp (part of the API) was hanging in the following while loop:
while ((next != '\r') && (next != '\n') && (http.available() || http.connected()))
{
next = http.read();
}
I imagine that the only reason it was ever coming out of this loop is because the connection is being closed by the server.
In order to make the function work for me I added the last two lines in the if statement just above the while loop and deleted the while loop.
if ((idBitfield & 1<<i) && (aFeed[i].idLength() == idIdx))
{
// We've found a matching datastream
// FIXME cope with any errors returned
aFeed[i].updateValue(http);
// When we get here we'll be at the end of the line, but if aFeed[i]
// was a string or buffer type, we'll have consumed the '\n'
next = '\n';
http.stop();
return ret;
}
I'm sure this is not an elegant solution but it works for me for now...

Resources