ESP8266 failing post requests - arduino

I'm hesitant to ask this on here; I have made an issue on GitHub, and I really need it resolved.
Basically, I'm sending posts to dweetPro every 15 seconds. It will work flawlessly for hours on end, then just randomly it will stop, send one successful one for every 20 failed responses.
I have tried everything I can think of, I'm pretty sure my code is right, not sure if it is something with the library or what. I can send the posts from Python on my PC indefinitely and never fails, so it's definitely on the ESP's side.
I'm using a clean 2A power supply, so it's not a power issue.
Here is the link of a couple Wireshark logs from the ESP sending posts. Both valid and invalid are on them.
One
Two
Some more info can be found on the issue on GitHub. This is the code I'm using, with debug lines used:
void send_to_server(String* time_sent, float magnitude, String status,
int earthquake_occured, float* data, int data_size)
{
int content_length;
String content = make_json_content(&content_length, magnitude,
status, earthquake_occured, data, data_size);
int s = -100;
int c = 0;
while (s<0 && c<10)
{
HTTPClient http;
http.setTimeout(1000);
bool suc = http.begin("http://dweetpro.io/v2/dweets");
Serial.print("Success?: "); Serial.println(suc);
http.addHeader("Content-Type", "application/json");
http.addHeader("Accept", "application/json");
http.addHeader("X-DWEET-AUTH", "xxx");
int timerr = millis();
Serial.println("Posting");
s = http.POST(content);
Serial.print("Posted: "); Serial.println(millis() - timerr);
Serial.print("Post len: "); Serial.println(s);
http.writeToStream(&Serial);
Serial.println("");
http.end();
++c;
}
Serial.println("Ended");
}
When successful, s prints 200, when failed, it print -1.
On Wireshark, every successful post will show data, but for unsuccessful posts, it will show some data for the first several failed ones, then it just stops. It's like it overloads itself or something.
Once again, sorry for this cluster of a question, but I just don't know what else to do. I have spent so much time on this with no end in sight. Thank you all so very much.
EDIT: Some other information I should have added is that if it isn't sending valid posts, I can reset the ESP and it will still not be working. Its really weird, that's why I am so at a loss. If it is on a roll sending valid posts, I can reset/unplug and it will work still. If it isn't working and I do the same, it won't be working still.
Is there any chance that it could just be some sort of interference at my home location?

Anytime I hear about an Arduino project that crashes after some time and I see the String class I get suspicious of it right away. In older versions for the Arduino it had a nasty memory leak. Current versions have fixed that leak but still leave the problem with concatenation. Basically, when you concatenate two Strings it has to deallocate the spot in memory where they were and allocate new space for the larger String. If nothing else has gone on the heap or stack since the last concatenation then that might not be a problem.
But what happens is that someone is building two Strings that are both growing larger and larger. And each time it has to allocate that memory further and further up in memory, leaving holes where the smaller Strings used to be. Eventually, the heap looks like swiss cheese and there's no garbage collection to come through and defragment it. After the project has run fine for quite some time then suddenly the heap and the stack get together and it's lights out.
I don't know if there is a function like freemem for the Arduino that works for the ESP8266, but it would be worth looking into finding a way to find out how much free memory you have. If it is steadily shrinking to 0 as the program runs then you've found your problem.

Related

QTextBrowser display issue in Qt 4.8.6 - always displayed wrongly

I hope to read some characters or strings and display them with QTextBrowse from serial port by Qt 4.8.6 and called the following functions( textBrowser is a object of QTextBrowser):
connect(com, SIGNAL(readyRead()), this, SLOT(readSerialPort()));
connect(textBrowser, SIGNAL(textChanged()), SimApplianceQtClass, SLOT(on_textBrowser_textChanged()));
void SimApplianceQt::on_textBrower_textChanged()
{
ui.textBrowser->moveCursor(QTextCursor::End);
}
void SimApplianceQt::readSerialPort()
{
QByteArray temp = com->readAll();
ui.textBrowser->insertPlainText(temp);
}
However, every time I cannot display characters or strings in the textBrowser rightly. Those input strings are always cut into smaller strings to be displayed in multiple lines in the textBrowser. For example, a string "0123456789" may be displayed as (in multiple lines):
01
2345
6789
How to deal with this issue? Many thanks.
What happens is that the readyRead signal is fired not after everything has been received, but after something has been received and is ready to read.
There is no guarantee that everything will have arrived or is readable by the time you receive the first readyRead.
This is a common "problem" for almost any kind of IO, especially if the data is larger than very few bytes. There is usually no automatic way to know when all the data has been received.
There are a few possible solutions:
All of them will require you to put the data in a buffer in readSerialPort() instead of adding it directly to the text browser. Maybe a simple QByteArray member variable in SimApplianceQt would already do the trick in your case.
The rest depends on the exact solution.
If you have access to the sender of the data, you could send the
number of bytes that will be sent before sending the actual string.
This must always be in an integer type of the same size (for
example, always a quint32). Then, in readSerialPort(), you would
first read that size, and then continue to read bytes to your buffer
in readSerialPort() until everything has been received. And then,
you could finally print it. I'd recommend that one. It is also what is used in almost all cases where this problem arises.
If you have access to the sender of the data, you could send some
kind of "ending sequence" at the end of the string. In your
readSerialPort(), you would then continue to read bytes into your
buffer until you receive that ending sequence. Once the ending
sequence has been received, you can print everything that came in
prior to it. Note that the ending sequence itself could be interrupted,
so you'd have to take care of that, too.
If you do not have access to the sender, the best idea I could come
up with would be to work with a timer. You put everything into a
buffer and re-start that timer each time you readSerialPort() is
called. When the timer runs out, that means no new data has been
sent for a while and you can probably print what you have so far.
This is... risky and I wouldn't recommend it if there is any other way.

Strange characters coming from Arduino when writing to SD card

I am connecting a SD card to an Arduino which is then communicating over serial to Visual studio. Everything works fine independently and 99% collectively. Now if i write this code in the setup in works fine. If i pop it into a function which is called when a specific character is sent from visual studio I get the strange characters at the bottom.
I have debugged each step of the code and nothing seems abnormal, unfortunately I cannot the code as
1) it's far too long...
2) it's confidential...
:(
I understand without code I cannot get a complete solution but what are those characters! why in the setup does it work perfectly and in a function I get all kinds of randomness?
myFile = SD.open("test.txt");
if (myFile) {
Serial.println("test.txt:");
// read from the file until there's nothing else in it:
while (myFile.available()) {
Serial.write(myFile.read());
}
// close the file:
myFile.close();
} else {
// if the file didn't open, print an error:
Serial.println("error opening test.txt");
}
}
整瑳湩⁧ⰱ㈠‬⸳ࠀ -- Copied straight from the text file
整瑳湩%E2%81%A7ⰱ㈠%E2%80%AC⸳ࠀ -- Output when pasted into google
Its the arduino DUE and yes lots of "String" including 4 x 2D string arrays we are using to upload to a tft screen. I ran into memory issues with the UNO but thought we would be ok with the DUE as its got considerably more ram?
Well, there's your problem. I like how you ended that with a question mark. :) Having extra RAM is no guarantee that the naughty String will behave, NOT EVEN ON SERVERS WITH GIGABYTES OF RAM. To summarize my Arduino forum post:
Don't use String™
The solutions using C strings (i.e., char arrays) are very efficient. If you're not sure how to convert a String usage to a character array, please post the snippet. You will get lots of help!
String will add at least 1600 bytes of FLASH to your program size and 10 bytes per String variable. Although String is fairly easy to use and understand, it will also cause random hangs and crashes as your program runs longer and/or grows in size and complexity. This is due to dynamic memory fragmentation caused by the heap management routines malloc and free.
Commentary from systems experts elsewhere on the net:
The Evils of Arduino Strings (required reading!)
Why is malloc harmful in embedded systems?
Dr Dobbs Journal
Memory Fragmentation in servers (MSDN)
Memory Fragmentation, your worst nightmare (nice graphics)
Another answer of mine.

Arduino WiFi shield stop working at WiFi.status()

UPDATE:
I pinpointed where the problem is coming from. To avoid any complication, I'm using ScanNetwork example, so I don't even have to put in SSID. The code stops functioning on the board as soon as it hits WiFi.status().
I have a Serial.println before and after it tries to get a WiFi.status(), the serial.println after wasn't performed, and of course, I'm still not connected.
I've downloaded fresh copy of the code, and the situation remains the same. I've really run out of idea....
I'm using the official arduino wifi shield, and I have the following code:
status = WiFi.begin([ssid],[pass]);
Serial.println(status);
Status is neither WL_CONNECTED nor WL_IDLE_STATUS, which are the two possible responses outlined in the official reference http://arduino.cc/en/Reference/WiFiBegin
Status is the number 4.
and of course, I couldn't connect to wifi.
What is this????
I've pressed the reset button a million times, is there a more powerful factory restore button?
I've figured it out.
Apparently, there's a jumper, that when you put it in, it'll kick the shield into a DFU-mode that enables reprogramming. And the shield wouldn't be present as a result.
According to WiFi.h The return values of the begin() functions (all three of them, one for each security scheme) are ints. It is not stated outright on this function but I believe that just as with the status() function the return type is a wl_status_t. wl_status_t is an enum declared in wl_definitions.h As:
typedef enum {
WL_NO_SHIELD = 255,
        WL_IDLE_STATUS = 0,
        WL_NO_SSID_AVAIL,
        WL_SCAN_COMPLETED,
        WL_CONNECTED,
        WL_CONNECT_FAILED,
        WL_CONNECTION_LOST,
        WL_DISCONNECTED
} wl_status_t;
So your 4 is WL_CONNECT_FAILED. Probably not surprising to you since, you know, you connection failed.
The hobbiest's debugger, AKA reset button, will only do so much. Printing the status was a good start. Be sure you are using the right flavor of begin() for you security type, you seem to be using the one for WPA. Consider shutting off you router's security completely (if safe to do so in your area), or using a spare router, to test the ability of the shield to communicate at all. Also this may sound obvious but check for a misspelling of the SSID.

Solaris de-referencing bus error

I have a function where I'm trying to take a 16-bit from a large chunk of data. I'm running this code on a Solaris box and I can compile without warning or error. When I run this code, however, whenever it gets to the part where I de-deference my pointer, I instantly get a bus error. The code looks something like:
void find_info(unsigned char* packet) {
int offset = 9;
uint16_t short_value = *(uint16_t*)(packet+offset);
}
The bus error occurs when I'm trying to de-reference that "packet+offset" pointer in order to get a short. I know for a fact there is data at packet[offset] and packet[offset+1]. On Linux and Cygwin this code works fine. As far as I know, I'm not doing anything revolutionary. What's going on here?
Sounds like an alignment problem. On the Sun SPARC processor, you can only access something like a short via a pointer that is divisible by some power of 2, typically 8. So the value of offset=9 is clearly going to cause a problem.
See http://blogs.oracle.com/d/entry/on_misaligned_memory_accesses for more details.
I can't recommend any way of fixing this without seeing more context; but if you're reading data from some input source, you could just read the bytes and convert to a short using ntohs (see the man page for ntohs for details).

Receiving image through winsocket

i have a proxy server running on my local machine used to cache images while surfing. I set up my browser with a proxy to 127.0.0.1, receive the HTTP requests, take the data and send it back to the browser. It works fine for everything except large images. When I receive the image info, it only displays half the image (ex.: the top half of the google logo) heres my code:
char buffer[1024] = "";
string ret("");
while(true)
{
valeurRetour = recv(socketClient_, buffer, sizeof(buffer), 0);
if(valeurRetour <= 0) break;
string t;
t.assign(buffer,valeurRetour);
ret += t;
longueur += valeurRetour;
}
closesocket(socketClient_);
valeurRetour = send(socketServeur_, ret.c_str(),longueur, 0);
the socketClient_ is non-blocking. Any idea how to fix this problem?
You're not making fine enough distinctions among the possible return values of recv.
There are two levels here.
The first is, you're lumping 0 and -1 together. 0 means the remote peer closed its sending half of the connection, so your code does the right thing here, closing its socket down, too. -1 means something happened besides data being received. It could be a permanent error, a temporary error, or just a notification from the stack that something happened besides data being received. Your code lumps all such possibilities together, and on top of that treats them the same as when the remote peer closes the connection.
The second level is that not all reasons for getting -1 from recv are "errors" in the sense that the socket is no longer useful. I think if you start checking for -1 and then calling WSAGetLastError to find out why you got -1, you'll get WSAEWOULDBLOCK, which is normal since you have a non-blocking socket. It means the recv call cannot return data because it would have to block your program's execution thread to do so, and you told Winsock you wanted non-blocking calls.
A naive fix is to not break out of the loop on WSAEWOULDBLOCK but that just means you burn CPU time calling recv again and again until it returns data. That goes against the whole point of non-blocking sockets, which is that they let your program do other things while the network is busy. You're supposed to use functions like select, WSAAsyncSelect or WSAEventSelect to be notified when a call to the API function is likely to succeed again. Until then, you don't call it.
You might want to visit The Winsock Programmer's FAQ. (Disclaimer: I'm its maintainer.)
Have you analyzed the transaction at the HTTP level i.e. checked Headers?
Are you accounting for things like Chunked transfers?
I do not have a definite answer in part because of the lack of details given here.

Resources