I'm hesitant to ask this on here; I have made an issue on GitHub, and I really need it resolved.
Basically, I'm sending posts to dweetPro every 15 seconds. It will work flawlessly for hours on end, then just randomly it will stop, send one successful one for every 20 failed responses.
I have tried everything I can think of, I'm pretty sure my code is right, not sure if it is something with the library or what. I can send the posts from Python on my PC indefinitely and never fails, so it's definitely on the ESP's side.
I'm using a clean 2A power supply, so it's not a power issue.
Here is the link of a couple Wireshark logs from the ESP sending posts. Both valid and invalid are on them.
One
Two
Some more info can be found on the issue on GitHub. This is the code I'm using, with debug lines used:
void send_to_server(String* time_sent, float magnitude, String status,
int earthquake_occured, float* data, int data_size)
{
int content_length;
String content = make_json_content(&content_length, magnitude,
status, earthquake_occured, data, data_size);
int s = -100;
int c = 0;
while (s<0 && c<10)
{
HTTPClient http;
http.setTimeout(1000);
bool suc = http.begin("http://dweetpro.io/v2/dweets");
Serial.print("Success?: "); Serial.println(suc);
http.addHeader("Content-Type", "application/json");
http.addHeader("Accept", "application/json");
http.addHeader("X-DWEET-AUTH", "xxx");
int timerr = millis();
Serial.println("Posting");
s = http.POST(content);
Serial.print("Posted: "); Serial.println(millis() - timerr);
Serial.print("Post len: "); Serial.println(s);
http.writeToStream(&Serial);
Serial.println("");
http.end();
++c;
}
Serial.println("Ended");
}
When successful, s prints 200, when failed, it print -1.
On Wireshark, every successful post will show data, but for unsuccessful posts, it will show some data for the first several failed ones, then it just stops. It's like it overloads itself or something.
Once again, sorry for this cluster of a question, but I just don't know what else to do. I have spent so much time on this with no end in sight. Thank you all so very much.
EDIT: Some other information I should have added is that if it isn't sending valid posts, I can reset the ESP and it will still not be working. Its really weird, that's why I am so at a loss. If it is on a roll sending valid posts, I can reset/unplug and it will work still. If it isn't working and I do the same, it won't be working still.
Is there any chance that it could just be some sort of interference at my home location?
Anytime I hear about an Arduino project that crashes after some time and I see the String class I get suspicious of it right away. In older versions for the Arduino it had a nasty memory leak. Current versions have fixed that leak but still leave the problem with concatenation. Basically, when you concatenate two Strings it has to deallocate the spot in memory where they were and allocate new space for the larger String. If nothing else has gone on the heap or stack since the last concatenation then that might not be a problem.
But what happens is that someone is building two Strings that are both growing larger and larger. And each time it has to allocate that memory further and further up in memory, leaving holes where the smaller Strings used to be. Eventually, the heap looks like swiss cheese and there's no garbage collection to come through and defragment it. After the project has run fine for quite some time then suddenly the heap and the stack get together and it's lights out.
I don't know if there is a function like freemem for the Arduino that works for the ESP8266, but it would be worth looking into finding a way to find out how much free memory you have. If it is steadily shrinking to 0 as the program runs then you've found your problem.
I am connecting a SD card to an Arduino which is then communicating over serial to Visual studio. Everything works fine independently and 99% collectively. Now if i write this code in the setup in works fine. If i pop it into a function which is called when a specific character is sent from visual studio I get the strange characters at the bottom.
I have debugged each step of the code and nothing seems abnormal, unfortunately I cannot the code as
1) it's far too long...
2) it's confidential...
:(
I understand without code I cannot get a complete solution but what are those characters! why in the setup does it work perfectly and in a function I get all kinds of randomness?
myFile = SD.open("test.txt");
if (myFile) {
Serial.println("test.txt:");
// read from the file until there's nothing else in it:
while (myFile.available()) {
Serial.write(myFile.read());
}
// close the file:
myFile.close();
} else {
// if the file didn't open, print an error:
Serial.println("error opening test.txt");
}
}
整瑳湩ⰱ㈠⸳ࠀ -- Copied straight from the text file
整瑳湩%E2%81%A7ⰱ㈠%E2%80%AC⸳ࠀ -- Output when pasted into google
Its the arduino DUE and yes lots of "String" including 4 x 2D string arrays we are using to upload to a tft screen. I ran into memory issues with the UNO but thought we would be ok with the DUE as its got considerably more ram?
Well, there's your problem. I like how you ended that with a question mark. :) Having extra RAM is no guarantee that the naughty String will behave, NOT EVEN ON SERVERS WITH GIGABYTES OF RAM. To summarize my Arduino forum post:
Don't use String™
The solutions using C strings (i.e., char arrays) are very efficient. If you're not sure how to convert a String usage to a character array, please post the snippet. You will get lots of help!
String will add at least 1600 bytes of FLASH to your program size and 10 bytes per String variable. Although String is fairly easy to use and understand, it will also cause random hangs and crashes as your program runs longer and/or grows in size and complexity. This is due to dynamic memory fragmentation caused by the heap management routines malloc and free.
Commentary from systems experts elsewhere on the net:
The Evils of Arduino Strings (required reading!)
Why is malloc harmful in embedded systems?
Dr Dobbs Journal
Memory Fragmentation in servers (MSDN)
Memory Fragmentation, your worst nightmare (nice graphics)
Another answer of mine.
What is a segmentation fault? Is it different in C and C++? How are segmentation faults and dangling pointers related?
Segmentation fault is a specific kind of error caused by accessing memory that “does not belong to you.” It’s a helper mechanism that keeps you from corrupting the memory and introducing hard-to-debug memory bugs. Whenever you get a segfault you know you are doing something wrong with memory – accessing a variable that has already been freed, writing to a read-only portion of the memory, etc. Segmentation fault is essentially the same in most languages that let you mess with memory management, there is no principal difference between segfaults in C and C++.
There are many ways to get a segfault, at least in the lower-level languages such as C(++). A common way to get a segfault is to dereference a null pointer:
int *p = NULL;
*p = 1;
Another segfault happens when you try to write to a portion of memory that was marked as read-only:
char *str = "Foo"; // Compiler marks the constant string as read-only
*str = 'b'; // Which means this is illegal and results in a segfault
Dangling pointer points to a thing that does not exist anymore, like here:
char *p = NULL;
{
char c;
p = &c;
}
// Now p is dangling
The pointer p dangles because it points to the character variable c that ceased to exist after the block ended. And when you try to dereference dangling pointer (like *p='A'), you would probably get a segfault.
It would be worth noting that segmentation fault isn't caused by directly accessing another process memory (this is what I'm hearing sometimes), as it is simply not possible. With virtual memory every process has its own virtual address space and there is no way to access another one using any value of pointer. Exception to this can be shared libraries which are same physical address space mapped to (possibly) different virtual addresses and kernel memory which is even mapped in the same way in every process (to avoid TLB flushing on syscall, I think). And things like shmat ;) - these are what I count as 'indirect' access. One can, however, check that they are usually located long way from process code and we are usually able to access them (this is why they are there, nevertheless accessing them in a improper way will produce segmentation fault).
Still, segmentation fault can occur in case of accessing our own (process) memory in improper way (for instance trying to write to non-writable space). But the most common reason for it is the access to the part of the virtual address space that is not mapped to physical one at all.
And all of this with respect to virtual memory systems.
A segmentation fault is caused by a request for a page that the process does not have listed in its descriptor table, or an invalid request for a page that it does have listed (e.g. a write request on a read-only page).
A dangling pointer is a pointer that may or may not point to a valid page, but does point to an "unexpected" segment of memory.
To be honest, as other posters have mentioned, Wikipedia has a very good article on this so have a look there. This type of error is very common and often called other things such as Access Violation or General Protection Fault.
They are no different in C, C++ or any other language that allows pointers. These kinds of errors are usually caused by pointers that are
Used before being properly initialised
Used after the memory they point to has been realloced or deleted.
Used in an indexed array where the index is outside of the array bounds. This is generally only when you're doing pointer math on traditional arrays or c-strings, not STL / Boost based collections (in C++.)
According to Wikipedia:
A segmentation fault occurs when a
program attempts to access a memory
location that it is not allowed to
access, or attempts to access a memory
location in a way that is not allowed
(for example, attempting to write to a
read-only location, or to overwrite
part of the operating system).
Segmentation fault is also caused by hardware failures, in this case the RAM memories. This is the less common cause, but if you don't find an error in your code, maybe a memtest could help you.
The solution in this case, change the RAM.
edit:
Here there is a reference: Segmentation fault by hardware
Wikipedia's Segmentation_fault page has a very nice description about it, just pointing out the causes and reasons. Have a look into the wiki for a detailed description.
In computing, a segmentation fault (often shortened to segfault) or access violation is a fault raised by hardware with memory protection, notifying an operating system (OS) about a memory access violation.
The following are some typical causes of a segmentation fault:
Dereferencing NULL pointers – this is special-cased by memory management hardware
Attempting to access a nonexistent memory address (outside process's address space)
Attempting to access memory the program does not have rights to (such as kernel structures in process context)
Attempting to write read-only memory (such as code segment)
These in turn are often caused by programming errors that result in invalid memory access:
Dereferencing or assigning to an uninitialized pointer (wild pointer, which points to a random memory address)
Dereferencing or assigning to a freed pointer (dangling pointer, which points to memory that has been freed/deallocated/deleted)
A buffer overflow.
A stack overflow.
Attempting to execute a program that does not compile correctly. (Some compilers will output an executable file despite the presence of compile-time errors.)
Segmentation fault occurs when a process (running instance of a program) is trying to access read-only memory address or memory range which is being used by other process or access the non-existent (invalid) memory address.
Dangling Reference (pointer) problem means that trying to access an object or variable whose contents have already been deleted from memory, e.g:
int *arr = new int[20];
delete arr;
cout<<arr[1]; //dangling problem occurs here
In simple words: segmentation fault is the operating system sending a signal to the program
saying that it has detected an illegal memory access and is prematurely terminating the program to prevent
memory from being corrupted.
There are several good explanations of "Segmentation fault" in the answers, but since with segmentation fault often there's a dump of the memory content, I wanted to share where the relationship between the "core dumped" part in Segmentation fault (core dumped) and memory comes from:
From about 1955 to 1975 - before semiconductor memory - the dominant technology in computer memory used tiny magnetic doughnuts strung on copper wires. The doughnuts were known as "ferrite cores" and main memory thus known as "core memory" or "core".
Taken from here.
"Segmentation fault" means that you tried to access memory that you do not have access to.
The first problem is with your arguments of main. The main function should be int main(int argc, char *argv[]), and you should check that argc is at least 2 before accessing argv[1].
Also, since you're passing in a float to printf (which, by the way, gets converted to a double when passing to printf), you should use the %f format specifier. The %s format specifier is for strings ('\0'-terminated character arrays).
Simple meaning of Segmentation fault is that you are trying to access some memory which doesn't belong to you. Segmentation fault occurs when we attempt to read and/or write tasks in a read only memory location or try to freed memory. In other words, we can explain this as some sort of memory corruption.
Below I mention common mistakes done by programmers that lead to Segmentation fault.
Use scanf() in wrong way(forgot to put &).
int num;
scanf("%d", num);// must use &num instead of num
Use pointers in wrong way.
int *num;
printf("%d",*num); //*num should be correct as num only
//Unless You can use *num but you have to point this pointer to valid memory address before accessing it.
Modifying a string literal(pointer try to write or modify a read only memory.)
char *str;
//Stored in read only part of data segment
str = "GfG";
//Problem: trying to modify read only memory
*(str+1) = 'n';
Try to reach through an address which is already freed.
// allocating memory to num
int* num = malloc(8);
*num = 100;
// de-allocated the space allocated to num
free(num);
// num is already freed there for it cause segmentation fault
*num = 110;
Stack Overflow -: Running out of memory on the stack
Accessing an array out of bounds'
Use wrong format specifiers when using printf() and scanf()'
Consider the following snippets of Code,
SNIPPET 1
int *number = NULL;
*number = 1;
SNIPPET 2
int *number = malloc(sizeof(int));
*number = 1;
I'd assume you know the meaning of the functions: malloc() and sizeof() if you are asking this question.
Now that that is settled,
SNIPPET 1 would throw a Segmentation Fault Error.
while SNIPPET 2 would not.
Here's why.
The first line of snippet one is creating a variable(*number) to store the address of some other variable but in this case it is initialized to NULL.
on the other hand,
The second line of snippet two is creating the same variable(*number) to store the address of some other and in this case it is given a memory address(because malloc() is a function in C/C++ that returns a memory address of the computer)
The point is you cannot put water inside a bowl that has not been bought OR a bowl that has been bought but has not been authorized for use by you.
When you try to do that, the computer is alerted and it throws a SegFault error.
You should only face this errors with languages that are close to low-level like C/C++. There is an abstraction in other High Level Languages that ensure you do not make this error.
It is also paramount to understand that Segmentation Fault is not language-specific.
There are enough definitions of segmentation fault, I would like to quote few examples which I came across while programming, which might seem like silly mistakes, but will waste a lot of time.
You can get a segmentation fault in below case while argument type mismatch in printf:
#include <stdio.h>
int main(){
int a = 5;
printf("%s",a);
return 0;
}
output : Segmentation Fault (SIGSEGV)
When you forgot to allocate memory to a pointer, but try to use it.
#include <stdio.h>
typedef struct{
int a;
} myStruct;
int main(){
myStruct *s;
/* few lines of code */
s->a = 5;
return 0;
}
output : Segmentation Fault (SIGSEGV)
In computing, a segmentation fault or access violation is a fault, or failure condition, raised by hardware with memory protection,
notifying an operating system the software has attempted to access a
restricted area of memory. -WIKIPEDIA
You might be accessing the computer memory with the wrong data type. Your case might be like the code below:
#include <stdio.h>
int main(int argc, char *argv[]) {
char A = 'asd';
puts(A);
return 0;
}
'asd' -> is a character chain rather than a single character char data type. So, storing it as a char causes the segmentation fault. Stocking some data at the wrong position.
Storing this string or character chain as a single char is trying to fit a square peg in a round hole.
Terminated due to signal: SEGMENTATION FAULT (11)
Segm. Fault is the same as trying to breath in under water, your lungs were not made for that. Reserving memory for an integer and then trying to operate it as another data type won't work at all.
Segmentation fault occurs when a process (running instance of a program) is trying to access a read-only memory address or memory range which is being used by another process or access the non-existent memory address.
seg fault,when type gets mismatched
A segmentation fault or access violation occurs when a program attempts to access a memory location that is not exist, or attempts to access a memory location in a way that is not allowed.
/* "Array out of bounds" error
valid indices for array foo
are 0, 1, ... 999 */
int foo[1000];
for (int i = 0; i <= 1000 ; i++)
foo[i] = i;
Here i[1000] not exist, so segfault occurs.
Causes of segmentation fault:
it arise primarily due to errors in use of pointers for virtual memory addressing, particularly illegal access.
De-referencing NULL pointers – this is special-cased by memory management hardware.
Attempting to access a nonexistent memory address (outside process’s address space).
Attempting to access memory the program does not have rights to (such as kernel structures in process context).
Attempting to write read-only memory (such as code segment).
I'm trying to write single bytes to a serial port in Vala using a FileOutputStream:
var dev = File.new_for_path("/dev/ttyACM0");
var dev_io = dev.open_readwrite();
var dev_o = dev_io.output_stream as FileOutputStream;
dev_o.write({0x13});
dev_o.flush();
My aim is to do this similar to echo -en '\x13' > /dev/ttyACM0 but it just behaves weirdly. The Byte 0x13 seems to be written multiple times, sometimes /dev/ttyACM0 is blocked for a few seconds, sometimes it's even blocked after the Vala program exited and sometimes it's not blocked at all. If i write my FileOutputStream to a file and send this to the serial port via cat byte_file > /dev/ttyACM0 everything is fine.
It seems to me that GIO struggles with the fact that the file is a device. My problem is that I need GIO to monitor /dev/ttyACM0 if it's plugged in and for asynchronous reading.
The problem is most likely that you have to configure the serial port to set things like baud rate, flow control, and parity. If you don't get all those options right there is a good chance that you'll end up with garbage data like you describe.
Basically, you first need an integer descriptor for the file; the easiest way to get one is probably to just open the file using Posix.open, but you can also use GLib.FileStream.fileno to get the integer descriptor of a GLib.FileStream, etc. Next, use Posix.cfmakeraw and Posix.cfsetspeed to configure it. Then, to get your nice GIO streams, just pass the integer descriptor to the default GLib.UnixInputStream/GLib.UnixOutputStream constructors.
I wrote a class to handle serial communication in Vala many years ago. As an example it is a bit horrible—it's convoluted (I had plans to use it as an abstraction layer), doesn't use GIO or async (Vala didn't have the async keyword), uses char[] instead of uint8[] (we hadn't yet standardized on uint8[]), etc., but it should help you understand what you need to do. Between that example and what I wrote above, you should be able to get it working, but if you are still having trouble after you've played with it let me know and I can throw together a quick example.
Sometimes, on various Unix architectures, recompiling a program while it is running causes the program to crash with a "Bus error". Can anyone explain under which conditions this happens? First, how can updating the binary on disk do anything to the code in memory? The only thing I can imagine is that some systems mmap the code into memory and when the compiler rewrites the disk image, this causes the mmap to become invalid. What would the advantages be of such a method? It seems very suboptimal to be able to crash running codes by changing the executable.
On local filesystems, all major Unix-like systems support solving this problem by removing the file. The old vnode stays open and, even after the directory entry is gone and then reused for the new image, the old file is still there, unchanged, and now unnamed, until the last reference to it (in this case the kernel) goes away.
But if you just start rewriting it, then yes, it is mmap(3)'ed. When the block is rewritten one of two things can happen depending on which mmap(3) options the dynamic linker uses:
the kernel will invalidate the corresponding page, or
the disk image will change but existing memory pages will not
Either way, the running program is possibly in trouble. In the first case, it is essentially guaranteed to blow up, and in the second case it will get clobbered unless all of the pages have been referenced, paged in, and are never dropped.
There were two mmap flags intended to fix this. One was MAP_DENYWRITE (prevent writes) and the other was MAP_COPY, which kept a pure version of the original and prevented writers from changing the mapped image.
But DENYWRITE has been disabled for security reasons, and COPY is not implemented in any major Unix-like system.
Well this is a bit complex scenario that might be happening in your case. The reason of this error is normally the Memory Alignment issue. The Bus Error is more common to FreeBSD based system. Consider a scenario that you have a structure something like,
struct MyStruct {
char ch[29]; // 29 bytes
int32 i; // 4 bytes
}
So the total size of this structure would be 33 bytes. Now consider a system where you have 32 byte cache lines. This structure cannot be loaded in a single cache line. Now consider following statements
Struct MyStruct abc;
char *cptr = &abc; // char point points at start of structure
int32 *iptr = (cptr + 1) // iptr points at 2nd byte of structure.
Now total structure size is 33 bytes your int pointer points at 2nd byte so you can 32 byte read data from int pointer (because total size of allocated memory is 33 bytes). But when you try to read it and if the structure is allocated at the border of a cache line then it is not possible for OS to read 32 bytes in a single call. Because current cache line only contains 31 bytes data and remaining 1 bytes is on next cache line. This will result into an invalid address and will give "Buss Error". Most operating systems handle this scenario by generating two memory read calls internally but some Unix systems don't handle this scenario. To avoid this, it is recommended take care of Memory Alignment. Mostly this scenario happen when you try to type cast a structure into another datatype and try reading the memory of that structure.
The scenario is bit complex, so I am not sure if I can explain it in simpler way. I hope you understand the scenario.