Mosquitto socket read error Arduino client - arduino

I have just downloaded the latest Arduino Library code from Github, and it's broken my MQTT client program. I'm using PubSubClient 1.91 on Arduino, and Mosquitto 1.1.2 (Build 2013-03-07) on Mac OSX. (I also tested against Mosquitto on Windows 7, same problem.)
The supplied Mosquitto clients work fine, (Mac over to Windows, Windows over to Mac) so it's some problem with what's coming from the Arduino end. A wireshark trace shows the Arduino client sending the following data packet:
10:15:ff:ff:4d:51:49:73:64:70:03:02:00:0f:00:07:41:72:64:75:69:6e:6f
And the Mosquitto broker shows:
New connection from 10.0.0.115
Socket read error on client (null), disconnecting.
Before I start to crawl through the MQTT spec, can anyone see anything wrong with the data packet being sent? It's got to be something to do with new Arduino library code...
* Update
Upon further investigation, it appears to be a code generation problem with avr-g++, although life experience tells me it will turn out not to be so. Here is a snippet of code from PubSubClient.cpp
boolean PubSubClient::connect(char *id, char *user, char *pass, char* willTopic, uint8_t willQos, uint8_t willRetain, char* willMessage) {
if (!connected()) {
int result = 0;
if (domain != NULL) {
result = _client->connect(this->domain, this->port);
} else {
result = _client->connect(this->ip, this->port);
}
if (result) {
nextMsgId = 1;
uint8_t d[9] = { 0x00, 0x06, 'M','Q','I','s','d','p',MQTTPROTOCOLVERSION};
// d[0] = 0;
// d[1] = 6;
Serial.print("d[0]="); Serial.println(d[0],HEX);
Now, the result of the Serial.print just above turns out to be 0xFF !!! So, the uint8_t array is not being initialised correctly. #knoleary Your pointer to the bad FF bytes lead me to this.
If I now uncomment the two lines above, and manually initialise the first 2 bytes to 0 and 6, all works fine, and my program communicates happily with Mosquitto.
I've looked at the generated code, but I'm not an Atmel expert.
Does anyone have any clue why this might be?
I'm compiling using the AVR-G++ toolset from Arduino 1.05, in Eclipse.
I'm going for a beer!

OK, I found it. It's a relatively subtle bug. Essentially, when the following line of source code is compiled;
uint8_t d[9] = { 0x00, 0x06, 'M','Q','I','s','d','p',MQTTPROTOCOLVERSION};
the 9 bytes get stored as a constant in the data section of the image. At runtime, a small loop copies the 9 bytes into the array (d[]) By looking at a combined Assembler / source listing, I could see where in the data section the 9 bytes were stored, and then print them out at regular intervals, until I found what was over-writing them. (A bit primitive, I know!)
It turns out the there's a bug in WiFi.cpp , the Arduino WiFi code. Here's the code:
uint8_t WiFiClient::connected() {
if (_sock == 255) {
return 0;
} else {
uint8_t s = status();
return !(s == LISTEN || s == CLOSED || s == FIN_WAIT_1 ||
s == FIN_WAIT_2 || s == TIME_WAIT ||
s == SYN_SENT || s== SYN_RCVD ||
(s == CLOSE_WAIT));
}
}
It turns out the the _sock variable is actually initialised like this:
WiFiClient::WiFiClient() : _sock(MAX_SOCK_NUM) {
}
and MAX_SOCK_NUM is 4, not 255. So, WiFiClient::status returned true, instead of false for an unused Socket.
This method was called by the MQTT Client like this:
boolean PubSubClient::connected() {
boolean rc;
if (_client == NULL ) {
rc = false;
} else {
rc = (int)_client->connected();
if (!rc) _client->stop();
}
return rc;
}
And, since the _client->connected() method erroneously returned true, the _client_stop() method was called. This resulted in a write to a non-existent socket array element, and so overwrote my string data.
#knolleary, I was wondering, is there any specific reason that your PubSubClient::connected() method does a disconnect? I use the ::connected method in a loop, to check that I'm still connected, and, of course it results in my getting a disconnect / reconnect each time round the loop. Any chance we could just make connected return true / false , and handle the disconnect in PuBSubClient::connect?

Nearly one and a half year later I ran into the same problem. Removing the
boolean PubSubClient::connected() {
int rc = (int)_client->connected();
if (!rc) _client->stop();
return rc;
}
the _client->stop() from the connected method of PubSubClient forehand fixed this problem for me. However, I'm not sure whether this is actually a solution or just a very dirty quick hack to localize the problem.
What have you done to fix this problem - your explanation of the problem above is fine however, I was not able to extract the solution easily ;-)

Related

Begin Transmission and Receiving Byte using I2C, PSOC

I'm new to the PSoC board and I'm trying to read the x,y,z values from a Digital Compass but I'm having a problem in beginning the Transmission with the compass itself.
I found some Arduino tutorial online here but since PSoC doesn't have the library I can't duplicate the code.
Also I was reading the HMC5883L datasheet here and I'm suppose to write bytes to the compass and obtain the values but I was unable to receive anything. All the values I received are zero which might be caused by reading values from wrong address.
Hoping for your answer soon.
PSoC is sorta tricky when you are first starting out with it. You need to read over the documentation carefully of both the device you want to talk to and the i2c module itself.
The datasheet for the device you linked states this on page 18:
All bus transactions begin with the master device issuing the start sequence followed by the slave address byte. The
address byte contains the slave address; the upper 7 bits (bits7-1), and the Least Significant bit (LSb). The LSb of the
address byte designates if the operation is a read (LSb=1) or a write (LSb=0). At the 9
th clock pulse, the receiving slave
device will issue the ACK (or NACK). Following these bus events, the master will send data bytes for a write operation, or
the slave will clock out data with a read operation. All bus transactions are terminated with the master issuing a stop
sequence.
If you use the I2C_MasterWriteBuf function, it wraps all that stuff the HMC's datasheet states above. The start command, dealing with that ack, the data handling, etc. The only thing you need to specify is how to transmit it.
If you refer to PSoC's I2C module datasheet, the MasterWriteBuf function takes in the device address, a pointer to the data you want to send, how many bytes you want to send, and a "mode". It shows what the various transfer modes in the docs.
I2C_MODE_COMPLETE_XFER Perform complete transfer from Start to Stop.
I2C_MODE_REPEAT_START Send Repeat Start instead of Start.
I2C_MODE_NO_STOP Execute transfer without a Stop
The MODE_COMPLETE_XFRE transfer will send the start and stop command for you if I'm not mistaken.
You can "bit-bang" this also if you want but calling directly on the I2C_MasterSendStart, WriteByte, SendStop, etc. But it's just easier to call on their writebuf functions.
Pretty much you need to write your code like follows:
// fill in your data or pass in the buffer of data you want to write
// if this is contained in a function call. I'm basing this off of HMC's docs
uint8 writeBuffer[3];
uint8 readBuffer[6];
writeBuffer[0] = 0x3C;
writeBuffer[1] = 0x00;
writeBuffer[2] = 0x70;
I2C_MasterWriteBuf(HMC_SLAVE_ADDRESS, &writeBuffer, 3, I2C_MODE_COMPLETE_XFER);
while((I2C_MasterStatus() & I2C_MSTAT_WR_CMPLT) == 0u)
{
// wait for operation to finish
}
writeBuffer[1] = 0x01;
writeBuffer[2] = 0xA0;
I2C_MasterWriteBuf(HMC_SLAVE_ADDRESS, &writeBuffer, 3, I2C_MODE_COMPLETE_XFER);
// wait for operation to finish
writeBuffer[1] = 0x02;
writeBuffer[2] = 0x00;
I2C_MasterWriteBuf(HMC_SLAVE_ADDRESS, &writeBuffer, 3, I2C_MODE_COMPLETE_XFER);
// wait for operation to finish
CyDelay(6); // docs state 6ms delay before you can start looping around to read
for(;;)
{
writeBuffer[0] = 0x3D;
writeBuffer[1] = 0x06;
I2C_MasterWriteBuf(HMC_SLAVE_ADDRESS, &writeBuffer, 2, I2C_MODE_COMPLETE_XFER);
// wait for operation to finish
// Docs don't state any different sort of bus transactions for reads.
// I'm assuming it'll be the same as a write
I2C_MasterReadBuf(HMC_SLAVE_ADDRESS, readBuffer, 6, I2C_MODE_COMPLETE_XFER);
// wait for operation to finish, wait on I2C_MSTAT_RD_CMPLT instead of WR_COMPLT
// You should have something in readBuffer to work with
CyDelay(67); // docs state to wait 67ms before reading again
}
I just sorta wrote that off the top of my head. I have no idea if that'll work or not, but I think that should be a good place to start and try. They have I2C example projects to look at also I think.
Another thing to look at so the WriteBuf function doesn't just seem like some magical command, if you right-click on the MasterWriteBuf function and click on "Find Definition" (after you build the project) it'll show you what it's doing.
Following are the samples for I2C read and write operation on PSoC,
simple Write operation:
//Dumpy data values to write
uint8 writebuffer[3]
writebuffer[0] = 0x23
writebuffer[1] = 0xEF
writebuffer[2] = 0x0F
uint8 I2C_MasterWrite(uint8 slaveAddr, uint8 nbytes)
{
uint8 volatile status;
status = I2C_MasterClearStatus();
if(!(status & I2C_MSTAT_ERR_XFER))
{
status = I2C_MasterWriteBuf(slaveAddr, (uint8 *)&writebuffer, nbytes, I2C_MODE_COMPLETE_XFER);
if(status == I2C_MSTR_NO_ERROR)
{
/* wait for write complete and no error */
do
{
status = I2C_MasterStatus();
} while((status & (I2C_MSTAT_WR_CMPLT | I2C_MSTAT_ERR_XFER)) == 0u);
}
else
{
/* translate from I2CM_MasterWriteBuf() error output to
* I2C_MasterStatus() error output */
status = I2C_MSTAT_ERR_XFER;
}
}
return status;
}
Read Operation:
void I2C_MasterRead(uint8 slaveaddress, uint8 nbytes)
{
uint8 volatile status;
status = I2C_MasterClearStatus();
if(!(status & I2C_MSTAT_ERR_XFER))
{
/* Then do the read */
status = I2C_MasterClearStatus();
if(!(status & I2C_MSTAT_ERR_XFER))
{
status = I2C_MasterReadBuf(slaveaddress,
(uint8 *)&(readbuffer),
nbytes, I2C_MODE_COMPLETE_XFER);
if(status == I2C_MSTR_NO_ERROR)
{
/* wait for reading complete and no error */
do
{
status = I2C_MasterStatus();
} while((status & (I2C_MSTAT_RD_CMPLT | I2C_MSTAT_ERR_XFER)) == 0u);
if(!(status & I2C_MSTAT_ERR_XFER))
{
/* Decrement all RW bytes in the EZI2C buffer, by different values */
for(uint8 i = 0u; i < nbytes; i++)
{
readbuffer[i] -= (i + 1);
}
}
}
else
{
/* translate from I2C_MasterReadBuf() error output to
* I2C_MasterStatus() error output */
status = I2C_MSTAT_ERR_XFER;
}
}
}
if(status & I2C_MSTAT_ERR_XFER)
{
/* add error handler code here */
}
}

arduino and esp8266 - how to get AT command response into variable

So i have my arduino and esp8266 wifi module. Everything is connected correctly and sending data to arduino lets me control connection via AT commands.
My skletch looks like this:
void setup()
{
Serial.begin(9600);
Serial1.begin(9600);
}
void loop()
{
if (Serial.available() > 0) {
char ch = Serial.read();
Serial1.print(ch);
}
if (Serial1.available() > 0) {
char ch = Serial1.read();
Serial.print(ch);
}
The code above lets me send commands and see response from esp. In spite of different response time and different answers I need to store response in variable when wifi module such a response creates. Unfortunatelly I cant do this because of Serial1.read() grabing only one char from Serial1.available() buffer instead of full buffer.
I tried approach like this:
if (Serial1.available() > 0) {
while (Serial1.available() > 0) {
char ch = Serial1.read();
Serial.print(ch);
response = response.concat(ch);
}
} else {
String response = "";
}
So as long ther eis something in a buffer it is sent to response variable that concatens last char with itself. And later it can be searched via indefOf command for "OK" marker or "ERROR". But that doesnt work as intended :( It for example may print my variable 8 times (dont know why).
I need full response from wifi module to analaze it for example to make the led on in my arduino board if proper command comes from wifi network, but also send some data if i press button on arduino to the network. Any ideas would be appreciated.
Kalreg.
Try this rather:
String response = ""; // No need to recreate the String each time no data is available
char ch; // No need to recreate the variable in a loop
while (Serial1.available() > 0) {
ch = Serial1.read();
response = response.concat(ch);
}
// Now do whatever you want with the string by first checking if it is empty or not. Then do something with it
Also remember to clear the buffer before you send a command like I suggested in your previous question: how to get AT response from ESP8266 connected to arduino

Sending char buffer through TCP socket incomplete

I'm just learning how to handle sockets and TCP connections in C. I've got an application (a long one) which basically sends and receives char arrays with the system call write from server to client and vice versa (two separate C applications of course). As long as I use it with a local connection, on the same PC, running the server on a terminal and the client on an another, everything just works fine and the data arrives at the destination. But if I try it with the server on one computer and the client on another but on the same internet line, passing to the client an address like 192.168.1.X (took from the machine on which the server is running), after the connection is established, I've got an error that tells me that the number of expected bytes (which I pass before sending the real char[]) isn't arrived. Same thing if I try the server on my PC, and the client on another one with a different line on a different provider.
There's something I'm missing, are there any limitations in sending a bunch of bytes in sequence?
The code where the error pops up.
SERVER SIDE:
r=htonl(lghstr);
w=write(myFd,&r,sizeof(int));//writes the number of incoming bytes
if(w<0) perror("writeServer4"),exit(-1);
w=write(myFd,tmp->string,lghstr);
if(w<0) perror("writeServer5"),exit(-1);
if(w!=lghstr) perror("ERROR");
CLIENT SIDE
rC=read(fdc,&cod,sizeof(int));//read incoming number of bytes
lghstr=ntohl(cod);
if(rC<0) perror("readClient3"),exit(-1);
rC=read(fdc,dest,lghstr);
if(rC<0) perror("readClient4"),exit(-1);
if(rC!=lghstr) perror("error : "), printf("didn't read the right number of bytes"),exit(-1);
Now this is basically repeated a lot of times, let's even say 300 times, and it's with big numbers that the program doesn't work.
This is the problem:
rC=read(fdc,dest,lghstr);
...
if(rC!=lghstr) perror("error : ")
The #1 fallacy with socket programming is expecting that recv() and read() will return exactly the same number of bytes corresponding to the write/send call made by the other side.
In reality, partial data is extremely likely and expected. The simple workaround is to loop on read/recv until you get the exact number of bytes expected:
size_t count = 0;
while (count < lghstr)
{
ssize_t readresult = read(fdc, dest+count, lghstr-count);
if (readresult == -1)
{
// socket error - handle appropriately (typically, just close the connection)
}
else if (readresult == 0)
{
// The other side closed the connection - handle appropriately (close the connection)
}
else
{
count += readresult;
}
}
The other alternative to looping is to the use the MSG_WAITALL flag with the socket. This means, using recv() instead of read(). You'll still need to handle the error cases.
rc = recv(fdc, dest, lghstr, MSG_WAITALL);
if (rc == -1)
{
// socket error
}
else if (rc == 0)
{
// socket closed by remote
}
else if (rc < lghstr)
{
// the other side likely closed the connection and this is residual data (next recv will return 0)
}
You do ntohl() on one side and not the other. That might be interpreting the bytes with the wrong value.
You should printf() the bytes on both sides and see what the int is being evaluated to.
Edit: I'm convinced this is a programming bug for the record.
If I had to guess, I'd say that you are not synchronous with the other side for some reason. You say this runs 'about 300 times'.
Try adding a magic integer to the protocol.
Heres an example of a client that sends in this order.
A magic integer which is always constant.
A lengh of bytes about to be sent.
The bytes to be sent.
This uses scatter gather mechanics (its nicer for serialization) but other than that it effectively is doing the same thing yours is doing, as a client, just adding a magic value.
When the receiver receives the data, it can validate that the data is coming in the right order, by checking what the magic number was that came in. If the magic is wrong it means the client or server has lost themselves positionally in the stream.
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/types.h>
#include <sys/uio.h>
#include <err.h>
#include <time.h>
#define MAGIC 0xDEADBEEFLU
#define GARBAGE_MAX 65536
const int iterations = 3000;
char * create_garbage_buf(
void)
{
int rc = -1;
int fd = -1;
char *buf = NULL;
buf = malloc(GARBAGE_MAX);
if (!buf)
err(1, "Cannot allocate buf");
fd = open("/dev/urandom", O_RDONLY);
if (fd < 0)
err(1, "Cannot open urandom");
rc = read(fd, buf, GARBAGE_MAX);
if (rc < 0)
err(1, "Cannot read from urandom");
else if (rc != GARBAGE_MAX)
errx(1, "Expected %d bytes, but got %d reading from urandom",
GARBAGE_MAX, rc);
close(fd);
return buf;
}
int main() {
int fd, offset, i, rc;
uint32_t magic = MAGIC;
uint32_t blen = 0;
char *buf = NULL;
struct iovec vecs[3];
/* Seed poor random number generator */
srand(time(NULL));
/* Use a file for demonstration, but a socket will do just fine */
fd = open("/dev/null", O_WRONLY);
/* Create some garbage to send */
buf = create_garbage_buf();
if (fd < 0)
err(1, "Cannot open file");
/* The first vector, is always the magic */
vecs[0].iov_len = sizeof(uint32_t);
vecs[0].iov_base = &magic;
for (i=0; i < iterations; i++) {
/* The second vector represents lengh of what we send
* in this demonstration it is a number between 0 and
* GARBAGE_MAX/2.
*/
blen = rand() % (GARBAGE_MAX / 2);
vecs[1].iov_len = sizeof(uint32_t);
vecs[1].iov_base = &blen;
/* The last record is the data to send. Its another random
* number between 0 and GARBAGE_MAX which represents the offset
* in our garbage data to send */
offset = rand() % (GARBAGE_MAX / 2);
vecs[2].iov_len = blen;
vecs[2].iov_base = &buf[offset];
rc = writev(fd, vecs, 3);
if (rc < 0)
err(1, "Could not write data");
if (rc != (sizeof(uint32_t)*2 + blen))
errx(1, "Did not write proper number of bytes to handle");
printf("Wrote %u bytes from offset %u in garbage\n", blen, offset);
}
free(buf);
printf("Done!\n");
return 0;
}
Closely read the documentation for read()/write() and learn that those two functions do not necessarily read()/write() as much bytes as they were told to, but few. So looping around such calls counting until all data expected had been read/written is a good idea, not to say an essential necessity.
For examples how this could be done for writing you might like to have look at this answer: https://stackoverflow.com/a/24260280/694576 and for reading on this answer: https://stackoverflow.com/a/20149925/694576

UDR register cleared before reading data

I am trying to simulate the uart using ATmega128. I have written this code in AVR STUDIO 4.
The PORTB0 is for used switch so that when it is pressed it is connected to 5v dc and it sends 'a' to uart1. at other times it is connected to ground. the reception of data is by interrupt.
Using debugger, when there is data in UDR1 and RXC1 is set, program jumps to ISR, and then UDR register is immediately cleared and nothing is retrieved. Can any one tell me why this happens?
Here is the code.
volatile unsigned char rxdata;
void uart_init(void)
{
UCSR1A = 0x00;
UCSR1B |= (1<<RXCIE1)|(1<<RXEN1)|(1<<TXEN1); //0b10011000;
UCSR1C |= (1<<7)|(1<<UCSZ11)|(UCSZ10); //0b10000110;
UBRR1H = 0;
UBRR1L = 103; //9600 baud rate
}
ISR(USART1_RX_vect)
{
rxdata = UDR1;
PORTC = rxdata;
}
void putch(char data)
{
while(!(UCSR1A & 0x20));
UDR1 = data;
}
And the main program is
void port_init(void)
{
DDRC = 0xFF;
}
int main(void)
{
port_init();
uart_init();
sei();
while(1)
{
if (PINB & 0x01){
putch('a');
}
}
}
I had this once. In my case, setting the breakpoint before the flag was evaluated in the code cleared it, because The AVR Studio "read" the flag (as I had the flag register open). Setting the breakpoint AFTER the line where the flag was read, helped. In your case, set the breakpoint on line PORTC = rxdata;
To get a better debug feeling, I read the flag into a variable right at the beginning of the ISR and set the breakpoint right after this.
It's been some years since this happened and I'm not even sure if this was really the case. So, maybe you can verify this ;)
I took a look at the AVR Studio 4 help section. Regarding known simulator issues with respect to UART functions it states:
The UART/USART UDR register can only be modified from the application. Input via stimuli files or by modifying the I/O view etc is not possible.

Unix Networking Programming - Client and Server. List Function That wait for input after 40 lines

I am currently in the process of making a Client and Server in the Unix/Windows environment but right now I am just working on the Unix side of it. One of the function we have to create for the program is similar to the list function in Unix which shows all files within a dir but we also have to show more information about the file such as its owner and creation date. Right now I am able to get all this information and print it to the client however we have to also add that once the program has printing 40 lines it waits for the client to push any key before it continues to print.
I have gotta the program to sort of do this but it will cause my client and server to become out of sync or at least the std out to become out of sync. This means that if i enter the command 'asdad' it should print invalid command but it won't print that message until i enter another command. I have added my list functions code below. I am open to suggestions how how to complete this requirement as the method I have chosen does not seem to be working out.
Thank-you in advance.
Server - Fork Function: This is called when the list command is enter. eg
fork_request(newsockfd, "list", buf);
int fork_request(int fd, char req[], char buf[])
{
#ifndef WIN
int pid = fork();
if (pid ==-1)
{
printf("Failed To Fork...\n");
return-1;
}
if (pid !=0)
{
wait(NULL);
return 10;
}
dup2(fd,1); //redirect standard output to the clients std output.
close(fd); //close the socket
execl(req, req, buf, NULL); //run the program
exit(1);
#else
#endif
}
Here is the function used to get all the info about a file in a dir
void longOutput(char str[])
{
char cwd[1024];
DIR *dip;
struct dirent *dit;
int total;
char temp[100];
struct stat FileAttrib;
struct tm *pTm;
int fileSize;
int lineTotal;
if(strcmp(str, "") == 0)
{
getcwd(cwd, sizeof(cwd));
}
else
{
strcpy (cwd, str);
}
if (cwd != NULL)
{
printf("\n Using Dir: %s\n", cwd);
dip = opendir(cwd);
if(dip != NULL)
{
while ((dit = readdir(dip)) != NULL)
{
printf("\n%s",dit->d_name);
stat(dit->d_name, &FileAttrib);
pTm = gmtime(&FileAttrib.st_ctime);
fileSize = FileAttrib.st_size;
printf("\nFile Size: %d Bytes", fileSize);
printf("\nFile created on: %.2i/%.2i/%.2i at %.2i:%.2i:%.2i GMT \n", (pTm->tm_mon + 1), pTm->tm_mday,(pTm->tm_year % 100),pTm->tm_hour,pTm->tm_min, pTm->tm_sec);;
lineTotal = lineTotal + 4;
if(lineTotal == 40)
{
printf("40 Lines: Waiting For Input!");
fflush(stdout);
gets(&temp);
}
}
printf("\n %d \n", lineTotal);
}
else
{
perror ("");
}
}
}
At here is the section of the client where i check that a ! was not found in the returned message. If there is it means that there were more lines to print.
if(strchr(command,'!') != NULL)
{
char temp[1000];
gets(&temp);
}
Sorry for the long post but if you need anything please just ask.
Although, I didn't see any TCP/IP code, I once had a similar problem when I wrote a server-client chat program in C++. In my case, the problem was that I didn't clearly define how messages were structured in my application. Once, I defined how my protocol was suppose to work--it was a lot easier to debug communication problems.
Maybe you should check how your program determines if a message is complete. In TCP, packets are guaranteed to arrive in order with no data loss, etc. Much like a conversation over a telephone. The only thing you have to be careful of is that it's possible to receive a message partially when you read the buffer for the socket. The only way you know to stop reading is when you determine a message is complete. This could be as simple as two '\n' characters or "\n\r".
If you are using UDP, then that is a completely different beast all together (i.e. messages can arrive out of order and can be lost in transit, et cetera).
Also, it looks like you are sending across strings and no binary data. If this is the case, then you don't have to worry about endianess.

Resources