Arduino - sending large amount of data over Serial - arduino

I am trying to build a simple FLASH memory programmer (for 39SF020A) using my arduino mega. I wrote the C code and Python script to send the data over (And it all works as expected).
I need to transfer about 32k of hexadecimal data, but with my settings only 10k of data took about 4 minutes (115200 BAUD), which i found unnecessary long. Currently, i am sending over serial (from Python) my value with a terminator (i chose '$'), so for exmple '3F$'. adresses are calulated on the arduino, so no need to send them.
In my arduino code, i have
String received_string = Serial.readStringUntil('$');
and after programming every byte to teh FLASH using arduino, it sends back a '\n' to let the Python know, that it is ready to receive next byte (the python is waiting for receiving a 'line' and then continues). I am not really sure if this is a way to do it, if sending only one byte at the time is good idea and if not, how many and how do i parse them on the arduino? Is the feedback loop useful?
Thanks.
Python Code:
('file' contains all data)
for item in file[1:]:
ser.write((item + "$").encode("ascii"))
line = ser.readline()
i += 1
if i >= top:
break
elif (i % 100) == 0:
print(i)
Arduino code (just part of it)
if (Serial.available() > 0){
String received_string = Serial.readStringUntil('$');
programData(received_string.toInt(),program_adress);
program_adress++;
}
void programData(char data_in, unsigned long adress)
{
digitalWrite(OE,HIGH);
digitalWrite(CE,LOW);
writeByte(0xAA, 0x5555);
writeByte(0x55, 0x2AAA);
writeByte(0xA0, 0x5555);
writeByte(data_in, adress);
Serial.print("\n"); // Feedback for Python
delayMicroseconds(30); // Just to be on the safe side
}
void writeByte(char data_in, unsigned long adress)
{
setDataAs(OUTPUT);
digitalWrite(OE,HIGH);
digitalWrite(WE,HIGH);
setAdress(adress);
setData(data_in);
digitalWrite(WE,LOW);
delayMicroseconds(1);
digitalWrite(WE,HIGH);
}
// Sets data BUS to input or output
void setDataAs(char dir){
for (unsigned int i = 0; i < data_size ;i++) pinMode(data[i],dir);
}
// Sets data to specific values
void setData(char data_i){
setDataAs(OUTPUT);
for (int i = 0; i < data_size;i++) { digitalWrite(data[i],bitRead(data_i,i)); }
}
void setAdress(long adr){
// Set all adresses
for (int i = 0; i < adresses_size;i++)
digitalWrite(adresses[i],bitRead(adr,i));
}

Related

Reading multiple bytes in Arduino sent using Pyserial

I'm constantly sending structs of int64 via Pyserial with:
with serial.Serial(port='COM4', baudrate=115200, timeout=.1) as arduino:
value = write_read(struct.pack(">q", int_array[1][i])) #this sends signed int.64 in bytes
print(value)
the struct.pack has this shape, for example:
b'\xff\xff\xff\xff\xff\xff\xff\xef'
and the function write_read consists of:
def write_read(x):
arduino.write((x))
data = arduino.readline()
#the idea is to receive an ACK from the Arduino after 8 bytes (the full
#number)
return data
The code I'm trying to develop in arduino is the following:
void loop() {
// send data only when you receive data:
if (Serial.available() \> 0) {
// read the incoming byte:
incomingByte = Serial.read();
//read 8 bytes and create the result
r= function_to_read_8_last_bytes // or similar
// say what you got:
Serial.print("I received: ");
Serial.printlesultn(r, DEC);
Serial.write("ACK");
}
}
I'm very curious how I could do a robust "read 8 bytes" function.
Should I add some especial character in the Python part to indentify when it ends one value?
Thanks! I'll appreciate any help :)
Given the discussion in the comments, it's hard to receive a stream of bytes and be sure that the receiver is completely synchronized. However let's make some assumptions to ease the problem:
The serial buffer is empty when you connect your laptop to Arduino. This ensures you won't receive spurious data with no meaning. I had this problem happens a lot when the serial connection was ended abruptly by any cause.
You are not constantly sending bytes, Arduino has time to process them until the start of the new sequence.
You only send this data, so there is no need to create a higher level protocol on top of it. Bare in mind that the serial communication is almost just an hardware stack, you receive bytes with no headers.
For assumption 1 you can write a simple piece of code to consume all the spurious bytes in the serial buffer as soon as your main starts from Arudino, so this will be done everytime you connect the serial (as this is also where the power supply comes from). Something like this:
void serialFlush(){
while(Serial.available() > 0) {
char t = Serial.read();
}
}
You can send a "READY" signal back to the Python interface, so that the program knows you are ready to receive data.
Going on with the solution you can implement an easy CRC in python, an additional byte which contains a XOR of all the previous bytes, and you check that in Arduino upon reception complete.
def xor_reduce_long_int(li):
res = 0;
for i in range(8):
mask = (0xFF)<<(i*8)
print(hex(mask))
masked = (li&mask)>>(i*8)
res ^= masked
return res
with serial.Serial(port='COM4', baudrate=115200, timeout=.1) as arduino:
crc=xor_reduce_long_int(int_array[1][i])
value = write_read(struct.pack(">qc", int_array[1][i],crc)) #this sends signed int.64 in bytes
print(value)
And with Arduino I would read 8 bytes when they are available and put them into an unsigned char buffer. I would then define a union that alias such buffer to interpret it as long long int.
typedef struct long_int_CRC
{
union
{
unsigned char bytes[8];
long int data;
};
unsigned char CRC;
}data_T;
// .. Later in main
data_T = received_data;
int received_bytes=0
unsigned char my_CRC = 0;
unsigned char rec_byte= 0;
while( received_bytes < 8 )
{
if(Serial.available() )
{
// Get the byte
rec_byte = Serial.read()
// Store the byte and calc CRC
received_data.bytes[received_bytes] = rec_byte;
my_CRC ^= rec_byte;
// Increment counter for next byte
received_bytes++;
}
}
// Reception complete, check CRC
unsigned char rec_CRC;
if(Serial.available() )
{
rec_CRC = Serial.read()
}
if( my_CRC != rec_CRC )
{
// Something was wrong!
}
// Now access your data as a long int
Serial.print("I received: ");
Serial.printlesultn(received_data.data, DEC);
Serial.write("ACK");

arduino and esp8266 - how to get AT command response into variable

So i have my arduino and esp8266 wifi module. Everything is connected correctly and sending data to arduino lets me control connection via AT commands.
My skletch looks like this:
void setup()
{
Serial.begin(9600);
Serial1.begin(9600);
}
void loop()
{
if (Serial.available() > 0) {
char ch = Serial.read();
Serial1.print(ch);
}
if (Serial1.available() > 0) {
char ch = Serial1.read();
Serial.print(ch);
}
The code above lets me send commands and see response from esp. In spite of different response time and different answers I need to store response in variable when wifi module such a response creates. Unfortunatelly I cant do this because of Serial1.read() grabing only one char from Serial1.available() buffer instead of full buffer.
I tried approach like this:
if (Serial1.available() > 0) {
while (Serial1.available() > 0) {
char ch = Serial1.read();
Serial.print(ch);
response = response.concat(ch);
}
} else {
String response = "";
}
So as long ther eis something in a buffer it is sent to response variable that concatens last char with itself. And later it can be searched via indefOf command for "OK" marker or "ERROR". But that doesnt work as intended :( It for example may print my variable 8 times (dont know why).
I need full response from wifi module to analaze it for example to make the led on in my arduino board if proper command comes from wifi network, but also send some data if i press button on arduino to the network. Any ideas would be appreciated.
Kalreg.
Try this rather:
String response = ""; // No need to recreate the String each time no data is available
char ch; // No need to recreate the variable in a loop
while (Serial1.available() > 0) {
ch = Serial1.read();
response = response.concat(ch);
}
// Now do whatever you want with the string by first checking if it is empty or not. Then do something with it
Also remember to clear the buffer before you send a command like I suggested in your previous question: how to get AT response from ESP8266 connected to arduino

How to read string using Serial.read outside of loop()?

I want to read a String in Arduino from the keyboard outside of the loop() method.
I have the following method:
void readFromKeyboard(byte arrayAddress[])
{
int count = 0, i = 0;
while ((count = Serial.available()) == 0);
while (i<count)
{
arrayAddress[i++] = Serial.read();
}
}
In the loop() method I am calling it like:
readFromKeyboard(userInput);
where userInput is a byte[];
The problem is that when I input more than one characters it read the 1st character initially and it call the readFromKeyboard again an then reads the rest.
Example; if I input "asdf":
--the 1st time it will do ==> userInput = "a"
--the 2nd time it will do ==> userInput = "sdf"
I have tryed many things but the same happens again and again...
Any suggestions??
So that's what worked:
In the loop():
while(Serial.available() == 0);
delay(100);
readInputFlag = readFromKeyboard(userInput);`
And in the readFromKeyboard method:
void readFromKeyboard(byte arrayAddress[])
{
int i = 0;
while (Serial.available() > 0)
{
arrayAddress[i++] = Serial.read();
}
}
This delay, in the loop method, somehow makes the Serial get the whole string instead of just the first letter.
I know you got it working, but I wanted to show you something that I use to deal with this issue. This is a two-tiered delay system for catching bytes that come in a bit late for whatever reason. It's designed to minimize the delay needed to accomplish that task.
int received_length = 0;
byte serial_incoming_buffer[200];
while(Serial.available()) {
serial_incoming_buffer[received_length++] = Serial.read();
if(!Serial.available()) {
delay(3);
if(!Serial.available()) {
delay(20);
}
}
}
Sometimes the Arduino falls behind in picking up serial from the sender and sometimes it grabs serial too fast. Sometimes the sender lags a little bit. This code will wait 3 ms for more bytes, and if they come in it goes back to receiving as many as are available having only had that very brief delay. This repeats as necessary, then when 3 ms goes by without anything being available, it waits a bit longer (20 ms here) for more bytes. If nothing comes in after the long delay, then the transmission is most likely done and you can safely move on.
I recommend tweaking the delays based on your baud rate.

How to send 4 Pot values via i2c from Arduino to Arduino? How to differentiate these values while receiving them?

I have one Arduino with 4 Pots. The other Arduino receives these 4 values via i2c and prints them on a Display. The problem is that I don't know how to send these 4 values that the Slave is able to know which value belongs to which Pot.
Slave Code:
#include <Wire.h>
#include <LiquidCrystal.h>
LiquidCrystal lcd(12, 11, 5, 4, 3, 2);
void setup()
{
Wire.begin(5);
Wire.onReceive(receiveEvent);
Serial.begin(9600);
lcd.begin(16,2);
}
void loop()
{
}
void receiveEvent(int)
{
while(Wire.available())
{
//How to create this part? How does the Slave know which value belongs to which pot?
}
}
Master Code:
#include <Wire.h>
void setup()
{
Serial.begin(9600);
Wire.begin();
delay(2000);
}
void loop()
{
int sensor1 = analogRead(A1);
Wire.beginTransmission(5);
Wire.write(sensor1);
Serial.print(sensor1);
Wire.endTransmission();
delay(100);
int sensor2 = analogRead(A2);
Wire.beginTransmission(5);
Wire.write(sensor2);
Serial.print(sensor2);
Wire.endTransmission();
delay(500);
}
Ahh what we have here is a basic question on how to design I2C communication. Unfortunately Examples for I2C master and slave included in Arduino IDE are IMO too limited to provide clear guidance on this matter.
First of all in your examples the master and slaves roles are exchanged and should be switched. Slave should read values from analog inputs and master should request them. Why? Because it's master which should decide when to request values and properly decode the request. Slave should provide proper answer to a given request eliminating the problem of data interpretation.
I2C communication is based on requestFunction-(wait)-requestResponse sequence controlled by the master.
Plese refer to the range finder example on arduino page. In a nutshell:
First: master requests a function to measure distance:
// step 3: instruct sensor to return a particular echo reading
Wire.beginTransmission(112); // transmit to device #112
Wire.write(byte(0x02)); // sets register pointer to echo #1 register (0x02)
Wire.endTransmission(); // stop transmitting
(sometimes slaves need some time e.g. 10 - 50 ms to process requests but in the example I'm refering to master doesn't delay read)
Second: master requests response:
// step 4: request reading from sensor
Wire.requestFrom(112, 2); // request 2 bytes from slave device #112
Third: master tries to read and analyze response.
You should design reliable I2C communication in a similar way.
Here is how I do it; you can follow my pattern and get extensible slave implementation which will support one function: read analog inputs but can be easily extended by adding additional function codes and required processing implementation to the slave main loop
Initial remarks
some kind of a simple protocol is needed to control slave - e.g. it should support requesting functions. Supporting functions requests is not absolutely needed in such simmple scenario as reading four analog inputs but what I'm describing is a more general pattern you may use in other projects.
Slave should not perform any additional actions (like reading inputs) on request response as I2C communication may break (due to delays) and you will get partial responses etc. This is very important requirement which affect the slave design.
response (and also request if needed) can contain CRC as if master waits not long enough it may get empty response. If nobody else is going to use your code such countermeasures are not needed and will not be described here. Other important thing is Wire library buffer limitation which is 32 bytes and implementing CRC checksum without modifying the buffer length limits the available data length by two bytes (if crc16 is used).
slave:
#include <WSWire.h> // look on the web for an improved wire library which improves reliability by performing re-init on lockups
// >> put this into a header file you include at the beginning for better clarity
enum {
I2C_CMD_GET_ANALOGS = 1
};
enum {
I2C_MSG_ARGS_MAX = 32,
I2C_RESP_LEN_MAX = 32
};
#define I2C_ADDR 0
#define TWI_FREQ_SETTING 400000L // 400KHz for I2C
#define CPU_FREQ 16000000L // 16MHz
extern const byte supportedI2Ccmd[] = {
1
};
// << put this into a header file you include at the beginning for better clarity
int argsCnt = 0; // how many arguments were passed with given command
int requestedCmd = 0; // which command was requested (if any)
byte i2cArgs[I2C_MSG_ARGS_MAX]; // array to store args received from master
int i2cArgsLen = 0; // how many args passed by master to given command
uint8_t i2cResponse[I2C_RESP_LEN_MAX]; // array to store response
int i2cResponseLen = 0; // response length
void setup()
{
// >> starting i2c
TWBR = ((CPU_FREQ / TWI_FREQ_SETTING) - 16) / 2;
Wire.begin(I2C_ADDR); // join i2c bus
Wire.onRequest(requestEvent); // register event
Wire.onReceive(receiveEvent);
// << starting i2c
}
void loop()
{
if(requestedCmd == I2C_CMD_GET_ANALOGS){
// read inputs and save to response array; example (not tested) below
i2cResponseLen = 0;
// analog readings should be averaged and not read one-by-one to reduce noise which is not done in this example
i2cResponseLen++;
i2cResponse[i2cResponseLen -1] = analogRead(A0);
i2cResponseLen++;
i2cResponse[i2cResponseLen -1] = analogRead(A1);
i2cResponseLen++;
i2cResponse[i2cResponseLen -1] = analogRead(A2);
i2cResponseLen++;
i2cResponse[i2cResponseLen -1] = analogRead(A3);
// now slave is ready to send back four bytes each holding analog reading from a specific analog input; you can improve robustness of the protocol by including e.g. crc16 at the end or instead of returning just 4 bytes return 8 where odd bytes indicate analog input indexes and even bytes their values; change master implementation accordingly
requestedCmd = 0; // set requestd cmd to 0 disabling processing in next loop
}
else if (requestedCmd != 0){
// log the requested function is unsupported (e.g. by writing to serial port or soft serial
requestedCmd = 0; // set requestd cmd to 0 disabling processing in next loop
}
}
// function that executes whenever data is requested by master
// this function is registered as an event, see setup()
void requestEvent(){
Wire.write(i2cResponse, i2cResponseLen);
}
// function that executes when master sends data (begin-end transmission)
// this function is registered as an event, see setup()
void receiveEvent(int howMany)
{
//digitalWrite(13,HIGH);
int cmdRcvd = -1;
int argIndex = -1;
argsCnt = 0;
if (Wire.available()){
cmdRcvd = Wire.read(); // receive first byte - command assumed
while(Wire.available()){ // receive rest of tramsmission from master assuming arguments to the command
if (argIndex < I2C_MSG_ARGS_MAX){
argIndex++;
i2cArgs[argIndex] = Wire.read();
}
else{
; // implement logging error: "too many arguments"
}
argsCnt = argIndex+1;
}
}
else{
// implement logging error: "empty request"
return;
}
// validating command is supported by slave
int fcnt = -1;
for (int i = 0; i < sizeof(supportedI2Ccmd); i++) {
if (supportedI2Ccmd[i] == cmdRcvd) {
fcnt = i;
}
}
if (fcnt<0){
// implement logging error: "command not supported"
return;
}
requestedCmd = cmdRcvd;
// now main loop code should pick up a command to execute and prepare required response when master waits before requesting response
}
master:
#include <WSWire.h>
#define I2C_REQ_DELAY_MS 2 // used for IO reads - from node's memory (fast)
#define I2C_REQ_LONG_DELAY_MS 5 //used for configuration etc.
#define TWI_FREQ_SETTING 400000L
#define CPU_FREQ 16000000L
enum {
I2C_CMD_GET_ANALOGS = 1
};
int i2cSlaveAddr = 0;
void setup(){
// joining i2c as a master
TWBR = ((CPU_FREQ / TWI_FREQ_SETTING) - 16) / 2;
Wire.begin();
}
void loop(){
//requesting analogs read:
Wire.beginTransmission(i2cSlaveAddr);
Wire.write((uint8_t)I2C_CMD_GET_ANALOGS);
Wire.endTransmission();
delay(I2C_REQ_DELAY_MS);
// master knows slave should return 4 bytes to the I2C_CMD_GET_ANALOGS command
int respVals[4];
Wire.requestFrom(i2cSlaveAddr, 4);
uint8_t respIoIndex = 0;
if(Wire.available())
for (byte r = 0; r < 4; r++)
if(Wire.available()){
respVals[respIoIndex] = (uint8_t)Wire.read();
respIoIndex++;
}
else{
// log or handle error: "missing read"; if you are not going to do so use r index instead of respIoIndex and delete respoIoIndex from this for loop
break;
}
// now the respVals array should contain analog values for each analog input in the same order as defined in slave (respVals[0] - A0, respVals[1] - A1 ...)
}
I hope my example will help. It's based on code working for weeks making 40 reads a second from multiple slaves however I have not compiled it to test the function you require.
Please use WSWire library as the Wire (at least as for Arduino 1.0.3) may occasionally freeze your master if for some reason slave will not respond to request.
EDIT: The WSWire lib requires external pull-up resistors for I2C unless you modify the source and enable internal pull-ups like Wire does.
EDIT: instead of creating i2c slave implementation you may try the EasyTransfer library. I haven't tried it but it may be easier to use it if sending four bytes is everything you need.
EDIT[12.2017]: There is a new player on the block - PJON - a library suited for easy multi-master communication ideal to exchange pot values (and much more). It's been around for some time but gained a substantial development speed in recent months. I'm partially involved in its development and switched all field-level and local buses I've used so far (I2C, MODBUS RTU) to PJON over single wire, hardware serial or RF.
Check out GitHub-I2CBus, I've done the exact same thing. Hope it can help

receiving data from pc to PIC 16F877A with compiler hi-tech

I try to send the data from pc to the pic microcontroller. I am a beginner in PIC.
I send the data from hyperterminal and the data will display in the led in port B of PIC.
I used 10Mhz clock and the connection in 9600 baudrate.
here my uart.h program:
char UART_Init(const long int baudrate)
{
unsigned int x;
x = (_XTAL_FREQ - baudrate*64)/(baudrate*64);
if(x>255)
{
x = (_XTAL_FREQ - baudrate*16)/(baudrate*16);
BRGH = 1;
}
if(x<256)
{
SPBRG = x;
SYNC = 0;
SPEN = 1;
TRISC7 = 1;
TRISC6 = 1;
CREN = 1;
TXEN = 1;
return 1;
}
return 0;
}
char UART_TX_Empty()
{
return TRMT;
}
char UART_Data_Ready()
{
return RCIF;
}
char UART_Read()
{
while(!RCIF);
return RCREG;
}
void UART_Read_Text(char *Output, unsigned int length)
{
int i;
for(int i=0;i<length;i++)
Output[i] = UART_Read();
}
void UART_Write(char data)
{
while(!TRMT);
TXREG = data;
}
void UART_Write_Text(char *text)
{
int i;
for(i=0;text[i]!='\0';i++)
UART_Write(text[i]);
}
and this is my main program:
#include<htc.h>
#include<pic.h>
#define _XTAL_FREQ 10000000 //Clock Frequency
#include "uart.h"
void main()
{
TRISB = 0x00; //PORTB as Output
UART_Init(9600);
do
{
if(UART_Data_Ready())
PORTB = UART_Read();
__delay_ms(1000);
}while(1);
}
in hyperteminal I send data say 10010010 but the led in port B do not respond, are there any error in my program?
You have several steps: initialize UART, initialize LEDs, communicate over UART and setup your PC's UART. Which components have you successfully written and tested? You say you're a beginner, so what is the smallest functional program you have successfully executed on a PIC? I've been working with microcontrollers for years, but I still schedule about a whole day to get a single LED to turn on because it could be a software problem, a hardware problem, a voltage problem, an oscillator problem, a PCB problem or a compiler problem.
Here are the steps I take for microchip bring up:
Go over the oscillator section, the configuration bits section, the watchdog section and the pinout section (looking for VDD and VSS) in the datasheet. These are some of the hardest parts to get right. (A gotcha about the oscillator: just because you can program a chip, doesn't mean the oscillator is working because the programmer provides it's own clock.)
Write the bare-minimum code to turn on a single LED.
Write the bare-minimum code to make the LED blink (just use a for-loop delay for now, timers come later)
Write UART initialization code and transmit a single character, I use captial U because it's pretty in binary. TXREG = 'U';
Connect the UART to a PC and see if the hyperterminal sees the U. If it doesn't, I connect an oscilscope to the lines to make sure that the PIC is transmitting, that the PC transmits when I type characters and that the timing of the edges matches.
Within the PIC code, have the UART echo characters from the terminal. (TXREG = RXREG;), and then type on the hyperterminal and make sure the characters are echoed back.
One more note:
Do not have the PIC perform the SPBRG calculation. PIC16 are 8-bit processors and 10000000 requires 32-bits to store. There might be hiccups with the integer divison. It might not have a bug in it, but there's no need to have the PIC calculate it each time. Calculate it before-hand and hard-code the value.

Resources