I'm using QSharedMemory to store some data and want to subsequently append data to what is contained there. So I call the following code several times with new data. The "audioBuffer" is new data given to this function. I can call this function about 4-7 times ( and it varies ) before it seg faults on the memcpy operation. The size of the QSharedMemory location is huge so in the few calls that I do before seg faulting, there is no issue of memcpy copying data beyond it's boundaries. Also, m_SharedAudioBuffer.errorString() gives no errors up to the memcpy operation. Currently, I only have one process using this QSharedMemory segment. I also tried to write continually without appending each time and that works fine, so something is happening when I try to append more data to the shared memory segment. Any ideas? Thanks!
// Get the buffer size for the current audio buffer in shared memory
int bufferAudioDataSizeBytes = readFromSharedAudioBufferSizeMemory(); // This in number of bytes
// Create a bytearray with our data currently in the shared buffer
char* bufferAudioData = readFromSharedAudioBufferMemory();
QByteArray currentAudioStream = QByteArray::fromRawData(bufferAudioData,bufferAudioDataSizeBytes);
QByteArray currentAudioStreamDeepCopy(currentAudioStream);
currentAudioStreamDeepCopy.append(audioBuffer);
dataSize = currentAudioStreamDeepCopy.size();
//#if DEBUG
qDebug() << "Inserting audio buffer, new size is: " << dataSize;
//#endif
writeToSharedAudioBufferSizeMemory( dataSize ); // Just the size of what we received
// Write into the shared memory
m_SharedAudioBuffer.lock();
// Clear the buffer and define the copy locations
memset(m_SharedAudioBuffer.data(), '\0', m_SharedAudioBuffer.size());
char *to = (char*)m_SharedAudioBuffer.data();
char *from = (char*)audioBuffer.data();
// Now perform the actual copy operation to store the buffer
memcpy( to, from, dataSize );
// Release the lock
m_SharedAudioBuffer.unlock();
EDIT: Perhaps, this is due to my target embedded device which is very small. The available RAM is large when I am trying to write to shared memory, but I notice that in the /tmp directory ( which is only given 4Mb ) I have the following entries - the size is not nearly consumed in /tmp though so I'm not sure why I couldn't allocate more memory, also the QSharedMemory::create method never fails for my maximum size of 960000:
# cd /tmp/
# ls
QtSettings
lib
qipc_sharedmemory_AudioBufferData2a7d5f1a29e3d27dac65b4f350d76a0dfd442222
qipc_sharedmemory_AudioBufferSizeData6b7acc119f94322a6794cbca37ed63df07b733ab
qipc_systemsem_AudioBufferData2a7d5f1a29e3d27dac65b4f350d76a0dfd442222
qipc_systemsem_AudioBufferSizeData6b7acc119f94322a6794cbca37ed63df07b733ab
qtembedded-0
run
The problem seemed to be that I was using QByteArray's ::fromRawData on the pointer returned by the shared memory segment. When I copied that data explicitly using memcpy on this pointer, and then constructed my QByteArray using the copied data, then the seg faults stopped.
Related
I am trying to allocate a 2GB buffer using huge TLB page (1GB) and bind the memory region to a specific numa node.
To allocate the buffer using huge TLB page, I am using the following code:
shmid = shmget (IPC_PRIVATE, buf_size,
SHM_HUGETLB | IPC_CREAT | SHM_R | SHM_W);
buf = (uint64_t *) shmat (shmid, 0, 0);
Then, I called:
numa_tonode_memory (buf, buf_size, 3);
to move the buffer to a specific node.
When I run the program, as soon as I access buffer offset larger than 1GB, the program would stop with "Bus error (core dumped)".
Removing numa_tonode_memory would avoid the error, however, it would also destroy the purpose of allocating memory on a specific node.
I am wondering if there is any work around on this problem,
Thank you,
Is it possible to put a sketch (.HEX file) to an SD card and run it from there?
My objective is to utilize SD storage instead of flash memory for a program.
If yes, are there any libraries doing exactly this?
All i found was "flashing arduino from sd card", which is not what i need.
UPDATE:
the sketch's loop calling is implemented in the bootloader.
so i assume there is something like this in the bootloader:
while(true)
{
call_sketch_loop();
}
can it be changed to this? :
//signature changed from void loop() to int loop()
while(true)
{
int retval = call_sketch_loop(); //get loop call's return value
if( 0 == retval )
continue; // if 0, iterate the loop as usual
else
{
//copy 1.HEX from sd to flash and reboot
copy_hex_from_sd_to_flash( retval + ".HEX" );
reboot();
}
}
change loop singature to int loop()
put {int}.HEX files to an SD card - 1.HEX , 2.HEX , 3.HEX
the loop() call returns 0
continue with next iteration as usual
the loop() call returns 2
copy file 2.HEX from SD card into program flash memory
reboot device
with this approach, we can run flash-capacity-exceeding programs if we split them up to smaller subprograms.
The technical term you are looking for is "SD card bootloader".
Have you looked into this: "https://github.com/thseiler/embedded/tree/master/avr/2boots"?
As far as I understand, 2boot will first load the hex into the flash and then execute it from there. This is not exactly what you are looking for (you want to load it directly to RAM, right?).
The problem with what you are looking for is that arduino's RAM is really small. And there is liittle advantage in loading directly to RAM. Therfore such library might not exist at all.
I can sugget a poor-mans approach for doing this. First write a sketch that contains a function that have an infinite loop inside it and inside this loop, put the code of your desired "loop". In the setup of the sketch take the pointer to this function and write sufficient ammount of bytes into a binary file on the SD card.
Then upload another sketch wich has an empty buffer. This sketch will load the binary file into it and refernce to it's beginning as a pointer to a function. Viola, you can now execute your "loop".
This is ugly and unless you have very specific and isoteric need for loading directly into RAM, I suggest to try the 2boot library.
I'm trying to exceed the shared memory object after shm_open and ftruncate successfully at fisrt. Here is the code,
char *uuid = GenerateUUID();
int fd = shm_open(uuid, O_RDWR|O_CREAT|O_EXCL, S_IRUSR|S_IWUSR);
if(fd == -1) perror("shm_open");
size_t shmSize = sizeof(container);
int ret = ftruncate(fd, shmSize);
perror("ftruncate first");
ret = ftruncate(fd, shmSize * 2);
perror("ftruncate second");
It could pass the first ftruncate, but for the second ftruncate, it exceeds failed with errno=22, "Invalid argument".
I also tried to ftruncate the memory object after mmap, refer to the ftruncate's man page, the shared memory should be formatted as zero to the new length.
Besides, I also tried to ftruncate the memory object in the child process (This is an IPC topic among two processes), the ftruncate returns "Invalid fd, no such file or directory" but I could shm_open and mmap successfully in child process.
Any ideas? Thanks!
I think this is a known "feature" of shm_open(), ftruncate(), mmap().
You have to ftruncate() the first time through to give the shared memory a length, but subsequent times ftruncate() gives that error number 22, which you can simply ignore.
The used implementation seems to conform to an older specification where returning an error is an allowed behavior for ftruncate(fd, length) when length exceeds the previous length:
If the file previously was smaller than this size, ftruncate() shall
either increase the size of the file or fail.
Can anybody tell me how to measure the consumed RAM for a particular code running on Arduino Mega or Due.
There is two kinds of numbers to this question:
Global static usage and current run time.
The static estimated usage can be determined by adding the following line to (if it does not already exist)
.\arduino-1.5.5\hardware\arduino\avr\boards.txt
uno.upload.maximum_ram_size=2048
This then allows the compiler to output the additional 2nd line in the following example in the IDE's result window
Binary sketch size: 25,880 bytes (of a 32,256 byte maximum)
Estimated used SRAM memory: 990 bytes (of a 2048 byte maximum)
To see the amount of memory used at any given point. Including memory space currently in use, that exists while only in functions and members. This includes the HEAP and such. I use the following MemoryFree library at specific points in the code to reveal the high-water. The readme explains how to save unnecessarily/unintentionally used RAM by prints.
Note: That while the original Arduino IDE 1.0.5's boards.txt files does contain these ram_sizes, it does not actually use display usage. Where the original Arduino IDE 1.5.5 does, along with Arduino ERW 1.0.5 does (an non-supported fork).
In my Arduino IDE 2.1.0
I edit the file: /usr/share/arduino/hardware/arduino/boards.txt
but the second line don't appear
After read:
check-ram-memory-usage-arduino-optimization
measuring-free-memory
I tried:
Show vervose output during compilation
and use avr-size /tmp/build4042914391435450796.tmp/XXXXXXX.cpp.elf
then i get my memory used
Best Regards!
int freeRam () {
extern int __heap_start, *__brkval;
int v;
int fr = (int) &v - (__brkval == 0 ? (int) &__heap_start : (int) __brkval);
Serial.print("Free ram: ");
Serial.println(fr);
}
I am having a doubt with the following code snippet…
const qint64 bytesReady = m_audioInput->bytesReady();
const qint64 bytesSpace = m_buffer.size() - m_dataLength;
const qint64 bytesToRead = qMin(bytesReady, bytesSpace);
const qint64 bytesRead = m_audioInputIODevice->read(m_buffer.data() + m_dataLength, bytesToRead);
bytesReady() method is giving me a particular number of bytes and am passing these number of bytes to the read() of QIODevice which will return me the number of bytes read.
The problem is that bytesRead is not equal to bytesToRead.
And am getting fixed number of bytes from read method i.e 320, 640, 960, 1280, etc. and this depends on byteToRead.
There is no direct relation between the QAudioInput::bytesReady() and the QIODevice on which it is writing its samples.
QAudioInput maintains internally a IO device (system dependent) to the audio system, which is analog to a read-only QIODevice.
When you call bytesReady, it returns the number of bytes available to be read, analog to QIODevice::bytesAvailable(). Those have not yet been written to the output QIODevice, so when you do the m_audioInputIODevice->read just after it, without processing events, you're actually getting the samples which have been written earlier, not the ones still in the audio buffer.
This, plus the buffering of the IODevice, explains why the numbers might be different, and I don't see a way to synchronize them together.
Actually, you should do:
const qint64 bytesRead = m_audioInputIODevice->read(m_buffer.data() + m_dataLength, bytesSpace);
to get whatever is available from the IODevice, up to your available buffer space.
Better yet:
Use ->readAll() which returns a QByteArray "qba"
...then use sz=qba.size() which will tell you what you GOT (may be zero)
...then do something with it using sz and qba.data();
...seems to be without bugs unlike much of QAudioInput, which is incredibly broken, for instance, arbitrary sample rate limit of 48000 under Windows but not OSX, pull mode uses foreground thread so will break up constantly in any real situation, bytesReady() is meaningless... Worst. Qt. Routine. Evar!