Qt - Get audio amplitude from QBytearray - qt

I'm trying to create a program, using Qt (c++), which can record audio from my microphone using QAudioinput and QIODevice. I made a research and I came up with an example located on the this page. This example does what I need.
Now, I am trying to create an audio waveform of the recorded sound. I want to extract audio amplitudes and save them on a QList. To do that I use the following code:
//Check the number of samples in input buffer
qint64 len = m_audioInput->bytesReady();
//Limit sample size
if(len > 4096)
len = 4096;
//Read sound samples from input device to buffer
qint64 l = m_input->read(m_buffer.data(), len);
if(l > 0)
{
//Assign sound samples to short array
short* resultingData = (short*)m_buffer.data();
for ( i=0; i < len; i++ )
{
btlist.append( resultingData[ i ]);
}
}
m_audioInput is QAudioinput | m_buffer is QBytearray | m_input is QIODevice | btlist is QList
I use the following QAudioFormat:
m_format.setFrequency(44100); //set frequency to 44100
m_format.setSampleRate(44100); //set sample rate to 44100
m_format.setChannels(1); //set channels to mono
m_format.setSampleSize(16); //set sample sze to 16 bit
m_format.setSampleType(QAudioFormat::SignedInt ); //signed integer sample
m_format.setByteOrder(QAudioFormat::LittleEndian); //Byte order
m_format.setCodec("audio/pcm"); //set codec as simple audio/pcm
When I print my QList, using qWarning() << btlist.at(int), I get some positive and negative numbers which represents my audio amplitudes. I used Microsoft Excel to plot the data and compare it with the actual sound waveform.
(EDIT BASED ON THE OP COMMENT)
I am drawing the waveform using QPainter in Qt like this
for(int i = 1; i < btlist.size(); i++){
double x1 = (i-(i/1.25))-0.2;
double y1 = btlist.at(i-1);
double x2 = i-(i/1.25);
double y2 = btlist.at(i);
painter.drawLine(x1,y1,x2, y2);
}
The problem is that I also get lots of zeros (0) in my QList between the amplitude data like this, which if I draw as a waveform they are a straight line, which is not normal because it causes corruption to my waveform.
My question is why is that happening? What these zeros (0) represent? Am I doing something wrong? Also, is there a better way to extract audio amplitudes from QBytearray?
Thank you.

The drawline method you are using take integer values. Which means most of the time both of your x indexes will be the same. By simplifiyng your formula the x value at a given i is (i/5.0). By itself it is not an issue because the lines will be superposed, and it is a perfect way of drawing (just to make sure that's what you want to do).
The zero you see can be perfectly valid. They represent silence.
The real issue is that the range of your 16 bits PCM values is [-32767 , 32768]. I doubt that the paint device you are using cover this range. You need to normalize your y-axis. Moreover, it seems taht the qt coordinated system doesn't have negative values (edit: Nevermind the negatives, its says logical coordinates are converted).
For instance, convert your pcm values using :
((btlist.at(i) / MAX_AMPLITUDE + 1.0) / 2) * paintDevice.height();
Edit:
Btw, you are not using l, which is the real amount of data you read. If it is inferior to len, you will read invalid values at the end of your buffer, possibly read garbage\ read zeros\crash.
And your buffer is a byte buffer. And you iterate using a short pointer. So whether you use l or len the maximum size need to be divided by two. This is probably the cause of the ling line of zero in your picture.
for ( i=0; i < l/2; i++ )
{
btlist.append( resultingData[ i ]);
}

Related

Failed conversion of a QImage image to a CV image

I am new to both opencv and opencv. What I am doing is to convert a QImage image to an opencv Mat image, and then display both of them. Here is my code for this conversion:
i = new QImage("lena.png");
QImage lena = i->scaled(labW,labH,Qt::IgnoreAspectRatio);
//Original
QImage lenaRGB = lena.convertToFormat(QImage::Format_RGB888);
ui->imgWindow->setPixmap(QPixmap::fromImage(lena,Qt::AutoColor));
//method 1
Mat lena_cv, out;
QImage lena2 = lenaRGB.rgbSwapped();
QImage swapped = lena2;
swapped = swapped.rgbSwapped();
lena_cv = Mat(swapped.width(),swapped.height(),CV_8UC3, swapped.bits(),swapped.bytesPerLine()).clone();
namedWindow("CV Image");
imshow("CV Image", lena_cv);
//method 2
Mat out2,out3;
out2.create(Size(lena2.width(),lena2.height()),CV_8UC3);
int width = lena2.width();
int height = lena2.height();
memcpy(out2.data, lena2.bits(), sizeof(char)*width*height*3);
cvtColor(out2,out3,CV_RGB2GRAY);
namedWindow("CV Image2");
imshow("CV Image2",out3);
Both of the above two conversions cannot yield desired images, as shown below:
It is also noted that the conversion cannot proceed without using rgbSwapped, i.e.,:
lena_cv = Mat(lenaRGB.width(),lenaRGB.height(),CV_8UC3, lenaRGB.bits(),lenaRGB.bytesPerLine());
because:
The resulting image lena_cv cannot be displayed. If an additional step to convert lena_cv to BGR format using cvtColor before image display:
Exception at 0x7ffdff394008, code: 0xe06d7363: C++ exception, flags=0x1
(execution cannot be continued) (first chance) at c:\opencv-3.2.0
\sources\modules\core\src\opencl\runtime\opencl_core.cpp:278
This indicates the post conversion to BGR fails. I am not sure RGB to BGR conversion (of QImage) is necessary or not for converting QImage to CV image.
Can anyone help identify the issue with the above codes. Thanks :)
The "skew" in the third image is almost likely a result of assuming that each scan line occupies exactly width*3 bytes. There's typically a "stride" (or "steps") factor with each row in many image formats image such that the number of bytes per row is on some 4-byte or 16-byte boundary. Fortunately, QImage has a helper method called bytesPerLine that tells you how long each source row is.
So instead of this:
memcpy(out2.data, lena2.bits(), sizeof(char)*width*height*3);
Do this:
unsigned char* src = lena2.bits();
unsigned char* dst = out2.data;
int stride = lena2.bytesPerLine();
for (int row = 0; row < height; row++)
{
memcpy(dst + width*3*row, src+row*stride, width*3); // copy a single row, accounting for stride bytes
}
All of this assume it's the QImage that has the stride bytes and not the target Mat image you are transforming the bits too. If I have this backwards, then adjust the code to account for the steps member of Mat. (I don't see you using this, so I'm willing to be the above code is what you need).
The "blue" image is mostly likely just the RGB color bytes needing to be swapped for every pixel. Not sure why you are calling rgbSwapped unless that was the effect you were going for. Oh wait, you're probably referring to that noise effect at the bottom of the image. I'm willing to bet you need to think about "stride" bytes as well here too.

Microcontroller, How to display decimal on LCD?

I have a microcontroller and I am sampling the values of an LM335 temperature sensor.
The LCD library that I have allows me to display the hexadecimal value sampled by the 10-bit ADC.
10bit ADC gives me values from 0x0000 to 0x03FF.
What I am having trouble is trying to convert the hexadecimal value to a format that can be understood by regular humans.
Any leads would be greatly appreciated, since I am completely lost on the issue.
You could create a "string" into which you construct the decimal number like this (constants depend on what size the value actually, I presume 0-255, whether You want it to be null-terminated, etc.):
char result[4];
char i = 3;
do {
result[i] = '0' + value % 10;
value /= 10;
i--;
}
while (value > 0);
Basically, your problem is how to split a number into decimal digits so you can use your LCD library and send one digit to each cell.
If your LCD is based on 7-segment cells, then you need to output a value from 0 to 9 for each digit, not an ASCII code. The solution by #Roman Hocke is fine for this, provided that you don't add '0' to value % 10
Another way to split a number into digits is to convert it into BCD. For that, there is an algorithm named "double dabble" which allows you to convert your number into BCD without using divisions nor module operations, which can be nice if your microcontroller has no provision for division operation, or this is slower than you need.
"Double dable" algorithm sounds perfect for microcontrollers without provision for the division operation. However, a quick oversight of such algorithm in the Wikipedia shows that it uses dynamic memory, which seems to be worst than a routine for division. Of course, there must be an implementation out there that are not using calls to malloc() and friends.
Just to point out that Roman Hocke's snippet code has a little mistake. This version works ok for decimals in the range 0-255. It can be easily expand it to any range:
void dec2str(uint8_t val, char * res)
{
uint8_t i = 2;
do {
res[i] = '0' + val % 10;
val /= 10;
i--;
} while (val > 0);
res[3] = 0;
}

using glDrawPixels to render bitmap raw data

I receive raw image data from server. The server uses MS Dib() function which returns in BGR format. Now, what i want to do is to read this raw data and use glDrawPixels to draw it in Linux.
I was advised that GetClrTabAddress function in MS and alike shall be used to get me the RGB values for each index of 800 by 600 image sent to me.
I do not know how to get these values using indices. Could anyone give some tips.
void func(QByteArray)
{
window_width = 800;
window_height = 600;
size = window_width * window_height;
pixels = new float[size*3];
memcpy(pixels, bytes, bytes.size());
}
void GlWidget::paintGL()
{
//! [5]
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
glDrawPixels(window_width,window_height,GL_RGB,GL_FLOAT,pixels);
}
You can use GL_BGR in glDrawPixels, which will do the conversion for you and will probably be faster since AFAIK the GPU will do the work.
QByteArray sounds like you should be using unsigned bytes/chars instead of floats, which means GL_UNSIGNED_BYTE.
I'd assert(size*3*sizeof(float) == bytes.size());.
In this case make sure to set glPixelStorei(GL_UNPACK_ALIGNMENT, 1) if your width doesn't align to the default 4-byte boundry. With GL_BGR very pixel is 3 bytes and by default each row of your pixels is assumed to be padded to the next 4-byte boundary.
[EDIT]
OK, it looks like the image uses a palette. This means every value inthe QByteArray maps to an rgb value in another array. I'm not 100% sure where the palette is and maybe it can be computed implicitly, but you mentioned GetClrTabAddress which sounds promising.
The code will then look something like this
for(int i = 0; i < size; ++i)
{
unsigned char index = btmp[i];
//and something like..
memcpy(bytes + i * 3, GetClrTabAddress() + index * 3, 3);
//or
bytes[i*3+0] = someOtherPaletteData[index].red;
bytes[i*3+1] = someOtherPaletteData[index].green;
bytes[i*3+2] = someOtherPaletteData[index].blue;
}

MSP430 not able to handle double

I am trying to program a MSP430 with a simple "FIR filter" program, that looks like the following:
#include "msp430x22x4.h"
#include "legacymsp430.h"
#define FILTER_LENGTH 4
#define TimerA_counter_value 12000 // 12000 counts/s -> 12000 counts ~ 1 Hz
int i;
double x[FILTER_LENGTH+1] = {0,0,0,0,0};
double y = 0;
double b[FILTER_LENGTH+1] = {0.0338, 0.2401, 0.4521, 0.2401, 0.0338};
signed char floor_and_convert(double y);
void setup(void)
{
WDTCTL = WDTPW + WDTHOLD; // Stop WDT
BCSCTL1 = CALBC1_8MHZ; // Set DCO
DCOCTL = CALDCO_8MHZ;
/* Setup Port 3 */
P3SEL |= BIT4 + BIT5; // P3.4,5 = USART0 TXD/RXD
P3DIR |= BIT4; // P3.4 output direction
/* UART */
UCA0CTL1 = UCSSEL_2; // SMCLK
UCA0BR0 = 0x41; // 9600 baud from 8Mhz
UCA0BR1 = 0x3;
UCA0MCTL = UCBRS_2;
UCA0CTL1 &= ~UCSWRST; // **Initialize USCI state machine**
IE2 |= UCA0RXIE; // Enable USCI_A0 RX interrupt
/* Setup TimerA */
BCSCTL3 |= LFXT1S_2; // LFXT1S_2: Mode 2 for LFXT1 = VLO
// VLO provides a typical frequency of 12kHz
TACCTL0 = CCIE; // TACCR0 Capture/compare interrupt enable
TACCR0 = TimerA_counter_value; // Timer A Capture/Compare 0: -> 25 Hz
TACTL = TASSEL_1; // TASSEL_1: Timer A clock source select: 1 - ACLK
TACTL |= MC_1; // Start Timer_A in up mode
__enable_interrupt();
}
void main(void) // Beginning of program
{
setup(); // Call Function setup (see above)
_BIS_SR(LPM3_bits); // Enter LPM0
}
/* USCIA interrupt service routine */
/*#pragma vector=USCIAB0RX_VECTOR;*/
/*__interrupt void USCI0RX_ISR(void)*/
interrupt (USCIAB0RX_VECTOR) USCI0RX_ISR(void)
{
TACTL |= MC_1; // Start Timer_A in up mode
x[0] = (double)((signed char)UCA0RXBUF); // Read received sample and perform type casts
y = 0;
for(i = 0;i <= FILTER_LENGTH;i++) // Run FIR filter for each received sample
{
y += b[i]*x[i];
}
for(i = FILTER_LENGTH-1;i >= 0;i--) // Roll x array in order to hold old sample inputs
{
x[i+1] = x[i];
}
while (!(IFG2&UCA0TXIFG)); // Wait until USART0 TX buffer is ready?
UCA0TXBUF = (signed char) y;
TACTL |= TACLR; // Clear TimerA (prevent interrupt during receive)
}
/* Timer A interrupt service routine */
/*#pragma vector=TIMERA0_VECTOR;*/
/*__interrupt void TimerA_ISR (void)*/
interrupt (TIMERA0_VECTOR) TimerA_ISR(void)
{
for(i = 0;i <= FILTER_LENGTH;i++) // Clear x array if no data has arrived after 1 sec
{
x[i] = 0;
}
TACTL &= ~MC_1; // Stops TimerA
}
The program interacts with a MatLab code, that sends 200 doubles to the MSP, for processing in the FIR filter. My problem is, that the MSP is not able to deal with the doubles.
I am using the MSPGCC to compile the code. When I send a int to the MSP it will respond be sending a int back again.
Your problem looks like it is in the way that the data is being sent to the MSP.
The communications from MATLAB is, according to your code, a sequence of 4 binary byte values that you then take from the serial port and cast it straight to a double. The value coming in will have a range -128 to +127.
If your source data is any other data size then your program will be broken. If your data source is providing binary "double" data then each value may be 4 or 8 bytes long depending upon its internal data representation. Sending one of these values over the serial port will be interpreted by the MSP as a full set of 4 input samples, resulting in absolute garbage for a set of answers.
The really big question is WHY ON EARTH ARE YOU DOING THIS IN FLOATING POINT - on a 16 bit integer processor that (many versions) have integer multiplier hardware.
As Ian said, You're taking an 8bit value (UCA0RXBUF is only 8 bits wide anyway) and expecting to get a 32bit or 64 bit value out of it.
In order to get a proper sample you would need to read UCA0RXBUF multiple times and then concatenate each 8 bit value into 32/64 bits which you then would cast to a double.
Like Ian I would also question the wisdom of doing floating point math in a Low power embedded microcontroller. This type of task is much better suited to a DSP.
At least you should use fixed point math, seewikipedia (even in a DSP you would use fixed point arithmetic).
Hmm. Actually the code is made of my teacher, I'm just trying to make it work on my Mac, and not in AIR :-)
MATLAB code is like this:
function FilterTest(comport)
Fs = 100; % Sampling Frequency
Ts = 1/Fs; % Sampling Periode
L = 200; % Number of samples
N = 4; % Filter order
Fcut = 5; % Cut-off frequency
B = fir1(N,Fcut/(Fs/2)) % Filter coefficients in length N+1 vector B
t = [0:L-1]*Ts; % time array
A_m = 80; % Amplitude of main component
F_m = 5; % Frequency of main component
P_m = 80; % Phase of main component
y_m = A_m*sin(2*pi*F_m*t - P_m*(pi/180));
A_s = 40; % Amplitude of secondary component
F_s = 40; % Frequency of secondary component
P_s = 20; % Phase of secondary component
y_s = A_s*sin(2*pi*F_s*t - P_s*(pi/180));
y = round(y_m + y_s); % sum of main and secondary components (rounded to integers)
y_filt = round(filter(B,1,y)); % filtered data (rounded to integers)
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Serial_port_object = serial(comport); % create Serial port object
set(Serial_port_object,'InputBufferSize',L) % set InputBufferSize to length of data
set(Serial_port_object,'OutputBufferSize',L) % set OutputBufferSize to length of data
fopen(Serial_port_object) % open Com Port
fwrite(Serial_port_object,y,'int8'); % send out data
data = fread(Serial_port_object,L,'int8'); % read back data
fclose(Serial_port_object) % close Com Port
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
subplot(2,1,1)
hold off
plot(t,y)
hold on
plot(t,y_filt,'r')
plot(t,y_filt,'ro')
plot(t,data,'k.')
ylabel('Amplitude')
legend('y','y filt (PC)','y filt (PC)','y filt (muP)')
subplot(2,1,2)
hold off
plot(t,data'-y_filt)
hold on
xlabel('time')
ylabel('muP - PC')
figure(1)
It is also not advised to keep interrupt routines doing long processing routines, because you will impact on interrupt latency. Bytes comming from the PC can get easily lost, because of buffer overrun on the serial port.
The best is to build a FIFO buffer holding a resonable number of input values. The USCI routine fills the FIFO while the main program keeps looking for data inside it and process them as they are available.
This way, while the data is being processed, the USCI can interrupt to handle new incomming bytes.
When the FIFO is empty, you can put the main process in a suitable LPM mode to conserve power (and this is the best MSP430 feature). The USCI routine will wake the CPU up when a data is ready (just put the WAKEUP attribute in the USCI handler if you are using MSPGCC).
In such a scenario be sure to declare volatile every variable that are shared between interrupt routines and the main process.

how to print a uint16 monochrome image in Qt?

I'm trying to print a image from a Dicom file. I pass the raw data to a convertToFormat_RGB888 function. As far as I know, Qt can't handle monochrome 16 bits images.
Here's the original image (converted to jpg here):
http://imageshack.us/photo/my-images/839/16bitc.jpg/
bool convertToFormat_RGB888(gdcm::Image const & gimage, char *buffer, QImage* &imageQt)
Inside this function, I get inside this...
...
else if (gimage.GetPixelFormat() == gdcm::PixelFormat::UINT16)
{
short *buffer16 = (short*)buffer;
unsigned char *ubuffer = new unsigned char[dimX*dimY*3];
unsigned char *pubuffer = ubuffer;
for (unsigned int i = 0; i < dimX*dimY; i++)
{
*pubuffer++ = *buffer16;
*pubuffer++ = *buffer16;
*pubuffer++ = *buffer16;
buffer16++;
}
imageQt = new QImage(ubuffer, dimX, dimY, QImage::Format_RGB888);
...
This code is a little adaptation from here:
gdcm.sourceforge.net/2.0/html/ConvertToQImage_8cxx-example.html
But the original one I got a execution error. Using mine at least I get an image, but it's not the same.
Here is the new image (converted to jpg here):
http://imageshack.us/photo/my-images/204/8bitz.jpg/
What am I doing wrong?
Thanks.
Try to get values of pixels from buffer manually and pass it to QImage::setPixel. It can be simplier.
You are assigning 16-bit integer to 8-bit variables here:
*pubuffer++ = *buffer16;
The result is undefined and most compilers just move the lower 8 bits to the destination. You want the upper 8 bits
*pubuffer++ = (*buffer16) >> 8;
The other issue is endianness. Depending to the endianness of the source data, you may need to call one of the QtEndian functions.
Lastly, you don't really need to use any of the 32 or 24-bit Qt image formats. Use 8-bit QImage::Format_Indexed8 and set the color table to grays.

Resources