Is there a way to receive data as unsigned char over UDP on Qt? - qt

I need to send floating point numbers using a UDP connection to a Qt application. Now in Qt the only function available is
qint64 readDatagram ( char * data, qint64 maxSize, QHostAddress * address = 0, quint16 * port = 0 )
which accepts data in the form of signed character buffer. I can convert my float into a string and send it but it will obviously not be very efficient converting a 4 byte float into a much longer sized character buffer.
I got hold of these 2 functions to convert a 4 byte float into an unsinged 32 bit integer to transfer over network which works fine for a simple C++ UDP program but for Qt I need to receive the data as unsigned char.
Is it possible to avoid converting the floatinf point data into a string and then sending it?
uint32_t htonf(float f)
{
uint32_t p;
uint32_t sign;
if (f < 0) { sign = 1; f = -f; }
else { sign = 0; }
p = ((((uint32_t)f)&0x7fff)<<16) | (sign<<31); // Whole part and sign.
p |= (uint32_t)(((f - (int)f) * 65536.0f))&0xffff; // Fraction.
return p;
}
float ntohf(uint32_t p)
{
float f = ((p>>16)&0x7fff); // Whole part.
f += (p&0xffff) / 65536.0f; // Fraction.
if (((p>>31)&0x1) == 0x1) { f = -f; } // Sign bit set.
return f;
}

Have you tried using readDatagram? Or converting the data to a QByteArray after reading? In many cases a char* is really just a byte array. This is one of those cases. Note that the writeDatagram can take a QByteArray.
Generally every thing sent across sockets is in bytes not strings, layers on either end do the conversions. Take a look here, especially the Broadcaster examples. They show how to create a QByteArray for broadcast and receive.

Not sure why the downvote, since the question is vague in requirements.
A 4-byte float is simply a 4 character buffer, if cast as one. If the systems are homogenous, the float can be sent as a signed char *, and bit for bit it'll be the same read into the signed char * on the receiver directly, no conversion needed. If the systems are heterogenous, then this won't work and you need to convert it to a portable format, anyway. IEEE format is often used, but my question is still, what are the requirements, is the float format the same between systems?

If I read it correctly, your primary question seems to be how to receive data of type unsigned char with QT's readDatagram function which uses a pointer to a buffer of type char.
The short answer is use a cast along these lines:
const size_t MAXSIZE = 1024;
unsigned char* data = malloc(MAXSIZE);
readDatagram ( (unsigned char *)data, MAXSIZE, address, port )
I'm going to assume you have multiple machines which use the same IEEE floating point format but some of which are big endian and some of which are little endian. See this SO post for a good discussion of this issue.
In that case you could do something a bit simpler like this:
const size_t FCOUNT = 256;
float* data = malloc(FCOUNT * sizeof(*data));
readDatagram ( (char *)data, FCOUNT * sizeof(*data), address, port )
for (int i = 0; i != FCOUNT; ++i)
data[i] = ntohf(*((uint32_t*)&data[i]));
The thing to remember is that as far as networking functions like readDatagram are concerned, the data is just a bunch of bits and it doesn't care what type those bits are interpreted as.

If both ends of your UDP connection use Qt, I would suggest looking at QDataStream. You can create this from a QByteArray each time you read a datagram, and then read whatever values you require - floats, maps, lists, QVariants, and of course string.
Similarly, on the sending side, you'd create a data stream, push data into it, then send the resulting QByteArray over writeDatagram.
Obviously this only works if both ends use Qt - the data encoding is well-defined, but non-trivial to generate by hand.
(If you want stream orientated behaviour, you could use the fact that QUDPSocket is a QIODevice with a data-stream, but it sounds as if you want per-datagram behaviour)

Related

How can i determine duration of wav file

I'm working with .wav files and I need to get their duration in seconds.
So far I've been determining it with:
File size / byte_rate
Byte_rate being (Sample Rate * BitsPerSample * Channels) / 8.
And it works, with smaller files, when I try to parse bigger files, I get more seconds than the actual duration.
Example:
Size(bytes): 45207622 Byte_rate: 176400 Duration: 256
(45207622 / 176400)
but the actual duration is 250...
FYI: I've double checked the size and byte_rate, they are correct.
Without a sample RIFF header or your code, it would be difficult to answer the specifics in your question. (i.e. Why your math isn't coming to your expected result.)
However, since you've specified that you're working in C in the comments, might I suggest using the sox library instead of parsing the headers with newly written code? In addition to catching a fair number of edge cases, this allows you to support any format sox supports reading without having to write any of the reading code yourself. (Though anyone inclined to do so should probably take a look at Can someone explain .wav(WAVE) file headers? and RIFF WAVE format specifications. The process should be roughly the method described in the question, at least in most cases. [Edit: That is chunk data length divided by the header's byte rate.])
Example code:
#include <sox.h>
#include <stdio.h>
int main(int argc, char **argv) {
sox_format_t *fmt;
if(argc < 2) {
printf("Please provide audio file.\n");
return 1;
}
fmt = sox_open_read(argv[1], NULL, NULL, NULL);
__uint64_t ws = fmt->signal.length / fmt->signal.channels;
if(fmt->signal.length) {
printf("%0.2f seconds long\n", (double)ws / fmt->signal.rate);
} else {
printf("Cannot determine duration from header.\n");
}
}
For anyone curious, I largely derived this from the sox command line tool's source code.
Thank you EPR for giving me the fix to timing in my program. I'm not using libsox, I've set up a struct trying to match the original at http://www.lightlink.com/tjweber/StripWav/Canon.html This is NOT the correct way to do it but it works for simple files. Another useful reference is at http://www-mmsp.ece.mcgill.ca/Documents/AudioFormats/WAVE/WAVE.html
Anyway I assume the header is 44 bytes and read() it into memory at the location of the struct. Then I can access fields of the struct, malloc space for the pcm data and read() it into the pcm space from where the file pointer was left. I'm just writing an audiogram program so it needs to be close to correct for the WAV files I generate with arecord, sox, Audacity. Always 2 channels, 44100 sample rate. My struct:
struct wavhdr { // defined by Microsoft, needs to match
char riff[4]; // should be "RIFF"
uint32_t len8; // file length - 8
char wave[4]; // should be "WAVE"
char fmt[4]; // should be "fmt "
uint32_t fdatalen; // should be 16 (0x10)
uint16_t ftag; // format tag, 1 = pcm
uint16_t channels; // 2 for stereo
uint32_t sps; // samples/sec
uint32_t srate; // sample rate in bytes/sec (block align)
uint16_t chan8; // channels * bits/sample / 8
uint16_t bps; // bits/sample
char data[4]; // should be "data"
uint32_t datlen; // length of data block
// pcm data follows this
} hdr;
I was trying to use the measured file size - header length / samples/sec, that didn't work, I was off by a factor of 6.

reading an array in a function

I am trying using the arduino IDE to write a sketch. I have data in progmem and want to move the data with a function to a memory address allocated using malloc. My code is below:
const uint8_t Data_6 [256] PROGMEM = { 0x11, 0x39......};
void setup() {
Serial.begin(57600);
oddBallData (Data_6, 0x00, 256);
}
void main() {
}
void oddBallData(const uint8_t *data, uint8_t mem, uint16_t bytes) {
uint8_t *buff1 = (uint8_t*)malloc(sizeof(bytes));
if (buff1 = 0) {
Serial.println(F("FATAL ERROR - NO MEMORY"));
}
else {
for (uint16_t x = 0; x < 6; x++ ) {
buff1[x] = data[x]; //edited from data[0] to [x] made a mistake in post
Serial.println(buff1[x],HEX);
}
}
buff1[0] = data[0];
Serial.println(buff1[0],HEX);
free(buff1);
}
I have some data saved in progmem and want to write that data to a second device using i2c protocol. I have multiple constant arrays of data saved to my progmem, with different sizes. So I have used malloc to reserve some memory from the heap, inside of the function.
I have not been able to write the data from the progmem so I have stripped things back to so that I am just trying to point to the progmem data using malloc and then print it.
This is where I found a the problem. If I print a single array entry from the data constant. It prints the correct value. If I use a loop I get mixed results, the loop works as long as the condition check value is below 3 or sometimes below 6!!!...?
If above this value the entire print is just garbage. Can anyone explain what I am seeing?
The culprit is probably
uint8_t *buff1 = (uint8_t*)malloc(sizeof(bytes));
sizeof(bytes) returns the size of the variable (which is probably 2 bytes) so you are just allocating 2 bytes of memory. You should use the value directly, eg:
uint8_t* buff1 = malloc(bytes);
Mind that the cast is not required in C since a void* is convertible to any other pointer type directly.
Again - AVR PROGMEM is not directly accessible from memory space, it needs different instruction than access into the RAM. If you are using it like this, you'll get RAM content on passed address, not the FLASH one. You have to use special functions for this. For example memcpy_P(ram_buff,flash_ptr); makes a copy from flash into the ram. Or you can read one byte by pgm_read_byte(flash_ptr + offset)
BTW: If you are using Data_6[0] and it's working, it's just because compiler sees it as a constant and constant can be replaced by its value compile time.
I Guess you just forgot to flush()
try to do Serial.flushI() after Serial.println(buff1[x],HEX);
you can also check flush documentation

Tuning QDataStream

I have a program that in outline processes binary data from afile.
Code outline is the following:
QFile fileIn ("the_file");
fileIn.open(QIODevice::ReadOnly);
The file has a mix of binary and text data.
The file contents are read using QDataStream:
QDataStream stream(&fileIn);
stream.setByteOrder(QDataStream::LittleEndian);
stream.setVersion(QDataStream::Qt_5_0);
I can read the data from the QDataStream into various data types. e.g.
QString the_value; // String
stream >> the_value;
qint32 the_num;
stream >> the_numm;
Nice and easy. Overall I read the file data byte by byte until I hit certain values that represent delimiters, e.g. 0x68 0x48. At this point I then next the next couple of bytes that tell me what type of data is next (floats, Strings, ints, etc) and extract as appropriate.
So, the data is orocessed (outline) like:
while ( ! stream.atEnd() )
{
qint8 byte1 = getInt8(stream);
qint8 byte2 = getInt8(stream);
if ( byte1 == 0x68 && byte2 == 0x48 )
{
qint8 byte3 = getInt8(stream);
qint8 byte4 = getInt8(stream);
if ( byte3 == 0x1 && byte4 == 0x7 )
{
do_this(stream);
}
else if ( byte3 == 0x2 && byte4 == 0x8 )
{
do_that(stream);
}
}
}
Some of this embedded data may be compressed, so we use
long dSize = 1024;
QByteArray dS = qUncompress( stream.device()->read(dSize));
QBuffer buffer;
buffer.setData(dS);
if (!buffer.open(QBuffer::ReadOnly)) {
qFatal("Buffer could not be opened. Something is very wrong!");
}
QDataStream stream2(&buffer);
stream2.setByteOrder(QDataStream::LittleEndian);
stream2.setVersion(QDataStream::Qt_5_0);
The convenience of QDataStream makes it easy to read the data, in terms of mapping to particular types but also in handling endianess easily, but it seems to be at the expense of speed. The issues is compounded by the fact that the processing is recursive - data being read could itself contain embedded file data, which needs to be read and processed in the same way.
Is there an alternative that is faster, and so if, how then to handle Endianess the same way?
Your code looks straight forward .. recursion should not be the show stopper ...
Do you have lots of strings ? Thousands ?
stream >> string allocates memory using new what is really slow. And needs to be freed manually afterwards. Refer to the Qt Docs for operator>>(char *&s) method. This is used when reading into QStrings.
Same is true for readBytes(char *&s, uint &l) which may be called internally slowing everything down !
The QString itself will also allocate memory (twice as much as it uses 16bit encoding) what slows down further.
If you use one of these functions often, consider rewriting that code parts for directly reading into a preallocated buffer using readRawData(char *s, int len) before further processing.
Overall, if you need high performance QDataStream itself may well be the show stopper.

How to convert double* data to const char* or QByteArray efficiently

I am trying to use the network programming APIs in Qt in my project. One part of my code requires me to convert double* data to QByteArray or a const char*.
I searched through the stackoverflow questions and could find many people suggesting this code :
QByteArray array(reinterpret_cast<const char*>(data), sizeof(double));
or, for an array of double :
QByteArray::fromRawData(reinterpret_cast<const char*>(data),s*sizeof(double));
When I use them in my function, It does notgive me the desired result. The output seems to be random characters.
Please Suggest an efficient way to implement it in Qt. Thank you very much for your time.
Regards
Alok
If you just need to encode and decode a double into a byte array, this works:
double value = 3.14159275;
// Encode the value into the byte array
QByteArray byteArray(reinterpret_cast<const char*>(&value), sizeof(double));
// Decode the value
double outValue;
// Copy the data from the byte array into the double
memcpy(&outValue, byteArray.data(), sizeof(double));
printf("%f", outValue);
However, that is not the best way to send data over the network, as it will depend on the platform specifics of how the machines encode the double type. I would recommend you look at the QDataStream class, which allows you to do this:
double value = 3.14159275;
// Encode the value into the byte array
QByteArray byteArray;
QDataStream stream(&byteArray, QIODevice::WriteOnly);
stream << value;
// Decode the value
double outValue;
QDataStream readStream(&byteArray, QIODevice::ReadOnly);
readStream >> outValue;
printf("%f", outValue);
This is now platform independent, and the stream operators make it very convenient and easy to read.
Assuming that you want to create a human readable string:
double d = 3.141459;
QString s = QString::number(d); // method has options for format and precision, see docs
or if you need localization where locale is a QLocale object:
s = locale.toString(d); // method has options for format and precision, see docs
You can easily convert the string into a QByteArray using s.toUtf8() or s.toLatin1() if really necessary. If speed is important there also is:
QByteArray ba = QByteArray::number(d); // method has options for format and precision, see docs

how to convert double between host and network byte order?

Could somebody tell me how to convert double precision into network byte ordering.
I tried
uint32_t htonl(uint32_t hostlong);
uint16_t htons(uint16_t hostshort);
uint32_t ntohl(uint32_t netlong);
uint16_t ntohs(uint16_t netshort);
functions and they worked well but none of them does double (float) conversion because these types are different on every architecture. And through the XDR i found double-float precision format representations (http://en.wikipedia.org/wiki/Double_precision) but no byte ordering there.
So, I would much appreciate if somebody helps me out on this (C code would be great!).
NOTE: OS is Linux kernel (2.6.29), ARMv7 CPU architecture.
You could look at IEEE 754 at the interchanging formats of floating points.
But the key should be to define a network order, ex. 1. byte exponent and sign, bytes 2 to n as mantissa in msb order.
Then you can declare your functions
uint64_t htond(double hostdouble);
double ntohd(uint64_t netdouble);
The implementation only depends of your compiler/plattform.
The best should be to use some natural definition,
so you could use at the ARM-platform simple transformations.
EDIT:
From the comment
static void htond (double &x)
{
int *Double_Overlay;
int Holding_Buffer;
Double_Overlay = (int *) &x;
Holding_Buffer = Double_Overlay [0];
Double_Overlay [0] = htonl (Double_Overlay [1]);
Double_Overlay [1] = htonl (Holding_Buffer);
}
This could work, but obviously only if both platforms use the same coding schema for double and if int has the same size of long.
Btw. The way of returning the value is a bit odd.
But you could write a more stable version, like this (pseudo code)
void htond (const double hostDouble, uint8_t result[8])
{
result[0] = signOf(hostDouble);
result[1] = exponentOf(hostDouble);
result[2..7] = mantissaOf(hostDouble);
}
This might be hacky (the char* hack), but it works for me:
double Buffer::get8AsDouble(){
double little_endian = *(double*)this->cursor;
double big_endian;
int x = 0;
char *little_pointer = (char*)&little_endian;
char *big_pointer = (char*)&big_endian;
while( x < 8 ){
big_pointer[x] = little_pointer[7 - x];
++x;
}
return big_endian;
}
For brevity, I've not include the range guards. Though, you should include range guards when working at this level.

Resources