I am trying to develop a multimedia program using Qt which gets an audio input stream from a microphone (using QAudioInput), then stores the sampled bytes in memory for 60 milliseconds, then play it back (using QAudioOutput).
The documentation for QAudioOuput has an example which does most of this, but it uses a file as its data source instead. How can I modify this to use an input stream instead?
QAudioInput::start() and QAudioOutput::start() both take a pointer to a QIODevice as a parameter. This could be pointer to a QFile (like in the example) or a QBuffer if you want to keep the data in the memory.
Related
I'm trying to monitor an audio input and record the audio to a file but only when a level threshold is exceeded. There seems to be two main options for recording in Qt; QAudioRecorder and QAudioInput.
Long story short: I'm trying to find the API that can take raw audio sample data read from QAudioInput and record it to a file just like QAudioRecorder does, but strangely it doesn't seem to exist. To give an example, the setup of the QAudioRecorder would be something like the following (but instead of specifying a input device with setAudioInput() you pass it sampled bytes):
QAudioEncoderSettings audioSettings;
QAudioRecorder recorder = new QAudioRecorder;
audioSettings.setCodec("audio/PCM");
audioSettings.setQuality(QMultimedia::HighQuality);
recorder.setEncodingSettings(audioSettings);
recorder.setContainerFormat("wav");
recorder.setOutputLocation(QUrl::fromLocalFile("/tmp/test.wav"));
recorder.setAudioInput("default");
recorder.record();
I'm using QAudioInput because I need access to the raw samples. The problem with QAudioInput is, Qt does not seem to provide an easy way to take the raw samples I get out of the QAudioInput and pipe them into a file encoding them along the way. QAudioRecorder does this nicely, but you can't feed raw samples into QAudioRecorder; you just tell it which device to record from and how you want it stored.
Note I tried using QAudioInput and QAudioRecorder together - QAudioInput for the raw access and QAudioRecorder whenever I need to record, but there is two main issues with that: A) Only one of these can be reading a given device at a time. B) I want to record the data at and just before the threshold is exceeded and not just after the threshold is exceeded.
I ended up using QAudioRecorder+QAudioProbe. There are a couple of limitations though:
Firstly the attached QAudioProbe only works if QAudioRecorder is actually recording, so I had to write a wrapper over QAudioRecorder to switch on/off recording by switching output device to|from actual_file|/dev/null.
Second, as I stated "I want to record the data at and just before the threshold is exceeded and not just after the threshold is exceeded". Well, I had to compromise on that. The probe is used to detect the recording condition, but there is no way to stuff the data from the probe back into the recorder. I mean, I guess you could record to a buffer file in idle state and somehow prepend part of that data ... but the complexity wasn't worth it for me.
Aside; there was another issue with QAudioRecorder that motivated me to write a wrapper over it. Basically I found QAudioRecorder::stop() sometimes hangs indefinitely. To get around this, I had to heap allocate a recorder and delete it and init a new one with each new recording.
I have 2 input streams of data that are being passed to a Haali Muxer (mp4 format).
Currently I stream these to Haali directly in a DirectShow graph without a clock. I wondered if I should be trying to write these to the muxer synchronised, or whether it happily accepts a stream of audio data that stops before the video data stream stops. (I have issues with the output file not playing audio after seeking, and I'm not sure why this could occur)
I can't find much in the way of documentation for muxing with the Haali muxer, does anyone know the best place to look for info on this filter?
To have the streams multiplexed into single MP4 file you need single instance of multiplexer (Haali, GDCL, commercial, wrapper over mp4v2 library, over Media Foundation sink etc) with two (or more) input pins on it connected to respective sources, which in turn are going to be written as tracks.
Filter graph clock does not matter. Clock is for presentation, and file writers accept incoming data and write it as soon as possible anyway. It is more accurate to remove the clock, as you seem to already be doing, but having standard clock is not going to be different.
Data is synchronized using time stamps on individual media samples, parts of media streams. Multiplexer builds internal queues for every stream and then consumes data from the streams to build single file, in a sort of way that original stream data is interleaved. If one stream supplies too much data, that is, if data is available too early while another stream supplies data slowly, multiplexer blocks further data reception on this particular stream by not returning from respective processing call (IPin::Receive) expecting that during this wait the slow stream provides additional input. Eventually, what multiplexer looks at when matching data from different streams is data time stamps.
To obtain synchronized data in resulting MP4 file you, thus, need to make sure the payload data is properly time stamped. Multiplexer will take care of the rest.
This also includes that the time stamps should be monotonously increasing within a stream, and key frames/splice points are respectively indicated. Otherwise some multiplexers might issue a failure immediately, other would produce the output file but it might have playback issues (esp. seeking).
I'm working on a speech recognition project and my program can recognize words from audio files. Now I need to work with the audio stream coming from microphone. I'm using QAudio for getting sound data from mic and QAudio has a function to start the process. This start(* QBuffer) function writes the data into a QBuffer(inherited from QByteArray) object. When I'm not dealing with continuous stream, I can stop recording from mic anytime I want and copy the whole data from QBuffer into a QByteArray and I can do whatever I wanna do with the data. But in continuous stream QBuffer's size increases by time and becomes 100Mb in 15 mins.
So I need to use some kind of circular buffer but I can't figure out how to do that especially with this start(*QBuffer) function. I also avoid of cutting the streaming sound at a point where the speech continues.
What is the basic way to handle streaming audio data for speech recognition?
Is it possible to change the start(*QBuffer) function into start(*QByteArray) and make the function to overwrite on that QByteArray to build and circular buffer?
Thanks in advance
boost.com is offering a circular buffer
http://www.boost.org/doc/libs/1_37_0/libs/circular_buffer/doc/circular_buffer.html#briefexample
It should meet what you need
Alain
My primary objective is to send a mp3 file over network using QDataStream, QTcpServer and QTcpSocket. But I have broken this task to smaller pieces. At first I need to get the mp3 file to the correct format so that It can be "fed" to the data stream.
How am I supposed to accomplish this? I figured it would be the easiest to use Phonon? But the MediaObject doesnt seem to be offering some sort of getData method.
Any help on how am I supposed to do that would be much appreciated. If needed I can explain more about this.
There is no "correct format". Also, your problem is not MP3-specific. You do the same for all files, regardless of what kind of data they contain. You open the file, read bytes from it and send those bytes until there's nothing left to send.
You don't need Phonon or anything MP3-related. You only need to open the file and read bytes from it. You then write those bytes to the socket using the write() function of your QTcpSocket object. You don't even need a QDataStream, since you're only dealing with data that you don't need to parse.
i'm programming an application (in C++/Qt Designer 4.6 and using some librairies like ffmpeg and v4l2) which capture from webcam and i want to send the captured QImage via IP, so i send it into a QTcpSocket and i succed in receiving it in my server application but the problem it's too slow, crearly because the QImage isn't compressed so i'm not getting the wanted result which is live video streaming via IP, my question is how can i compress the QImage? I think in converting it to the YUV format but i can't realize it, this is some lines from my code to send the available QImage:
QImage image;
QByteArray ba;
QBuffer buffer(&ba);
image.save(&buffer, "PNG");
imsocket->write(ba);
You would be better off sending an MPEG compressed video stream instead of compressing each frame as QImage and sending it over network.
MPEG compressed video streams use a technique which uses a full scale image for first frame and then only records changed pixels for each proceeding frame thus providing maximum compression as well as smooth video playback over the network.
On the client end, you will playback by getting frames generated by the library from the compressed MPEG stream.
I had a similar problem as the above, my code looked like this:
QBuffer buffer;
buffer.open(QIOdevice::WriteOnly);
QDataStream stream(&buffer);
stream << image;
Using QElapsedTimer I found that encoding the image took more than 500 msec for a 1024x768 sized image. Using valgrind I found that the time was spend in png_write_image which was called from QImageWriter::write(QImage)
Changing the code to encode as a JPG instead, made the code execute in only 5ms for the same image. The resulting code looked like this:
QBuffer buffer;
buffer.open(QIODevice::WriteOnly);
image.save(&buffer,"JPEG");
Unfortunately I found no way to tell QDataStream to encode images in JPG rather than PNG, so the decoding side obviously needed to be adapted to decode the image like:
image.load(&buffer, "JPEG");
Before you send off the image, you can compress the byte array and then de-compress it on the server. This should provide a nice boost for you.
QByteArray arr, cArr;
cArr = qCompress(arr);
You should adhere to the first rule of optimization: Don't optimize when you have not measured where your application spends its time. I asume you are just guessing which part pf your application is slow.