I'm trying to monitor an audio input and record the audio to a file but only when a level threshold is exceeded. There seems to be two main options for recording in Qt; QAudioRecorder and QAudioInput.
Long story short: I'm trying to find the API that can take raw audio sample data read from QAudioInput and record it to a file just like QAudioRecorder does, but strangely it doesn't seem to exist. To give an example, the setup of the QAudioRecorder would be something like the following (but instead of specifying a input device with setAudioInput() you pass it sampled bytes):
QAudioEncoderSettings audioSettings;
QAudioRecorder recorder = new QAudioRecorder;
audioSettings.setCodec("audio/PCM");
audioSettings.setQuality(QMultimedia::HighQuality);
recorder.setEncodingSettings(audioSettings);
recorder.setContainerFormat("wav");
recorder.setOutputLocation(QUrl::fromLocalFile("/tmp/test.wav"));
recorder.setAudioInput("default");
recorder.record();
I'm using QAudioInput because I need access to the raw samples. The problem with QAudioInput is, Qt does not seem to provide an easy way to take the raw samples I get out of the QAudioInput and pipe them into a file encoding them along the way. QAudioRecorder does this nicely, but you can't feed raw samples into QAudioRecorder; you just tell it which device to record from and how you want it stored.
Note I tried using QAudioInput and QAudioRecorder together - QAudioInput for the raw access and QAudioRecorder whenever I need to record, but there is two main issues with that: A) Only one of these can be reading a given device at a time. B) I want to record the data at and just before the threshold is exceeded and not just after the threshold is exceeded.
I ended up using QAudioRecorder+QAudioProbe. There are a couple of limitations though:
Firstly the attached QAudioProbe only works if QAudioRecorder is actually recording, so I had to write a wrapper over QAudioRecorder to switch on/off recording by switching output device to|from actual_file|/dev/null.
Second, as I stated "I want to record the data at and just before the threshold is exceeded and not just after the threshold is exceeded". Well, I had to compromise on that. The probe is used to detect the recording condition, but there is no way to stuff the data from the probe back into the recorder. I mean, I guess you could record to a buffer file in idle state and somehow prepend part of that data ... but the complexity wasn't worth it for me.
Aside; there was another issue with QAudioRecorder that motivated me to write a wrapper over it. Basically I found QAudioRecorder::stop() sometimes hangs indefinitely. To get around this, I had to heap allocate a recorder and delete it and init a new one with each new recording.
Related
I'm trying to build simple app that would stream video from camera using browser to the remote server.
For the camera access from browser I've found a wonderful WebRTC API: getUserMedia.
Now for the streaming it to the server IIUC the best way would be to use some of the WebRTC_API for transporting and then use some server side library to deal with it.
However, at first I went with a bit different approach:
I've user MediaRecorder based on the stream from camera. And then I was setting the timeslice for the MediaRecorder.start() to be few hundred Ms, e.g. 200. And I had some assumptions in wrt MediaRecorder which are not in sync with what I was observing:
I've observed weird behaviour(wrt to my assumptions about MediaRecorder):
If there was only 1 chunk uploaded to server -> it opens just fine.
If there are multiple chunks -> none of them loads correctly, they give errors: Could not determine type of stream. But then if I use ffmpeg to concat all the chunks - resulting file is fine. Same happens if I'm concatenating the blobs from MediaRecorder.ondataavailable on the client.
Thus the question:
Can the chunks in theory be independent video files? Or it is not what MediaRecorder was designed for? If it is not - then why do we even have the option to give timeslice parameter in its start() method?
Bonus question
If we're setting timeslice comparatively small, e.g. 10ms -> lots of data blobs that are sent to MediaRecorder.ondataavailable are of size 0. Where we can find some sort of guarantees/specs on the minimal timeslice that we can use, so that the data blobs are meaningful?
In the documentation there are the following:
If timeslice is not undefined, then once a minimum of timeslice milliseconds of data have been collected, or some minimum time slice imposed by the UA, whichever is greater, start gathering data into a new Blob blob, and queue a task, using the DOM manipulation task source, that fires a blob event named dataavailable at recorder with blob.
So, my guess is that it is somehow related to some data blobs being of 0 size. What does it "some minimum time slice imposed by the UA" mean?
PS
Happy to provide code if needed. But the question is not about some specific code. It is to get understanding of the assumptions behind the MediaRecorder API and why they are there.
The timeslice parameter does not allow to create independent media chunks; instead, it gives an opportunity to save data (e.g. on the filesystem, or uploaded to a server) on a regular basis, rather than holding potentially large media content in memory.
I have been writing lots of different kinds of Sketches for Arduino, but there is something I have not heard if it is possible.
I would like to be able to get an Arduino to save preferences that can be restored back when the Arduino restarts. The kind of data that you want to change (in the field) without having to hardcode it in the sketch, or needing to upload changes from the IDE.
Examples:
Thermostat settings
Time lapse camera frame rate
So if I set the thermostat for 68° and the power goes out, I want it to remember what temperature I set when the power comes back on.
Unless you have a large amount of settings that you wish to save, EEPROM should suffice for your purposes. What I would do is define a struct to store the relevant options, and use the EEPROM library to read/write the entire struct.
A wonderful example of how to do that exists on the Arduino reference pages:
EEPROM Put
I was hoping to do some kind of open file, read/write operation like you can do in php, where I can read/write whole strings at a time
There is really no need to do this unless you have a large amount of options. Otherwise your best bet is to use something like an SD card, which does support FileIO operations
Seems you looking for Arduino EEPROM
EEPROM: memory whose values are kept when the board is turned off (like a tiny hard drive). This library enables you to read and write those bytes.
I have 2 input streams of data that are being passed to a Haali Muxer (mp4 format).
Currently I stream these to Haali directly in a DirectShow graph without a clock. I wondered if I should be trying to write these to the muxer synchronised, or whether it happily accepts a stream of audio data that stops before the video data stream stops. (I have issues with the output file not playing audio after seeking, and I'm not sure why this could occur)
I can't find much in the way of documentation for muxing with the Haali muxer, does anyone know the best place to look for info on this filter?
To have the streams multiplexed into single MP4 file you need single instance of multiplexer (Haali, GDCL, commercial, wrapper over mp4v2 library, over Media Foundation sink etc) with two (or more) input pins on it connected to respective sources, which in turn are going to be written as tracks.
Filter graph clock does not matter. Clock is for presentation, and file writers accept incoming data and write it as soon as possible anyway. It is more accurate to remove the clock, as you seem to already be doing, but having standard clock is not going to be different.
Data is synchronized using time stamps on individual media samples, parts of media streams. Multiplexer builds internal queues for every stream and then consumes data from the streams to build single file, in a sort of way that original stream data is interleaved. If one stream supplies too much data, that is, if data is available too early while another stream supplies data slowly, multiplexer blocks further data reception on this particular stream by not returning from respective processing call (IPin::Receive) expecting that during this wait the slow stream provides additional input. Eventually, what multiplexer looks at when matching data from different streams is data time stamps.
To obtain synchronized data in resulting MP4 file you, thus, need to make sure the payload data is properly time stamped. Multiplexer will take care of the rest.
This also includes that the time stamps should be monotonously increasing within a stream, and key frames/splice points are respectively indicated. Otherwise some multiplexers might issue a failure immediately, other would produce the output file but it might have playback issues (esp. seeking).
I'm working on a speech recognition project and my program can recognize words from audio files. Now I need to work with the audio stream coming from microphone. I'm using QAudio for getting sound data from mic and QAudio has a function to start the process. This start(* QBuffer) function writes the data into a QBuffer(inherited from QByteArray) object. When I'm not dealing with continuous stream, I can stop recording from mic anytime I want and copy the whole data from QBuffer into a QByteArray and I can do whatever I wanna do with the data. But in continuous stream QBuffer's size increases by time and becomes 100Mb in 15 mins.
So I need to use some kind of circular buffer but I can't figure out how to do that especially with this start(*QBuffer) function. I also avoid of cutting the streaming sound at a point where the speech continues.
What is the basic way to handle streaming audio data for speech recognition?
Is it possible to change the start(*QBuffer) function into start(*QByteArray) and make the function to overwrite on that QByteArray to build and circular buffer?
Thanks in advance
boost.com is offering a circular buffer
http://www.boost.org/doc/libs/1_37_0/libs/circular_buffer/doc/circular_buffer.html#briefexample
It should meet what you need
Alain
I made a console application, using directshow, that record from a live source (now a webcam, then a tv capture card), add current date and time in overlay and then save audio and video as .asf.
Now I want that the output file is going to change every 60 minutes without stopping the graph. I must not loose any seconds of the live stream.
The graph is something like this one:
http://imageshack.us/photo/my-images/543/graphp.jpg/
I took a look at the GMFBridge but I have some compiling problem with their examples.
I am wondering if there is a way to split what exist from the overlay filter and audio source, connect them to another asf writer (paused) and then switch them every 60 minutes.
The paused asf filter's file name must change (pp.asf, pp2.asf, pp4.asf ...). Something like this:
http://imageshack.us/photo/my-images/546/graph1f.jpg/
with pp1 paused. I found some people in internet that say that the asf writer deletes the current file if the graph does not go in stop mode.
Well, I have the product (http://www.videophill.com) that does exactly what you described (its used for broadcast compliance recording purposes) - and I found that only way to do that is this:
create a dshow graph that will be used only to capture the audio and video
then, at the end of the graph, insert samplegrabber filters, both for audio and video
then, use IWMWritter to create and save wmv file, using samples fetched from samplegrabber filters
when time comes, close one IWMWritter and create another one.
That way, you won't lose single frame when switching the output files.
Of course, there is also question of queue-ing and storing the samples (when switching the writters) and properly re-aligning the audio/video timestamps, but from my research, that's the only 'normal' way to do it, and I used in practice.
The solution is in writing a custom DShow filter with two input pins in your case. One for audio stream and the other for video stream. Inside that filter (doesn't have to be inside from the architecture point of view, because you can also use callbacks for example and do the job somewhere else) you should create asf files. While switching files, A/V data would be stored in cache (e.g. big enough circular buffer). You can also watch and modify A/V sync in that filter. For writing ASF files I would recommend Windows Media Format SDK.You can also add output pins if you like to pass A/V data further if necessary for preview, parallel streaming etc...
GMFBridge is a viable, but complicated solution, a more direct approach I have implemented in the past is querying your ASF Writer for the IWMWriterAdvanced2 interface and setting a custom sink. Within that interface you have methods to remove and add sinks to your ASF writer. The sink automatically connected will write to the file that you speficifed. One way to write whereever you want to is
1.) remove all default sinks:
pWriterAdv->RemoveSink(NULL);
2.) register a custom sink:
pWriterAdv->AddSink((IWMWriterSink*)&streamSink);
The custom sink can be a class that implements IWMWriterSink, which requires implementing callback methods that are called i.e. when the ASF header is written (OnHeader(/* [in] */ INSSBuffer *pHeader);) and when a data packet is written (OnDataUnit(/* [in] */ INSSBuffer *pDataUnit);) - in your implementation you can then write them wherever you want, for example offer additional methods on this class where you can specify the file name you want to write to.
Note that this solution does not quite get you were you want to if you need to write out the header information in each of the 60 minute files - after the initial header you will only get ASF packet data. A workaround for that could be to re-write the intial header before any packet data of each file, however this will produce an unindexed (non-seekable) ASF file.