I have to implement AEC in Flex video conferencing application. I know that it can not be done directly in Flash. I'm thinking to write a native application or library that will do AEC outside Flex.
As far I know Adobe Connect Pro is using some external library that do AEC. I'd like to follow this direction and do something similiar.
There comes two possibilities in my mind:
1) Independend custom AEC filter.
The custom AEC filter will be run in the background and will be processing all data from microphone and sound card and doing AEC. The echo signal will be the data that is to be played but delayed in a temporary buffer, something like this: aec(input, delayed_input, output).
I don't know if it work, because Flash microphone input doesn't know about my custom input and the same with the speakers. But how it is done in WinXP/Vista that have AEC at the system level?
2) AEC filter interacting with Flex's NativeProcess.
The filter will be run by NativeProcess. Flex app will pass microphone sound, received remote voice to the filter and return voice with echo removed and then Flex will output this to the speakers. Will it be correct?
If the 2) method is OK then I have following questions:
I will start with Speex's AEC so in what format should be the data pased from Flex? If I choose Flex Speex codec for sound compression can I pass it directly to the speex library?
Will be the performance OK? Calling NativeProcess library every sound frame buffer can be time consuming I think.
Why there is no AEC implementation in AS3 that can be used directly from Flash? As far I know to do AEC you must have echo data and microphone data (in Flash 10 there is direct access to the microphone). There is definetly something that is stopping doing it like this but I don't know what it is yet :)
Is there some free AEC implementation or a DSP library with in except Speex's that I can use?
PS I'm now a Flash/Flex expert so sorry for some stupid questions :)
Thank in advance for help,
ternyk
Good news - it's built into Flash Player 10.3 now :) Read the release notes and make sure to compile against Flex 4.5 SDK
In Java AEC is Possible and you can access java functionalities in AIR using Merpi.
May be you should try this.
Thanks
Related
Given qt3d's structure, is it possible to integrate the oculus sdk with a qt3d application?
I have tried but my two main obstacles are:
I cant use the textures from the texture swap chain created by the oculus sdk as a render target attachment
I am not able to call ovr_SubmitFrame at the end of each frame since qt3d doesnt have a signal that would allow me to do so.
Has anyone successfully gotten the oculus sdk to work with qt3d? If so, how did you overcome these issues?
Are there any plans for allowing the integration of VR SDKs (not just oculus') in qt3d in further releases?
You could probably do it with some sort of custom framegraph that encapsulated the stereo rendering functionality and included a custom component that could take the currently rendered content and submitted it to the SDK prior to the swapbuffer call.
Alternatively you could dive into the code that processes the framegraph itself and see how hard it would be to customize it to work against a VR API. I've done significant work with integrating Qt apps with VR, but not specifically with Qt3D.
The frame graph will indeed provide one part of the solution for the stereoscopic rendering setup. There is already an anaglyphic stereo example showing most of what you need that ships with Qt 3D.
To integrate the swap chain of the occulus SDK will require deeper integration. I do not know the details of the Occulus SDK as yet but we can take a look.
From what I can see you should be able to do something analogous to the Scene3D custom Qt Quick 2 item to be able to render to the textures provided by the Occulus SDK and to tell Qt 3D which OpenGL Context to use. See
http://code.qt.io/cgit/qt/qt3d.git/tree/src/quick3d/imports/scene3d?h=5.7
Nicolas, I also do not appreciate you publicly saying that KDAB are not much help. I only received an email from Karsten on Friday which I responded to despite being on vacation saying that we can help but it will be on a best efforts basis since you are not paying and I have a very full workload preparing Qt 3D for release at the end of the month along with Qt 5.7. Today is a public holiday in the UK, as you are aware, yet you are already saying detrimental things about us.
You were also directed to post to the interest#qt-project.org mailing list on the qt-forums as I do not tend to monitor SO or the qt-forums on a regular basis. You could have also emailed us directly or via the development#qt-interest mailing list.
We would be more than happy to set up a support agreement with you.
I am using using libsndfile read and manipulate sound files.
I am wandering what whould be the simplest way to play these sounds in real time
in Qt.
I have tried to look at QtAudioOutput, however has to use QtAudioFormat, which is
quite cumbersome to configure.
All i want is to
1) Read data using libsndfile
2) manipulate the data
3) send the buffer to an audio output library.
Try using sndfile-play to play audio files using libsndfile, or a library based on libsndfile such as ALURE
http://kcat.strangesoft.net/alure.html
You could load the audio file yourself manually using libsndfile and play the file using OpenAL. This is essentially what ALURE is doing.
You can output the audio using Phonon. Here is the piece of the docu from Phonon module
The type of the source is set by the media source itself, and is dependent on the constructor used to create it. Note that it is possible to provide data from any source by implementing a QIODevice.
Basically you have to implement your own QIODevice if you want to do it Qt way
I´m working on an application based on directshow that has to convert an AVI source file to to an mp4-file that can be played back with Quicktime.
Since 3ivx, according to my web research the most popular way to fulfill this task, has become commercial (and my budget is quite limited), I decided to use a solution based on ffdshow.
I created a simple graph in graphedit, using LAME for audio encoding and GDCL MPEG 4 Multiplexor for the muxing, but everytime I try to play the movie with Quicktime, I´m getting an error indicating a wrong "sample description".
Playback with Windows Media Player is working, except that there is no sound.
My guess is that there´s a problem with the muxer, because every time I try to add audio encoding, graphedit automatically adds an decoder after the encoding unit (see picture link).
http://imageshack.us/photo/my-images/39/graphjrgr.png/
Any ideas on how to integrate ffdshow in a better way, tips for alternative mp4 muxers, or a complete different approach are appreciated!
The GDCL muxer has limited number of audio formats that it supports, probably you should check the source code for the muxer to see if the formats you are using are in fact supported. Basically, you need to choose an audio encoder that the mux recognizes as valid. It might be possible to use GraphEdit to choose different properties for the encoder filter that allow things to work better.
I have had some luck with the Monogram x264(video) and AAC(audio) encoders. See http://blog.monogram.sk/janos/directshow-filters/
Finally, try the debug version of the GDCL mp4 muxer.
Also, you must be aware of MPEG-4 LA licensing requirements for x264 http://www.mpegla.com/main/programs/AVC/Pages/FAQ.aspx
I would like to accomplish 2 tasks in AIR:
Determine, whether a defined program is running (for example firefox.exe)
If its running, then get the current dimensions of its window - i want to make a screenshot of the window, so i'd need other parameters too i guess: Is it minimized? is it behind some other window?
Is this possible to accomplish in AIR? Im using the latest version (2.6)
No it's not possible out of the box. What you're going to need to do is write a native application in something like C# or C++ and then interface with that application using the NativeProcess API. Here is a video tutorial to something close to what you want to do, and should have you well under way.
http://gotoandlearn.com/play.php?id=125
http://gotoandlearn.com/play.php?id=126
Having had a quick look at the Flex docs I can't seem to find any reference to providing audio content to be played from a custom (possibly encrypted - don't worry, it's not that evil) container format. Is this possible and if so, could someone point me in the right direction.
Or if that's not possible, some way to hook into the disk/network (disk is much more important in this case) I/O of the sound playing mechanism to provide a supported container in memory from a custom wrapper.
Since Flash Player 10, it's posible to write PCM / raw audio data to a Sound Object.
Basically, you call play on an "empty" Sound Object and it will start dispatching periodically a SampleDataEvent, requesting data. You then can write to the audio stream through the data ByteArray exposed by the event object.
http://help.adobe.com/en_US/FlashPlatform//reference/actionscript/3/flash/events/SampleDataEvent.html?filter_flex=4
http://www.adobe.com/devnet/flash/articles/dynamic_sound_generation/index.html
Also, if you're interested in good articles and reference for audio programming in Actionscript, you might want to check out Andre Michelle's stuf:
http://blog.andre-michelle.com/
http://lab.andre-michelle.com/
A flash.media.Sound must either be:
constructed/loaded with a URLRequest,
inherit its data through embedding
There currently is no provision for directly piping mp3 (or aac, or video) data to a any "media" object, such as Sound. You can only get the Sound object to download the data for itself. There are people who are upset about this, including myself; you are not alone!
I say "at this stage" because it's not unthinkable that Adobe will update the API to make this possible in a future version. For the now, you're best to go with the decoding-to-a-dynamic-sound workaround mentioned by Juan, if you really need to be able to do this.
And post a feature request at Adobe's bug tracker, or vote on an existing one!