I have a legacy file format that contains sounds embedded in it (in various encodings). I would like to be able to play these sounds in Flash (Air?) by reading the sound bytes out of the file and instantiating a Sound object with them.
If the sound is unencoded (e.g., raw pcm), I've found that I can use the new flex 4 SampleDataEvent.SAMPLE_DATA event to play the sound.
However, if the sound is encoded (e.g., mp3), then I'm at a loss. The sound expected by SampleDataEvent.SAMPLE_DATA has to be raw pcm. From what I've seen, encoded Sounds can only be instantiated by [Embed]ing them, or by using a URLRequest with Sound.load().
Surely there's a third way? AMF or e4x?
There are really only two routes for you to go. The first is to write a decoder in ActionScript. You may be able to use Alchemy to port over some C/C++ code to make this job significantly easier (and possibly more performant). This is exactly how I got Ogg Vorbis playback to work with Flash.
The other option is to dynamically create a valid SWF inside of a ByteArray. That SWF could contain an embedded sound object that was made up of your sound data. A number of folks have pulled off similar hacks in the past before Flash Player 10 was available. I believe you can find a good place to start in Andre Michelle's and Joa Ebert's PopForge codebase.
Related
I'm trying to understand how I can generate a waveform from an audio (or video) file to display to the user.
I've been googling around for quite a while now and can't determine if this is even possible in Qt without using something like FFmpeg. I've seen all of these classes: QMediaPlayer, QMediaContent, QMediaResource, QAudioProbe and experimented with the Qt Media Player Example but am just not seeing where I can access the actual audio buffer.
So I have 2 questions:
Is what I want to do even possible without 3rd party libraries?
If it is possible, can some kind soul outline what I need to read and understand in order to access the audio data
I have tried the suggestions from this question (Audio visualization with QMediaPlayer) but the result of audioProbe->setSource(player) is always false and the method processBuffer never gets called.
audioProbe = new QAudioProbe(this);
bool success = audioProbe->setSource(player);
qDebug() << success;
connect(audioProbe, SIGNAL(audioBufferProbed(QAudioBuffer)), this, SLOT(processBuffer(QAudioBuffer)));
Update: Adding some additional detail in the hope of clarifying things.
For testing/learning I am using the Media Player Example which ships with Qt, so it is set up correctly with Q_OBJECT etc.
For audio, I tested with both .mp3 and .wav files. FWIW, the player example won't play video for some reason (.mp4, .avi were tested)
The player in the code is QMediaPlayer – which inherits from QMediaObject. The example code for the Player class is here. I added my code (in original comment above) right after the player is instantiated. I also tried adding it once media is loaded.
I tried declaring my slot first as private, then as public – either way, it is never called.
Frustrating that such a simple thing is so hard.
Going the "no external library" route will likely just lead to more of a headache and more work than is necessary. The other advantage of going with an established library is you won't be bound to one file format, as not all formats store their data the same way. If the audio format is uncompressed (wav or other) you can read the header until you get to the data chunk. An answer to this question here details this in C. You should be able to get an idea for the file format from this to apply it to another language.
You will want to understand how many channels are in the wav file, bit depth, and also the sampling rate before you can do anything worthwhile with the data. All this info can be grabbed from the header.
It turns out that QAudioProbe is not supported on OSX – the platform I am working on. Took quite a while (a "Qt while. . .") to ferret that info out so I am posting it here explicitly.
See this document for full details: Qt 5.5.0 Multimedia Backends
I´m working on an application based on directshow that has to convert an AVI source file to to an mp4-file that can be played back with Quicktime.
Since 3ivx, according to my web research the most popular way to fulfill this task, has become commercial (and my budget is quite limited), I decided to use a solution based on ffdshow.
I created a simple graph in graphedit, using LAME for audio encoding and GDCL MPEG 4 Multiplexor for the muxing, but everytime I try to play the movie with Quicktime, I´m getting an error indicating a wrong "sample description".
Playback with Windows Media Player is working, except that there is no sound.
My guess is that there´s a problem with the muxer, because every time I try to add audio encoding, graphedit automatically adds an decoder after the encoding unit (see picture link).
http://imageshack.us/photo/my-images/39/graphjrgr.png/
Any ideas on how to integrate ffdshow in a better way, tips for alternative mp4 muxers, or a complete different approach are appreciated!
The GDCL muxer has limited number of audio formats that it supports, probably you should check the source code for the muxer to see if the formats you are using are in fact supported. Basically, you need to choose an audio encoder that the mux recognizes as valid. It might be possible to use GraphEdit to choose different properties for the encoder filter that allow things to work better.
I have had some luck with the Monogram x264(video) and AAC(audio) encoders. See http://blog.monogram.sk/janos/directshow-filters/
Finally, try the debug version of the GDCL mp4 muxer.
Also, you must be aware of MPEG-4 LA licensing requirements for x264 http://www.mpegla.com/main/programs/AVC/Pages/FAQ.aspx
I'm trying to use Adobe's OggVorbis library. But I can't seem to get the Sound object to loop.
I even tried looping the _sound object inside the AudioDecoder.as in the "com.automatastudios.audio.audiodecoder" package.
Do you really have to reload the file and stream it over and over?
If you're streaming, then yes you'll have to jump back the beginning of the stream. A stream, by definition, is a constant link to the server and does minimal loading of files locally.
But, if you're not really streaming, you should have no problem loading up a file and caching it locally then playing it over and over.
Since you mention Alchemy, there may be other unknown issues if you're trying to use a converted C library, as opposed to native ActionScript.
[Note; I didn't know the OggVorbis library for Flex before now].
Having had a quick look at the Flex docs I can't seem to find any reference to providing audio content to be played from a custom (possibly encrypted - don't worry, it's not that evil) container format. Is this possible and if so, could someone point me in the right direction.
Or if that's not possible, some way to hook into the disk/network (disk is much more important in this case) I/O of the sound playing mechanism to provide a supported container in memory from a custom wrapper.
Since Flash Player 10, it's posible to write PCM / raw audio data to a Sound Object.
Basically, you call play on an "empty" Sound Object and it will start dispatching periodically a SampleDataEvent, requesting data. You then can write to the audio stream through the data ByteArray exposed by the event object.
http://help.adobe.com/en_US/FlashPlatform//reference/actionscript/3/flash/events/SampleDataEvent.html?filter_flex=4
http://www.adobe.com/devnet/flash/articles/dynamic_sound_generation/index.html
Also, if you're interested in good articles and reference for audio programming in Actionscript, you might want to check out Andre Michelle's stuf:
http://blog.andre-michelle.com/
http://lab.andre-michelle.com/
A flash.media.Sound must either be:
constructed/loaded with a URLRequest,
inherit its data through embedding
There currently is no provision for directly piping mp3 (or aac, or video) data to a any "media" object, such as Sound. You can only get the Sound object to download the data for itself. There are people who are upset about this, including myself; you are not alone!
I say "at this stage" because it's not unthinkable that Adobe will update the API to make this possible in a future version. For the now, you're best to go with the decoding-to-a-dynamic-sound workaround mentioned by Juan, if you really need to be able to do this.
And post a feature request at Adobe's bug tracker, or vote on an existing one!
I am embedding an mp3 into my Flex project for use as a sound effect, but I am finding that every time I play it, there is a delay of about half a second from when I call .play() to when you can hear the sound. This makes it weird because I want the sound effects to sync to game events. My mp3 itself is only about a fifth of a second long so it isn't because of the contents of the mp3.
I'm embedding with
[Embed(source="assets/Tock.mp3")]
[Bindable]
public static var TockSound:Class;
public var tock_sound:SoundAsset;
and then playing with
if (tock_sound == null) {
tock_sound = new TockSound() as SoundAsset;
}
Alert.show("tock");
tock_sound.play();
I know there's a delay because the sound plays about a half second after the Alert displays. I did consider that maybe it was the initial loading time of constructing the TockSound, but the delay is there on all the subsequent calls as well.
How can I avoid this delay on playing a sound?
Update: It turns out this delay is only present when playing the swf on Linux. I believe it is a Linux-specific flaw in Adobe's flash player.
Not sure about the reason, other than Flash always has had some bad audio latency issues. Read Tinic's blog to stay on top of this stuff: http://www.kaourantin.net/
One thing that might help: make sure your MP3 is 44.1kHz or else Flash will need to resample it.
You can actually embed a WAV file, it just takes work. You embed it as a byte array, and in FP9, dynamically construct a SWF file on the fly. Pretty horrible, but doable. :-) In FP10, you can use the dynamic sound API, so it's easy.
Try StandingWave
http://code.google.com/p/standingwave/
It has the ability to "cache" the sound before playing getting rid of those delays and clicks you normally hear
I haven't worked with audio in Flash too much but it sounds like the half second delay might be the Flash Player opening up the file and reading it into memory. You could try doing a play() and a stop() when you load the application. That might push it into memory.
The other option is using the StandingWave library which was built by the guys at Noteflight. You can get some additional control over the audio files with that library and hopefully it'll help your delay problem.
The problem is that all MP3s have a random amount of blank time at the beginning of the file that is put there during the compression process. Modern software jukeboxes(itunes, songbird etc...) compensate for this by scanning the file before its played and determining the songs actual starting point. Your best bet for sound effects is to use .wav files as their format allows for instant playback, but with a filesize hit.
you might also try: http://www.mptrim.com/ <- they claim to be able to trim the space off the mp3.