My application must read one video track and several audio tracks, and be able to specify one section of the file and play it in loop. I have created a setup with Media Foundation, using the sequencer source and creating several topologies with the start and end point of the section I want to loop. It works, except for the fact that there is a 0.5 to 1 sec time of stabilization of the playback just when it goes back to the starting point.
First, I made it with individual audio files and one video file. This was quite bad for some files, sometimes all the files were completely out of sync, sometimes the video was frozen for several seconds, then went very fast to catch with the audio.
I had a good improvement using only one file, that includes the video and the multiple audio tracks. However, for most files, there is still a problem about the smoothness of the transition.
With a poor quality video AVI file, I could make it work smoothly, which would mean that the method I use is correct. I have noticed that the quality of the loop smoothness is strongly related to the CPU used on a file when simply playing it.
I use the "SetTopology" on the session, using a series of topologies, so normally it should preroll the next one during the playback of the current one, right ? Or am I missing something there ?
My app works also on Mac, where I have used a similar setup with AVFoundation, and it works fine with the same media files I use on Windows.
What can I do to have the looping work smoothly with better quality video on Windows ? Is there something to do about it ?
When I play the media file without looping, I notice that when I preroll it to some point, then when I hit the START button, the media starts instantly and with no glitch. Could it work better if I was using two independent simple playback setups, start the first, preroll the second, then stop the first and start the second programmatically at the looping point ?
Related
I was recently playing a video on YouTube, and a thought came to my mind. As videos are played, a user can skip ahead in the video and the video just resumes at that point without any trouble.
What I can't seem to find is how this works, I know that when I request a file through HTTP it downloads the entire thing, so starting a binary stream halfway through the video doesn't seem possible using HTTP. Is there any RFC or related document on how browsers do this?
Thank you
There are a couple of different technologies but they all essentially allow you specific an offset in the video and then download a 'chunk' from there.
The simple way to do this is with byte ranges and HTTP progressive download 0 there is an RFC which covers this:
https://www.rfc-editor.org/rfc/rfc7233
A similar but slightly more complex mechanism is behind the various adaptive bit rate protocols, such as HLS, MPEG-DASH, Smooth-streaming etc. These protocols break a video into multiple chunks (e..g 10 second long segments) and also create several different encoding of the video each at different bit rates.
The client can then request the next chunk based on current network conditions - if the network is busy or if the client is using a low band witch connection it can request the next chunk from a low bit rate encoding of the video. If network connectivity improves then it can request from progressively higher bit rates until it reaches the maximum.
You can see this in action if you look at the 'stats for nerds' available in YouTube if you right click on the video - look at the connection speed graph.
This mechanism also means the client can request chunks from further ahead (or behind) than the current position in the video - so long as it is not live obviously!
It also allows faster start up if you do jump ahead as the playback can start from a lower bit rate which is faster to download and work up to the higher bit rate again. You can often see this when playing around with services like Netflix - if you jump ahead it may be lower quality for a little time initially.
YouTube stores the videos in several chunks. Once each chunk completes its download, you can play that chunk of video. Think of them as individual split videos.
When you try to jump into the middle, they will start downloading the necessary chunk of video and start playing. Therefore, you can jump to the middle.
I have code that generates thumbnails from JPEGs. It pulls an image from S3 and then generates the thumbs.
One in about every 3000 files ends up looking like this. It happens in batches. The high res looks like this and they're all resized down to low res. It does not fail on resize. I can go to my S3 bucket and see that the original file is indeed intact.
I had this code written in Ruby and ported it all over to clojure hoping it would just fix my issue but it's still happening.
What would result in a JPEG that looks like this?
I'm using standard image copying code like so
(with-open [in (clojure.java.io/input-stream uri)
out (clojure.java.io/output-stream file)]
(clojure.java.io/copy in out))
Would there be any way to detect the transfer didn't go well in clojure? Imagemagick? Any other command line tool?
My guess is it is one of 2 possible issues (you know your code, so you can probably rule one out quickly):
You are running out of memory. If the whole batch of processing is happening at once, the first few are probably not being released until the whole process is completed.
You are running out of time. You may be reaching your maximum execution time for the script.
Implementing some logging as the batches are processed could tell you when the issue happens and what the overall state is at that moment.
I'm using two custom push filters to inject audio and video (uncompressed RGB) into a DirectShow graph. I'm making a video capture application, so I'd like to encode the frames as they come in and store them in a file.
Up until now, I've used the ASF Writer to encode the input to a WMV file, but it appears the renderer is too slow to process high resolution input (such as 1920x1200x32). At least, FillBuffer() seems to only be able to process around 6-15 FPS, which obviously isn't fast enough.
I've tried increasing the cBuffers count in DecideBufferSize(), but that only pushes the problem to a later point, of course.
What are my options to speed up the process? What's the right way to do live high res encoding via DirectShow? I eventually want to end up with a WMV video, but maybe that has to be a post-processing step.
You have great answers posted here to your question: High resolution capture and encoding too slow. The task is too complex for the CPU in your system, which is just not fast enough to perform realtime video encoding in the configuration you set it to work.
I'm using a simple DirectShow graph to convert some videos to WMV format, which is working fine. I'm now trying to use a filter based on the Synth Filter sample to supply a silent audio track to the videos and I'm running into some problems.
Essentially, I don't know how to stop the graph when this filter (the synth filter) is connected. I guess because it just provides samples forever until somebody tells it to stop, the usual approach of calling IMediaEvent::WaitForCompletion on the filter graph doesn't work (the graph never stops). What I want it to do of course is stop as soon as the video source filter is finished.
I've tried tracking the position of the graph with IMediaSeeking::GetPositions and then manually stopping the graph when this exceeds the duration of the source file, but the accuracy of the stop time with this approach isn't great.
Can anyone think of a better way to do this? Do I need to have another filter that monitors the output from the video source and also has a pointer to the audio source so it can stop it as soon as the video source delivers EndOfStream? Is there no way to accomplish this from purely application-side code?
I've done something not too different myself in the past. I added support for IMediaSeeking to the silence generator filter, and then you need to make sure that you set start and stop times for the conversion (even if it's just 0 and duration), so that the silence generator can generate the right amount of audio and then send EOS.
G
I am attempting to use DirectShow to play two AVI files consecutively (one after the other) so that there is no interruption in the audio or video when the player transitions from one file to the next.
I have two custom controls on my form. Each one is pre-loaded with an AVI file, and before playback begins I set up all the DirectShow interfaces, set the video windows and resize them, call IMediaControl.Run(), then IMediaControl.Pause(), then IMediaSeeking.SetPositions to reset to frame 0, on both controls. On the form, you can see that both files are paused at their initial frames.
I then call IMediaControl.Run() on the first control, and wait for it to complete before calling Run() on the second control. Initially, I hooked into the first video's EC_COMPLETE notification message, and used this to start the second. Thinking that this event might be slow to arrive (turns out it is, but for a weird reason), I tried two other approaches:
Check the first video's current position inside a timer that goes off every second or so (using IMediaPosition.get_CurrentPosition). When the current position is within a second of the video's stop time (known in advance from IMediaPosition.get_StopTime), I go into a tight while loop and wait for the current position to equal the stop time, and then call Run() on the second video.
Same as the first, except I replace the while loop with a call to timeSetEvent from winmm.dll, with a delay set so that it fires right when the first file is supposed to end. I use the callback to Run() the second file.
Either of these two methods substantially cuts down the delay between the end of the first file and the beginning of the second, indicating that the EC_COMPLETE message doesn't arrive immediately after the file is complete (I also tried hooking the EC_SEGMENT_COMPLETE message, which is supposed to be used for looping within a file, but apparently nobody supports this - it never occurs on my machine, at least).
Doing all of the above has cut the transition delay from as much as a second, down to a barely perceptible glitch; about a third of the time the files transition with no interruption at all, which suggests there's no fundamental reason I can't get this to work all the time.
The slight delay is still unacceptable, unfortunately. I assume (and I could easily be wrong) that the remaining delay is due to a slight variable delay between the call to IMediaControl.Run() and when the video actually starts playing.
Does anybody know anything I can do to eliminate this little lag? It would also help to be told this is fundamentally impossible for whatever reason, which wouldn't surprise me. I've never encountered a video player in Windows that doesn't have this problem, so it may not be doable.
More info: the AVI files I'm playing are completely uncompressed (video and audio are uncompressed), so I don't think the lag is due to DirectShow's having to uncompress the video ahead of play start, although it may still buffer ahead as matter of course (and this may be the source of the problem). I would have though that starting play, pausing and then rewinding to the beginning would fix this.
Also, the way I'm handling the transition is to actually have the second control underneath the first; when the first completes playing, I start the second and then call BringToFront on it, creating the appearance of a single video transitioning between the two originals. I don't think the glitch is due to this, because it works perfectly some of the time, and even if this were problematic, it wouldn't explain the matching audio glitch.
Even more: I just tried starting the second video 30-50 milliseconds "early" and that seemed to eliminate even more of the gap, so I'm guessing that the lag in Run() is about that long. It appears to be variable, though, so this is still not where I need it to be.
Still more: perhaps I could eliminate this delay by loading the AVIs from memory rather than from a file. Unfortunately, I have no idea how to do this. IMediaControl only has a RenderFile() method, not something like a RenderStream or RenderMemory method.
If you call IMediaControl::Run on a stopped graph, the graph manager will post the call to a worker thread (so there's some variability). On the worker thread, the graph will be paused. Render filters only complete a pause transition once they have received data, so once GetState() returns S_OK, the graph manager knows that the graph is fully cued. At this point, it picks a time roughly 10ms into the future, and calls Run on each filter with that time as the start point. Since it takes time to tell each filter to Run, the dshow Run method has a parameter which is the refclock time at which a sample timestamped zero should be played -- i.e. the time at which the actual transition to run mode should take place.
To synchronise this with another graph, you first have to ensure that both graphs have the same clock. Query the graph (not the filter) for IMediaFilter, and call GetSyncSource on one graph and SetSyncSource on the other. Then you need to pause the second graph, so that it is cued and ready. When you want to start it, call IMediaFilter::Run instead of IMediaControl::Run, and you can pass your own start time. This still has to be a few milliseconds into the future, so the best thing might be to set the start time of the second graph to be the first graph's start time plus its duration (for an indexed container of uncompressed streams, the duration should be accurate).
Another approach is to use multiple graphs. Separating source from rendering would allow you to switch seamlessly between sources since they feed into a common render graph. There is sample source code for this approach at www.gdcl.co.uk/gmfbridge.
G