In short
For a project I need an audio/video-chat for 2 people, with the ability to record (part of) the session. I am running into issues where the 2nd user's recorded video gets messed up, with massive amounts of (seemingly) skipping frames and/or audio loss, most likely caused by the audio stream (when not recording the audio, the problem doesn't appear to occur).
Overview
For a project I need a setup where two clients can video(+audio, obviously :) chat with one another. Also, the 'host' should be able to record the session when (s)he presses a button, and stop recording in the same way.
While this setup is far from rocket science, I've been experiencing issues that I can't seem to figure out. The clients connect fine, and each other's videos show up propely and they can even hear each other just fine. I chose to re-initialize the connection when the host starts the recording, which also works just fine. The recorded files, however, at times experience issues.
The possible cause
Usually the hosts' recording is great, with audio and video working as it should. The client video shows problems, with the video literally jumping when played back (in a number of applications such as VLC, KMPlayer, Adobe's own media player, etc.). I tried to debug the situation and it seems to be directly related to the audio, though how this happens I'm not sure. When I did tests with no recorded audio, both videos played back fine.
The problem
I'll try to explain what happens in the client video; the video plays back normally, with audio working fine too. But in seemingly random locations of the video, playback suddenly skips seconds of the video (and audio), so a one minute video sometimes lasts mere seconds (even though the playback bar and the such show that the video in fact lasts for a minute).
I have not found any logic in how it skips --some videos show only 4/5 gaps of a few seconds each, others jump 20-30 seconds ahead--, so I'm assuming it's a random thing.
Scenarios
I have tested multiple scenarios, and the problem seems consistent enough (as in, it occurs every time, just not when I dont record audio). I have used a local (developer edition) FMS server, the hosted influxis service, and I have used two local computers, one local machine (using 2 webcams) and 1 local, 1 (really) remote computer. All setups seem to have the same issues with the recorded (client) video.
In closing
I'm not sure what details you need, so please, ask me for anything you might need to help me find a solution to this. I have searched and debugged like there was no tomorrow, and haven't been able to figure out what is causing this.
Many, many thanks in advance!
-Dave
I've had some pretty extensive experience with FMS and specifically with influxis... Also did recording of both audio and video as the system I had needed it for security reasons. However, in all cases the video and audio were recorded on the server - not on the same machine as either client.
The issues I've seen include a vast amount of tweaking of both audio and video quality - to find the correct mix to get optimal results. That's where I'd start, adjust quality down and see where that gets you.
For a test environment, I'd suggest using different machines so the CPU doesn't become your issue - as the actual environment would have the clients on different boxes with more CPU.
Contact me with addition information, etc - I and several friends have a system working and I'm happy to help.
<>
Looked over my notes - we were able to talk to influxis about the lagging audio - and there were a couple of server setting they played with AND it was our frames-per-second and other quality guys...
I'm not sure if my problem was the same as yours, but we found a solution.
The flash piece needed to set the silence level (setSilenceLevel) such that it didn't insert audio packets that signal no audio. These audio packets caused recording problems that produced random skips and audio drops.
Dave, have you ruled out latency? are the computers with both cams connected to a high speed network? I would also check the processor utilisation on the machine which is recording.
Related
We are trying to optimize our ASP.NET MVC app and get a big time difference between our server side logs and client side delay.
When refresh the page in Chrome in Timeline it shows 4.47s:
As I understand from the picture, the time for server side code execution should be 3.34s, but in our server logs we have the following:
Begin Request 15:41:52.421
End Request 15:41:53.218
Pre Send Request Headers 15:41:53.218
Pre Send Request Content 15:41:53.218
So, according to server side logs code execution took only 797ms in total.
It does not happen all the time and very often the Chrome timeline shows times very close to server logs. But sometimes we have this couple of seconds delay.
Where could this delay come from?
There is lot of stuff that can affect the time sporadically to such an extent, even though addition of almost three seconds is sort of excessive for this scenario. Since you don't mention much about how is your network set up, what operating system u use etc,
I'll try to sum up a list of what comes to my mind when dealing with this sort of a delay, sorted by probability.
The main problem here is the Waiting part of the total time there you should concentrate your detective talent.
Please note that the answer is very general since the question says virtually nothing about configuration of the server, client computer or the network (if any) between them. Since you say the delay is not present all the time, there are one or more moving targets you need to aim at.
Antivirus
If you have an internet shield or similarly named component, it is not uncommon that the antivirus can seemingly randomly delay some connections while leaving other virtually untouched. For the browser this is transparent (it's just a delay, whatever may have caused it), hence the Waiting.
Network issue
Especially if you are connected through a wireless network or poorly configured wired network, a few seconds delay may occur even though the label on the network device says TurboSpeedTM.
Server side issue
Server may be overloaded with previous requests in a manner not covered by your in-application timer, since there are many steps the server performs before and after your script is executed.
Client OS issue
Just like the antivirus, the OS can delay your packets virtually randomly for various reasons.
When hunting down such issue, I would recommend trying to perform the query on the server itself and compare resulting times, try as may combinations of network setup and operation systems as possible, prefer well planned network environments to those with many unknown or external factors (read wireless) and make use of some packet sniffing software (like wireshark) to check whether the browser doesn't lie. And that would be just the start of it :)
I'm not sure if this is the right place to ask this, but I'll do it anyway.
I have developed an uploader for the Umbraco CMS that lets people upload a queue of files in one go. This uses some simple flash app that just calls a .NET ashx to upload the files one at a time. When one is done, the next one starts.
Recently I've had a user hit a problem where 1 or 2 uploads will go up fine, but then the rest fail. This happens for himself and a client of his. After some debugging, he thinks he's found the problem, but it seems weird so was wondering if anyone else has had this problem?
Both him and his client are on a fibre optic broadband connection so have got really fast upload speeds. When it was tested on a lesser speed broadband connection, all the files were uploaded no problem. According to one of his developer friends, apparently they had come across it before and had to put a slight delay in the upload script to make it work.
Does this sound possible? Had anyone else hit this problem? Is there a known workaround to prevent the uploads from failing?
I have not struck this precise problem before, but I have done a lot of diagnosis of DSL and broadband troubleshooting before, so will do my best to answer this.
There are 2 possible causes for this particular symptom, both generally outside of your network control (I would have thought).
1) Packet loss
Of course where some links receive a very high volume of traffic then they can chose to just dump a lot of data (eg all that is over that link maximum set size), but TCP/IP should be controlling that, and also expecting that sort of thing to drop from time to time, so this seems less likely.
2) The receiving server
May have some HTTP bottlenecks into that server or even the receiving server CPU / RAM etc, may be at capacity.
From a troubleshooting perspective, even if these symptoms shouldn't (in theory) exist, the fact that they do, and you have a specific
Next steps if you really need to understand how it is all working might be to get some sort of packet sniffer (like WireShark) to try to work out at a packet level what exactly is happening.
Also Socket programming can often program directly to the TCP/IP sockets, so you would be processing at the lower network layers, and seeing the responses and timeouts etc.
Also if you control the receiving server, then you can do the same from that end, or at least review the error logs to see what is getting thrown up as a problem.
A really basic method could be to send a pathping to the receiving server if that is possible, and that might highlight slow nodes getting the server, or packet loss between your local machine and the end server.
The upshot? Put in a slow down function in the upload code, and that should at least make the code work.
Get in touch if you need any analysis of the WireShark stuff.
I have run into a similar problem with an MVC2 website using Flash uploader and Firefox. The servers were load balanced with a Big-IP load balancer. What we found out in debugging this is that Flash, in Firefox, did not send the session ID on continuation requests and the load balancer would send continuation requests off to another server. Because the user had no session on the new server, the request failed.
If a file could be sent in one chunk, it would upload fine. If it required a second chunk, it failed. Because of this the upload would fail after an undetermined number of files being uploaded.
To fix it, I wrote a Silverlight uploader.
I'm trying to create a Karaoke app in my spare time, but i'm having some problem syncing the recording and the backing track.
Basically, after i start to publish to an external media server (wowza), when i play back with the recording (mp3, ripped from the recording flv) and the backing track (mp3 too), i notice a delay of 0-800 ms between the 2 track, which is a big time for this type of application.
This delay is always random, as bigger in windows than macs.
I already tried many solution, amongs the others:
Playback with the Sound.extract method, taking 2048 samples each time, to have less latency for the audio start/processing;
Set the microphone silence level to 0;
Sync by cuepoint, in different cases in flv or in the mp3 (extending the sound class)
But every time, i get mixed results (worse results in windows, best results in mac)
Anyone has some suggestion to give? Any help would be appreciated :-)
I'm developing a service where people can stream multiple audio files at the same time.
Unfortunately, when streaming about 4 streams simultaneously, Chrome's HTTP connection limit seems to kick in: new stream requests only arive at the server when a previous connection was closed.
Interestingly enough, I can play 10+ videos at the same time on YouTube.
What kind of technique could YouTube have used here to circumvent the browser's simultaneous http connection limit?
The crucial point here I suppose is that YouTube streams are not directly controlled by the browser, they are embedded Flash players which use streams that are handled by Flash. If you want to hand off the streaming process to an external app/library (Flash, Java etc) you can circumvent these limitations quite easily.
The other point is that YouTube has a huge CDN, so there is no guarantee you're getting any two videos from the same server, which would also help to circumvent concurrency limitations (to a point, at least).
I'm not surprised that Chrome stops you after a while because Google did a load of research and experiments regarding browser concurrency and relative efficiency a while ago, and I remember reading somewhere that they concluded that 3-4 concurrent connections to the same server represented the the most efficient data transfer architecture over straight HTTP. Annoyingly, I can't find a reputable source to reference with that (although I got it from one in the first place), however this is related and probably part of the same research program.
This is also the sort of research Facebook get quite heavily involved in, and you might find some useful information in over at http://developers.facebook.com/ if you can be bothered sifting through the rubbish to find it...
I'm no expert in audio, so if any of you folks are, I'd appreciate your insights on this.
My client has a handful of MP3 podcasts stored at a relatively high bit rate, and I'd like to be able to serve those files to her users at "different" bit rates depending on that user's credentials. (For example, if you're an authenticated user, you might get the full, unaltered stream, but if you're not, you'd get a lower-bit-rate version -- or at least a purposely tweaked lower-quality version than the original.)
Seems like there are two options: downsampling at the source and downsampling at the client. In this case, knowing of course that the source stream would arrive at the client at a high bit rate (and that there are considerations to be made about that, which I realize), I'd prefer to alter the stream at the client somehow, rather than on the server, for several reasons.
Is doing so possible with the Flash Player and ActionScript alone, at runtime (even with a third-party library), or does a scenario like this one require a server-based solution? If the latter, can Flash Media Server handle this requirement specifically? Again, I'd like to avoid using FMS if I can, since she doesn't really have the budget for it, but if that's the only option and it's really an option, I'm open to considering it.
Thanks in advance...
Note: Please don't question the sanity of the request -- I realize it might sound a bit strange, but the requirements are what they are. In that light, for purposes of answering the question, you can ignore the source and delivery path of the bits; all I'm really looking for is an explanation of whether (and ideally how) a Flash client can downsample an MP3 audio stream at runtime, irrespective of whether the audio's arriving over a network connection or being read directly from disk. Thanks much!
I'd prefer to alter the stream at the client somehow, rather than on the server, for several reasons.
Please elucidate the reasons, because resampling on the client end would normally be considered crazy: wasting bandwidth sending the higher-quality version to a user who cannot hear it, and risking a canny user ripping the higher-quality stream at it comes in through the network.
In any case the Flash Player doesn't give you the tools to process audio, only play it.
You shouldn't need FMS to process audio at the server end. You could have a server-side script that loaded the newly-uploaded podcasts and saved them back out as lower-bitrate files which could be served to lowly users via a normal web server. For Python see eg. PyMedia, py-lame; or even a shell script using lame or ffmpeg or something from the command line should be pretty easy to pull off.
If storage is at a premium, have you looked into AAC audio? I believe Flash 9 and 10 on desktop browsers will play it. AAC in my experience takes only half of the size of the comparable MP3 (i.e. a 80kbps AAC will sound the same as a 160kbps MP3).
As for playback quality, if I recall correctly there's audio playback settings in the Publish Settings section in the Flash editor. Wether or not the playback bitrate can be changed at runtime is something I'm not sure of.