Surround Sound handling by folding 5 channels down to stereo.

Surround Sound handling by folding 5 channels down to stereo.

BUG=16026
TEST=play a movie trailer with 5.1 audio.  It should sound roughly the same as quicktime.


Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=21613

[fbarchard, 2009-07-21 23:49:15]

just the api and some comments about the surround sound folding

[fbarchard, 2009-07-22 05:00:17]

Surround sound 5.1 folding down to stereo.
Includes volume which reduces clipping if done in 2 steps.

Caveats:

Only supports 5 or 6 channel down to stereo.  Other channels are future work,
but we can try falling back on Alpha's change that mutes the audio.

This version always does the folding.  Future version could attempt 5.1 first,
and then fallback on stereo.

Performance is slow, mainly in the cast to integer.

[fbarchard, 2009-07-23 00:51:17]

any comments on the code?

[cpu_(ooo_6.6-7.5), 2009-07-23 17:49:51]

http://codereview.chromium.org/155894/diff/1010/13
File media/audio/audio_util.cc (right):

http://codereview.chromium.org/155894/diff/1010/13#newcode82
Line 82: #define CHANNEL_SR 4
#define considered harmful. Use a const.

I am unconfortable with this assumption. At the very least we need to expose the
magic order in the audio_output header (as an enum for example) so the layer
above (when it has more info) can reorder the channels.

http://codereview.chromium.org/155894/diff/1010/13#newcode103
Line 103: (left + surround_left + center_half) * volume + mid_value,
maybe I am not reading this correctly but it seems it will overflow for in32
format

http://codereview.chromium.org/155894/diff/1010/13#newcode111
Line 111: }
I am not sure I agree with this form of mixing. For example when there is a
significant signal in both the C and L channel we will distort (clamp). Second,
the perceived loudness is based on the rms power, reducing the C channel by 1/2
in amplitude reduces the power much more (4 times).  That is why you see some
people mixing by 1/sqrt(2) or 0.707 which should give you half the rms power in
each L R. Of course this causes even bigger clamping if you just change the 0.5
by 0.707 above.
I assumed that you would pre-scale all channels so the even in full swing the
sum will not clamp. As is it would only sound acceptable if the signal is
pre-scaled which I assume is not or it will only sound good if there is very
little combined signal in SL, C, SR which takes us back to then why  do it?
One thing that you can use to your advantage is that people rarely feel signal
reduction up to -3db.

Is there any document that you can point to validate this approach? So far this
goes against my gut feeling.

http://codereview.chromium.org/155894/diff/1010/14
File media/audio/audio_util_unittest.cc (right):

http://codereview.chromium.org/155894/diff/1010/14#newcode104
Line 104: 
Please test with values close to full swing as well. So we check that we don't
underflow or overflow.

[fbarchard, 2009-07-24 00:27:54]

fix for overflow
center channel gets .707 scaling
const instead of macro
fixed up unittests to match

http://codereview.chromium.org/155894/diff/1010/13
File media/audio/audio_util.cc (right):

http://codereview.chromium.org/155894/diff/1010/13#newcode82
Line 82: #define CHANNEL_SR 4
Done const - but I avoided const because it tends to have a performance impact.
The layer above only support mono and stereo, and we only support, at this
level, one ordering.  Which I hope is what ffmpeg adapts into from anything, but
will need to look more into that with other file formats.  For now I know that
if I convert mp4 to ogv and play that, it sounds the same.


On 2009/07/23 17:49:51, cpu wrote:
> #define considered harmful. Use a const.
> 
> I am unconfortable with this assumption. At the very least we need to expose
the
> magic order in the audio_output header (as an enum for example) so the layer
> above (when it has more info) can reorder the channels.

http://codereview.chromium.org/155894/diff/1010/13#newcode103
Line 103: (left + surround_left + center_half) * volume + mid_value,
doh!  The multiple by float volume does the cast too late.  Good catch.
Done.

http://codereview.chromium.org/155894/diff/1010/13#newcode111
Line 111: }
no I havent found good reference to how to fold.
The key element here is the center channel, which is the main voice track needs
to be loud, and certainly not dropped.  And I'm reluctant to drop the rear
channels, just because they might have important sound on them... but its okay
if they're quiet.
So in the next draft, I'll use 1/sqrt(2) to drive the center channel volume.
As for clipping, I do the math in float, then the volume, and then clip.  We
could reduce the volume... ultimately using 1/3 would be just an average and not
clip.  But my concern there is that laptops are already too quiet, and movies
are often authored with sound that is too quiet.
Also reducing the volume introduces quantization noise, since we're working with
limited precision integers.
I could gather some stats on how much clipping occurs on actual movie trailers.
I could also build you versions of chrome with different methods of folding, so
your sound experts could listen to the differences.
If we get more complicated with the math, performance will become a concern, and
a different type of quality concern will occur - dropped frames.  Currently its
3% of the CPU.  Putting sqrt or pow into the loop would make it 10x slower... so
about 30% of a cpu.  This is just the quick fix for compatibility.  The good
solution is proper 5.1 hardware support, maybe via directsound.

Using 0.707 I was surprised when one of the unittests didnt fail.  .5 and .7 are
close enough on u8 to not make much difference.

[fbarchard, 2009-07-25 02:43:59]

I've done quite a bit more testing, but have no more code changes to suggest at
this point.

Are we good to go?

[fbarchard, 2009-07-25 03:23:12]

we need to move on.  i think this is the best compromise for now, and we should
definately improve on it in the future.

its fast enough, but could be faster.  I suggest optimizing the top 5
bottlenecks first and revisit this later.  But the thread on "a slowish audio
function" had some hints on what to do.

it doesnt do ogg properly, but neither does ffmpeg, mplayer, safari or firefox. 
The fix, I think will be in the the decoder - we'll detect ogg and remap to aac
style.  This is unrelated to folding.

its not perceptually correct, but it matches safari.  the center channel is
scaled by .707.
Doing a perceptually correct folder would introduce risk on performance, and not
sound the same as safari.  More research required.  But doing really good
folding, is not as good as actually supporting surround sound.

its not proper 5.1 support. The best solution will be when we can produce full
5.1 or 7.1 output to a home theatre.  But this likely requires a change to
directsound for PC.  We may want to do mac/linux first which hopefully have
better sound API's that pass thru 6 channels properly.