Collapse AudioRendererMixer and OnMoreDataResampler into AudioTransform.

media/base/audio_transform.cc

Index: media/base/audio_transform.cc
diff --git a/media/base/audio_transform.cc b/media/base/audio_transform.cc
new file mode 100644
index 0000000000000000000000000000000000000000..bd2bf8368aab06bf0c9933f253718d739a1d3062
--- /dev/null
+++ b/media/base/audio_transform.cc
@@ -0,0 +1,196 @@
+// Copyright (c) 2012 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#include "media/base/audio_transform.h"
+
+#include <algorithm>
+
+#include "base/bind.h"
+#include "base/bind_helpers.h"
+#include "media/base/audio_pull_fifo.h"
+#include "media/base/channel_mixer.h"
+#include "media/base/multi_channel_resampler.h"
+#include "media/base/vector_math.h"
+
+namespace media {
+
+AudioTransform::AudioTransform(const AudioParameters& input_params,

[DaleCurtis, 2012/11/12 20:15:23]

I'll add micro-benchmark and run it against the old ARM code, to see how this
new AudioTransform performs.  I'll attempt to isolate into something like:
1. x fake render() calls.
2. x Transforms() with n input unresampled, unfifo'd.
4. x Transforms() with n input channel mixed.
5. x Transforms() with n input rebuffered.
6. x Transforms() with n input resampled.
7. x Transforms() with n input resampled, channel mixed.
8. x Transforms() with n input resampled, channel mixed, refifo'd.

+                               const AudioParameters& output_params)
+    : downmix_early_(false) {
+  CHECK(input_params.IsValid());
+  CHECK(output_params.IsValid());
+
+  // Handle different input and output channel layouts.
+  if (input_params.channel_layout() != output_params.channel_layout()) {
+    DVLOG(1) << "Remixing channel layout from " << input_params.channel_layout()
+             << " to " << output_params.channel_layout() << "; from "
+             << input_params.channels() << " channels to "
+             << output_params.channels() << " channels.";
+    channel_mixer_.reset(new ChannelMixer(
+        input_params.channel_layout(), output_params.channel_layout()));
+
+    // Pare off data as early as we can for efficiency.
+    downmix_early_ = input_params.channels() > output_params.channels();
+    if (downmix_early_) {
+      DVLOG(1) << "Remixing channel layout prior to resampling.";
+      // If we're downmixing early we need a temporary AudioBus which matches
+      // the the input channel count and input frame size since we're passing
+      // |unmixed_audio_| directly to the |source_callback_|.
+      unmixed_audio_ = AudioBus::Create(input_params);
+    } else {
+      // Instead, if we're not downmixing early we need a temporary AudioBus
+      // which matches the input channel count but uses the output frame size
+      // since we'll mix into the AudioBus from the output stream.
+      unmixed_audio_ = AudioBus::Create(
+          input_params.channels(), output_params.frames_per_buffer());
+    }
+  }
+
+  // Only resample if necessary since it's expensive.
+  if (input_params.sample_rate() != output_params.sample_rate()) {
+    DVLOG(1) << "Resampling from " << input_params.sample_rate() << " to "
+             << output_params.sample_rate();
+    double io_sample_rate_ratio = input_params.sample_rate() /
+        static_cast<double>(output_params.sample_rate());
+    resampler_.reset(new MultiChannelResampler(
+        downmix_early_ ? output_params.channels() :
+            input_params.channels(),
+        io_sample_rate_ratio, base::Bind(
+            &AudioTransform::ProvideInput, base::Unretained(this))));
+  }
+
+  // Since the resampler / output device may want a different buffer size than
+  // the caller asked for, we need to use a FIFO to ensure that both sides
+  // read in chunk sizes they're configured for.  Clients may optionally choose
+  // to support arbitrary downstream requests by disabling rebuffering.
+  if (resampler_.get() ||
+      input_params.frames_per_buffer() != output_params.frames_per_buffer()) {
+    DVLOG(1) << "Rebuffering from " << input_params.frames_per_buffer()
+             << " to " << output_params.frames_per_buffer();
+    audio_fifo_.reset(new AudioPullFifo(
+        downmix_early_ ? output_params.channels() :
+            input_params.channels(),
+        input_params.frames_per_buffer(), base::Bind(
+            &AudioTransform::SourceCallback,
+            base::Unretained(this))));
+  }
+
+  // Temporary AudioBus for mixing inputs together.
+  mixer_input_audio_bus_ = AudioBus::Create(input_params);
+
+  input_frame_duration_ = base::TimeDelta::FromMicroseconds(
+      base::Time::kMicrosecondsPerSecond /
+      static_cast<double>(input_params.sample_rate()));
+  output_frame_duration_ = base::TimeDelta::FromMicroseconds(
+      base::Time::kMicrosecondsPerSecond /
+      static_cast<double>(output_params.sample_rate()));
+}
+
+AudioTransform::~AudioTransform() {}
+
+void AudioTransform::AddInput(AudioTransformInput* input) {

[DaleCurtis, 2012/11/12 20:15:23]

What do you think about making AudioTransform thread safe w/ a lock and
providing a Map(<callback>) operation that can be applied to all
AudioTransformInputs.

This would allow us to remove the lock and list from AudioRendererMixer with
something like AudioTransform::Map(OnRenderErrorCallback) to issue down stream
errors for each input.

I suspect CloudView may need to do similar thread safe operations on the list.

[miu, 2012/11/12 20:51:59]

I'm not familiar with how this all works at the higher level (i.e., glue between
inputs and single output, and also the threading scheme), but I'd like to be
educated.  :)  Can we chat on Wed?

[DaleCurtis, 2012/11/12 21:23:53]

I'm not sure I'll be in KIR this Wednesday, but we can chat whenever, send out a
mtg invite.

[DaleCurtis, 2012/11/16 23:51:05]

I'll add the Map() operation in a follow up CL and we can discuss there.

+  transform_inputs_.push_back(input);
+}
+
+void AudioTransform::RemoveInput(AudioTransformInput* input) {
+  DCHECK(std::find(transform_inputs_.begin(), transform_inputs_.end(), input) !=
+         transform_inputs_.end());
+  transform_inputs_.remove(input);
+
+  if (transform_inputs_.empty())
+    Reset();
+}
+
+void AudioTransform::Reset() {
+  if (audio_fifo_)
+    audio_fifo_->Clear();
+  if (resampler_)
+    resampler_->Flush();
+}
+
+void AudioTransform::Transform(AudioBus* dest) {
+  if (transform_inputs_.empty()) {
+    dest->Zero();
+    return;
+  }
+
+  bool needs_mixing = channel_mixer_ && !downmix_early_;
+  AudioBus* temp_dest = needs_mixing ? unmixed_audio_.get() : dest;
+
+  if (!resampler_ && !audio_fifo_) {
+    SourceCallback(temp_dest);
+  } else {
+    if (resampler_)
+      resampler_->Resample(temp_dest, temp_dest->frames());
+    else
+      ProvideInput(temp_dest);
+  }
+
+  if (needs_mixing) {
+    DCHECK_EQ(temp_dest->frames(), dest->frames());
+    channel_mixer_->Transform(temp_dest, dest);
+  }
+}
+
+void AudioTransform::SourceCallback(AudioBus* dest) {
+  bool needs_downmix = channel_mixer_ && downmix_early_;
+  AudioBus* temp_dest = needs_downmix ? unmixed_audio_.get() : dest;

[DaleCurtis, 2012/11/12 20:15:23]

Next CL Optimization: Make FIFO optional for clients, by automatically sizing
the mixer_input_audio_bus_ and unmixed_audio_ buses on the fly.

+
+  // Sanity check our inputs.
+  DCHECK_EQ(temp_dest->frames(), mixer_input_audio_bus_->frames());
+  DCHECK_EQ(temp_dest->channels(), mixer_input_audio_bus_->channels());
+
+  // Calculate the buffer delay for this callback.
+  base::TimeDelta buffer_delay;
+  if (resampler_) {
+    buffer_delay += base::TimeDelta::FromMicroseconds(
+        resampler_->output_frames_ready() *
+        output_frame_duration_.InMicroseconds());
+  }
+  if (audio_fifo_) {
+    buffer_delay += base::TimeDelta::FromMicroseconds(
+        audio_fifo_->output_frames_ready() *
+        input_frame_duration_.InMicroseconds());
+  }
+
+  // Have each mixer render its data into an output buffer then mix the result.
+  for (AudioTransformInputSet::iterator it = transform_inputs_.begin();
+       it != transform_inputs_.end(); ++it) {
+    AudioTransformInput* input = *it;
+
+    float volume = input->ProvideAudioTransformInput(

[DaleCurtis, 2012/11/12 20:15:23]

Next CL Optimization: Write directly to output bus for the first input and using
vector_math::FMUL / memset for volume < 1 and volume == 0 respectively.

[DaleCurtis, 2012/11/16 23:51:05]

Tried this and it didn't make a huge impact. ~100ms over 50000 iterations over 3
inputs.  I suspect it would matter more with larger channel counts.

+        mixer_input_audio_bus_.get(), buffer_delay);
+
+    // Optimize the most common single input, full volume case.
+    if (it == transform_inputs_.begin()) {
+      if (volume == 1.0f) {
+        mixer_input_audio_bus_->CopyTo(temp_dest);
+        continue;
+      }
+
+      // Zero |temp_dest| otherwise, so we're mixing into a clean buffer.
+      temp_dest->Zero();
+    }
+
+    // Volume adjust and mix each mixer input into |temp_dest| after rendering.
+    if (volume > 0) {
+      for (int i = 0; i < mixer_input_audio_bus_->channels(); ++i) {
+        vector_math::FMAC(
+            mixer_input_audio_bus_->channel(i), volume,
+            mixer_input_audio_bus_->frames(), temp_dest->channel(i));
+      }
+    }
+  }
+
+  if (needs_downmix) {
+    DCHECK_EQ(temp_dest->frames(), dest->frames());
+    channel_mixer_->Transform(temp_dest, dest);
+  }
+}
+
+void AudioTransform::ProvideInput(AudioBus* dest) {
+  audio_fifo_->Consume(dest, dest->frames());
+}
+
+}  // namespace media