Integrate HTMLMediaElement with Web Audio API's MediaElementAudioSourceNode

content/renderer/render_audiosourceprovider.cc

Index: content/renderer/render_audiosourceprovider.cc
===================================================================
--- content/renderer/render_audiosourceprovider.cc	(revision 0)
+++ content/renderer/render_audiosourceprovider.cc	(revision 0)
@@ -0,0 +1,95 @@
+// Copyright (c) 2011 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#include "content/renderer/render_audiosourceprovider.h"
+
+#include "base/basictypes.h"
+#include "base/logging.h"
+#include "third_party/WebKit/Source/WebKit/chromium/public/WebAudioSourceProviderClient.h"
+
+using std::vector;
+using WebKit::WebVector;
+
+RenderAudioSourceProvider::RenderAudioSourceProvider(
+    media::AudioRendererSink::RenderCallback* renderer)
+    : is_running_(false),
+      volume_(1.0),
+      renderer_(renderer),
+      client_(0) {

[tommi (sloooow) - chröme, 2011/12/19 15:26:24]

use NULL for initializing pointers.

[Chris Rogers, 2011/12/19 21:40:35]

Righto! Too much time spent in WebKit :)

On 2011/12/19 15:26:24, tommi wrote:

+}
+
+void RenderAudioSourceProvider::Start() {
+  if (!is_running_) {

[tommi (sloooow) - chröme, 2011/12/19 15:26:24]

nit: just set is_running_ to true and skip the if()

[Chris Rogers, 2011/12/19 21:40:35]

Done.

+    is_running_ = true;
+  }
+}
+
+void RenderAudioSourceProvider::Stop() {
+  if (is_running_) {

[tommi (sloooow) - chröme, 2011/12/19 15:26:24]

ditto

[Chris Rogers, 2011/12/19 21:40:35]

Done.

+    is_running_ = false;
+  }
+}
+
+void RenderAudioSourceProvider::Play() {
+  is_running_ = true;

[tommi (sloooow) - chröme, 2011/12/19 15:26:24]

what's the difference between Start() and Play()?  If Start() must be called
before Play() can be called, then should is_running_ perhaps already be true?

[Chris Rogers, 2011/12/19 21:40:35]

The Start()/Play()/Pause()/Stop() state is described in the AudioDevice
interface - now moved to AudioRendererSink with this patch

There *is* the possibility that Pause() could have been called at some point
after Start() was called, so we still need to set is_running_ to true here.

On 2011/12/19 15:26:24, tommi wrote:

+}
+
+void RenderAudioSourceProvider::Pause(bool flush) {
+  is_running_ = false;

[tommi (sloooow) - chröme, 2011/12/19 15:26:24]

same here with Stop vs Pause... could you call Pause() and then Stop()?

Actually... should Play() call Start() and Pause() Stop()?

[Chris Rogers, 2011/12/19 21:40:35]

I realize that it's unfortunate to have four calls (Start/Play/Pause/Stop)
instead of just two (Start/Stop).  The reason we have it is because of the
recent simplification of AudioRendererImpl.  For more details, you can look at
the review comments for that CL:
http://codereview.chromium.org/8477037/

We added a TODO in audio_device.h:
  // TODO(crogers): it should be possible to remove the extra complexity
  // of Play() and Pause() with additional re-factoring work in
  // AudioRendererImpl.

On 2011/12/19 15:26:24, tommi wrote:

[tommi (sloooow) - chröme, 2011/12/20 15:05:46]

ah, I see. Thanks for the info.  Two methods would be nice moving forward :)

+}
+
+bool RenderAudioSourceProvider::SetVolume(double volume) {
+  volume_ = volume;
+  return true;
+}
+
+void RenderAudioSourceProvider::Initialize(
+    size_t buffer_size,
+    int channels,
+    double sample_rate,
+    AudioParameters::Format latency_format,
+    RenderCallback* renderer) {
+  // We already know what the renderer is from our constructor.
+  DCHECK_EQ(renderer, renderer_);
+
+  // Inform WebKit about the audio stream format.
+  if (client_)
+    client_->setFormat(channels, sample_rate);
+}
+
+void RenderAudioSourceProvider::setClient(
+    WebKit::WebAudioSourceProviderClient* client) {
+  client_ = client;
+
+  if (client) {
+    // Detach the audio renderer from normal playback and make us be the sink.
+    // The renderer will call Initialize() when the stream format is known.
+    renderer_->SetAudioRendererSink(this);
+  } else {
+    // Restore normal playback.
+    renderer_->SetAudioRendererSink(0);

[tommi (sloooow) - chröme, 2011/12/19 15:26:24]

NULL

[Chris Rogers, 2011/12/19 21:40:35]

Done.

+  }
+}
+
+void RenderAudioSourceProvider::provideInput(
+    const WebVector<float*>& audio_data, size_t number_of_frames) {
+  DCHECK(renderer_);
+  DCHECK(client_);
+
+  if (renderer_ && is_running_) {
+    // Wrap WebVector as std::vector.
+    vector<float*> v(audio_data.size());
+    for (size_t i = 0; i < audio_data.size(); ++i)

[tommi (sloooow) - chröme, 2011/12/19 15:26:24]

memcpy?

[Chris Rogers, 2011/12/19 21:40:35]

This is copying pointer values (one per audio channel - 2 in the stereo case),
not copying blocks of data.

On 2011/12/19 15:26:24, tommi wrote:

[tommi (sloooow) - chröme, 2011/12/20 15:05:46]

Yes, I know, but since the pointers are stored in a contiguous vector memcpy
should still work.  However, it's not going make a dramatic difference, so this
is fine.

I wonder though if we could get rid of some of these alloc+copy operations
purely for the sake of being able to call a single method.  Could Render() be
changed to accept a const float** with a size or is the additional vector<>
adding anything of value?

If so, then we could call Render() directly with the webkit vector:

renderer_->Render(&audio_data[0], audio_data.size(), number_of_frames, 0);

+      v[i] = audio_data[i];
+
+    // TODO(crogers): figure out if we should volume scale here or in common
+    // WebAudio code.  In any case we need to take care of volume.
+    renderer_->Render(v, number_of_frames, 0);
+  } else {
+    // Provide silence if the source is not running.
+    for (size_t i = 0; i < audio_data.size(); ++i) {

[tommi (sloooow) - chröme, 2011/12/19 15:26:24]

nit: no {} like above or add {} there

[Chris Rogers, 2011/12/19 21:40:35]

Done.

+      memset(audio_data[i], 0, sizeof(float) * number_of_frames);
+    }
+  }
+}