Integrate HTMLMediaElement with Web Audio API's MediaElementAudioSourceNode

content/renderer/render_audiosourceprovider.cc

Index: content/renderer/render_audiosourceprovider.cc
===================================================================
--- content/renderer/render_audiosourceprovider.cc	(revision 0)
+++ content/renderer/render_audiosourceprovider.cc	(revision 0)
@@ -0,0 +1,90 @@
+// Copyright (c) 2011 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#include "content/renderer/render_audiosourceprovider.h"
+
+#include "base/basictypes.h"
+#include "base/logging.h"
+#include "third_party/WebKit/Source/WebKit/chromium/public/WebAudioSourceProviderClient.h"
+
+using std::vector;
+using WebKit::WebVector;
+
+RenderAudioSourceProvider::RenderAudioSourceProvider(
+    media::AudioRendererSink::RenderCallback* renderer)

[acolwell GONE FROM CHROMIUM, 2011/12/19 23:01:49]

It feels to me like this should be an AudioRendererSink and all the switching
logic should be in here. 

This object could be passed to the AudioRendererImpl and the AudioDevice could
be the AudioRendererSink passed to this constructor. Calls could be forwarded to
the sink(AudioDevice) until setClient() is called.

[Chris Rogers, 2011/12/19 23:28:14]

I agree that I should switch this to take an AudioRendererImpl as the argument,
but I think the logic still needs to remain in AudioRendererImpl (see my comment
there about dynamic changes being made to sink during its lifetime).  Also,
there's thread-safety/locking which needs to be in-place directly in 
AudioRendererImpl, so I don't believe this logic can be moved into this class.

[acolwell GONE FROM CHROMIUM, 2011/12/20 01:04:06]

I'm suggesting AudioRendererSink not AudioRendererImpl. Why not handle switching
here since this appears to be the code that seems to detect/control it?
Isn't the thread-safety issues only because sink_ can change? I'm saying move
that logic in here and don't let sink_ change in AudioRendererImpl. 

[Chris Rogers, 2011/12/21 01:38:17]

Done.

+    : is_running_(false),
+      volume_(1.0),
+      renderer_(renderer),
+      client_(NULL) {
+}
+
+void RenderAudioSourceProvider::Start() {
+  is_running_ = true;
+}
+
+void RenderAudioSourceProvider::Stop() {
+  is_running_ = false;
+}
+
+void RenderAudioSourceProvider::Play() {
+  is_running_ = true;
+}
+
+void RenderAudioSourceProvider::Pause(bool flush) {
+  is_running_ = false;
+}
+
+bool RenderAudioSourceProvider::SetVolume(double volume) {
+  volume_ = volume;
+  return true;
+}
+
+void RenderAudioSourceProvider::Initialize(
+    size_t buffer_size,
+    int channels,
+    double sample_rate,
+    AudioParameters::Format latency_format,
+    RenderCallback* renderer) {
+  // We already know what the renderer is from our constructor.
+  DCHECK_EQ(renderer, renderer_);
+
+  // Inform WebKit about the audio stream format.
+  if (client_)
+    client_->setFormat(channels, sample_rate);
+}
+
+void RenderAudioSourceProvider::setClient(
+    WebKit::WebAudioSourceProviderClient* client) {

[no longer working on chromium, 2011/12/20 15:47:19]

Do we allow this client to be NULL?
If it is a valid user case, why we have DCHECK(client_) in provideInput()?

[Chris Rogers, 2011/12/21 01:38:17]

passing in NULL for client effectively means that there's no client, so we
expect that there will be no more calls to provideInput().

In other words, it would be a bug if provideInput() were called after
setClient(NULL)

On 2011/12/20 15:47:19, xians1 wrote:

+  client_ = client;
+
+  if (client) {
+    // Detach the audio renderer from normal playback and make us be the sink.
+    // The renderer will call Initialize() when the stream format is known.
+    renderer_->SetAudioRendererSink(this);
+  } else {
+    // Restore normal playback.
+    renderer_->SetAudioRendererSink(NULL);
+  }
+}
+
+void RenderAudioSourceProvider::provideInput(
+    const WebVector<float*>& audio_data, size_t number_of_frames) {
+  DCHECK(renderer_);
+  DCHECK(client_);
+
+  if (renderer_ && is_running_) {
+    // Wrap WebVector as std::vector.
+    vector<float*> v(audio_data.size());
+    for (size_t i = 0; i < audio_data.size(); ++i)
+      v[i] = audio_data[i];
+
+    // TODO(crogers): figure out if we should volume scale here or in common
+    // WebAudio code.  In any case we need to take care of volume.
+    renderer_->Render(v, number_of_frames, 0);

[no longer working on chromium, 2011/12/20 15:47:19]

Please correct me if I am wrong here.
Is there a casting here? If there is, what about using
reinterpret_cast<int16*>(v)

[Chris Rogers, 2011/12/21 01:38:17]

No, there's no cast here.  The type is std::vector<float*>

On 2011/12/20 15:47:19, xians1 wrote:

+  } else {
+    // Provide silence if the source is not running.
+    for (size_t i = 0; i < audio_data.size(); ++i)
+      memset(audio_data[i], 0, sizeof(float) * number_of_frames);
+  }
+}