Feed the render data to MediaStreamAudioProcessor and used AudioBus in render callback

content/renderer/media/webrtc_audio_device_impl.h

Index: content/renderer/media/webrtc_audio_device_impl.h
diff --git a/content/renderer/media/webrtc_audio_device_impl.h b/content/renderer/media/webrtc_audio_device_impl.h
index f9279f5f6ff5c4986b1afd62e5223228426ae690..9985c6cb9131db81a5f0236fe2124a5b56cfd8ff 100644
--- a/content/renderer/media/webrtc_audio_device_impl.h
+++ b/content/renderer/media/webrtc_audio_device_impl.h
@@ -186,15 +186,10 @@ class WebRtcAudioRenderer;
 class WebRtcAudioRendererSource {
  public:
   // Callback to get the rendered interleaved data.

[tommi (sloooow) - chröme, 2014/01/31 13:58:32]

the data isn't interleaved anymore, right?

[no longer working on chromium, 2014/02/02 16:50:16]

Done.

-  // TODO(xians): Change uint8* to int16*.
-  virtual void RenderData(uint8* audio_data,
-                          int number_of_channels,
-                          int number_of_frames,
+  virtual void RenderData(media::AudioBus* audio_bus,
+                          int sample_rate,
                           int audio_delay_milliseconds) = 0;
 
-  // Set the format for the capture audio parameters.
-  virtual void SetRenderFormat(const media::AudioParameters& params) = 0;
-
   // Callback to notify the client that the renderer is going away.
   virtual void RemoveAudioRenderer(WebRtcAudioRenderer* renderer) = 0;
 
@@ -305,6 +300,12 @@ class CONTENT_EXPORT WebRtcAudioDeviceImpl
   void AddAudioCapturer(const scoped_refptr<WebRtcAudioCapturer>& capturer);
   void RemoveAudioCapturer(const scoped_refptr<WebRtcAudioCapturer>& capturer);
 
+  // Adds/Removes the observer of WebRtcAudioRendererSource to the ADM.
+  // These methods are used by the MediaStreamAudioProcesssor to get the
+  // render data for AEC.
+  void AddRenderDataObserver(WebRtcAudioRendererSource* observer);

[tommi (sloooow) - chröme, 2014/01/31 13:58:32]

I'm a bit confused about this... WebRtcAudioDeviceImpl is a
WebRtcAudioRendererSource and this adds a way for a WebRtcAudioRendererSource to
observe what an WebRtcAudioRendererSource is doing?

If this is a generic way to chain together WebRtcAudioRendererSource instances,
should these methods then be a part of the WebRtcAudioRendererSource interface?

[no longer working on chromium, 2014/02/02 16:50:16]

This solution is an intermedia step to get the rendered data.
Today, there is only one WebRtcAudioRendererSource, so the WebRtcAudioDeviceImpl
is the only option for the observers to get the rendered data from.

In the current version, I am reusing the WebRtcAudioRendererSource interface to
allow the observers to get the rendered data, so Add/RemoveRenderDataObserver
can't be interfaces in WebRtcAudioRendererSource.

If we want to put these two methods to interfaces, I need to add new interfaces
like:
WebRtcAudioRenderedDataHost {
// Implement by MediaStreamAudioProcessor.
class WebRtcAudioRenderedData {
  virtual void RenderData(media::AudioBus* audio_bus, int sample_rate, int
delay_ms) = 0;
}

// Implement by WebRtcAudioDeviceImpl.
virtual void AddRenderDataObserver(WebRtcAudioRendererSource* observer) = 0;
virtual void AddRenderDataObserver(WebRtcAudioRendererSource* observer) = 0;

}

I am actually fine with both, please let me know which solution you prefer.

[tommi (sloooow) - chröme, 2014/02/04 14:39:14]

It sounds like what you want is a sink to a source and not an observer.

+  void RemoveRenderDataObserver(WebRtcAudioRendererSource* observer);
+
   // Gets paired device information of the capture device for the audio
   // renderer. This is used to pass on a session id, sample rate and buffer
   // size to a webrtc audio renderer (either local or remote), so that audio
@@ -318,18 +319,11 @@ class CONTENT_EXPORT WebRtcAudioDeviceImpl
   const scoped_refptr<WebRtcAudioRenderer>& renderer() const {
     return renderer_;
   }
-  int output_buffer_size() const {
-    return output_audio_parameters_.frames_per_buffer();
-  }
-  int output_channels() const {
-    return output_audio_parameters_.channels();
-  }
-  int output_sample_rate() const {
-    return output_audio_parameters_.sample_rate();
-  }
 
  private:
   typedef std::list<scoped_refptr<WebRtcAudioCapturer> > CapturerList;
+  typedef std::list<WebRtcAudioRendererSource* > RenderDataObservers;

[tommi (sloooow) - chröme, 2014/01/31 13:58:32]

no space before >

Also document lifetime/ownership management.

[no longer working on chromium, 2014/02/02 16:50:16]

Done with adding comment in line 379, where the RenderDataObservers is being
used.

+  class RenderBuffer;
 
   // Make destructor private to ensure that we can only be deleted by Release().
   virtual ~WebRtcAudioDeviceImpl();
@@ -353,13 +347,11 @@ class CONTENT_EXPORT WebRtcAudioDeviceImpl
   // WebRtcAudioRendererSource implementation.
 
   // Called on the AudioInputDevice worker thread.
-  virtual void RenderData(uint8* audio_data,
-                          int number_of_channels,
-                          int number_of_frames,
+  virtual void RenderData(media::AudioBus* audio_bus,
+                          int sample_rate,
                           int audio_delay_milliseconds) OVERRIDE;
 
   // Called on the main render thread.
-  virtual void SetRenderFormat(const media::AudioParameters& params) OVERRIDE;
   virtual void RemoveAudioRenderer(WebRtcAudioRenderer* renderer) OVERRIDE;
 
   // Helper to get the default capturer, which is the last capturer in
@@ -378,14 +370,14 @@ class CONTENT_EXPORT WebRtcAudioDeviceImpl
   // Provides access to the audio renderer in the browser process.
   scoped_refptr<WebRtcAudioRenderer> renderer_;
 
+  // List of observers which requires access to the render data.
+  RenderDataObservers render_data_observers_;

[tommi (sloooow) - chröme, 2014/01/31 13:58:32]

What's the difference between "render data" and "data" or even "audio" (or
"audio data")?  Is the concept of 'render' needed to explain the context of at
what point the callback is made? (since 'rendering' is an action that's
performed on data, this throws me off a bit)

[no longer working on chromium, 2014/02/02 16:50:16]

Probably it should be rendered data. This WebRtcAudioDeviceImpl is being used by
both capture side and render side, so the documents have been made explicitly to
tell which side the members are used for.

Done with changing it to rendered data.

+
   // Weak reference to the audio callback.
   // The webrtc client defines |audio_transport_callback_| by calling
   // RegisterAudioCallback().
   webrtc::AudioTransport* audio_transport_callback_;
 
-  // Cached values of used output audio parameters. Platform dependent.
-  media::AudioParameters output_audio_parameters_;
-
   // Cached value of the current audio delay on the input/capture side.
   int input_delay_ms_;
 
@@ -412,6 +404,10 @@ class CONTENT_EXPORT WebRtcAudioDeviceImpl
   // Range is [0, 255].
   uint32_t microphone_volume_;
 
+  // Buffer used for temporary storage during render callback.
+  // It is only accessed by the audio render thread.
+  std::vector<int16> render_buffer_;
+
   DISALLOW_COPY_AND_ASSIGN(WebRtcAudioDeviceImpl);
 };