Added an "enable-audio-processor" flag and WebRtcAudioProcessor class

content/renderer/media/webrtc_audio_processor.h

+// Copyright 2013 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#ifndef CONTENT_RENDERER_MEDIA_WEBRTC_AUDIO_PROCESSOR_H_
+#define CONTENT_RENDERER_MEDIA_WEBRTC_AUDIO_PROCESSOR_H_
+
+#include "base/synchronization/lock.h"
+#include "base/threading/thread_checker.h"
+#include "content/common/content_export.h"
+#include "media/base/audio_converter.h"
+#include "third_party/libjingle/source/talk/app/webrtc/mediaconstraintsinterface.h"
+#include "third_party/webrtc/modules/audio_processing/include/audio_processing.h"
+#include "third_party/webrtc/modules/interface/module_common_types.h"
+
+namespace media {
+class AudioBus;
+class AudioFifo;
+class AudioParameters;
+}  // namespace media
+
+namespace webrtc {
+class AudioFrame;
+}
+
+namespace content {
+
+// This class owns an object of webrtc::AudioProcessing which contains signal
+// processing components like AGC, AEC and NS. It enables the components based
+// on the constraints, processes the data and outputs it in a unit of 10 ms
+// data chunk.
+class CONTENT_EXPORT WebRtcAudioProcessor {
+ public:
+  explicit WebRtcAudioProcessor(
+      const webrtc::MediaConstraintsInterface* constraints);
+  ~WebRtcAudioProcessor();
+
+  // Pushes capture data in |audio_source| to the internal FIFO.
+  // Called on the capture audio thread.
+  void PushCaptureData(media::AudioBus* audio_source);
+
+  // Processes a block of 10 ms data from the internal FIFO and outputs it via
+  // |out|.
+  // Returns true if the internal FIFO has at least 10ms data for processing,
+  // otherwise false.
+  // Called on the capture audio thread.
+  bool ProcessAndConsumeData(int capture_audio_delay_ms,
+                             int volume,
+                             bool key_pressed,
+                             int16** out);

[Jói, 2013/11/08 13:36:40]

The semantics of |out| need to be documented.

[no longer working on chromium, 2013/11/08 15:39:30]

Done.

+
+  // Called when the format of the capture data has changed.
+  // Called on the main render thread.
+  void SetCaptureFormat(const media::AudioParameters& source_params);
+
+  // Push the render audio to WebRtc::AudioProcessing for analysis. This is
+  // needed iff echo processing is enabled.
+  // Called on the render audio thread.
+  void PushRenderData(const int16* render_audio,

[Jói, 2013/11/08 13:36:40]

The semantics of |render_audio| need to be documented, in particular how many
bytes does it need to have available, who is the owner and what guarantees are
made about lifetime (presumably just that it's alive during the method call).

[no longer working on chromium, 2013/11/08 15:39:30]

adding a comment like:
// |render_audio| is the pointer to the render audio data, its format
// is specified by |sample_rate|, |number_of_channels| and |number_of_frames|.
// Called on the render audio thread.

Let me know if it is insufficient.

[Jói, 2013/11/08 16:44:22]

Looks good.

+                      int sample_rate,
+                      int number_of_channels,
+                      int number_of_frames,
+                      int render_delay_ms);
+
+  // The audio format of the output from the processor.
+  const media::AudioParameters& OutputFormat() const;
+
+  // Accessor to check if the audio processing is enabled or not.
+  bool has_audio_processing() const { return audio_processing_.get() != NULL; }
+
+ private:
+  class WebRtcAudioConverter;
+
+  // Helper to initialize the WebRtc AudioProcessing.
+  void InitializeAudioProcessingModule(
+      const webrtc::MediaConstraintsInterface* constraints);
+
+  // Helper to initialize the render converter.
+  void InitializeRenderConverterIfNeeded(int sample_rate,
+                                         int number_of_channels,
+                                         int frames_per_buffer);
+
+  // Called by ProcessAndConsumeData().
+  void ProcessData(webrtc::AudioFrame* audio_frame,
+                   int audio_delay_milliseconds,
+                   int volume,
+                   bool key_pressed);
+
+  // Called when the processor is going away.
+  void StopAudioProcessing();
+
+  // Cached value for the render delay latency.
+  int render_delay_ms_;
+
+  // Protects |render_delay_ms_|.
+  // TODO(xians): Can we get rid of the lock?

[Jói, 2013/11/08 13:36:40]

It looks like you just need base::subtle::NoBarrier_Load() and
base::subtle::NoBarrier_Store().

[no longer working on chromium, 2013/11/08 15:39:30]

Dale would like to change all the xxx_ms into base::TimeDelta, so
NoBarrier_Load() and NoBarrier_Store() might not work.

[Jói, 2013/11/08 16:44:22]

Your private cache could be an int64 that gets shifted to/from TimeDelta using
TimeDelta::ToInternalValue/FromInternalValue.

Alternatively, if you are going to keep using the delta as a millisecond value
(currently you retrieve it using InMilliseconds()) then maybe your cache should
stay as an integer value of milliseconds while the interface to your class uses
only TimeDelta.

A lock would be OK if contention was minimal, but it seems like there might be a
fair bit of contention?

[DaleCurtis, 2013/11/08 21:00:48]

I think avoiding lock contention (which may be an issue here given the 3
threads) is more important than keeping everything as a base::TimeDelta.  Feel
free to change it back to an int or Atomic32 or whatever you need.

[no longer working on chromium, 2013/11/11 14:35:25]

Thanks for the tips. I am happy to change it to Atomic32 to remove the lock,
please take another look.

[no longer working on chromium, 2013/11/11 14:35:25]

Done with changing the internal member to Atomic32 and remove the lock.

+  mutable base::Lock lock_;
+
+  // WebRtc AudioProcessing module which does AEC, AGC, NS, HighPass filter,
+  // ..etc.
+  scoped_ptr<webrtc::AudioProcessing> audio_processing_;
+
+  // Converter used for the down-mixing and resampling of the capture data.
+  scoped_ptr<WebRtcAudioConverter> capture_converter_;
+
+  // AudioFrame used to hold the output of |capture_converter_|.
+  webrtc::AudioFrame capture_frame_;
+
+  // Converter used for the down-mixing and resampling of the render data when
+  // the AEC is enabled.
+  scoped_ptr<WebRtcAudioConverter> render_converter_;
+
+  // AudioFrame used to hold the output of |render_converter_|.
+  webrtc::AudioFrame render_frame_;
+
+  // Data bus to help converting interleaved data to an AudioBus.
+  scoped_ptr<media::AudioBus> render_data_bus_;
+
+  // Used to DCHECK that some methods are called on the correct thread.

[Jói, 2013/11/08 13:36:40]

Document which thread that is, since you do so for the other ThreadChecker
objects.

[no longer working on chromium, 2013/11/08 15:39:30]

Done.

+  base::ThreadChecker thread_checker_;
+
+  // Used to DCHECK that some methods are called on the capture audio thread.
+  base::ThreadChecker capture_thread_checker_;
+  bool capture_thread_detach_;
+
+  // Used to DCHECK that PushRenderData() is called on the render audio thread.
+  base::ThreadChecker render_thread_checker_;
+  bool render_thread_detach_;
+};
+
+}  // namespace content
+
+#endif  // CONTENT_RENDERER_MEDIA_WEBRTC_AUDIO_PROCESSOR_H_