Feed the WebRTC APM with empty far-end frames for the number of frames skipped by OS.

content/renderer/media/media_stream_audio_processor.cc

Index: content/renderer/media/media_stream_audio_processor.cc
diff --git a/content/renderer/media/media_stream_audio_processor.cc b/content/renderer/media/media_stream_audio_processor.cc
index 5e4c1c0569d0c01044246d82c01ab07a12b74392..e0ddd634820afcf020e07a81be78f30fdfee8b28 100644
--- a/content/renderer/media/media_stream_audio_processor.cc
+++ b/content/renderer/media/media_stream_audio_processor.cc
@@ -432,7 +432,8 @@ void MediaStreamAudioProcessor::OnIpcClosing() {
 
 void MediaStreamAudioProcessor::OnPlayoutData(media::AudioBus* audio_bus,
                                               int sample_rate,
-                                              int audio_delay_milliseconds) {
+                                              int audio_delay_milliseconds,
+                                              uint32_t skipped_frames) {
   DCHECK(render_thread_checker_.CalledOnValidThread());
   DCHECK(audio_processing_->echo_control_mobile()->is_enabled() ^
          audio_processing_->echo_cancellation()->is_enabled());
@@ -447,6 +448,18 @@ void MediaStreamAudioProcessor::OnPlayoutData(media::AudioBus* audio_bus,
 
   render_fifo_->Push(
       *audio_bus, base::TimeDelta::FromMilliseconds(audio_delay_milliseconds));
+
+  // Feed the APM with empty frames corresponding to |skipped_frames|.
+  if (skipped_frames > 0) {
+    MediaStreamAudioBus empty_bus(audio_bus->channels(), skipped_frames);

[peah, 2015/12/16 10:26:57]

Does this call also set all samples in the empty_bus to zero?

[Henrik Grunell, 2015/12/18 10:17:33]

I doesn't. Fixed.

+    audio_processing_->AnalyzeReverseStream(
+        empty_bus.channel_ptrs(),
+        empty_bus.bus()->frames(),

[peah, 2015/12/16 10:26:57]

Is there any restriction on the values of skipped_frames? AnalyzeReverseStream
only allows multiples of 10 ms of samples to be received.

[Henrik Grunell, 2015/12/18 10:17:33]

No, it can be any value. OK, I added storing the number of skipped frames and
feeding empty 10 ms chunks when reaching that.

[peah, 2015/12/18 13:34:12]

What does this mean? Does it mean that typically any number of samples can be
skipped?

In that case, feeding the AEC zero frames will not work, and may actually even
make things worse. 

If we are to compensate for the delay change, that compensation needs to be done
before the AEC starts seeing the delay change in the captured microphone signal
as when that happens, the AEC starts to readjust to the delay change.

Therefore, if the skipped samples are not sent to the AEC as a zero frame until
at least a full frame has been skipped, the extra zero frame passed to the AEC
will likely arrive after the AEC has started to readjusting and will therefore
cause even further readjustment to occur. Compared to that it may be better not
to do anything.

+        sample_rate,
+        ChannelsToLayout(empty_bus.bus()->channels()));
+  }
+
+  // Pull data from the fifo and feed the APM.
   MediaStreamAudioBus* analysis_bus;
   base::TimeDelta audio_delay;
   while (render_fifo_->Consume(&analysis_bus, &audio_delay)) {
@@ -455,7 +468,7 @@ void MediaStreamAudioProcessor::OnPlayoutData(media::AudioBus* audio_bus,
         analysis_bus->channel_ptrs(),
         analysis_bus->bus()->frames(),
         sample_rate,
-        ChannelsToLayout(audio_bus->channels()));
+        ChannelsToLayout(analysis_bus->bus()->channels()));
   }
 }