Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(301)

Issue 1977553002: RELAND: MediaStream audio: Refactor 3 separate "glue" impls into one. (Closed)

Created:
4 years, 7 months ago by miu
Modified:
4 years, 7 months ago
CC:
chromium-reviews, mlamouri+watch-content_chromium.org, imcheng+watch_chromium.org, posciak+watch_chromium.org, avayvod+watch_chromium.org, jam, feature-media-reviews_chromium.org, darin-cc_chromium.org, mcasas+watch+vc_chromium.org, jasonroberts+watch_google.com, mkwst+moarreviews-renderer_chromium.org, xjz+watch_chromium.org, isheriff+watch_chromium.org, miu+watch_chromium.org
Base URL:
https://chromium.googlesource.com/chromium/src.git@master
Target Ref:
refs/pending/heads/master
Project:
chromium
Visibility:
Public.

Description

RELAND: MediaStream audio: Refactor 3 separate "glue" impls into one. This is a re-land of https://codereview.chromium.org/1834323002 with locking changes made in ProcessedLocalAudioSource to mitigate a deadlock condition at shutdown, identified by the TSAN bots. The root-cause of the issue was due to both AudioInputDevice::Thread and PLAS acquiring two locks in a different order under certain circumstances. The solution to this was to eliminate locking in PLAS::Capture() and take a different approach to thread-safe concurrency in PLAS::SetVolume(). -------Description of original change follows------- This is a refactoring of the MediaStreamAudioSource/Track object graph such that life-cycle and audio flow control are unified into a single architecture. Previously, each of three implementations solved this same problem in three different ways; and this has made it difficult to maintain the code in src/content/renderer/media across all product features, WebRTC or otherwise. Diagram of post-refactoring class relationships: https://docs.google.com/drawings/d/1yTsXvRMIyMlXjIEeQOVqXPxMPWgekB5ePWUVSI5-zwo/edit?usp=sharing The new architeture is as follows: 1. MediaStreamAudioSource becomes a base class implementation for creating MediaStreamAudioTracks and delivering audio to them from the source (i.e., an AudioInputDevice, a PeerConnection remote source, or a WebAudio source). All of its methods are now assumed to be run on the main thread, while audio flow may optionally occur on a separate thread. 2. MediaStreamAudioTrack becomes a base class implementation for connecting/disconnecting MediaStreamAudioSinks, and delivering audio to them from the source. 3. Both MediaStreamAudioSource and MediaStreamAudioTrack are owned by their blink counterparts (WebMediaStreamSource and WebMediaStreamTrack), and their destruction may safely occur any time the blink implementation requires it. Following the new architeture, the refactoring is: 1. WebRtcAudioCapturer becomes a sub-class of MediaStreamAudioSource and is renamed to ProcessedLocalAudioSource. This new class owns and manages the WebRTC-specific audio processing on the source data. Also: a) Significant implementation from PeerConnectionDependencyFactory was consolidated/moved to this class. b) The "EnablePeerConnectionMode" functionality, which re-started the audio input with a different buffer size, has been removed to simplify the implementation. Now, buffer size is determined BEFORE the first time the audio input device is started. c) Currently, all local audio sources (i.e., via AudioInputDevice) are routed through this pipeline, but a soon-upcoming change will split the WebRTC-specific cases from the ones that should not go through audio processing. 2. MediaStreamRemoteAudioTrack becomes a sub-class of MediaStreamAudioSource and is renamed to PeerConnectionRemoteAudioSource. This new class owns and manages the flow of audio data from a PeerConnection into the MediaStream framework. 3. WebAudioCapturer becomes a sub-class of MediaStreamAudioSource and is renamed to WebAudioMediaStreamSource. As a blink::WebAudioDestinationConsumer, it manages the flow of audio from a WebAudio destination node into the MediaStream framework. BUG=577881, 577874 TBR=olka@chromium.org,perkj@chromium.org,tommi@chromium.org,nick@chromium.org Committed: https://crrev.com/7f68ab4de698d37bb159682995d7bfbf6d701b67 Cr-Commit-Position: refs/heads/master@{#393343}

Patch Set 1 : Final Patch Set from original change #

Patch Set 2 : Alternative approach to PLAS::SetVolume() thread-safety. #

Unified diffs Side-by-side diffs Delta from patch set Stats (+2846 lines, -4883 lines) Patch
M content/content_renderer.gypi View 7 chunks +15 lines, -19 lines 0 comments Download
M content/content_tests.gypi View 2 chunks +4 lines, -3 lines 0 comments Download
M content/public/renderer/media_stream_utils.cc View 2 chunks +9 lines, -25 lines 0 comments Download
M content/renderer/media/audio_track_recorder_unittest.cc View 2 chunks +8 lines, -10 lines 0 comments Download
A + content/renderer/media/external_media_stream_audio_source.h View 1 chunk +42 lines, -41 lines 0 comments Download
A content/renderer/media/external_media_stream_audio_source.cc View 1 chunk +81 lines, -0 lines 0 comments Download
A content/renderer/media/media_stream_audio_deliverer.h View 1 chunk +156 lines, -0 lines 0 comments Download
M content/renderer/media/media_stream_audio_processor.cc View 1 chunk +3 lines, -0 lines 0 comments Download
M content/renderer/media/media_stream_audio_processor_options.cc View 1 chunk +3 lines, -3 lines 0 comments Download
M content/renderer/media/media_stream_audio_processor_unittest.cc View 1 chunk +1 line, -0 lines 0 comments Download
D content/renderer/media/media_stream_audio_sink_owner.h View 1 chunk +0 lines, -47 lines 0 comments Download
D content/renderer/media/media_stream_audio_sink_owner.cc View 1 chunk +0 lines, -49 lines 0 comments Download
M content/renderer/media/media_stream_audio_source.h View 1 chunk +106 lines, -55 lines 0 comments Download
M content/renderer/media/media_stream_audio_source.cc View 2 chunks +104 lines, -51 lines 0 comments Download
M content/renderer/media/media_stream_audio_track.h View 2 chunks +69 lines, -30 lines 0 comments Download
M content/renderer/media/media_stream_audio_track.cc View 2 chunks +90 lines, -16 lines 0 comments Download
D content/renderer/media/media_stream_audio_track_sink.h View 1 chunk +0 lines, -61 lines 0 comments Download
A content/renderer/media/media_stream_audio_unittest.cc View 1 chunk +450 lines, -0 lines 0 comments Download
M content/renderer/media/media_stream_center.h View 2 chunks +2 lines, -4 lines 0 comments Download
M content/renderer/media/media_stream_center.cc View 5 chunks +35 lines, -27 lines 0 comments Download
M content/renderer/media/media_stream_renderer_factory_impl.cc View 2 chunks +2 lines, -5 lines 0 comments Download
M content/renderer/media/media_stream_source.h View 2 chunks +24 lines, -20 lines 0 comments Download
M content/renderer/media/media_stream_source.cc View 1 chunk +30 lines, -3 lines 0 comments Download
A content/renderer/media/mock_audio_device_factory.h View 1 chunk +82 lines, -0 lines 0 comments Download
A + content/renderer/media/mock_audio_device_factory.cc View 1 chunk +16 lines, -22 lines 0 comments Download
M content/renderer/media/mock_media_stream_dispatcher.cc View 2 chunks +4 lines, -0 lines 0 comments Download
M content/renderer/media/mock_media_stream_registry.cc View 3 chunks +33 lines, -13 lines 0 comments Download
M content/renderer/media/remote_media_stream_impl.cc View 2 chunks +5 lines, -6 lines 0 comments Download
M content/renderer/media/rtc_peer_connection_handler.cc View 3 chunks +12 lines, -9 lines 0 comments Download
M content/renderer/media/rtc_peer_connection_handler_unittest.cc View 9 chunks +54 lines, -14 lines 0 comments Download
M content/renderer/media/speech_recognition_audio_sink_unittest.cc View 7 chunks +49 lines, -33 lines 0 comments Download
D content/renderer/media/tagged_list.h View 1 chunk +0 lines, -95 lines 0 comments Download
M content/renderer/media/user_media_client_impl.h View 3 chunks +8 lines, -4 lines 0 comments Download
M content/renderer/media/user_media_client_impl.cc View 5 chunks +25 lines, -17 lines 0 comments Download
M content/renderer/media/user_media_client_impl_unittest.cc View 5 chunks +29 lines, -6 lines 0 comments Download
D content/renderer/media/webaudio_capturer_source.h View 1 chunk +0 lines, -101 lines 0 comments Download
D content/renderer/media/webaudio_capturer_source.cc View 1 chunk +0 lines, -136 lines 0 comments Download
A + content/renderer/media/webaudio_media_stream_source.h View 3 chunks +32 lines, -51 lines 0 comments Download
A + content/renderer/media/webaudio_media_stream_source.cc View 3 chunks +52 lines, -77 lines 0 comments Download
D content/renderer/media/webrtc/media_stream_remote_audio_track.h View 1 chunk +0 lines, -89 lines 0 comments Download
D content/renderer/media/webrtc/media_stream_remote_audio_track.cc View 1 chunk +0 lines, -237 lines 0 comments Download
M content/renderer/media/webrtc/mock_peer_connection_dependency_factory.h View 2 chunks +5 lines, -12 lines 0 comments Download
M content/renderer/media/webrtc/mock_peer_connection_dependency_factory.cc View 4 chunks +6 lines, -21 lines 0 comments Download
M content/renderer/media/webrtc/peer_connection_dependency_factory.h View 5 chunks +7 lines, -40 lines 0 comments Download
M content/renderer/media/webrtc/peer_connection_dependency_factory.cc View 5 chunks +0 lines, -189 lines 0 comments Download
A + content/renderer/media/webrtc/peer_connection_remote_audio_source.h View 1 chunk +70 lines, -57 lines 0 comments Download
A content/renderer/media/webrtc/peer_connection_remote_audio_source.cc View 1 chunk +153 lines, -0 lines 0 comments Download
A + content/renderer/media/webrtc/processed_local_audio_source.h View 1 3 chunks +87 lines, -146 lines 0 comments Download
A + content/renderer/media/webrtc/processed_local_audio_source.cc View 1 6 chunks +238 lines, -398 lines 0 comments Download
A + content/renderer/media/webrtc/processed_local_audio_source_unittest.cc View 3 chunks +163 lines, -88 lines 0 comments Download
A + content/renderer/media/webrtc/webrtc_audio_sink.h View 1 chunk +149 lines, -73 lines 0 comments Download
A + content/renderer/media/webrtc/webrtc_audio_sink.cc View 2 chunks +126 lines, -93 lines 0 comments Download
D content/renderer/media/webrtc/webrtc_audio_sink_adapter.h View 1 chunk +0 lines, -51 lines 0 comments Download
D content/renderer/media/webrtc/webrtc_audio_sink_adapter.cc View 1 chunk +0 lines, -51 lines 0 comments Download
D content/renderer/media/webrtc/webrtc_local_audio_track_adapter.h View 1 chunk +0 lines, -107 lines 0 comments Download
D content/renderer/media/webrtc/webrtc_local_audio_track_adapter.cc View 1 chunk +0 lines, -161 lines 0 comments Download
D content/renderer/media/webrtc/webrtc_local_audio_track_adapter_unittest.cc View 1 chunk +0 lines, -102 lines 0 comments Download
M content/renderer/media/webrtc/webrtc_media_stream_adapter.h View 5 chunks +10 lines, -6 lines 0 comments Download
M content/renderer/media/webrtc/webrtc_media_stream_adapter.cc View 3 chunks +70 lines, -31 lines 0 comments Download
M content/renderer/media/webrtc/webrtc_media_stream_adapter_unittest.cc View 4 chunks +28 lines, -12 lines 0 comments Download
D content/renderer/media/webrtc_audio_capturer.h View 1 chunk +0 lines, -207 lines 0 comments Download
D content/renderer/media/webrtc_audio_capturer.cc View 1 chunk +0 lines, -566 lines 0 comments Download
D content/renderer/media/webrtc_audio_capturer_unittest.cc View 1 chunk +0 lines, -152 lines 0 comments Download
M content/renderer/media/webrtc_audio_device_impl.h View 4 chunks +8 lines, -7 lines 0 comments Download
M content/renderer/media/webrtc_audio_device_impl.cc View 4 chunks +7 lines, -7 lines 0 comments Download
M content/renderer/media/webrtc_audio_renderer.h View 1 chunk +1 line, -0 lines 0 comments Download
M content/renderer/media/webrtc_audio_renderer.cc View 3 chunks +9 lines, -7 lines 0 comments Download
M content/renderer/media/webrtc_audio_renderer_unittest.cc View 2 chunks +4 lines, -1 line 0 comments Download
M content/renderer/media/webrtc_local_audio_source_provider.h View 1 chunk +7 lines, -3 lines 0 comments Download
M content/renderer/media/webrtc_local_audio_source_provider_unittest.cc View 5 chunks +8 lines, -13 lines 0 comments Download
D content/renderer/media/webrtc_local_audio_track.h View 1 chunk +0 lines, -106 lines 0 comments Download
D content/renderer/media/webrtc_local_audio_track.cc View 1 chunk +0 lines, -169 lines 0 comments Download
D content/renderer/media/webrtc_local_audio_track_unittest.cc View 1 chunk +0 lines, -579 lines 0 comments Download
M content/renderer/pepper/pepper_media_stream_video_track_host.h View 5 chunks +9 lines, -18 lines 0 comments Download
M content/renderer/pepper/pepper_media_stream_video_track_host.cc View 6 chunks +46 lines, -31 lines 0 comments Download

Messages

Total messages: 14 (8 generated)
miu
mcasas: Would you please sanity-check the difference between PS2 and PS1 before I try to ...
4 years, 7 months ago (2016-05-12 19:25:16 UTC) #4
mcasas
On 2016/05/12 19:25:16, miu wrote: > mcasas: Would you please sanity-check the difference between PS2 ...
4 years, 7 months ago (2016-05-12 19:36:08 UTC) #7
miu
On 2016/05/12 19:36:08, mcasas wrote: > On 2016/05/12 19:25:16, miu wrote: > > mcasas: Would ...
4 years, 7 months ago (2016-05-12 19:40:35 UTC) #8
commit-bot: I haz the power
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/1977553002/40001 View timeline at https://chromium-cq-status.appspot.com/patch-timeline/1977553002/40001
4 years, 7 months ago (2016-05-12 19:41:13 UTC) #10
commit-bot: I haz the power
Committed patchset #2 (id:40001)
4 years, 7 months ago (2016-05-12 20:29:16 UTC) #12
commit-bot: I haz the power
4 years, 7 months ago (2016-05-12 20:30:38 UTC) #14
Message was sent while issue was closed.
Patchset 2 (id:??) landed as
https://crrev.com/7f68ab4de698d37bb159682995d7bfbf6d701b67
Cr-Commit-Position: refs/heads/master@{#393343}

Powered by Google App Engine
This is Rietveld 408576698