Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(140)

Issue 2383473002: [scheduler] Teach scheduler about audio state (Closed)

Created:
4 years, 2 months ago by altimin
Modified:
4 years, 1 month ago
CC:
DaleCurtis, blink-reviews, blink-reviews-api_chromium.org, chromium-reviews, creis+watch_chromium.org, darin-cc_chromium.org, dglazkov+blink, feature-media-reviews_chromium.org, jam, miu+watch_chromium.org, miu, mlamouri+watch-content_chromium.org, nasko+codewatch_chromium.org, posciak+watch_chromium.org, scheduler-bugs_chromium.org
Target Ref:
refs/pending/heads/master
Project:
chromium
Visibility:
Public.

Description

[scheduler] Teach scheduler about audio state This patch wires information about audio state to renderer scheduler and stops all throttling (background and offscreen) while audio is playing and for a short period after audio is stopped. BUG=642321, 616519, 656019 CQ_INCLUDE_TRYBOTS=master.tryserver.blink:linux_precise_blink_rel Committed: https://crrev.com/a7a651d546b76499821b4ba47b8f017d4b8becac Committed: https://crrev.com/d8bd26c6124f76be5a0c7cfacc39c3996dd29adf Cr-Original-Commit-Position: refs/heads/master@{#423405} Cr-Commit-Position: refs/heads/master@{#429861}

Patch Set 1 #

Total comments: 36

Patch Set 2 : Addressed comments from skyostil@ and alexclarke@ #

Patch Set 3 : Reverted DEPS #

Total comments: 13

Patch Set 4 : Switched approach to use WebContentsImpl::WasRecentlyAudible #

Total comments: 12

Patch Set 5 : Address comments from skyostil@ #

Patch Set 6 : Support out-of-process frames #

Total comments: 2

Patch Set 7 : WebFrames! #

Total comments: 1

Patch Set 8 : Fixed nit for clamy@ #

Patch Set 9 : Switched to page messages #

Total comments: 1

Patch Set 10 : constexpr! #

Patch Set 11 : Rebased #

Total comments: 6
Unified diffs Side-by-side diffs Delta from patch set Stats (+245 lines, -58 lines) Patch
M cc/test/ordered_simple_task_runner.cc View 1 2 3 4 1 chunk +3 lines, -1 line 0 comments Download
M content/browser/media/audio_stream_monitor.cc View 1 2 3 1 chunk +1 line, -1 line 0 comments Download
M content/browser/web_contents/web_contents_impl.h View 1 2 3 4 5 6 7 8 9 10 1 chunk +1 line, -0 lines 0 comments Download
M content/browser/web_contents/web_contents_impl.cc View 1 2 3 4 5 6 7 8 9 10 2 chunks +9 lines, -2 lines 2 comments Download
M content/common/page_messages.h View 1 2 3 4 5 6 7 8 1 chunk +2 lines, -0 lines 2 comments Download
M content/public/browser/web_contents.h View 1 2 3 4 5 6 7 8 9 10 1 chunk +3 lines, -0 lines 2 comments Download
M content/renderer/render_view_impl.h View 1 2 3 4 5 6 7 8 9 10 1 chunk +1 line, -0 lines 0 comments Download
M content/renderer/render_view_impl.cc View 1 2 3 4 5 6 7 8 9 10 2 chunks +5 lines, -0 lines 0 comments Download
M third_party/WebKit/Source/platform/scheduler/renderer/renderer_scheduler_impl.h View 1 2 3 4 5 6 7 8 9 6 chunks +13 lines, -1 line 0 comments Download
M third_party/WebKit/Source/platform/scheduler/renderer/renderer_scheduler_impl.cc View 1 2 3 4 5 6 7 8 9 10 8 chunks +77 lines, -2 lines 0 comments Download
M third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.h View 1 2 3 4 5 6 7 8 9 10 2 chunks +8 lines, -4 lines 0 comments Download
M third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.cc View 1 2 3 4 5 6 7 8 9 10 10 chunks +61 lines, -45 lines 0 comments Download
M third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler_unittest.cc View 1 2 3 4 5 6 7 8 9 10 1 chunk +39 lines, -0 lines 0 comments Download
M third_party/WebKit/Source/platform/scheduler/renderer/web_view_scheduler_impl.h View 1 2 3 4 5 6 7 8 9 10 2 chunks +3 lines, -2 lines 0 comments Download
M third_party/WebKit/Source/platform/scheduler/renderer/web_view_scheduler_impl.cc View 1 2 3 4 5 6 7 8 9 10 2 chunks +9 lines, -0 lines 0 comments Download
M third_party/WebKit/Source/web/WebViewImpl.h View 1 2 3 4 5 6 7 8 9 10 1 chunk +1 line, -0 lines 0 comments Download
M third_party/WebKit/Source/web/WebViewImpl.cpp View 1 2 3 4 5 6 7 8 9 10 1 chunk +4 lines, -0 lines 0 comments Download
M third_party/WebKit/public/platform/WebViewScheduler.h View 1 2 3 4 5 7 8 1 chunk +2 lines, -0 lines 0 comments Download
M third_party/WebKit/public/web/WebView.h View 1 2 3 4 5 6 7 8 9 10 1 chunk +3 lines, -0 lines 0 comments Download

Messages

Total messages: 84 (47 generated)
altimin
PTAL
4 years, 2 months ago (2016-09-29 10:47:00 UTC) #4
alex clarke (OOO till 29th)
https://codereview.chromium.org/2383473002/diff/1/third_party/WebKit/Source/platform/scheduler/base/delayed_switch.cc File third_party/WebKit/Source/platform/scheduler/base/delayed_switch.cc (right): https://codereview.chromium.org/2383473002/diff/1/third_party/WebKit/Source/platform/scheduler/base/delayed_switch.cc#newcode32 third_party/WebKit/Source/platform/scheduler/base/delayed_switch.cc:32: delayed_disable_task_.Cancel(); DCHECK(task_runner_->RunsTasksOnCurrentThread()); https://codereview.chromium.org/2383473002/diff/1/third_party/WebKit/Source/platform/scheduler/base/delayed_switch.cc#newcode44 third_party/WebKit/Source/platform/scheduler/base/delayed_switch.cc:44: if (state_ == State::DISABLED) DCHECK(task_runner_->RunsTasksOnCurrentThread()); ...
4 years, 2 months ago (2016-09-29 11:16:54 UTC) #5
Sami
https://codereview.chromium.org/2383473002/diff/1/content/browser/renderer_host/media/audio_renderer_host.cc File content/browser/renderer_host/media/audio_renderer_host.cc (right): https://codereview.chromium.org/2383473002/diff/1/content/browser/renderer_host/media/audio_renderer_host.cc#newcode79 content/browser/renderer_host/media/audio_renderer_host.cc:79: bool is_audio_active) { nit: also call this is_playing? (elsewhere ...
4 years, 2 months ago (2016-09-29 13:36:08 UTC) #6
alex clarke (OOO till 29th)
https://codereview.chromium.org/2383473002/diff/1/third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.h File third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.h (right): https://codereview.chromium.org/2383473002/diff/1/third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.h#newcode240 third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.h:240: void UpdateGlobalThrottlingSetting(); On 2016/09/29 13:36:07, Sami wrote: > On ...
4 years, 2 months ago (2016-09-29 13:46:18 UTC) #7
altimin
PTAL https://codereview.chromium.org/2383473002/diff/1/content/browser/renderer_host/media/audio_renderer_host.cc File content/browser/renderer_host/media/audio_renderer_host.cc (right): https://codereview.chromium.org/2383473002/diff/1/content/browser/renderer_host/media/audio_renderer_host.cc#newcode79 content/browser/renderer_host/media/audio_renderer_host.cc:79: bool is_audio_active) { On 2016/09/29 13:36:07, Sami wrote: ...
4 years, 2 months ago (2016-09-29 16:29:44 UTC) #9
commit-bot: I haz the power
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.org/2383473002/40001
4 years, 2 months ago (2016-09-29 17:24:54 UTC) #11
commit-bot: I haz the power
No L-G-T-M from a valid reviewer yet. CQ run can only be started by full ...
4 years, 2 months ago (2016-09-29 17:24:56 UTC) #13
DaleCurtis
Keep in mind this will prevent throttling even for silent audio streams. To avoid this ...
4 years, 2 months ago (2016-09-29 17:26:35 UTC) #14
alex clarke (OOO till 29th)
That's much simpler now, thanks. Scheduler LGTM https://codereview.chromium.org/2383473002/diff/40001/third_party/WebKit/Source/platform/scheduler/renderer/renderer_scheduler_impl.cc File third_party/WebKit/Source/platform/scheduler/renderer/renderer_scheduler_impl.cc (right): https://codereview.chromium.org/2383473002/diff/40001/third_party/WebKit/Source/platform/scheduler/renderer/renderer_scheduler_impl.cc#newcode822 third_party/WebKit/Source/platform/scheduler/renderer/renderer_scheduler_impl.cc:822: if (MainThreadOnly().last_audio_state_change ...
4 years, 2 months ago (2016-09-30 10:33:47 UTC) #15
alex clarke (OOO till 29th)
On 2016/09/29 17:26:35, DaleCurtis_OOO_Until_Oct_18 wrote: > Keep in mind this will prevent throttling even for ...
4 years, 2 months ago (2016-09-30 10:34:23 UTC) #16
Sami
Could we also do WebContentsImpl::WasRecentlyAudible() as part of this patch? https://codereview.chromium.org/2383473002/diff/40001/third_party/WebKit/Source/platform/scheduler/renderer/renderer_scheduler_impl.cc File third_party/WebKit/Source/platform/scheduler/renderer/renderer_scheduler_impl.cc (right): https://codereview.chromium.org/2383473002/diff/40001/third_party/WebKit/Source/platform/scheduler/renderer/renderer_scheduler_impl.cc#newcode996 ...
4 years, 2 months ago (2016-09-30 11:52:09 UTC) #17
DaleCurtis
cc:miu since I'll be OOO in case any review questions come up about WasRecentlyAudible.
4 years, 2 months ago (2016-09-30 17:03:26 UTC) #19
altimin
PTAL https://codereview.chromium.org/2383473002/diff/40001/third_party/WebKit/Source/platform/scheduler/renderer/renderer_scheduler_impl.cc File third_party/WebKit/Source/platform/scheduler/renderer/renderer_scheduler_impl.cc (right): https://codereview.chromium.org/2383473002/diff/40001/third_party/WebKit/Source/platform/scheduler/renderer/renderer_scheduler_impl.cc#newcode822 third_party/WebKit/Source/platform/scheduler/renderer/renderer_scheduler_impl.cc:822: if (MainThreadOnly().last_audio_state_change && On 2016/09/30 10:33:46, alex clarke ...
4 years, 2 months ago (2016-10-03 11:15:26 UTC) #24
altimin
+clamy@ for content/ changes +dcheng@ for IPC and non-scheduler blink changes PTAL
4 years, 2 months ago (2016-10-03 14:28:23 UTC) #28
Sami
lgtm with a couple of nits. https://codereview.chromium.org/2383473002/diff/40001/third_party/WebKit/Source/platform/scheduler/renderer/renderer_scheduler_impl.cc File third_party/WebKit/Source/platform/scheduler/renderer/renderer_scheduler_impl.cc (right): https://codereview.chromium.org/2383473002/diff/40001/third_party/WebKit/Source/platform/scheduler/renderer/renderer_scheduler_impl.cc#newcode996 third_party/WebKit/Source/platform/scheduler/renderer/renderer_scheduler_impl.cc:996: ShouldDisableThrottlingBecauseOfAudio(now) || On ...
4 years, 2 months ago (2016-10-03 14:36:32 UTC) #29
Sami
https://codereview.chromium.org/2383473002/diff/60001/cc/test/ordered_simple_task_runner.cc File cc/test/ordered_simple_task_runner.cc (right): https://codereview.chromium.org/2383473002/diff/60001/cc/test/ordered_simple_task_runner.cc#newcode252 cc/test/ordered_simple_task_runner.cc:252: if (advance_now_ && now_src_->NowTicks() < time) { Not sure ...
4 years, 2 months ago (2016-10-03 14:37:30 UTC) #30
altimin
PTAL https://codereview.chromium.org/2383473002/diff/60001/cc/test/ordered_simple_task_runner.cc File cc/test/ordered_simple_task_runner.cc (right): https://codereview.chromium.org/2383473002/diff/60001/cc/test/ordered_simple_task_runner.cc#newcode252 cc/test/ordered_simple_task_runner.cc:252: if (advance_now_ && now_src_->NowTicks() < time) { On ...
4 years, 2 months ago (2016-10-03 15:53:39 UTC) #33
Sami
lgtm. https://codereview.chromium.org/2383473002/diff/60001/cc/test/ordered_simple_task_runner.cc File cc/test/ordered_simple_task_runner.cc (right): https://codereview.chromium.org/2383473002/diff/60001/cc/test/ordered_simple_task_runner.cc#newcode252 cc/test/ordered_simple_task_runner.cc:252: if (advance_now_ && now_src_->NowTicks() < time) { On ...
4 years, 2 months ago (2016-10-03 16:25:29 UTC) #34
altimin
https://codereview.chromium.org/2383473002/diff/60001/cc/test/ordered_simple_task_runner.cc File cc/test/ordered_simple_task_runner.cc (right): https://codereview.chromium.org/2383473002/diff/60001/cc/test/ordered_simple_task_runner.cc#newcode252 cc/test/ordered_simple_task_runner.cc:252: if (advance_now_ && now_src_->NowTicks() < time) { On 2016/10/03 ...
4 years, 2 months ago (2016-10-03 16:28:22 UTC) #35
dcheng
How should this work with OOPIF? Right now, this only unthrottles one process (the one ...
4 years, 2 months ago (2016-10-03 17:42:12 UTC) #36
altimin
PTAL
4 years, 2 months ago (2016-10-04 16:46:46 UTC) #41
clamy
Thanks! As explained in the comment, I think the render_view_messages may not be the most ...
4 years, 2 months ago (2016-10-05 12:11:36 UTC) #44
altimin
PTAL https://codereview.chromium.org/2383473002/diff/100001/content/browser/web_contents/web_contents_impl.cc File content/browser/web_contents/web_contents_impl.cc (right): https://codereview.chromium.org/2383473002/diff/100001/content/browser/web_contents/web_contents_impl.cc#newcode885 content/browser/web_contents/web_contents_impl.cc:885: int WebContentsImpl::SendToAllViews(IPC::Message* message) { On 2016/10/05 12:11:35, clamy ...
4 years, 2 months ago (2016-10-05 13:26:37 UTC) #46
clamy
Thanks! content/ lgtm with a nit. https://codereview.chromium.org/2383473002/diff/120001/content/public/browser/web_contents.h File content/public/browser/web_contents.h (right): https://codereview.chromium.org/2383473002/diff/120001/content/public/browser/web_contents.h#newcode410 content/public/browser/web_contents.h:410: // Method for ...
4 years, 2 months ago (2016-10-05 13:31:21 UTC) #48
altimin
Started to use PageMsg as discussed with dcheng@. PTAL.
4 years, 2 months ago (2016-10-05 18:15:35 UTC) #49
dcheng
lgtm https://codereview.chromium.org/2383473002/diff/160001/third_party/WebKit/Source/platform/scheduler/renderer/renderer_scheduler_impl.h File third_party/WebKit/Source/platform/scheduler/renderer/renderer_scheduler_impl.h (right): https://codereview.chromium.org/2383473002/diff/160001/third_party/WebKit/Source/platform/scheduler/renderer/renderer_scheduler_impl.h#newcode400 third_party/WebKit/Source/platform/scheduler/renderer/renderer_scheduler_impl.h:400: const base::TimeDelta throttling_delay_after_audio_is_played; Nit: this can be a ...
4 years, 2 months ago (2016-10-05 18:58:03 UTC) #50
commit-bot: I haz the power
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.org/2383473002/180001
4 years, 2 months ago (2016-10-05 23:42:40 UTC) #61
commit-bot: I haz the power
Committed patchset #10 (id:180001)
4 years, 2 months ago (2016-10-06 02:40:41 UTC) #62
commit-bot: I haz the power
Patchset 10 (id:??) landed as https://crrev.com/a7a651d546b76499821b4ba47b8f017d4b8becac Cr-Commit-Position: refs/heads/master@{#423405}
4 years, 2 months ago (2016-10-06 02:43:00 UTC) #64
DaleCurtis
AudioStreamMonitor is not enabled on Android, so I think this is not working as intended ...
4 years, 2 months ago (2016-10-18 22:15:32 UTC) #66
altimin
On 2016/10/18 22:15:32, DaleCurtis wrote: > AudioStreamMonitor is not enabled on Android, so I think ...
4 years, 2 months ago (2016-10-20 12:54:23 UTC) #68
commit-bot: I haz the power
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.org/2383473002/200001
4 years, 1 month ago (2016-11-04 10:31:59 UTC) #76
commit-bot: I haz the power
Committed patchset #11 (id:200001)
4 years, 1 month ago (2016-11-04 11:45:24 UTC) #78
commit-bot: I haz the power
Patchset 11 (id:??) landed as https://crrev.com/d8bd26c6124f76be5a0c7cfacc39c3996dd29adf Cr-Commit-Position: refs/heads/master@{#429861}
4 years, 1 month ago (2016-11-04 11:47:35 UTC) #80
magjed_chromium
A revert of this CL (patchset #11 id:200001) has been created in https://codereview.chromium.org/2483843002/ by magjed@chromium.org. ...
4 years, 1 month ago (2016-11-07 14:58:09 UTC) #81
miu
https://codereview.chromium.org/2383473002/diff/200001/content/browser/web_contents/web_contents_impl.cc File content/browser/web_contents/web_contents_impl.cc (right): https://codereview.chromium.org/2383473002/diff/200001/content/browser/web_contents/web_contents_impl.cc#newcode1322 content/browser/web_contents/web_contents_impl.cc:1322: // Notification for UI updates in response to the ...
4 years, 1 month ago (2016-11-15 22:04:04 UTC) #83
altimin
4 years, 1 month ago (2016-11-16 12:22:33 UTC) #84
Message was sent while issue was closed.
https://codereview.chromium.org/2383473002/diff/200001/content/browser/web_co...
File content/browser/web_contents/web_contents_impl.cc (right):

https://codereview.chromium.org/2383473002/diff/200001/content/browser/web_co...
content/browser/web_contents/web_contents_impl.cc:1322: // Notification for UI
updates in response to the changed muting state.
On 2016/11/15 22:04:04, miu wrote:
> nit: The wording here could be confusing. There are two concepts in the UI: 1)
> Whether an audio stream is audible (i.e., not silent); and 2) Whether an audio
> stream has been muted (i.e., user or extension API controls were used to mute
> audio stream). Can you fix this?

Addressed in crrev.com/2496173003

https://codereview.chromium.org/2383473002/diff/200001/content/common/page_me...
File content/common/page_messages.h (right):

https://codereview.chromium.org/2383473002/diff/200001/content/common/page_me...
content/common/page_messages.h:46:
IPC_MESSAGE_ROUTED1(PageMsg_AudioStateChanged, bool /* is_audio_playing */)
On 2016/11/15 22:04:04, miu wrote:
> OOC, why are we doing this whole "round about" approach: send sound to the
> browser process, then have it do audible detection (takes time), and finally
> send the renderer back a message to determine whether it should disable
> throttling?
> 
> Instead, it seems like anything that causes audio to be generated should
> directly call DisableThrottling() (e.g., from HTMLMediaElement.cpp, the
WebAudio
> modules, etc.). Then, correct behavior would not be dependent on the browser
> process.

Because we do not want to disable throttling when page is not audible. Also this
approach makes easy to explain when we avoid throttling -- the rules are the
same as for displaying sound notification in browser UI. (Also controlling
scheduler settings is not uncommon - for example, foreground/background page
stage is forwarded to the scheduler via browser process).

If you have further questions, let's continue discussion in
crrev.com/2496173003.

https://codereview.chromium.org/2383473002/diff/200001/content/public/browser...
File content/public/browser/web_contents.h (right):

https://codereview.chromium.org/2383473002/diff/200001/content/public/browser...
content/public/browser/web_contents.h:401: virtual void OnAudioStateChanged(bool
is_audio_playing) = 0;
On 2016/11/15 22:04:04, miu wrote:
> Throughout this change "playing" is inaccurate. "playing" means samples are
> being delivered to the OS sound system. The audio samples may represent silent
> or audible sound. So, can you please fix the naming to "audible" everywhere?

Addressed in crrev.com/2496173003

Powered by Google App Engine
This is Rietveld 408576698