Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(258)

Issue 2258133002: [scheduler] Implement time-based cpu throttling. (Closed)

Created:
4 years, 4 months ago by altimin
Modified:
4 years, 2 months ago
CC:
blink-reviews, chromium-reviews, scheduler-bugs_chromium.org
Base URL:
https://chromium.googlesource.com/chromium/src.git@master
Target Ref:
refs/pending/heads/master
Project:
chromium
Visibility:
Public.

Description

[scheduler] Implement time-based cpu throttling. This patch adds TimeBudgetPool API to renderer scheduler, which allows to group task queues into pools and set shared limits on how long this pool of queues is allowed to run (as per cent of wall time). Queues violating this limit will be throttled. This patch also renames throttling_helper.cc to task_queue_throttler.cc. BUG=639852 Committed: https://crrev.com/906d4a1e806b27681bad0ec63f64c0c9fc239a77 Cr-Commit-Position: refs/heads/master@{#420842}

Patch Set 1 #

Patch Set 2 : Tests #

Patch Set 3 : Removed "Locked" from MaybeSchedulePumpThrottledTasksLocked #

Patch Set 4 : Changed similarity #

Total comments: 35

Patch Set 5 : Addressed comments #

Patch Set 6 : Use double instead of base::TimeTicks #

Total comments: 74

Patch Set 7 : Addressed comments #

Patch Set 8 : Formatted #

Total comments: 10

Patch Set 9 : Addressed comments #

Patch Set 10 : Renamed Enable/Disable to Enable/DisableThrottling. #

Total comments: 56

Patch Set 11 : Rebased & addressed comments #

Total comments: 4

Patch Set 12 : More fixes #

Total comments: 6

Patch Set 13 : Addressed comments #

Total comments: 4

Patch Set 14 : One more fix #

Total comments: 33

Patch Set 15 : Fixed compilation #

Patch Set 16 : git cl try #

Patch Set 17 : Fix trybot failures (hopefully) #

Patch Set 18 : Rebased #

Unified diffs Side-by-side diffs Delta from patch set Stats (+1052 lines, -874 lines) Patch
M third_party/WebKit/Source/core/inspector/InspectorWebPerfAgent.h View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 1 chunk +4 lines, -1 line 0 comments Download
M third_party/WebKit/Source/core/inspector/InspectorWebPerfAgent.cpp View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 1 chunk +4 lines, -1 line 0 comments Download
M third_party/WebKit/Source/core/inspector/InspectorWebPerfAgentTest.cpp View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 4 chunks +5 lines, -5 lines 0 comments Download
M third_party/WebKit/Source/platform/BUILD.gn View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 2 chunks +3 lines, -3 lines 0 comments Download
M third_party/WebKit/Source/platform/scheduler/DEPS View 1 2 3 4 5 2 chunks +4 lines, -2 lines 0 comments Download
M third_party/WebKit/Source/platform/scheduler/base/task_queue_impl.h View 1 2 3 4 5 6 7 8 9 10 15 16 17 1 chunk +1 line, -0 lines 0 comments Download
M third_party/WebKit/Source/platform/scheduler/base/task_queue_impl.cc View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 2 chunks +25 lines, -14 lines 0 comments Download
M third_party/WebKit/Source/platform/scheduler/base/task_queue_manager.cc View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 2 chunks +8 lines, -3 lines 0 comments Download
M third_party/WebKit/Source/platform/scheduler/base/test_task_time_observer.h View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 1 chunk +3 lines, -1 line 0 comments Download
M third_party/WebKit/Source/platform/scheduler/base/time_domain.h View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 2 chunks +5 lines, -2 lines 0 comments Download
M third_party/WebKit/Source/platform/scheduler/base/time_domain.cc View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 2 chunks +2 lines, -2 lines 0 comments Download
M third_party/WebKit/Source/platform/scheduler/base/time_domain_unittest.cc View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 2 chunks +4 lines, -4 lines 0 comments Download
M third_party/WebKit/Source/platform/scheduler/child/compositor_worker_scheduler.cc View 1 2 3 4 5 6 7 8 9 10 1 chunk +5 lines, -0 lines 0 comments Download
M third_party/WebKit/Source/platform/scheduler/child/scheduler_helper.h View 1 1 chunk +1 line, -1 line 0 comments Download
M third_party/WebKit/Source/platform/scheduler/renderer/renderer_scheduler_impl.h View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 7 chunks +12 lines, -5 lines 0 comments Download
M third_party/WebKit/Source/platform/scheduler/renderer/renderer_scheduler_impl.cc View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 14 chunks +40 lines, -18 lines 0 comments Download
M third_party/WebKit/Source/platform/scheduler/renderer/renderer_scheduler_impl_unittest.cc View 1 5 chunks +5 lines, -5 lines 0 comments Download
A + third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.h View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 4 chunks +157 lines, -29 lines 0 comments Download
A + third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.cc View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 4 chunks +429 lines, -77 lines 0 comments Download
A + third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler_unittest.cc View 1 2 3 4 5 6 7 8 9 10 11 18 chunks +307 lines, -74 lines 0 comments Download
M third_party/WebKit/Source/platform/scheduler/renderer/throttled_time_domain.h View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 1 chunk +1 line, -1 line 0 comments Download
M third_party/WebKit/Source/platform/scheduler/renderer/throttled_time_domain.cc View 1 2 chunks +3 lines, -2 lines 0 comments Download
D third_party/WebKit/Source/platform/scheduler/renderer/throttling_helper.h View 1 1 chunk +0 lines, -124 lines 0 comments Download
D third_party/WebKit/Source/platform/scheduler/renderer/throttling_helper_unittest.cc View 1 2 3 4 5 6 7 8 9 10 1 chunk +0 lines, -494 lines 0 comments Download
M third_party/WebKit/Source/platform/scheduler/renderer/web_frame_scheduler_impl.cc View 1 2 3 4 5 6 2 chunks +3 lines, -3 lines 0 comments Download
M third_party/WebKit/Source/platform/scheduler/renderer/web_view_scheduler_impl.h View 1 2 3 4 5 6 1 chunk +1 line, -0 lines 0 comments Download
M third_party/WebKit/public/platform/scheduler/DEPS View 1 2 3 4 5 1 chunk +1 line, -0 lines 0 comments Download
M third_party/WebKit/public/platform/scheduler/base/task_queue.h View 1 2 3 4 5 6 7 8 9 10 2 chunks +9 lines, -2 lines 0 comments Download
M third_party/WebKit/public/platform/scheduler/base/task_time_observer.h View 1 2 3 4 5 6 1 chunk +10 lines, -1 line 0 comments Download

Messages

Total messages: 73 (32 generated)
altimin
4 years, 3 months ago (2016-09-07 13:12:30 UTC) #3
alex clarke (OOO till 29th)
https://codereview.chromium.org/2258133002/diff/60001/third_party/WebKit/Source/platform/scheduler/base/task_queue_impl.cc File third_party/WebKit/Source/platform/scheduler/base/task_queue_impl.cc (right): https://codereview.chromium.org/2258133002/diff/60001/third_party/WebKit/Source/platform/scheduler/base/task_queue_impl.cc#newcode543 third_party/WebKit/Source/platform/scheduler/base/task_queue_impl.cc:543: if (wakeup) { I don't think we should allow ...
4 years, 3 months ago (2016-09-07 15:13:08 UTC) #4
Sami
Overall looks great! My main suggestion is to improve the debuggability a bit. I think ...
4 years, 3 months ago (2016-09-07 15:20:44 UTC) #5
altimin
PTAL. (WebPerfAgentInspector thing is still TBD). https://codereview.chromium.org/2258133002/diff/60001/third_party/WebKit/Source/platform/scheduler/base/task_queue_impl.cc File third_party/WebKit/Source/platform/scheduler/base/task_queue_impl.cc (right): https://codereview.chromium.org/2258133002/diff/60001/third_party/WebKit/Source/platform/scheduler/base/task_queue_impl.cc#newcode543 third_party/WebKit/Source/platform/scheduler/base/task_queue_impl.cc:543: if (wakeup) { ...
4 years, 3 months ago (2016-09-09 15:43:58 UTC) #6
alex clarke (OOO till 29th)
Overall I like what you're trying to do but actually reviewing this code is proving ...
4 years, 3 months ago (2016-09-12 17:45:27 UTC) #7
Sami
Thanks for adding the tracing support! https://codereview.chromium.org/2258133002/diff/100001/third_party/WebKit/Source/core/inspector/InspectorWebPerfAgent.h File third_party/WebKit/Source/core/inspector/InspectorWebPerfAgent.h (right): https://codereview.chromium.org/2258133002/diff/100001/third_party/WebKit/Source/core/inspector/InspectorWebPerfAgent.h#newcode39 third_party/WebKit/Source/core/inspector/InspectorWebPerfAgent.h:39: scheduler::TaskQueue*, Could you ...
4 years, 3 months ago (2016-09-12 17:49:10 UTC) #8
altimin
PTAL. Intended usage of this API can be found in crrev.com/2345483002 https://codereview.chromium.org/2258133002/diff/100001/third_party/WebKit/Source/core/inspector/InspectorWebPerfAgent.h File third_party/WebKit/Source/core/inspector/InspectorWebPerfAgent.h (right): ...
4 years, 3 months ago (2016-09-14 11:23:18 UTC) #9
altimin
PTAL. Intended usage of this API can be found in crrev.com/2345483002 https://codereview.chromium.org/2258133002/diff/100001/third_party/WebKit/Source/core/inspector/InspectorWebPerfAgent.h File third_party/WebKit/Source/core/inspector/InspectorWebPerfAgent.h (right): ...
4 years, 3 months ago (2016-09-14 11:23:19 UTC) #10
alex clarke (OOO till 29th)
https://codereview.chromium.org/2258133002/diff/140001/third_party/WebKit/Source/platform/scheduler/base/task_queue_impl.cc File third_party/WebKit/Source/platform/scheduler/base/task_queue_impl.cc (right): https://codereview.chromium.org/2258133002/diff/140001/third_party/WebKit/Source/platform/scheduler/base/task_queue_impl.cc#newcode543 third_party/WebKit/Source/platform/scheduler/base/task_queue_impl.cc:543: MoveReadyDelayedTasksToDelayedWorkQueue(&lazy_now); Per offline discussion lets drop the call to ...
4 years, 3 months ago (2016-09-14 12:36:35 UTC) #11
Sami
https://codereview.chromium.org/2258133002/diff/100001/third_party/WebKit/Source/core/inspector/InspectorWebPerfAgent.h File third_party/WebKit/Source/core/inspector/InspectorWebPerfAgent.h (right): https://codereview.chromium.org/2258133002/diff/100001/third_party/WebKit/Source/core/inspector/InspectorWebPerfAgent.h#newcode39 third_party/WebKit/Source/core/inspector/InspectorWebPerfAgent.h:39: scheduler::TaskQueue*, On 2016/09/14 11:23:14, altimin wrote: > On 2016/09/12 ...
4 years, 3 months ago (2016-09-14 13:50:33 UTC) #12
altimin
PTAL https://codereview.chromium.org/2258133002/diff/140001/third_party/WebKit/Source/platform/scheduler/base/task_queue_impl.cc File third_party/WebKit/Source/platform/scheduler/base/task_queue_impl.cc (right): https://codereview.chromium.org/2258133002/diff/140001/third_party/WebKit/Source/platform/scheduler/base/task_queue_impl.cc#newcode543 third_party/WebKit/Source/platform/scheduler/base/task_queue_impl.cc:543: MoveReadyDelayedTasksToDelayedWorkQueue(&lazy_now); On 2016/09/14 12:36:34, alex clarke wrote: > ...
4 years, 3 months ago (2016-09-14 18:34:50 UTC) #13
alex clarke (OOO till 29th)
https://codereview.chromium.org/2258133002/diff/100001/third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.h File third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.h (right): https://codereview.chromium.org/2258133002/diff/100001/third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.h#newcode206 third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.h:206: std::unordered_map<TaskQueue*, TimeBudgetPool*> time_budget_pool_for_queue_; On 2016/09/14 11:23:17, altimin wrote: > ...
4 years, 3 months ago (2016-09-15 12:19:36 UTC) #14
altimin
PTAL https://codereview.chromium.org/2258133002/diff/180001/third_party/WebKit/Source/platform/scheduler/base/task_queue_impl.cc File third_party/WebKit/Source/platform/scheduler/base/task_queue_impl.cc (right): https://codereview.chromium.org/2258133002/diff/180001/third_party/WebKit/Source/platform/scheduler/base/task_queue_impl.cc#newcode517 third_party/WebKit/Source/platform/scheduler/base/task_queue_impl.cc:517: LazyNow lazy_now = main_thread_only().time_domain->CreateLazyNow(); On 2016/09/15 12:19:34, alex ...
4 years, 3 months ago (2016-09-15 15:52:11 UTC) #15
alex clarke (OOO till 29th)
https://codereview.chromium.org/2258133002/diff/180001/third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.cc File third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.cc (right): https://codereview.chromium.org/2258133002/diff/180001/third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.cc#newcode63 third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.cc:63: if (!task_queue_throttler_->IsThrottled(queue)) On 2016/09/15 15:52:11, altimin wrote: > On ...
4 years, 3 months ago (2016-09-15 16:18:40 UTC) #16
Sami
I think this is looking pretty neat now. One question about the set of pool ...
4 years, 3 months ago (2016-09-15 16:30:30 UTC) #17
altimin
PTAL Alex, thanks for the thorough review! https://codereview.chromium.org/2258133002/diff/180001/third_party/WebKit/Source/platform/scheduler/base/task_queue_impl.cc File third_party/WebKit/Source/platform/scheduler/base/task_queue_impl.cc (right): https://codereview.chromium.org/2258133002/diff/180001/third_party/WebKit/Source/platform/scheduler/base/task_queue_impl.cc#newcode733 third_party/WebKit/Source/platform/scheduler/base/task_queue_impl.cc:733: // We ...
4 years, 3 months ago (2016-09-16 13:38:49 UTC) #18
alex clarke (OOO till 29th)
https://codereview.chromium.org/2258133002/diff/180001/third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.cc File third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.cc (right): https://codereview.chromium.org/2258133002/diff/180001/third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.cc#newcode63 third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.cc:63: if (!task_queue_throttler_->IsThrottled(queue)) On 2016/09/16 13:38:48, altimin wrote: > On ...
4 years, 3 months ago (2016-09-16 14:36:18 UTC) #21
altimin
PTAL https://codereview.chromium.org/2258133002/diff/180001/third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.cc File third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.cc (right): https://codereview.chromium.org/2258133002/diff/180001/third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.cc#newcode63 third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.cc:63: if (!task_queue_throttler_->IsThrottled(queue)) On 2016/09/16 14:36:18, alex clarke wrote: ...
4 years, 3 months ago (2016-09-16 14:47:00 UTC) #22
alex clarke (OOO till 29th)
Feels like it's close now. https://codereview.chromium.org/2258133002/diff/240001/third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.cc File third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.cc (right): https://codereview.chromium.org/2258133002/diff/240001/third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.cc#newcode63 third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.cc:63: if (metadata->throttling_ref_count == 0) ...
4 years, 3 months ago (2016-09-16 15:23:38 UTC) #23
altimin
PTAL https://codereview.chromium.org/2258133002/diff/240001/third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.cc File third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.cc (right): https://codereview.chromium.org/2258133002/diff/240001/third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.cc#newcode63 third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.cc:63: if (metadata->throttling_ref_count == 0) On 2016/09/16 15:23:38, alex ...
4 years, 3 months ago (2016-09-16 16:32:25 UTC) #24
alex clarke (OOO till 29th)
lgtm
4 years, 3 months ago (2016-09-16 17:04:29 UTC) #25
altimin
+caseq@ for core/inspector +haraken@ for platform/BUILD.gn PTAL
4 years, 3 months ago (2016-09-21 22:55:44 UTC) #27
caseq
inspector/ lgtm + a bunch of drive-by comments. https://codereview.chromium.org/2258133002/diff/260001/third_party/WebKit/Source/platform/scheduler/base/test_task_time_observer.h File third_party/WebKit/Source/platform/scheduler/base/test_task_time_observer.h (right): https://codereview.chromium.org/2258133002/diff/260001/third_party/WebKit/Source/platform/scheduler/base/test_task_time_observer.h#newcode18 third_party/WebKit/Source/platform/scheduler/base/test_task_time_observer.h:18: double ...
4 years, 3 months ago (2016-09-22 00:52:34 UTC) #28
caseq
https://codereview.chromium.org/2258133002/diff/260001/third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.cc File third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.cc (right): https://codereview.chromium.org/2258133002/diff/260001/third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.cc#newcode202 third_party/WebKit/Source/platform/scheduler/renderer/task_queue_throttler.cc:202: const char* tracing_category) Ouch -- just noticed this! Trace ...
4 years, 3 months ago (2016-09-22 01:25:26 UTC) #29
haraken
platform/BUILD.gn LGTM Add more explanation to the CL description so that others can follow what ...
4 years, 3 months ago (2016-09-22 06:17:07 UTC) #30
Sami
Nice, couple of small simplification suggestions. https://codereview.chromium.org/2258133002/diff/260001/third_party/WebKit/Source/platform/scheduler/base/test_task_time_observer.h File third_party/WebKit/Source/platform/scheduler/base/test_task_time_observer.h (right): https://codereview.chromium.org/2258133002/diff/260001/third_party/WebKit/Source/platform/scheduler/base/test_task_time_observer.h#newcode18 third_party/WebKit/Source/platform/scheduler/base/test_task_time_observer.h:18: double end_time) override{}; ...
4 years, 3 months ago (2016-09-22 12:34:08 UTC) #31
altimin
Sami, PTAL https://codereview.chromium.org/2258133002/diff/260001/third_party/WebKit/Source/platform/scheduler/base/test_task_time_observer.h File third_party/WebKit/Source/platform/scheduler/base/test_task_time_observer.h (right): https://codereview.chromium.org/2258133002/diff/260001/third_party/WebKit/Source/platform/scheduler/base/test_task_time_observer.h#newcode18 third_party/WebKit/Source/platform/scheduler/base/test_task_time_observer.h:18: double end_time) override{}; On 2016/09/22 00:52:34, caseq ...
4 years, 3 months ago (2016-09-22 13:45:59 UTC) #34
Sami
lgtm, thank you. Please coordinate with Alex to make sure this collides in a controlled ...
4 years, 3 months ago (2016-09-22 13:48:29 UTC) #35
commit-bot: I haz the power
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.org/2258133002/320001
4 years, 3 months ago (2016-09-22 16:42:21 UTC) #51
commit-bot: I haz the power
Try jobs failed on following builders: linux_chromium_rel_ng on master.tryserver.chromium.linux (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_rel_ng/builds/302401)
4 years, 3 months ago (2016-09-22 17:49:02 UTC) #53
commit-bot: I haz the power
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.org/2258133002/320001
4 years, 3 months ago (2016-09-22 18:06:09 UTC) #55
commit-bot: I haz the power
Try jobs failed on following builders: linux_chromium_rel_ng on master.tryserver.chromium.linux (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_rel_ng/builds/302496)
4 years, 3 months ago (2016-09-22 19:17:22 UTC) #57
commit-bot: I haz the power
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.org/2258133002/340001
4 years, 2 months ago (2016-09-23 22:46:13 UTC) #60
commit-bot: I haz the power
Try jobs failed on following builders: win_chromium_rel_ng on master.tryserver.chromium.win (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_rel_ng/builds/299069)
4 years, 2 months ago (2016-09-24 00:07:27 UTC) #62
commit-bot: I haz the power
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.org/2258133002/340001
4 years, 2 months ago (2016-09-24 11:39:06 UTC) #64
commit-bot: I haz the power
Committed patchset #18 (id:340001)
4 years, 2 months ago (2016-09-24 13:22:00 UTC) #66
commit-bot: I haz the power
Patchset 18 (id:??) landed as https://crrev.com/906d4a1e806b27681bad0ec63f64c0c9fc239a77 Cr-Commit-Position: refs/heads/master@{#420842}
4 years, 2 months ago (2016-09-24 13:23:47 UTC) #68
ojan
Drive-by after the fact comment: Please try to do these big changes in smaller patches. ...
4 years, 2 months ago (2016-10-05 17:37:34 UTC) #70
Sami
On 2016/10/05 17:37:34, ojan wrote: > Drive-by after the fact comment: Please try to do ...
4 years, 2 months ago (2016-10-05 17:41:31 UTC) #71
altimin
On 2016/10/05 17:41:31, Sami wrote: > On 2016/10/05 17:37:34, ojan wrote: > > Drive-by after ...
4 years, 2 months ago (2016-10-05 17:49:43 UTC) #72
ojan
4 years, 2 months ago (2016-10-05 17:57:26 UTC) #73
Message was sent while issue was closed.
No worries. I just make a point of commenting everytime I see big patches in the
hopes of maintaining a small patch culture. :)

Powered by Google App Engine
This is Rietveld 408576698