|
|
Created:
5 years, 10 months ago by Ian Vollick Modified:
5 years, 9 months ago CC:
cc-bugs_chromium.org, chromium-reviews, creis+watch_chromium.org, darin-cc_chromium.org, davemoore+watch_chromium.org, dzhioev+watch_chromium.org, jam, mkwst+moarreviews-renderer_chromium.org, mlamouri+watch-content_chromium.org, nasko+codewatch_chromium.org, nkostylev+watch_chromium.org, oshima+watch_chromium.org, piman+watch_chromium.org, stevenjb+watch_chromium.org, telemetry-reviews_chromium.org Base URL:
https://chromium.googlesource.com/chromium/src.git@master Target Ref:
refs/pending/heads/master Project:
chromium Visibility:
Public. |
DescriptionAdd a telemetry measurement for property tree performance.
This required plumbing a flag to enable property tree verification. I've
also had to disable the opacity check for skipping VisibleContentRect
computation.
BUG=386810
Committed: https://crrev.com/d127498b38a2a59303e5e2c5ce54382e8f82911a
Cr-Commit-Position: refs/heads/master@{#318611}
Patch Set 1 : . #
Total comments: 6
Patch Set 2 : include pre calculate meta in the trace events #Patch Set 3 : . #Patch Set 4 : fix opacity issue. add top 25 benchmark. #Patch Set 5 : . #Patch Set 6 : . #Patch Set 7 : . #
Total comments: 1
Patch Set 8 : . #
Total comments: 8
Patch Set 9 : . #Patch Set 10 : . #
Messages
Total messages: 35 (12 generated)
Patchset #1 (id:1) has been deleted
vollick@chromium.org changed reviewers: + enne@chromium.org
https://codereview.chromium.org/918633003/diff/20001/cc/trees/draw_property_u... File cc/trees/draw_property_utils.cc (left): https://codereview.chromium.org/918633003/diff/20001/cc/trees/draw_property_u... cc/trees/draw_property_utils.cc:161: layer->opacity() == 0.0f || Is it bogus to calculate visible content rects for layers with zero opacity, or just inefficient? This check was causing some failures for me (missing VCRs). I assume that this will all be subsumed with your modifications to this method in your switch-to-property-trees patch.
vollick@chromium.org changed reviewers: + hartmanng@chromium.org
hartmanng@chromium.org changed reviewers: + ernstm@chromium.org
Telemetry part lg2m, but I'm pretty rusty (and no longer an OWNER). +ernstm for a better review.
https://codereview.chromium.org/918633003/diff/20001/cc/trees/draw_property_u... File cc/trees/draw_property_utils.cc (left): https://codereview.chromium.org/918633003/diff/20001/cc/trees/draw_property_u... cc/trees/draw_property_utils.cc:161: layer->opacity() == 0.0f || On 2015/02/15 at 06:27:18, vollick_slow_to_review wrote: > Is it bogus to calculate visible content rects for layers with zero opacity, or just inefficient? This check was causing some failures for me (missing VCRs). I assume that this will all be subsumed with your modifications to this method in your switch-to-property-trees patch. I don't think this function goes away. Where was it failing? https://codereview.chromium.org/918633003/diff/20001/cc/trees/layer_tree_host... File cc/trees/layer_tree_host_common.cc (right): https://codereview.chromium.org/918633003/diff/20001/cc/trees/layer_tree_host... cc/trees/layer_tree_host_common.cc:2510: TRACE_EVENT0(TRACE_DISABLED_BY_DEFAULT("cc.debug.cdp-perf"), You should include the precalculate meta-information step here too, I think.
I thought I'd corrected some bugs in my measurement, but I've got some conflicting data that makes me doubt my numbers. I'll continue digging, but I thought it would be worth posting my WIP. The telemetry benchmark now shows that property trees are slower. Not by a lot in the absolute values, but still crummier. Here are the results on my desktop: Avg CDP_reduction: -14.950058 % Sd CDP_reduction: 7.323942 % Avg PT_avg_cost: 0.025954ms Sd PT_avg_cost: 0.005798ms Avg CDP_avg_cost: 0.022533ms Sd CDP_avg_cost: 0.004497ms Oddly, when running the unit tests, the property tree approach seems to always be faster. When running with the arguments --test-launcher-print-test-stdio=always --trace-to-console=-*,disabled-by-default-cc.debug.cdp-perf, you see a spew that looks like this. [ RUN ] LayerTreeHostReadbackPixelTests/LayerTreeHostReadbackPixelTest.HiddenSubtreeNotVisibleWhenDrawnForReadback/1 [27468:27468:0218/001236:56994352158:ERROR:trace_event_impl.cc(2008)] Main: LayerTreeHostCommon::CalculateDrawProperties[disabled-by-default-cc.debug.cdp-perf] [27468:27468:0218/001236:56994352213:ERROR:trace_event_impl.cc(2153)] Main: LayerTreeHostCommon::CalculateDrawProperties[disabled-by-default-cc.debug.cdp-perf] (0.062 ms) [27468:27468:0218/001236:56994352240:ERROR:trace_event_impl.cc(2008)] Main: LayerTreeHostCommon::ComputeVisibleRectsWithPropertyTrees[disabled-by-default-cc.debug.cdp-perf] [27468:27468:0218/001236:56994352280:ERROR:trace_event_impl.cc(2153)] Main: LayerTreeHostCommon::ComputeVisibleRectsWithPropertyTrees[disabled-by-default-cc.debug.cdp-perf] (0.038 ms) [ OK ] I didn't see an example where the property approach is slower (in fact they look a fair bit faster) and I'm not sure what to make of this yet. Would you expect CDP to be faster in the wild than in unit tests? https://codereview.chromium.org/918633003/diff/20001/cc/trees/draw_property_u... File cc/trees/draw_property_utils.cc (left): https://codereview.chromium.org/918633003/diff/20001/cc/trees/draw_property_u... cc/trees/draw_property_utils.cc:161: layer->opacity() == 0.0f || On 2015/02/17 17:40:14, enne wrote: > On 2015/02/15 at 06:27:18, vollick_slow_to_review wrote: > > Is it bogus to calculate visible content rects for layers with zero opacity, > or just inefficient? This check was causing some failures for me (missing VCRs). > I assume that this will all be subsumed with your modifications to this method > in your switch-to-property-trees patch. > > I don't think this function goes away. Where was it failing? Sorry, I didn't mean to imply that it was going away. I think the problem is in the conflation of the I-need-a-visible-content-rect and I-need-to-be-in-the-RSLL concepts and that your patch will fix that. I constantly tripping the checks in the property tree verification code due to zero opacity layers that needed VCRs. This seemed strange to me at first (what were these zero opacity layers) but it looks like these were just overlay scollbars. Here's a portion of the layer tree from NTP: 7 Root Transform Layer, (hidden) 8 Inner Viewport Container Layer, (hidden) bounds: (1270, 940) 9 Overscroll Elasticity Layer, (hidden) 10 Page Scale Layer, (hidden) 11 Inner Viewport Scroll Layer, (hidden) bounds: (1270, 940) 2 Overflow Controls Host Layer, (hidden) bounds: (1270, 940) 3 LocalFrame Clipping Layer, (hidden) bounds: (1270, 940) 4 LocalFrame Scrolling Layer, (hidden) bounds: (1270, 940) 1 Content Root Layer, (hidden) bounds: (1270, 940) 19 RenderView #document, reasons(Is the root layer) bounds: (1270, 940) transform originr: (635, 470, 0) 12 Overlay Scrollbar Horizontal Layer, (hidden) bounds: (1260, 10) position: (0, 930) draw_transform: 1 0 0 0 0 1 0 930 0 0 1 0 0 0 0 1 14 ContentsLayer for Overlay Scrollbar Horizontal Layer bounds: (1260, 10) draw_transform: 1 0 0 0 0 1 0 930 0 0 1 0 0 0 0 1 opacity:0 13 Overlay Scrollbar Vertical Layer, (hidden) bounds: (10, 930) position: (1260, 0) draw_transform: 1 0 0 1260 0 1 0 0 0 0 1 0 0 0 0 1 [*] 15 ContentsLayer for Overlay Scrollbar Vertical Layer bounds: (10, 930) draw_transform: 1 0 0 1260 0 1 0 0 0 0 1 0 0 0 0 1 opacity:0 https://codereview.chromium.org/918633003/diff/20001/cc/trees/layer_tree_host... File cc/trees/layer_tree_host_common.cc (right): https://codereview.chromium.org/918633003/diff/20001/cc/trees/layer_tree_host... cc/trees/layer_tree_host_common.cc:2510: TRACE_EVENT0(TRACE_DISABLED_BY_DEFAULT("cc.debug.cdp-perf"), On 2015/02/17 17:40:14, enne wrote: > You should include the precalculate meta-information step here too, I think. Done.
Patchset #3 (id:60001) has been deleted
Patchset #4 (id:100001) has been deleted
I've added a benchmark that uses the top 25 page set in case the slowdown was due to trivial scrolling pages, but the reported results were still bad. Strangely, running with --enable-property-tree-verification --trace-to-console=-*,disabled-by-default-cc.debug.cdp-perf by hand on a complex site (I tried facebook) shows similar improvements to the ones I saw when dumping the trace to console this during the unit tests. Something's fishy. :/ https://codereview.chromium.org/918633003/diff/20001/cc/trees/draw_property_u... File cc/trees/draw_property_utils.cc (left): https://codereview.chromium.org/918633003/diff/20001/cc/trees/draw_property_u... cc/trees/draw_property_utils.cc:161: layer->opacity() == 0.0f || On 2015/02/18 05:22:43, vollick wrote: > On 2015/02/17 17:40:14, enne wrote: > > On 2015/02/15 at 06:27:18, vollick_slow_to_review wrote: > > > Is it bogus to calculate visible content rects for layers with zero opacity, > > or just inefficient? This check was causing some failures for me (missing > VCRs). > > I assume that this will all be subsumed with your modifications to this method > > in your switch-to-property-trees patch. > > > > I don't think this function goes away. Where was it failing? > > Sorry, I didn't mean to imply that it was going away. I think the problem is in > the conflation of the I-need-a-visible-content-rect and I-need-to-be-in-the-RSLL > concepts and that your patch will fix that. I constantly tripping the checks in > the property tree verification code due to zero opacity layers that needed VCRs. > This seemed strange to me at first (what were these zero opacity layers) but it > looks like these were just overlay scollbars. Here's a portion of the layer tree > from NTP: > > > 7 Root Transform Layer, (hidden) > 8 Inner Viewport Container Layer, (hidden) > bounds: (1270, 940) > 9 Overscroll Elasticity Layer, (hidden) > 10 Page Scale Layer, (hidden) > 11 Inner Viewport Scroll Layer, (hidden) > bounds: (1270, 940) > 2 Overflow Controls Host Layer, (hidden) > bounds: (1270, 940) > 3 LocalFrame Clipping Layer, (hidden) > bounds: (1270, 940) > 4 LocalFrame Scrolling Layer, (hidden) > bounds: (1270, 940) > 1 Content Root Layer, (hidden) > bounds: (1270, 940) > 19 RenderView #document, reasons(Is the root layer) > bounds: (1270, 940) > transform originr: (635, 470, 0) > 12 Overlay Scrollbar Horizontal Layer, (hidden) > bounds: (1260, 10) > position: (0, 930) > draw_transform: > 1 0 0 0 > 0 1 0 930 > 0 0 1 0 > 0 0 0 1 > 14 ContentsLayer for Overlay Scrollbar Horizontal Layer > bounds: (1260, 10) > draw_transform: > 1 0 0 0 > 0 1 0 930 > 0 0 1 0 > 0 0 0 1 > opacity:0 > 13 Overlay Scrollbar Vertical Layer, (hidden) > bounds: (10, 930) > position: (1260, 0) > draw_transform: > 1 0 0 1260 > 0 1 0 0 > 0 0 1 0 > 0 0 0 1 > [*] 15 ContentsLayer for Overlay Scrollbar Vertical Layer > bounds: (10, 930) > draw_transform: > 1 0 0 1260 > 0 1 0 0 > 0 0 1 0 > 0 0 0 1 > opacity:0 Stepping through this example, it looks like the problem is that the scrollbars have impl-only-animations. Matching the opacity check in LTHC's SubtreeShouldBeSkipped catches this case and we no longer crash. Should be fixed now.
On 2015/02/19 07:07:45, vollick wrote: > I've added a benchmark that uses the top 25 page set in case the slowdown was > due to trivial scrolling pages, but the reported results were still bad. > > Strangely, running with --enable-property-tree-verification > --trace-to-console=-*,disabled-by-default-cc.debug.cdp-perf by hand on a complex > site (I tried facebook) shows similar improvements to the ones I saw when > dumping the trace to console this during the unit tests. Something's fishy. :/ > > https://codereview.chromium.org/918633003/diff/20001/cc/trees/draw_property_u... > File cc/trees/draw_property_utils.cc (left): > > https://codereview.chromium.org/918633003/diff/20001/cc/trees/draw_property_u... > cc/trees/draw_property_utils.cc:161: layer->opacity() == 0.0f || > On 2015/02/18 05:22:43, vollick wrote: > > On 2015/02/17 17:40:14, enne wrote: > > > On 2015/02/15 at 06:27:18, vollick_slow_to_review wrote: > > > > Is it bogus to calculate visible content rects for layers with zero > opacity, > > > or just inefficient? This check was causing some failures for me (missing > > VCRs). > > > I assume that this will all be subsumed with your modifications to this > method > > > in your switch-to-property-trees patch. > > > > > > I don't think this function goes away. Where was it failing? > > > > Sorry, I didn't mean to imply that it was going away. I think the problem is > in > > the conflation of the I-need-a-visible-content-rect and > I-need-to-be-in-the-RSLL > > concepts and that your patch will fix that. I constantly tripping the checks > in > > the property tree verification code due to zero opacity layers that needed > VCRs. > > This seemed strange to me at first (what were these zero opacity layers) but > it > > looks like these were just overlay scollbars. Here's a portion of the layer > tree > > from NTP: > > > > > > 7 Root Transform Layer, (hidden) > > 8 Inner Viewport Container Layer, (hidden) > > bounds: (1270, 940) > > 9 Overscroll Elasticity Layer, (hidden) > > 10 Page Scale Layer, (hidden) > > 11 Inner Viewport Scroll Layer, (hidden) > > bounds: (1270, 940) > > 2 Overflow Controls Host Layer, (hidden) > > bounds: (1270, 940) > > 3 LocalFrame Clipping Layer, (hidden) > > bounds: (1270, 940) > > 4 LocalFrame Scrolling Layer, (hidden) > > bounds: (1270, 940) > > 1 Content Root Layer, (hidden) > > bounds: (1270, 940) > > 19 RenderView #document, reasons(Is the root layer) > > bounds: (1270, 940) > > transform originr: (635, 470, 0) > > 12 Overlay Scrollbar Horizontal Layer, (hidden) > > bounds: (1260, 10) > > position: (0, 930) > > draw_transform: > > 1 0 0 0 > > 0 1 0 930 > > 0 0 1 0 > > 0 0 0 1 > > 14 ContentsLayer for Overlay Scrollbar Horizontal Layer > > bounds: (1260, 10) > > draw_transform: > > 1 0 0 0 > > 0 1 0 930 > > 0 0 1 0 > > 0 0 0 1 > > opacity:0 > > 13 Overlay Scrollbar Vertical Layer, (hidden) > > bounds: (10, 930) > > position: (1260, 0) > > draw_transform: > > 1 0 0 1260 > > 0 1 0 0 > > 0 0 1 0 > > 0 0 0 1 > > [*] 15 ContentsLayer for Overlay Scrollbar Vertical Layer > > bounds: (10, 930) > > draw_transform: > > 1 0 0 1260 > > 0 1 0 0 > > 0 0 1 0 > > 0 0 0 1 > > opacity:0 > > Stepping through this example, it looks like the problem is that the scrollbars > have impl-only-animations. Matching the opacity check in LTHC's > SubtreeShouldBeSkipped catches this case and we no longer crash. Should be fixed > now. Well, I've resolved the discrepancy, but the result is sad. Turns out it's the trace-to-console that was skewing the results. Doing the same experiment but capturing the trace using about:tracing rather than dumping to the console showed results consistent with the telemetry test. I'll have to do some more investigation to see where the time is being spent in the property tree code (hopefully it's the tree building). On the plus side, the absolute change in time is a tiny portion of the frame budget.
Patchset #5 (id:140001) has been deleted
If you rebaseline this, I think it should be ready to go?
On 2015/02/25 19:53:12, enne wrote: > If you rebaseline this, I think it should be ready to go? Ali was noticing super noisy results. I'd like to see if I can fix that, but yeah, hopefully this is nearly done!
On 2015/02/25 20:01:29, vollick wrote: > On 2015/02/25 19:53:12, enne wrote: > > If you rebaseline this, I think it should be ready to go? > > Ali was noticing super noisy results. I'd like to see if I can fix that, but > yeah, hopefully this is nearly done! Ok, I've rebased and I think this is ready to go. There are still issues unrelated to the benchmark (we get visible content rect mismatches on 3 pages), but I don't think there's a reason to hold up the telemetry stuff on those fixes which can come later.
https://codereview.chromium.org/918633003/diff/200001/cc/trees/layer_tree_hos... File cc/trees/layer_tree_host_common.cc (right): https://codereview.chromium.org/918633003/diff/200001/cc/trees/layer_tree_hos... cc/trees/layer_tree_host_common.cc:2507: if (!inputs->verify_property_trees) { I'm not understanding why you moved this and then had to duplicate the meta-information and CDP?
On 2015/02/26 19:00:42, enne wrote: > https://codereview.chromium.org/918633003/diff/200001/cc/trees/layer_tree_hos... > File cc/trees/layer_tree_host_common.cc (right): > > https://codereview.chromium.org/918633003/diff/200001/cc/trees/layer_tree_hos... > cc/trees/layer_tree_host_common.cc:2507: if (!inputs->verify_property_trees) { > I'm not understanding why you moved this and then had to duplicate the > meta-information and CDP? The code used to be structure like this: { TRACE_EVENT0(...); PreCalc...; CDPInternal...; } if (verify_prop_trees) { { TRACE_EVENT0(...); PropTreesStuff...; } Problem was that CDP (external) is called without the verify_property_trees in other places than the renderer (to avoid conflating effects on page work and ui work), and this was causing extra counting of CDPInternal. This caused that function to be unfairly penalized. To be sure that it was an apples to apples comparison I changed the code to if (not verifying) { do the old thing } else { measure the old thing measure the new thing } To guarantee that they're in sync. This seems to eliminate a class of bugs that the brittleness of the old approach permitted.
Oh! Ok! lgtm
The CQ bit was checked by vollick@chromium.org
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/918633003/220001
The CQ bit was unchecked by commit-bot@chromium.org
Try jobs failed on following builders: chromium_presubmit on tryserver.chromium.linux (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presub...)
vollick@chromium.org changed reviewers: + sullivan@chromium.org
On 2015/02/26 21:17:41, I haz the power (commit-bot) wrote: > Try jobs failed on following builders: > chromium_presubmit on tryserver.chromium.linux (JOB_FAILED, > http://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presub...) sullivan, Could you please take a look at my measurement? It's my first one. Might be crummy :)
https://codereview.chromium.org/918633003/diff/220001/tools/perf/benchmarks/d... File tools/perf/benchmarks/draw_properties.py (right): https://codereview.chromium.org/918633003/diff/220001/tools/perf/benchmarks/d... tools/perf/benchmarks/draw_properties.py:19: """Measures rendering statistics while scrolling down the top 25 web pages. Is this actually meant to scroll? I don't see any code that scrolls anywhere. https://codereview.chromium.org/918633003/diff/220001/tools/perf/measurements... File tools/perf/measurements/draw_properties.py (right): https://codereview.chromium.org/918633003/diff/220001/tools/perf/measurements... tools/perf/measurements/draw_properties.py:35: return tab.EvaluateJavaScript('isDone') I think this function is unneeded? I don't see it called and I don't think the pagesets have this JS set up. https://codereview.chromium.org/918633003/diff/220001/tools/perf/measurements... tools/perf/measurements/draw_properties.py:55: pt_avg = pt_sum / pt_count Could put this duplicated code into a function.
https://codereview.chromium.org/918633003/diff/220001/tools/perf/benchmarks/d... File tools/perf/benchmarks/draw_properties.py (right): https://codereview.chromium.org/918633003/diff/220001/tools/perf/benchmarks/d... tools/perf/benchmarks/draw_properties.py:19: """Measures rendering statistics while scrolling down the top 25 web pages. On 2015/02/26 21:59:02, sullivan wrote: > Is this actually meant to scroll? I don't see any code that scrolls anywhere. I'm honestly not sure the mechanism by which scrolling is specified, but this certainly does cause scrolling in my tests. Perhaps the interaction is baked into the page set? https://codereview.chromium.org/918633003/diff/220001/tools/perf/measurements... File tools/perf/measurements/draw_properties.py (right): https://codereview.chromium.org/918633003/diff/220001/tools/perf/measurements... tools/perf/measurements/draw_properties.py:35: return tab.EvaluateJavaScript('isDone') On 2015/02/26 21:59:02, sullivan wrote: > I think this function is unneeded? I don't see it called and I don't think the > pagesets have this JS set up. Done. https://codereview.chromium.org/918633003/diff/220001/tools/perf/measurements... tools/perf/measurements/draw_properties.py:55: pt_avg = pt_sum / pt_count On 2015/02/26 21:59:02, sullivan wrote: > Could put this duplicated code into a function. Done.
lgtm I'm assuming I can add you as an owner for this benchmark? If it regresses or breaks, you'll be cc-ed on the bug to help triage. Also cc-ed on regression bugs marked WontFix. https://codereview.chromium.org/918633003/diff/220001/tools/perf/benchmarks/d... File tools/perf/benchmarks/draw_properties.py (right): https://codereview.chromium.org/918633003/diff/220001/tools/perf/benchmarks/d... tools/perf/benchmarks/draw_properties.py:19: """Measures rendering statistics while scrolling down the top 25 web pages. On 2015/02/26 22:59:39, vollick wrote: > On 2015/02/26 21:59:02, sullivan wrote: > > Is this actually meant to scroll? I don't see any code that scrolls anywhere. > > I'm honestly not sure the mechanism by which scrolling is specified, but this > certainly does cause scrolling in my tests. Perhaps the interaction is baked > into the page set? Oops, yeah, it's coded into the page interactions of both pagesets. FYI, the class-level comment you put here and above gets uploaded to the perf dashboard as the benchmark documentation. So please double-check that it's what you'd like the people viewing graphs to see!
I'm certainly fine with owning this measurement/benchmark, but I wasn't sure where to indicate that. (Not the OWNERS file, right?) https://codereview.chromium.org/918633003/diff/220001/tools/perf/benchmarks/d... File tools/perf/benchmarks/draw_properties.py (right): https://codereview.chromium.org/918633003/diff/220001/tools/perf/benchmarks/d... tools/perf/benchmarks/draw_properties.py:19: """Measures rendering statistics while scrolling down the top 25 web pages. On 2015/02/28 15:58:05, sullivan wrote: > On 2015/02/26 22:59:39, vollick wrote: > > On 2015/02/26 21:59:02, sullivan wrote: > > > Is this actually meant to scroll? I don't see any code that scrolls > anywhere. > > > > I'm honestly not sure the mechanism by which scrolling is specified, but this > > certainly does cause scrolling in my tests. Perhaps the interaction is baked > > into the page set? > > Oops, yeah, it's coded into the page interactions of both pagesets. > > FYI, the class-level comment you put here and above gets uploaded to the perf > dashboard as the benchmark documentation. So please double-check that it's what > you'd like the people viewing graphs to see! Yeah, my comment was pretty crummy. Updated.
The CQ bit was checked by vollick@chromium.org
The patchset sent to the CQ was uploaded after l-g-t-m from enne@chromium.org, sullivan@chromium.org Link to the patchset: https://codereview.chromium.org/918633003/#ps260001 (title: ".")
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/918633003/260001
On 2015/02/28 22:22:14, vollick wrote: > I'm certainly fine with owning this measurement/benchmark, but I wasn't sure > where to indicate that. (Not the OWNERS file, right?) Currently just this spreadsheet: https://docs.google.com/a/chromium.org/spreadsheets/d/1R_1BAOd3xeVtR0jn6wB5HH... We're hoping to integrate this better so you add a bit of metadata to the benchmark and it is automatically integrated with the perf dashboard. (Haven't done it yet because there are weird edge cases, like different metrics on different platforms can be owned by different people).
Message was sent while issue was closed.
Committed patchset #10 (id:260001)
Message was sent while issue was closed.
Patchset 10 (id:??) landed as https://crrev.com/d127498b38a2a59303e5e2c5ce54382e8f82911a Cr-Commit-Position: refs/heads/master@{#318611} |