Chromium Code Reviews
|
|
Chromium Code Reviews
Issue
2271793003:
Add a new benchmark for cpu power measurements on steady state sites. (Closed)
|
Created:
4 years, 4 months ago by erikchen Modified:
4 years, 3 months ago Reviewers:
nednguyen, Camillo Bruni, petrcermak, charliea (OOO until 10-5), fmeawad, kouhei (in TOK) CC:
chromium-reviews, telemetry-reviews_chromium.org, rnephew (Reviews Here) Base URL:
https://chromium.googlesource.com/chromium/src.git@master Target Ref:
refs/pending/heads/master Project:
chromium Visibility:
Public. |
DescriptionAdd a new benchmark for cpu power measurements on steady state sites.
The site in question is known to have high CPU usage, even though there appears
to be no visual changes to the site.
BUG=640398, 638365
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.perf:android_s5_perf_cq;master.tryserver.chromium.perf:linux_perf_cq;master.tryserver.chromium.perf:mac_retina_perf_cq;master.tryserver.chromium.perf:winx64_10_perf_cq
Committed: https://crrev.com/9c3c0a38de1c83ac9e6a9f683bad1291e12e2885
Cr-Commit-Position: refs/heads/master@{#415654}
Patch Set 1 #
Total comments: 2
Patch Set 2 : Naming from nednguyen. #Patch Set 3 : Add wpr files. #
Total comments: 2
Patch Set 4 : Add comment. #
Dependent Patchsets: Messages
Total messages: 48 (16 generated)
Description was changed from ========== Add a new benchmark for cpu power measurements on steady state sites. The site in question is known to have high CPU usage, even though there appears to be no visual changes to the site. BUG=640398, 638365 ========== to ========== Add a new benchmark for cpu power measurements on steady state sites. The site in question is known to have high CPU usage, even though there appears to be no visual changes to the site. BUG=640398, 638365 CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.perf:android_s5_perf_cq;master.tryserver.chromium.perf:linux_perf_cq;master.tryserver.chromium.perf:mac_retina_perf_cq;master.tryserver.chromium.perf:winx64_10_perf_cq ==========
erikchen@chromium.org changed reviewers: + charliea@chromium.org, nednguyen@google.com
nednguyen: Please review.
nednguyen@google.com changed reviewers: + rnephew@chromium.org
https://codereview.chromium.org/2271793003/diff/1/tools/perf/page_sets/steady... File tools/perf/page_sets/steady_state.py (right): https://codereview.chromium.org/2271793003/diff/1/tools/perf/page_sets/steady... tools/perf/page_sets/steady_state.py:8: class SteadyStateStorySet(story.StorySet): Why create new story set & benchmark instead of add this page to existing one?
https://codereview.chromium.org/2271793003/diff/1/tools/perf/page_sets/steady... File tools/perf/page_sets/steady_state.py (right): https://codereview.chromium.org/2271793003/diff/1/tools/perf/page_sets/steady... tools/perf/page_sets/steady_state.py:8: class SteadyStateStorySet(story.StorySet): On 2016/08/24 19:51:43, nednguyen wrote: > Why create new story set & benchmark instead of add this page to existing one? Assuming this has low noise, I'm going to add another 40 or so WPR-ed pages to this set. This is a different type of story set from trivial pages, which doesn't use WPR, and is therefore an entire level of complexity simpler. Separating this benchmark will also make it easier to view differences in noise between these pages, and the trivial pages.
On 2016/08/24 22:13:32, erikchen wrote: > https://codereview.chromium.org/2271793003/diff/1/tools/perf/page_sets/steady... > File tools/perf/page_sets/steady_state.py (right): > > https://codereview.chromium.org/2271793003/diff/1/tools/perf/page_sets/steady... > tools/perf/page_sets/steady_state.py:8: class > SteadyStateStorySet(story.StorySet): > On 2016/08/24 19:51:43, nednguyen wrote: > > Why create new story set & benchmark instead of add this page to existing one? > > Assuming this has low noise, I'm going to add another 40 or so WPR-ed pages to > this set. This is a different type of story set from trivial pages, which > doesn't use WPR, and is therefore an entire level of complexity simpler. Then should these be renamed as real_world_power_cases? Also have you checked out system health user stories? We have a bunch of realistic scenarios there. > > Separating this benchmark will also make it easier to view differences in noise > between these pages, and the trivial pages.
Description was changed from ========== Add a new benchmark for cpu power measurements on steady state sites. The site in question is known to have high CPU usage, even though there appears to be no visual changes to the site. BUG=640398, 638365 CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.perf:android_s5_perf_cq;master.tryserver.chromium.perf:linux_perf_cq;master.tryserver.chromium.perf:mac_retina_perf_cq;master.tryserver.chromium.perf:winx64_10_perf_cq ========== to ========== Add a new benchmark for cpu power measurements on steady state sites. The site in question is known to have high CPU usage, even though there appears to be no visual changes to the site. BUG=640398, 638365 CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.perf:android_s5_perf_cq;master.tryserver.chromium.perf:linux_perf_cq;master.tryserver.chromium.perf:mac_retina_perf_cq;master.tryserver.chromium.perf:winx64_10_perf_cq ==========
nednguyen@google.com changed reviewers: + petrcermak@chromium.org
On 2016/08/25 00:27:16, nednguyen wrote: > On 2016/08/24 22:13:32, erikchen wrote: > > > https://codereview.chromium.org/2271793003/diff/1/tools/perf/page_sets/steady... > > File tools/perf/page_sets/steady_state.py (right): > > > > > https://codereview.chromium.org/2271793003/diff/1/tools/perf/page_sets/steady... > > tools/perf/page_sets/steady_state.py:8: class > > SteadyStateStorySet(story.StorySet): > > On 2016/08/24 19:51:43, nednguyen wrote: > > > Why create new story set & benchmark instead of add this page to existing > one? > > > > Assuming this has low noise, I'm going to add another 40 or so WPR-ed pages to > > this set. This is a different type of story set from trivial pages, which > > doesn't use WPR, and is therefore an entire level of complexity simpler. > > Then should these be renamed as real_world_power_cases? Also have you checked > out system health user stories? We have a bunch of realistic scenarios there. > > > > Separating this benchmark will also make it easier to view differences in > noise > > between these pages, and the trivial pages. Yes, I have looked at System Health Stories. See: https://docs.google.com/document/d/15kC6bPPLH81uoUfG192xCkJoezX-sWGuWQlduk7r0... That's where the name "SteadyState" comes from. Would you prefer "SteadyStateRealWorldPowerCases"? Seems a bit of a mouthful, but I will choose whatever name you want.
nednguyen@google.com changed reviewers: + cbruni@chromium.org, fmeawad@chromium.org, kouhei@chromium.org - rnephew@chromium.org
If you're looking into create a bunch of realistic pages that just wait x seconds after loading, we can put these stories in s.t like idle_loading_stories.py which are a good set of URLs that can be reused by loading-dev & v8 teams. For the context, loading-dev is looking into TimeToInteractive metric which is "how long does it take for the pages to become relatively idle after navigation" & v8 team is optimizing for js during that periods. I think these spaces are similar & the cost of maintaining these stories can be high, so let's try to come up with a share stories that are valuable to all parties. +Kouhei, Fadi & Camillo: what do you folks think?
On 2016/08/25 00:40:08, nednguyen wrote: > If you're looking into create a bunch of realistic pages that just wait x > seconds after loading, we can put these stories in s.t like > idle_loading_stories.py which are a good set of URLs that can be reused by > loading-dev & v8 teams. > > For the context, loading-dev is looking into TimeToInteractive metric which is > "how long does it take for the pages to become relatively idle after navigation" > & v8 team is optimizing for js during that periods. This sounds like loading-dev will want to target a different set of pages? I have a specific set of 40 pages that I want: https://docs.google.com/spreadsheets/d/1U_P_4QOgyvOg8127gY8nINyggM2qOkBphz-Ou... These pages have high idle CPU usage after they're fully "loaded". I don't know anything about the performance of the load itself, and I'm not particularly interested. > > I think these spaces are similar & the cost of maintaining these stories can be > high, so let's try to come up with a share stories that are valuable to all > parties. Could you be more specific about the cost of maintaining these stories? Shouldn't this be fire and forget? (Make a WPR recording, reuse forever). > > +Kouhei, Fadi & Camillo: what do you folks think?
On 2016/08/25 00:45:35, erikchen wrote: > On 2016/08/25 00:40:08, nednguyen wrote: > > If you're looking into create a bunch of realistic pages that just wait x > > seconds after loading, we can put these stories in s.t like > > idle_loading_stories.py which are a good set of URLs that can be reused by > > loading-dev & v8 teams. > > > > For the context, loading-dev is looking into TimeToInteractive metric which is > > "how long does it take for the pages to become relatively idle after > navigation" > > & v8 team is optimizing for js during that periods. > This sounds like loading-dev will want to target a different set of pages? I > have a specific set of 40 pages that I want: > https://docs.google.com/spreadsheets/d/1U_P_4QOgyvOg8127gY8nINyggM2qOkBphz-Ou... > > These pages have high idle CPU usage after they're fully "loaded". I don't know > anything about the performance of the load itself, and I'm not particularly > interested. I think this correlates well with whether the page is idle after load. I would expect pages with high idle CPU usage to have high TimeToInteractive metrics values as well. > > > > > I think these spaces are similar & the cost of maintaining these stories can > be > > high, so let's try to come up with a share stories that are valuable to all > > parties. > Could you be more specific about the cost of maintaining these stories? > Shouldn't this be fire and forget? (Make a WPR recording, reuse forever). The main cost of maintenance is triaging/fixing bugs when Chrome crashes when it to load these pages. For an ideas of how often that happens, you can look at the number of blocking bugs in https://bugs.chromium.org/p/chromium/issues/detail?id=589726 > > > > > +Kouhei, Fadi & Camillo: what do you folks think?
> I think this correlates well with whether the page is idle after load. I would > expect > pages with high idle CPU usage to have high TimeToInteractive metrics values as > well. I do not have this expectation. From everything we've seen so far, most pages with high idle CPU usage are due to too many Timer firings, excessive relayouts, etc. See the "Potential Areas for Investigation" section under https://docs.google.com/document/d/1QlX7BJDjDmHH2BuysnwYA5QqqGOW24bj6OpQzqWHF... > The main cost of maintenance is triaging/fixing bugs when Chrome crashes when it > to load these pages. For an ideas of how often that happens, > you can look at the number of blocking bugs in > https://bugs.chromium.org/p/chromium/issues/detail?id=589726 I started going through the blocking bugs in https://bugs.chromium.org/p/chromium/issues/detail?id=589726. I'm failing to see any that are caused by Chrome crashes associated with loading pages from WPR. Could you point out some specific examples? There are several possibilities that I see: 1) Pages fail to load/quiesce/whatever under WPR. (I've seen this problem before). This is a serious issue, and I could see an argument about wanting to have fewer WPR-ed pages to avoid this issue. Although the right solution here is to rely less on WPR. 2) Chrome crashes under WPR-ed pages (your claim). This is huge! If we're catching real Chrome crashes that are making it past the Chrome CQ, we should find a set of WPR pages to add to the CQ. I'm sure everyone would want to see us catch more crashes before they make it live.
On 2016/08/25 01:01:51, erikchen wrote: > > I think this correlates well with whether the page is idle after load. I would > > expect > > pages with high idle CPU usage to have high TimeToInteractive metrics values > as > > well. > > I do not have this expectation. From everything we've seen so far, most pages > with high idle CPU usage are due to too many Timer firings, excessive relayouts, > etc. See the "Potential Areas for Investigation" section under > https://docs.google.com/document/d/1QlX7BJDjDmHH2BuysnwYA5QqqGOW24bj6OpQzqWHF... Wont't these tasks run on the main threads? If the main threads is quiet, we consider the page to be interactive. (see more on https://github.com/tdresser/time-to-interactive/blob/master/README.md) To be clear, my main question here is: "if a page exhibit interesting power problems around loading period, would that page also be interesting for loading & v8 to track their performance?" If we believe the nature of these pages are different, I am fine with not sharing the pages but we should discuss first. > > > The main cost of maintenance is triaging/fixing bugs when Chrome crashes when > it > > to load these pages. For an ideas of how often that happens, > > you can look at the number of blocking bugs in > > https://bugs.chromium.org/p/chromium/issues/detail?id=589726 > > I started going through the blocking bugs in > https://bugs.chromium.org/p/chromium/issues/detail?id=589726. I'm failing to see > any that are caused by Chrome crashes associated with loading pages from WPR. > Could you point out some specific examples? > > There are several possibilities that I see: > > 1) Pages fail to load/quiesce/whatever under WPR. (I've seen this problem > before). This is a serious issue, and I could see an argument about wanting to > have fewer WPR-ed pages to avoid this issue. Although the right solution here is > to rely less on WPR. > > 2) Chrome crashes under WPR-ed pages (your claim). This is huge! If we're > catching real Chrome crashes that are making it past the Chrome CQ, we should > find a set of WPR pages to add to the CQ. I'm sure everyone would want to see us > catch more crashes before they make it live. Yes. We already enable running all the system health stories on CQ and catch a bunch of Chrome crashes with those. Examples: https://bugs.chromium.org/p/chromium/issues/detail?id=624474 https://bugs.chromium.org/p/chromium/issues/detail?id=624587 https://bugs.chromium.org/p/chromium/issues/detail?id=624701 https://bugs.chromium.org/p/chromium/issues/detail?id=612135 https://bugs.chromium.org/p/chromium/issues/detail?id=637230 ... I would want to enable more tests but stip@ from infra team is having some concern with telemetry_perf_unittests currently is taking most of resources in android_chromium_rel_ng. +__+ I will advocate for more tests anyway, but we need to adjust the number of tests we enable with android infra's capacity.
> +Kouhei, Fadi & Camillo: what do you folks think? I'm supportive of erikchen@'s idea of separating idle-time busy stories and loading-time busy stories.
On 2016/08/25 01:28:40, nednguyen wrote: > On 2016/08/25 01:01:51, erikchen wrote: > > > I think this correlates well with whether the page is idle after load. I > would > > > expect > > > pages with high idle CPU usage to have high TimeToInteractive metrics values > > as > > > well. > > > > I do not have this expectation. From everything we've seen so far, most pages > > with high idle CPU usage are due to too many Timer firings, excessive > relayouts, > > etc. See the "Potential Areas for Investigation" section under > > > https://docs.google.com/document/d/1QlX7BJDjDmHH2BuysnwYA5QqqGOW24bj6OpQzqWHF... > > Wont't these tasks run on the main threads? If the main threads is quiet, we > consider the page to be interactive. (see more on > https://github.com/tdresser/time-to-interactive/blob/master/README.md) > > To be clear, my main question here is: > "if a page exhibit interesting power problems around loading period, would that > page also be interesting for loading & v8 to track their performance?" > > If we believe the nature of these pages are different, I am fine with not > sharing the pages but we should discuss first. > > > > > > > The main cost of maintenance is triaging/fixing bugs when Chrome crashes > when > > it > > > to load these pages. For an ideas of how often that happens, > > > you can look at the number of blocking bugs in > > > https://bugs.chromium.org/p/chromium/issues/detail?id=589726 > > > > I started going through the blocking bugs in > > https://bugs.chromium.org/p/chromium/issues/detail?id=589726. I'm failing to > see > > any that are caused by Chrome crashes associated with loading pages from WPR. > > Could you point out some specific examples? > > > > There are several possibilities that I see: > > > > 1) Pages fail to load/quiesce/whatever under WPR. (I've seen this problem > > before). This is a serious issue, and I could see an argument about wanting to > > have fewer WPR-ed pages to avoid this issue. Although the right solution here > is > > to rely less on WPR. > > > > 2) Chrome crashes under WPR-ed pages (your claim). This is huge! If we're > > catching real Chrome crashes that are making it past the Chrome CQ, we should > > find a set of WPR pages to add to the CQ. I'm sure everyone would want to see > us > > catch more crashes before they make it live. > > Yes. We already enable running all the system health stories on CQ and catch a > bunch of Chrome crashes with those. Examples: > https://bugs.chromium.org/p/chromium/issues/detail?id=624474 > https://bugs.chromium.org/p/chromium/issues/detail?id=624587 > https://bugs.chromium.org/p/chromium/issues/detail?id=624701 > https://bugs.chromium.org/p/chromium/issues/detail?id=612135 > https://bugs.chromium.org/p/chromium/issues/detail?id=637230 > ... > > I would want to enable more tests but stip@ from infra team is having some > concern with telemetry_perf_unittests currently is taking most of resources in > android_chromium_rel_ng. +__+ > I will advocate for more tests anyway, but we need to adjust the number of tests > we enable with android infra's capacity. It's a good thing these tests will only be running on Mac for now. I went through the 5 bugs you posted - 3 of them were chrome crashes, and 2 were webpage load timeouts (guessing WPR failure?)
On 2016/08/25 15:02:03, erikchen wrote: > On 2016/08/25 01:28:40, nednguyen wrote: > > On 2016/08/25 01:01:51, erikchen wrote: > > > > I think this correlates well with whether the page is idle after load. I > > would > > > > expect > > > > pages with high idle CPU usage to have high TimeToInteractive metrics > values > > > as > > > > well. > > > > > > I do not have this expectation. From everything we've seen so far, most > pages > > > with high idle CPU usage are due to too many Timer firings, excessive > > relayouts, > > > etc. See the "Potential Areas for Investigation" section under > > > > > > https://docs.google.com/document/d/1QlX7BJDjDmHH2BuysnwYA5QqqGOW24bj6OpQzqWHF... > > > > Wont't these tasks run on the main threads? If the main threads is quiet, we > > consider the page to be interactive. (see more on > > https://github.com/tdresser/time-to-interactive/blob/master/README.md) > > > > To be clear, my main question here is: > > "if a page exhibit interesting power problems around loading period, would > that > > page also be interesting for loading & v8 to track their performance?" > > > > If we believe the nature of these pages are different, I am fine with not > > sharing the pages but we should discuss first. > > > > > > > > > > > The main cost of maintenance is triaging/fixing bugs when Chrome crashes > > when > > > it > > > > to load these pages. For an ideas of how often that happens, > > > > you can look at the number of blocking bugs in > > > > https://bugs.chromium.org/p/chromium/issues/detail?id=589726 > > > > > > I started going through the blocking bugs in > > > https://bugs.chromium.org/p/chromium/issues/detail?id=589726. I'm failing to > > see > > > any that are caused by Chrome crashes associated with loading pages from > WPR. > > > Could you point out some specific examples? > > > > > > There are several possibilities that I see: > > > > > > 1) Pages fail to load/quiesce/whatever under WPR. (I've seen this problem > > > before). This is a serious issue, and I could see an argument about wanting > to > > > have fewer WPR-ed pages to avoid this issue. Although the right solution > here > > is > > > to rely less on WPR. > > > > > > 2) Chrome crashes under WPR-ed pages (your claim). This is huge! If we're > > > catching real Chrome crashes that are making it past the Chrome CQ, we > should > > > find a set of WPR pages to add to the CQ. I'm sure everyone would want to > see > > us > > > catch more crashes before they make it live. > > > > Yes. We already enable running all the system health stories on CQ and catch a > > bunch of Chrome crashes with those. Examples: > > https://bugs.chromium.org/p/chromium/issues/detail?id=624474 > > https://bugs.chromium.org/p/chromium/issues/detail?id=624587 > > https://bugs.chromium.org/p/chromium/issues/detail?id=624701 > > https://bugs.chromium.org/p/chromium/issues/detail?id=612135 > > https://bugs.chromium.org/p/chromium/issues/detail?id=637230 > > ... > > > > I would want to enable more tests but stip@ from infra team is having some > > concern with telemetry_perf_unittests currently is taking most of resources in > > android_chromium_rel_ng. +__+ > > I will advocate for more tests anyway, but we need to adjust the number of > tests > > we enable with android infra's capacity. > It's a good thing these tests will only be running on Mac for now. > > I went through the 5 bugs you posted - 3 of them were chrome crashes, and 2 were > webpage load timeouts (guessing WPR failure?) Ok. Let's name these stories: busy_idle_loading_stories.py?
On 2016/08/25 15:19:49, nednguyen wrote: > On 2016/08/25 15:02:03, erikchen wrote: > > On 2016/08/25 01:28:40, nednguyen wrote: > > > On 2016/08/25 01:01:51, erikchen wrote: > > > > > I think this correlates well with whether the page is idle after load. I > > > would > > > > > expect > > > > > pages with high idle CPU usage to have high TimeToInteractive metrics > > values > > > > as > > > > > well. > > > > > > > > I do not have this expectation. From everything we've seen so far, most > > pages > > > > with high idle CPU usage are due to too many Timer firings, excessive > > > relayouts, > > > > etc. See the "Potential Areas for Investigation" section under > > > > > > > > > > https://docs.google.com/document/d/1QlX7BJDjDmHH2BuysnwYA5QqqGOW24bj6OpQzqWHF... > > > > > > Wont't these tasks run on the main threads? If the main threads is quiet, we > > > consider the page to be interactive. (see more on > > > https://github.com/tdresser/time-to-interactive/blob/master/README.md) > > > > > > To be clear, my main question here is: > > > "if a page exhibit interesting power problems around loading period, would > > that > > > page also be interesting for loading & v8 to track their performance?" > > > > > > If we believe the nature of these pages are different, I am fine with not > > > sharing the pages but we should discuss first. > > > > > > > > > > > > > > > The main cost of maintenance is triaging/fixing bugs when Chrome crashes > > > when > > > > it > > > > > to load these pages. For an ideas of how often that happens, > > > > > you can look at the number of blocking bugs in > > > > > https://bugs.chromium.org/p/chromium/issues/detail?id=589726 > > > > > > > > I started going through the blocking bugs in > > > > https://bugs.chromium.org/p/chromium/issues/detail?id=589726. I'm failing > to > > > see > > > > any that are caused by Chrome crashes associated with loading pages from > > WPR. > > > > Could you point out some specific examples? > > > > > > > > There are several possibilities that I see: > > > > > > > > 1) Pages fail to load/quiesce/whatever under WPR. (I've seen this problem > > > > before). This is a serious issue, and I could see an argument about > wanting > > to > > > > have fewer WPR-ed pages to avoid this issue. Although the right solution > > here > > > is > > > > to rely less on WPR. > > > > > > > > 2) Chrome crashes under WPR-ed pages (your claim). This is huge! If we're > > > > catching real Chrome crashes that are making it past the Chrome CQ, we > > should > > > > find a set of WPR pages to add to the CQ. I'm sure everyone would want to > > see > > > us > > > > catch more crashes before they make it live. > > > > > > Yes. We already enable running all the system health stories on CQ and catch > a > > > bunch of Chrome crashes with those. Examples: > > > https://bugs.chromium.org/p/chromium/issues/detail?id=624474 > > > https://bugs.chromium.org/p/chromium/issues/detail?id=624587 > > > https://bugs.chromium.org/p/chromium/issues/detail?id=624701 > > > https://bugs.chromium.org/p/chromium/issues/detail?id=612135 > > > https://bugs.chromium.org/p/chromium/issues/detail?id=637230 > > > ... > > > > > > I would want to enable more tests but stip@ from infra team is having some > > > concern with telemetry_perf_unittests currently is taking most of resources > in > > > android_chromium_rel_ng. +__+ > > > I will advocate for more tests anyway, but we need to adjust the number of > > tests > > > we enable with android infra's capacity. > > It's a good thing these tests will only be running on Mac for now. > > > > I went through the 5 bugs you posted - 3 of them were chrome crashes, and 2 > were > > webpage load timeouts (guessing WPR failure?) > > Ok. Let's name these stories: busy_idle_loading_stories.py? There are many names I am okay with but this is not one of them. Busy and Idle have the exact opposite meaning. And the whole point is that there is no loading involved at all. If you don't like the term "steady state", we can go with "idle_stories.py". Note that the fact that Chrome has high CPU usage on these sites is almost certainly a chrome bug, rather than a feature of the pages themselves. Once these bugs are fixed, then they will just be "idle pages"
On 2016/08/25 15:22:07, erikchen wrote: > On 2016/08/25 15:19:49, nednguyen wrote: > > On 2016/08/25 15:02:03, erikchen wrote: > > > On 2016/08/25 01:28:40, nednguyen wrote: > > > > On 2016/08/25 01:01:51, erikchen wrote: > > > > > > I think this correlates well with whether the page is idle after load. > I > > > > would > > > > > > expect > > > > > > pages with high idle CPU usage to have high TimeToInteractive metrics > > > values > > > > > as > > > > > > well. > > > > > > > > > > I do not have this expectation. From everything we've seen so far, most > > > pages > > > > > with high idle CPU usage are due to too many Timer firings, excessive > > > > relayouts, > > > > > etc. See the "Potential Areas for Investigation" section under > > > > > > > > > > > > > > > https://docs.google.com/document/d/1QlX7BJDjDmHH2BuysnwYA5QqqGOW24bj6OpQzqWHF... > > > > > > > > Wont't these tasks run on the main threads? If the main threads is quiet, > we > > > > consider the page to be interactive. (see more on > > > > https://github.com/tdresser/time-to-interactive/blob/master/README.md) > > > > > > > > To be clear, my main question here is: > > > > "if a page exhibit interesting power problems around loading period, would > > > that > > > > page also be interesting for loading & v8 to track their performance?" > > > > > > > > If we believe the nature of these pages are different, I am fine with not > > > > sharing the pages but we should discuss first. > > > > > > > > > > > > > > > > > > > The main cost of maintenance is triaging/fixing bugs when Chrome > crashes > > > > when > > > > > it > > > > > > to load these pages. For an ideas of how often that happens, > > > > > > you can look at the number of blocking bugs in > > > > > > https://bugs.chromium.org/p/chromium/issues/detail?id=589726 > > > > > > > > > > I started going through the blocking bugs in > > > > > https://bugs.chromium.org/p/chromium/issues/detail?id=589726. I'm > failing > > to > > > > see > > > > > any that are caused by Chrome crashes associated with loading pages from > > > WPR. > > > > > Could you point out some specific examples? > > > > > > > > > > There are several possibilities that I see: > > > > > > > > > > 1) Pages fail to load/quiesce/whatever under WPR. (I've seen this > problem > > > > > before). This is a serious issue, and I could see an argument about > > wanting > > > to > > > > > have fewer WPR-ed pages to avoid this issue. Although the right solution > > > here > > > > is > > > > > to rely less on WPR. > > > > > > > > > > 2) Chrome crashes under WPR-ed pages (your claim). This is huge! If > we're > > > > > catching real Chrome crashes that are making it past the Chrome CQ, we > > > should > > > > > find a set of WPR pages to add to the CQ. I'm sure everyone would want > to > > > see > > > > us > > > > > catch more crashes before they make it live. > > > > > > > > Yes. We already enable running all the system health stories on CQ and > catch > > a > > > > bunch of Chrome crashes with those. Examples: > > > > https://bugs.chromium.org/p/chromium/issues/detail?id=624474 > > > > https://bugs.chromium.org/p/chromium/issues/detail?id=624587 > > > > https://bugs.chromium.org/p/chromium/issues/detail?id=624701 > > > > https://bugs.chromium.org/p/chromium/issues/detail?id=612135 > > > > https://bugs.chromium.org/p/chromium/issues/detail?id=637230 > > > > ... > > > > > > > > I would want to enable more tests but stip@ from infra team is having some > > > > concern with telemetry_perf_unittests currently is taking most of > resources > > in > > > > android_chromium_rel_ng. +__+ > > > > I will advocate for more tests anyway, but we need to adjust the number of > > > tests > > > > we enable with android infra's capacity. > > > It's a good thing these tests will only be running on Mac for now. > > > > > > I went through the 5 bugs you posted - 3 of them were chrome crashes, and 2 > > were > > > webpage load timeouts (guessing WPR failure?) > > > > Ok. Let's name these stories: busy_idle_loading_stories.py? > > There are many names I am okay with but this is not one of them. Busy and Idle > have the exact opposite meaning. And the whole point is that there is no loading > involved at all. If you don't like the term "steady state", we can go with > "idle_stories.py". Note that the fact that Chrome has high CPU usage on these > sites is almost certainly a chrome bug, rather than a feature of the pages > themselves. Once these bugs are fixed, then they will just be "idle pages" sgtm.
On 2016/08/25 15:25:05, nednguyen wrote: > On 2016/08/25 15:22:07, erikchen wrote: > > On 2016/08/25 15:19:49, nednguyen wrote: > > > On 2016/08/25 15:02:03, erikchen wrote: > > > > On 2016/08/25 01:28:40, nednguyen wrote: > > > > > On 2016/08/25 01:01:51, erikchen wrote: > > > > > > > I think this correlates well with whether the page is idle after > load. > > I > > > > > would > > > > > > > expect > > > > > > > pages with high idle CPU usage to have high TimeToInteractive > metrics > > > > values > > > > > > as > > > > > > > well. > > > > > > > > > > > > I do not have this expectation. From everything we've seen so far, > most > > > > pages > > > > > > with high idle CPU usage are due to too many Timer firings, excessive > > > > > relayouts, > > > > > > etc. See the "Potential Areas for Investigation" section under > > > > > > > > > > > > > > > > > > > > > https://docs.google.com/document/d/1QlX7BJDjDmHH2BuysnwYA5QqqGOW24bj6OpQzqWHF... > > > > > > > > > > Wont't these tasks run on the main threads? If the main threads is > quiet, > > we > > > > > consider the page to be interactive. (see more on > > > > > https://github.com/tdresser/time-to-interactive/blob/master/README.md) > > > > > > > > > > To be clear, my main question here is: > > > > > "if a page exhibit interesting power problems around loading period, > would > > > > that > > > > > page also be interesting for loading & v8 to track their performance?" > > > > > > > > > > If we believe the nature of these pages are different, I am fine with > not > > > > > sharing the pages but we should discuss first. > > > > > > > > > > > > > > > > > > > > > > > The main cost of maintenance is triaging/fixing bugs when Chrome > > crashes > > > > > when > > > > > > it > > > > > > > to load these pages. For an ideas of how often that happens, > > > > > > > you can look at the number of blocking bugs in > > > > > > > https://bugs.chromium.org/p/chromium/issues/detail?id=589726 > > > > > > > > > > > > I started going through the blocking bugs in > > > > > > https://bugs.chromium.org/p/chromium/issues/detail?id=589726. I'm > > failing > > > to > > > > > see > > > > > > any that are caused by Chrome crashes associated with loading pages > from > > > > WPR. > > > > > > Could you point out some specific examples? > > > > > > > > > > > > There are several possibilities that I see: > > > > > > > > > > > > 1) Pages fail to load/quiesce/whatever under WPR. (I've seen this > > problem > > > > > > before). This is a serious issue, and I could see an argument about > > > wanting > > > > to > > > > > > have fewer WPR-ed pages to avoid this issue. Although the right > solution > > > > here > > > > > is > > > > > > to rely less on WPR. > > > > > > > > > > > > 2) Chrome crashes under WPR-ed pages (your claim). This is huge! If > > we're > > > > > > catching real Chrome crashes that are making it past the Chrome CQ, we > > > > should > > > > > > find a set of WPR pages to add to the CQ. I'm sure everyone would want > > to > > > > see > > > > > us > > > > > > catch more crashes before they make it live. > > > > > > > > > > Yes. We already enable running all the system health stories on CQ and > > catch > > > a > > > > > bunch of Chrome crashes with those. Examples: > > > > > https://bugs.chromium.org/p/chromium/issues/detail?id=624474 > > > > > https://bugs.chromium.org/p/chromium/issues/detail?id=624587 > > > > > https://bugs.chromium.org/p/chromium/issues/detail?id=624701 > > > > > https://bugs.chromium.org/p/chromium/issues/detail?id=612135 > > > > > https://bugs.chromium.org/p/chromium/issues/detail?id=637230 > > > > > ... > > > > > > > > > > I would want to enable more tests but stip@ from infra team is having > some > > > > > concern with telemetry_perf_unittests currently is taking most of > > resources > > > in > > > > > android_chromium_rel_ng. +__+ > > > > > I will advocate for more tests anyway, but we need to adjust the number > of > > > > tests > > > > > we enable with android infra's capacity. > > > > It's a good thing these tests will only be running on Mac for now. > > > > > > > > I went through the 5 bugs you posted - 3 of them were chrome crashes, and > 2 > > > were > > > > webpage load timeouts (guessing WPR failure?) > > > > > > Ok. Let's name these stories: busy_idle_loading_stories.py? > > > > There are many names I am okay with but this is not one of them. Busy and Idle > > have the exact opposite meaning. And the whole point is that there is no > loading > > involved at all. If you don't like the term "steady state", we can go with > > "idle_stories.py". Note that the fact that Chrome has high CPU usage on these > > sites is almost certainly a chrome bug, rather than a feature of the pages > > themselves. Once these bugs are fixed, then they will just be "idle pages" > > sgtm. sgtm to the current name or idle_stories.py?
On 2016/08/25 15:28:30, erikchen wrote: > On 2016/08/25 15:25:05, nednguyen wrote: > > On 2016/08/25 15:22:07, erikchen wrote: > > > On 2016/08/25 15:19:49, nednguyen wrote: > > > > On 2016/08/25 15:02:03, erikchen wrote: > > > > > On 2016/08/25 01:28:40, nednguyen wrote: > > > > > > On 2016/08/25 01:01:51, erikchen wrote: > > > > > > > > I think this correlates well with whether the page is idle after > > load. > > > I > > > > > > would > > > > > > > > expect > > > > > > > > pages with high idle CPU usage to have high TimeToInteractive > > metrics > > > > > values > > > > > > > as > > > > > > > > well. > > > > > > > > > > > > > > I do not have this expectation. From everything we've seen so far, > > most > > > > > pages > > > > > > > with high idle CPU usage are due to too many Timer firings, > excessive > > > > > > relayouts, > > > > > > > etc. See the "Potential Areas for Investigation" section under > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://docs.google.com/document/d/1QlX7BJDjDmHH2BuysnwYA5QqqGOW24bj6OpQzqWHF... > > > > > > > > > > > > Wont't these tasks run on the main threads? If the main threads is > > quiet, > > > we > > > > > > consider the page to be interactive. (see more on > > > > > > https://github.com/tdresser/time-to-interactive/blob/master/README.md) > > > > > > > > > > > > To be clear, my main question here is: > > > > > > "if a page exhibit interesting power problems around loading period, > > would > > > > > that > > > > > > page also be interesting for loading & v8 to track their performance?" > > > > > > > > > > > > If we believe the nature of these pages are different, I am fine with > > not > > > > > > sharing the pages but we should discuss first. > > > > > > > > > > > > > > > > > > > > > > > > > > > The main cost of maintenance is triaging/fixing bugs when Chrome > > > crashes > > > > > > when > > > > > > > it > > > > > > > > to load these pages. For an ideas of how often that happens, > > > > > > > > you can look at the number of blocking bugs in > > > > > > > > https://bugs.chromium.org/p/chromium/issues/detail?id=589726 > > > > > > > > > > > > > > I started going through the blocking bugs in > > > > > > > https://bugs.chromium.org/p/chromium/issues/detail?id=589726. I'm > > > failing > > > > to > > > > > > see > > > > > > > any that are caused by Chrome crashes associated with loading pages > > from > > > > > WPR. > > > > > > > Could you point out some specific examples? > > > > > > > > > > > > > > There are several possibilities that I see: > > > > > > > > > > > > > > 1) Pages fail to load/quiesce/whatever under WPR. (I've seen this > > > problem > > > > > > > before). This is a serious issue, and I could see an argument about > > > > wanting > > > > > to > > > > > > > have fewer WPR-ed pages to avoid this issue. Although the right > > solution > > > > > here > > > > > > is > > > > > > > to rely less on WPR. > > > > > > > > > > > > > > 2) Chrome crashes under WPR-ed pages (your claim). This is huge! If > > > we're > > > > > > > catching real Chrome crashes that are making it past the Chrome CQ, > we > > > > > should > > > > > > > find a set of WPR pages to add to the CQ. I'm sure everyone would > want > > > to > > > > > see > > > > > > us > > > > > > > catch more crashes before they make it live. > > > > > > > > > > > > Yes. We already enable running all the system health stories on CQ and > > > catch > > > > a > > > > > > bunch of Chrome crashes with those. Examples: > > > > > > https://bugs.chromium.org/p/chromium/issues/detail?id=624474 > > > > > > https://bugs.chromium.org/p/chromium/issues/detail?id=624587 > > > > > > https://bugs.chromium.org/p/chromium/issues/detail?id=624701 > > > > > > https://bugs.chromium.org/p/chromium/issues/detail?id=612135 > > > > > > https://bugs.chromium.org/p/chromium/issues/detail?id=637230 > > > > > > ... > > > > > > > > > > > > I would want to enable more tests but stip@ from infra team is having > > some > > > > > > concern with telemetry_perf_unittests currently is taking most of > > > resources > > > > in > > > > > > android_chromium_rel_ng. +__+ > > > > > > I will advocate for more tests anyway, but we need to adjust the > number > > of > > > > > tests > > > > > > we enable with android infra's capacity. > > > > > It's a good thing these tests will only be running on Mac for now. > > > > > > > > > > I went through the 5 bugs you posted - 3 of them were chrome crashes, > and > > 2 > > > > were > > > > > webpage load timeouts (guessing WPR failure?) > > > > > > > > Ok. Let's name these stories: busy_idle_loading_stories.py? > > > > > > There are many names I am okay with but this is not one of them. Busy and > Idle > > > have the exact opposite meaning. And the whole point is that there is no > > loading > > > involved at all. If you don't like the term "steady state", we can go with > > > "idle_stories.py". Note that the fact that Chrome has high CPU usage on > these > > > sites is almost certainly a chrome bug, rather than a feature of the pages > > > themselves. Once these bugs are fixed, then they will just be "idle pages" > > > > sgtm. > > sgtm to the current name or idle_stories.py? as per discussion, going with idle_after_loading_stories.py
On 2016/08/25 15:28:30, erikchen wrote: > On 2016/08/25 15:25:05, nednguyen wrote: > > On 2016/08/25 15:22:07, erikchen wrote: > > > On 2016/08/25 15:19:49, nednguyen wrote: > > > > On 2016/08/25 15:02:03, erikchen wrote: > > > > > On 2016/08/25 01:28:40, nednguyen wrote: > > > > > > On 2016/08/25 01:01:51, erikchen wrote: > > > > > > > > I think this correlates well with whether the page is idle after > > load. > > > I > > > > > > would > > > > > > > > expect > > > > > > > > pages with high idle CPU usage to have high TimeToInteractive > > metrics > > > > > values > > > > > > > as > > > > > > > > well. > > > > > > > > > > > > > > I do not have this expectation. From everything we've seen so far, > > most > > > > > pages > > > > > > > with high idle CPU usage are due to too many Timer firings, > excessive > > > > > > relayouts, > > > > > > > etc. See the "Potential Areas for Investigation" section under > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://docs.google.com/document/d/1QlX7BJDjDmHH2BuysnwYA5QqqGOW24bj6OpQzqWHF... > > > > > > > > > > > > Wont't these tasks run on the main threads? If the main threads is > > quiet, > > > we > > > > > > consider the page to be interactive. (see more on > > > > > > https://github.com/tdresser/time-to-interactive/blob/master/README.md) > > > > > > > > > > > > To be clear, my main question here is: > > > > > > "if a page exhibit interesting power problems around loading period, > > would > > > > > that > > > > > > page also be interesting for loading & v8 to track their performance?" > > > > > > > > > > > > If we believe the nature of these pages are different, I am fine with > > not > > > > > > sharing the pages but we should discuss first. > > > > > > > > > > > > > > > > > > > > > > > > > > > The main cost of maintenance is triaging/fixing bugs when Chrome > > > crashes > > > > > > when > > > > > > > it > > > > > > > > to load these pages. For an ideas of how often that happens, > > > > > > > > you can look at the number of blocking bugs in > > > > > > > > https://bugs.chromium.org/p/chromium/issues/detail?id=589726 > > > > > > > > > > > > > > I started going through the blocking bugs in > > > > > > > https://bugs.chromium.org/p/chromium/issues/detail?id=589726. I'm > > > failing > > > > to > > > > > > see > > > > > > > any that are caused by Chrome crashes associated with loading pages > > from > > > > > WPR. > > > > > > > Could you point out some specific examples? > > > > > > > > > > > > > > There are several possibilities that I see: > > > > > > > > > > > > > > 1) Pages fail to load/quiesce/whatever under WPR. (I've seen this > > > problem > > > > > > > before). This is a serious issue, and I could see an argument about > > > > wanting > > > > > to > > > > > > > have fewer WPR-ed pages to avoid this issue. Although the right > > solution > > > > > here > > > > > > is > > > > > > > to rely less on WPR. > > > > > > > > > > > > > > 2) Chrome crashes under WPR-ed pages (your claim). This is huge! If > > > we're > > > > > > > catching real Chrome crashes that are making it past the Chrome CQ, > we > > > > > should > > > > > > > find a set of WPR pages to add to the CQ. I'm sure everyone would > want > > > to > > > > > see > > > > > > us > > > > > > > catch more crashes before they make it live. > > > > > > > > > > > > Yes. We already enable running all the system health stories on CQ and > > > catch > > > > a > > > > > > bunch of Chrome crashes with those. Examples: > > > > > > https://bugs.chromium.org/p/chromium/issues/detail?id=624474 > > > > > > https://bugs.chromium.org/p/chromium/issues/detail?id=624587 > > > > > > https://bugs.chromium.org/p/chromium/issues/detail?id=624701 > > > > > > https://bugs.chromium.org/p/chromium/issues/detail?id=612135 > > > > > > https://bugs.chromium.org/p/chromium/issues/detail?id=637230 > > > > > > ... > > > > > > > > > > > > I would want to enable more tests but stip@ from infra team is having > > some > > > > > > concern with telemetry_perf_unittests currently is taking most of > > > resources > > > > in > > > > > > android_chromium_rel_ng. +__+ > > > > > > I will advocate for more tests anyway, but we need to adjust the > number > > of > > > > > tests > > > > > > we enable with android infra's capacity. > > > > > It's a good thing these tests will only be running on Mac for now. > > > > > > > > > > I went through the 5 bugs you posted - 3 of them were chrome crashes, > and > > 2 > > > > were > > > > > webpage load timeouts (guessing WPR failure?) > > > > > > > > Ok. Let's name these stories: busy_idle_loading_stories.py? > > > > > > There are many names I am okay with but this is not one of them. Busy and > Idle > > > have the exact opposite meaning. And the whole point is that there is no > > loading > > > involved at all. If you don't like the term "steady state", we can go with > > > "idle_stories.py". Note that the fact that Chrome has high CPU usage on > these > > > sites is almost certainly a chrome bug, rather than a feature of the pages > > > themselves. Once these bugs are fixed, then they will just be "idle pages" > > > > sgtm. > > sgtm to the current name or idle_stories.py? Chatted offline, we both agree on idle_after_loading_stories.py
nednguyen: PTAL
lgtm
https://codereview.chromium.org/2271793003/diff/40001/tools/perf/page_sets/id... File tools/perf/page_sets/idle_after_loading_stories.py (right): https://codereview.chromium.org/2271793003/diff/40001/tools/perf/page_sets/id... tools/perf/page_sets/idle_after_loading_stories.py:17: 'http://www.labradortraininghq.com/labrador-training/how-to-crate-train' nits: can you add comment or bug link about why this URL is interesting?
https://codereview.chromium.org/2271793003/diff/40001/tools/perf/page_sets/id... File tools/perf/page_sets/idle_after_loading_stories.py (right): https://codereview.chromium.org/2271793003/diff/40001/tools/perf/page_sets/id... tools/perf/page_sets/idle_after_loading_stories.py:17: 'http://www.labradortraininghq.com/labrador-training/how-to-crate-train' On 2016/08/27 00:10:38, nednguyen (ooo til 8-29) wrote: > nits: can you add comment or bug link about why this URL is interesting? Done.
The CQ bit was checked by erikchen@chromium.org
The patchset sent to the CQ was uploaded after l-g-t-m from nednguyen@google.com Link to the patchset: https://codereview.chromium.org/2271793003/#ps60001 (title: "Add comment.")
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Try jobs failed on following builders: winx64_10_perf_cq on master.tryserver.chromium.perf (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64_10_perf_c...)
The CQ bit was checked by erikchen@chromium.org
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
lgtm
The CQ bit was unchecked by commit-bot@chromium.org
Try jobs failed on following builders: android_s5_perf_cq on master.tryserver.chromium.perf (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.perf/builders/android_s5_perf_...)
The CQ bit was checked by erikchen@chromium.org
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Exceeded global retry quota
The CQ bit was checked by erikchen@chromium.org
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
Message was sent while issue was closed.
Description was changed from ========== Add a new benchmark for cpu power measurements on steady state sites. The site in question is known to have high CPU usage, even though there appears to be no visual changes to the site. BUG=640398, 638365 CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.perf:android_s5_perf_cq;master.tryserver.chromium.perf:linux_perf_cq;master.tryserver.chromium.perf:mac_retina_perf_cq;master.tryserver.chromium.perf:winx64_10_perf_cq ========== to ========== Add a new benchmark for cpu power measurements on steady state sites. The site in question is known to have high CPU usage, even though there appears to be no visual changes to the site. BUG=640398, 638365 CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.perf:android_s5_perf_cq;master.tryserver.chromium.perf:linux_perf_cq;master.tryserver.chromium.perf:mac_retina_perf_cq;master.tryserver.chromium.perf:winx64_10_perf_cq ==========
Message was sent while issue was closed.
Committed patchset #4 (id:60001)
Message was sent while issue was closed.
Description was changed from ========== Add a new benchmark for cpu power measurements on steady state sites. The site in question is known to have high CPU usage, even though there appears to be no visual changes to the site. BUG=640398, 638365 CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.perf:android_s5_perf_cq;master.tryserver.chromium.perf:linux_perf_cq;master.tryserver.chromium.perf:mac_retina_perf_cq;master.tryserver.chromium.perf:winx64_10_perf_cq ========== to ========== Add a new benchmark for cpu power measurements on steady state sites. The site in question is known to have high CPU usage, even though there appears to be no visual changes to the site. BUG=640398, 638365 CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.perf:android_s5_perf_cq;master.tryserver.chromium.perf:linux_perf_cq;master.tryserver.chromium.perf:mac_retina_perf_cq;master.tryserver.chromium.perf:winx64_10_perf_cq Committed: https://crrev.com/9c3c0a38de1c83ac9e6a9f683bad1291e12e2885 Cr-Commit-Position: refs/heads/master@{#415654} ==========
Message was sent while issue was closed.
Patchset 4 (id:??) landed as https://crrev.com/9c3c0a38de1c83ac9e6a9f683bad1291e12e2885 Cr-Commit-Position: refs/heads/master@{#415654} |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
