Chromium Code Reviews
|
|
Chromium Code Reviews
Issue
2589173004:
Stop bad results on first memory benchmark run (Closed)
DescriptionStop bad results on first memory benchmark run
The first memory system-health story ran in a set has been reporting
memory use differently all subsequent runs. This is due to how we
flush the system caches.
Just before we measure memory we flush the system caches unfortunately
this doesn't immediately take effect, instead the next story run is
effected. Due to this the first story run has anomalous results.
This option causes us to flush caches each time before Chrome starts
so we effect even the first story - avoiding the bug.
*************** Note to Perf Sheriff ****************
Regressions across several memory metrics are
expected for system_health.memory_mobile,
system_health.memory_desktop and memory.top10_mobile
as we stop under counting memory use.
*****************************************************
BUG=chromium:671156
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.perf:linux_perf_cq;master.tryserver.chromium.perf:mac_retina_perf_cq
Review-Url: https://codereview.chromium.org/2589173004
Cr-Commit-Position: refs/heads/master@{#443273}
Committed: https://chromium.googlesource.com/chromium/src/+/3afb7e7da06df087b3e9fac2ae68593f75a051ed
Patch Set 1 #Patch Set 2 : Stop bad results on first memory benchmark run #Patch Set 3 : Fix extra newline. #Patch Set 4 : rebase #Patch Set 5 : Stop browse:social:twitter scrolling wrong element #Patch Set 6 : Update reviewers. #Patch Set 7 : Oops. #Patch Set 8 : rebase #
Messages
Total messages: 32 (17 generated)
Description was changed from ========== Stop bad results on first memory benchmark run The first memory system-health story ran in a set has been reporting memory use differently all subsequent runs. This is due to how we flush the system caches. Just before we measure memory we flush the system caches unfortunately this doesn't immediately take effect, instead the next story run is effected. Due to this the first story run has anomalous results. This option causes us to flush caches each time before Chrome starts so we effect even the first story - avoiding the bug. BUG=chromium:671156 ========== to ========== Stop bad results on first memory benchmark run The first memory system-health story ran in a set has been reporting memory use differently all subsequent runs. This is due to how we flush the system caches. Just before we measure memory we flush the system caches unfortunately this doesn't immediately take effect, instead the next story run is effected. Due to this the first story run has anomalous results. This option causes us to flush caches each time before Chrome starts so we effect even the first story - avoiding the bug. BUG=chromium:671156 CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.perf:linux_perf_cq;master.tryserver.chromium.perf:mac_retina_perf_cq;master.tryserver.chromium.perf:winx64_10_perf_cq ==========
Description was changed from ========== Stop bad results on first memory benchmark run The first memory system-health story ran in a set has been reporting memory use differently all subsequent runs. This is due to how we flush the system caches. Just before we measure memory we flush the system caches unfortunately this doesn't immediately take effect, instead the next story run is effected. Due to this the first story run has anomalous results. This option causes us to flush caches each time before Chrome starts so we effect even the first story - avoiding the bug. BUG=chromium:671156 CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.perf:linux_perf_cq;master.tryserver.chromium.perf:mac_retina_perf_cq;master.tryserver.chromium.perf:winx64_10_perf_cq ========== to ========== Stop bad results on first memory benchmark run The first memory system-health story ran in a set has been reporting memory use differently all subsequent runs. This is due to how we flush the system caches. Just before we measure memory we flush the system caches unfortunately this doesn't immediately take effect, instead the next story run is effected. Due to this the first story run has anomalous results. This option causes us to flush caches each time before Chrome starts so we effect even the first story - avoiding the bug. BUG=chromium:671156 CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.perf:linux_perf_cq;master.tryserver.chromium.perf:mac_retina_perf_cq;master.tryserver.chromium.perf:winx64_10_perf_cq ==========
hjd@chromium.org changed reviewers: + perezju@chromium.org
ptal, what do you think of enabling the option here? :)
this sounds good! could you try running: tools/perf/run_benchmark try android-nexus5 system_health.memory_mobile --pageset-repeat 5 on this patch? Also, could you apply the same change to _MemoryInfra class on: https://chromium.googlesource.com/chromium/src/+/master/tools/perf/benchmarks... And update the description of this CL with a warning to perf sheriffs. When it lands it's probably going to fire alerts all over the place.
Description was changed from ========== Stop bad results on first memory benchmark run The first memory system-health story ran in a set has been reporting memory use differently all subsequent runs. This is due to how we flush the system caches. Just before we measure memory we flush the system caches unfortunately this doesn't immediately take effect, instead the next story run is effected. Due to this the first story run has anomalous results. This option causes us to flush caches each time before Chrome starts so we effect even the first story - avoiding the bug. BUG=chromium:671156 CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.perf:linux_perf_cq;master.tryserver.chromium.perf:mac_retina_perf_cq;master.tryserver.chromium.perf:winx64_10_perf_cq ========== to ========== Stop bad results on first memory benchmark run The first memory system-health story ran in a set has been reporting memory use differently all subsequent runs. This is due to how we flush the system caches. Just before we measure memory we flush the system caches unfortunately this doesn't immediately take effect, instead the next story run is effected. Due to this the first story run has anomalous results. This option causes us to flush caches each time before Chrome starts so we effect even the first story - avoiding the bug. *************** Note to Perf Sheriff **************** Regressions across several memory metrics are expected for system_health.memory_mobile, system_health.memory_desktop and memory.top10_mobile as we stop under counting memory use. ***************************************************** BUG=chromium:671156 CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.perf:linux_perf_cq;master.tryserver.chromium.perf:mac_retina_perf_cq;master.tryserver.chromium.perf:winx64_10_perf_cq ==========
On 2016/12/21 13:07:49, perezju wrote: > this sounds good! could you try running: > > tools/perf/run_benchmark try android-nexus5 system_health.memory_mobile > --pageset-repeat 5 > > on this patch? > > Also, could you apply the same change to _MemoryInfra class on: > https://chromium.googlesource.com/chromium/src/+/master/tools/perf/benchmarks... Ahh I was wondering where top10 mobile was, thanks :) > And update the description of this CL with a warning to perf sheriffs. When it > lands it's probably going to fire alerts all over the place. Updated, ptal! :)
perezju@chromium.org changed reviewers: + nednguyen@google.com
looking good to me, but I would like to see the results of that try job before making a call. +Ned since we'll also need owners approval.
lgtm
The CQ bit was checked by hjd@chromium.org
The patchset sent to the CQ was uploaded after l-g-t-m from nednguyen@google.com Link to the patchset: https://codereview.chromium.org/2589173004/#ps120001 (title: "Oops.")
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Try jobs failed on following builders: android_n5x_swarming_rel on master.tryserver.chromium.android (JOB_FAILED, https://build.chromium.org/p/tryserver.chromium.android/builders/android_n5x_...)
The CQ bit was checked by hjd@chromium.org
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Try jobs failed on following builders: mac_retina_perf_cq on master.tryserver.chromium.perf (JOB_TIMED_OUT, http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_retina_perf_...)
The CQ bit was checked by hjd@chromium.org
The patchset sent to the CQ was uploaded after l-g-t-m from nednguyen@google.com Link to the patchset: https://codereview.chromium.org/2589173004/#ps140001 (title: "rebase")
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Try jobs failed on following builders: winx64_10_perf_cq on master.tryserver.chromium.perf (JOB_TIMED_OUT, http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64_10_perf_c...)
On 2017/01/12 16:20:29, commit-bot: I haz the power wrote: > Try jobs failed on following builders: > winx64_10_perf_cq on master.tryserver.chromium.perf (JOB_TIMED_OUT, > http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64_10_perf_c...) Failure is unrelated to this patch, so remove master.tryserver.chromium.perf:winx64_10_perf_cq
Description was changed from ========== Stop bad results on first memory benchmark run The first memory system-health story ran in a set has been reporting memory use differently all subsequent runs. This is due to how we flush the system caches. Just before we measure memory we flush the system caches unfortunately this doesn't immediately take effect, instead the next story run is effected. Due to this the first story run has anomalous results. This option causes us to flush caches each time before Chrome starts so we effect even the first story - avoiding the bug. *************** Note to Perf Sheriff **************** Regressions across several memory metrics are expected for system_health.memory_mobile, system_health.memory_desktop and memory.top10_mobile as we stop under counting memory use. ***************************************************** BUG=chromium:671156 CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.perf:linux_perf_cq;master.tryserver.chromium.perf:mac_retina_perf_cq;master.tryserver.chromium.perf:winx64_10_perf_cq ========== to ========== Stop bad results on first memory benchmark run The first memory system-health story ran in a set has been reporting memory use differently all subsequent runs. This is due to how we flush the system caches. Just before we measure memory we flush the system caches unfortunately this doesn't immediately take effect, instead the next story run is effected. Due to this the first story run has anomalous results. This option causes us to flush caches each time before Chrome starts so we effect even the first story - avoiding the bug. *************** Note to Perf Sheriff **************** Regressions across several memory metrics are expected for system_health.memory_mobile, system_health.memory_desktop and memory.top10_mobile as we stop under counting memory use. ***************************************************** BUG=chromium:671156 CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.perf:linux_perf_cq;master.tryserver.chromium.perf:mac_retina_perf_cq ==========
The CQ bit was checked by nednguyen@google.com
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
CQ is committing da patch.
Bot data: {"patchset_id": 140001, "attempt_start_ts": 1484239359177910,
"parent_rev": "ffa982507bece3662d3b48dd73f8b1c1b85999f2", "commit_rev":
"3afb7e7da06df087b3e9fac2ae68593f75a051ed"}
Message was sent while issue was closed.
Description was changed from ========== Stop bad results on first memory benchmark run The first memory system-health story ran in a set has been reporting memory use differently all subsequent runs. This is due to how we flush the system caches. Just before we measure memory we flush the system caches unfortunately this doesn't immediately take effect, instead the next story run is effected. Due to this the first story run has anomalous results. This option causes us to flush caches each time before Chrome starts so we effect even the first story - avoiding the bug. *************** Note to Perf Sheriff **************** Regressions across several memory metrics are expected for system_health.memory_mobile, system_health.memory_desktop and memory.top10_mobile as we stop under counting memory use. ***************************************************** BUG=chromium:671156 CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.perf:linux_perf_cq;master.tryserver.chromium.perf:mac_retina_perf_cq ========== to ========== Stop bad results on first memory benchmark run The first memory system-health story ran in a set has been reporting memory use differently all subsequent runs. This is due to how we flush the system caches. Just before we measure memory we flush the system caches unfortunately this doesn't immediately take effect, instead the next story run is effected. Due to this the first story run has anomalous results. This option causes us to flush caches each time before Chrome starts so we effect even the first story - avoiding the bug. *************** Note to Perf Sheriff **************** Regressions across several memory metrics are expected for system_health.memory_mobile, system_health.memory_desktop and memory.top10_mobile as we stop under counting memory use. ***************************************************** BUG=chromium:671156 CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.perf:linux_perf_cq;master.tryserver.chromium.perf:mac_retina_perf_cq Review-Url: https://codereview.chromium.org/2589173004 Cr-Commit-Position: refs/heads/master@{#443273} Committed: https://chromium.googlesource.com/chromium/src/+/3afb7e7da06df087b3e9fac2ae68... ==========
Message was sent while issue was closed.
Committed patchset #8 (id:140001) as https://chromium.googlesource.com/chromium/src/+/3afb7e7da06df087b3e9fac2ae68...
Message was sent while issue was closed.
A revert of this CL (patchset #8 id:140001) has been created in https://codereview.chromium.org/2627323002/ by xlai@chromium.org. The reason for reverting is: Suspecting that this CL is causing many failures on benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest. Example builds: https://build.chromium.org/p/chromium.mac/builders/Mac10.10%20Tests/builds/11962 https://build.chromium.org/p/chromium.mac/builders/Mac10.11%20Tests/builds/6323. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
