[Telemetry] Power metric: subtract quiescent power draw from result

[Telemetry] Power metric: subtract quiescent power draw from result

Some power measurement backends can only measure global power draw for the whole system and not per-process. Try to get a more exact power measurement by measuring the power draw without the browser running and then subtracting that number from the measurement in the test to try to get a more exact number for how much additional power is being used by the test case.

BUG=339180
Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=280994

[jeremy, 2014-06-25 08:27:23]

Initially just try this with the tab_switching measurement, if this improve
results then we can enable it for more tests.

For other tests where we don't know the amount of time the test is going to take
I was thinking of measuring quiescent power draw for a set amount of time and
scaling that value up for the measurement duration.

In addition the power measurement backends need to be able to report whether
they provide per-process or global measurements but I didn't want to make that
change until this CL landed and we could tell if this approach actually works.

WDYT?

[qsr, 2014-06-25 10:41:23]

On 2014/06/25 08:27:23, jeremy wrote:
> Initially just try this with the tab_switching measurement, if this improve
> results then we can enable it for more tests.
> 
> For other tests where we don't know the amount of time the test is going to
take
> I was thinking of measuring quiescent power draw for a set amount of time and
> scaling that value up for the measurement duration.
> 
> In addition the power measurement backends need to be able to report whether
> they provide per-process or global measurements but I didn't want to make that
> change until this CL landed and we could tell if this approach actually works.
> 
> WDYT?

 For this case, that seems fine, as it was already a long test, and you made so
that it still mostly takes the same time. But doubling the time of all tests
seems not optimal to me...

[jeremy, 2014-06-25 11:05:04]

Thanks, for other tests the thought I had was to measure for a small period
of time and then extrapolate from that number.

Could you please review this change as is?


On Wed, Jun 25, 2014 at 1:41 PM, <qsr@chromium.org> wrote:

> On 2014/06/25 08:27:23, jeremy wrote:
>
>> Initially just try this with the tab_switching measurement, if this
>> improve
>> results then we can enable it for more tests.
>>
>
>  For other tests where we don't know the amount of time the test is going
>> to
>>
> take
>
>> I was thinking of measuring quiescent power draw for a set amount of time
>> and
>> scaling that value up for the measurement duration.
>>
>
>  In addition the power measurement backends need to be able to report
>> whether
>> they provide per-process or global measurements but I didn't want to make
>> that
>> change until this CL landed and we could tell if this approach actually
>> works.
>>
>
>  WDYT?
>>
>
>  For this case, that seems fine, as it was already a long test, and you
> made so
> that it still mostly takes the same time. But doubling the time of all
> tests
> seems not optimal to me...
>
> https://codereview.chromium.org/350763005/
>

To unsubscribe from this group and stop receiving emails from it, send an email
to chromium-reviews+unsubscribe@chromium.org.

[qsr, 2014-06-25 11:10:26]

LGTM, but each time I see the number of change we have to made because we need
to cut and paste the same thing on all test suite, I'm thinking we are doing
something wrong.

[jeremy, 2014-06-25 11:15:04]

On 2014/06/25 11:10:26, qsr wrote:
> LGTM, but each time I see the number of change we have to made because we need
> to cut and paste the same thing on all test suite, I'm thinking we are doing
> something wrong.

That bugs me too :/ the only way I can think of around this is to have
measurements register with a test and have callbacks called directly by the base
class.  The problem with that is that currently the constructors for some tests
do a non-trivial amount of work and we need to explicitly manage order of
intialization/measurement e.g. the memory measurement does a bunch of processing
which we probably don't want to measure within the power measurement.

If you have some ideas on how to structure this better I think it's definitely
worth a chat with dtu@ or tonyg@.

[qsr, 2014-06-25 11:16:15]

On 2014/06/25 11:15:04, jeremy wrote:
> On 2014/06/25 11:10:26, qsr wrote:
> > LGTM, but each time I see the number of change we have to made because we
need
> > to cut and paste the same thing on all test suite, I'm thinking we are doing
> > something wrong.
> 
> That bugs me too :/ the only way I can think of around this is to have
> measurements register with a test and have callbacks called directly by the
base
> class.  The problem with that is that currently the constructors for some
tests
> do a non-trivial amount of work and we need to explicitly manage order of
> intialization/measurement e.g. the memory measurement does a bunch of
processing
> which we probably don't want to measure within the power measurement.
> 
> If you have some ideas on how to structure this better I think it's definitely
> worth a chat with dtu@ or tonyg@.

 I had this: https://chromiumcodereview.appspot.com/230163004/ but didn't pursue
at the time because I finally only needed a single change. We can restart from
there if there is interest.

[tonyg, 2014-06-26 16:19:05]

lgtm assuming you're okay w/ the suggested approach

https://codereview.chromium.org/350763005/diff/1/tools/perf/metrics/power.py
File tools/perf/metrics/power.py (right):

https://codereview.chromium.org/350763005/diff/1/tools/perf/metrics/power.py#...
tools/perf/metrics/power.py:100: results.AddValue(scalar.ScalarValue(
WDYT about just reporting two things:

energy_consumption_mwh: Remains unchanged and reports full system energy
consumption (we should document this in the API, as some of the providers I
think are trying to report just chrome here).
application_energy_consumption_mwh: Add this new metric which aims to estimate
just the target application (Chrome's) energy consumption. Some providers may
pass this directly in the platform API. The logic in this metric would be to
compute it if it is not given directly.

[dtu, 2014-06-26 23:33:03]

On 2014/06/26 16:19:05, tonyg wrote:
> lgtm assuming you're okay w/ the suggested approach
> 
> https://codereview.chromium.org/350763005/diff/1/tools/perf/metrics/power.py
> File tools/perf/metrics/power.py (right):
> 
>
https://codereview.chromium.org/350763005/diff/1/tools/perf/metrics/power.py#...
> tools/perf/metrics/power.py:100: results.AddValue(scalar.ScalarValue(
> WDYT about just reporting two things:
> 
> energy_consumption_mwh: Remains unchanged and reports full system energy
> consumption (we should document this in the API, as some of the providers I
> think are trying to report just chrome here).
> application_energy_consumption_mwh: Add this new metric which aims to estimate
> just the target application (Chrome's) energy consumption. Some providers may
> pass this directly in the platform API. The logic in this metric would be to
> compute it if it is not given directly.

+1, makes sense for platforms that don't need to measure quiescent power.

[jeremy, 2014-06-30 08:58:22]

Tony/dtu: Could you take another look please?

I needed to add a CanMeasurePerApplicationPower() to power monitor so the
measurement can figure out whether it needs to do the quiescent power
measurement up front.

[qsr, 2014-07-01 09:02:24]

lgtm

https://codereview.chromium.org/350763005/diff/40001/tools/perf/measurements/...
File tools/perf/measurements/page_cycler.py (right):

https://codereview.chromium.org/350763005/diff/40001/tools/perf/measurements/...
tools/perf/measurements/page_cycler.py:85: """Initialize metrics once right
before the browser has been launched."""
Nit: Why is there a comment on 2 of them, and not on the others?

https://codereview.chromium.org/350763005/diff/40001/tools/perf/metrics/power.py
File tools/perf/metrics/power.py (right):

https://codereview.chromium.org/350763005/diff/40001/tools/perf/metrics/power...
tools/perf/metrics/power.py:20: in seconds."""
Maybe document that 0 means not doing it at all...

[jeremy, 2014-07-01 11:49:27]

The CQ bit was checked by jeremy@chromium.org

[commit-bot: I haz the power, 2014-07-01 11:50:20]

CQ is trying da patch. Follow status at
 https://chromium-status.appspot.com/cq/jeremy@chromium.org/350763005/60001

[commit-bot: I haz the power, 2014-07-01 13:01:56]

FYI, CQ is re-trying this CL (attempt #1).
The failing builders are:
  linux_chromium_rel on tryserver.chromium
(http://build.chromium.org/p/tryserver.chromium/builders/linux_chromium_rel/bu...)
  mac_chromium_rel on tryserver.chromium
(http://build.chromium.org/p/tryserver.chromium/builders/mac_chromium_rel/buil...)
  win_chromium_rel on tryserver.chromium
(http://build.chromium.org/p/tryserver.chromium/builders/win_chromium_rel/buil...)
  win_chromium_x64_rel on tryserver.chromium
(http://build.chromium.org/p/tryserver.chromium/builders/win_chromium_x64_rel/...)

[commit-bot: I haz the power, 2014-07-01 13:22:15]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2014-07-01 13:22:16]

Try jobs failed on following builders:
  linux_chromium_rel on tryserver.chromium
(http://build.chromium.org/p/tryserver.chromium/builders/linux_chromium_rel/bu...)

[jeremy, 2014-07-02 08:03:35]

The CQ bit was checked by jeremy@chromium.org

[commit-bot: I haz the power, 2014-07-02 08:04:53]

CQ is trying da patch. Follow status at
 https://chromium-status.appspot.com/cq/jeremy@chromium.org/350763005/80001

[commit-bot: I haz the power, 2014-07-02 10:11:32]

Change committed as 280994