Add documentation on authoring metrics.

Add documentation on authoring metrics.

BUG=None

Review-Url: https://codereview.chromium.org/2973213002
Cr-Commit-Position: refs/heads/master@{#488299}
Committed: https://chromium.googlesource.com/chromium/src/+/2cda996dd6938ee9200abf28b391407ef9bfb2b3

[tdresser, 2017-07-07 15:35:58]

tdresser@chromium.org changed reviewers:
+ sullivan@chromium.org

[tdresser, 2017-07-07 15:35:59]

PTAL

Rendered html is here:

https://drive.google.com/file/d/0B7mjRvOU-oG-RkRqVmJBRlhnS1k/view?usp=sharing

https://drive.google.com/file/d/0B7mjRvOU-oG-cVVGU2YyQ0l6ZTg/view?usp=sharing

[sullivan, 2017-07-13 13:12:27]

lgtm

These docs are great!

Adding Ben, Simon, and Ethan: the types of diagnostics are an important thing to
think about for histogram diagnostics.

https://codereview.chromium.org/2973213002/diff/1/docs/speed/good_toplevel_me...
File docs/speed/good_toplevel_metrics.md (right):

https://codereview.chromium.org/2973213002/diff/1/docs/speed/good_toplevel_me...
docs/speed/good_toplevel_metrics.md:26: * Measure the error between the
implementation results and the hand picked results. Ideally, our error
measurement would be more forgiving in cases where humans were unsure of the
correct point in time. We don’t have a concrete plan here yet.
I really liked the analysis ksakamoto did here:
https://docs.google.com/document/d/1eMA9YWTLFCJomyiKUO4Be4PeVIOBFMfXMdE8bpudL...

Maybe link it?

Actually noticed the example below, it stands on its own.

[benjhayden, 2017-07-13 17:35:37]

benjhayden@chromium.org changed reviewers:
+ benjhayden@chromium.org

[benjhayden, 2017-07-13 17:35:38]

https://codereview.chromium.org/2973213002/diff/1/docs/speed/diagnostic_metri...
File docs/speed/diagnostic_metrics.md (right):

https://codereview.chromium.org/2973213002/diff/1/docs/speed/diagnostic_metri...
docs/speed/diagnostic_metrics.md:5: We want our metrics to be reflective of user
experience, so we know we’re optimizing for the right thing. However, metrics
which accurately reflect user experience are often so high level that they
aren’t very actionable. Diagnostic metrics are submetrics which enable us to act
on our high level user experience metrics. Also see the document on constructing
a [good toplevel metric](good_toplevel_metrics.md) for guidance on constructing
high quality user experience metrics.
Do you want to point readers to RelatedHistogramBreakdown for summations and
RelatedHistogramMap for slices and proxies for further reading?

https://codereview.chromium.org/2973213002/diff/1/docs/speed/diagnostic_metri...
docs/speed/diagnostic_metrics.md:17: Summations enable us to answer these
questions. In a Summation diagnostic, the diagnostic metrics sum up to the
higher level metric. For example, a Summation diagnostic for First Meaningful
Paint (FMP) might be the durations the main thread spent doing various tasks,
such as Style, Layout, V8, Idle, etc before FMP fired. These diagnostics often
to lead to hierarchies, where the top level metric, such as FMP, has a
diagnostic metric, such as time spent in V8 before FMP, which has further
diagnostic metrics, such as the time spent parsing, compiling, or executing JS.
"often to lead to"

[tdresser, 2017-07-20 16:16:38]

https://codereview.chromium.org/2973213002/diff/1/docs/speed/diagnostic_metri...
File docs/speed/diagnostic_metrics.md (right):

https://codereview.chromium.org/2973213002/diff/1/docs/speed/diagnostic_metri...
docs/speed/diagnostic_metrics.md:5: We want our metrics to be reflective of user
experience, so we know we’re optimizing for the right thing. However, metrics
which accurately reflect user experience are often so high level that they
aren’t very actionable. Diagnostic metrics are submetrics which enable us to act
on our high level user experience metrics. Also see the document on constructing
a [good toplevel metric](good_toplevel_metrics.md) for guidance on constructing
high quality user experience metrics.
On 2017/07/13 17:35:38, benjhayden wrote:
> Do you want to point readers to RelatedHistogramBreakdown for summations and
> RelatedHistogramMap for slices and proxies for further reading?

Done.

https://codereview.chromium.org/2973213002/diff/1/docs/speed/diagnostic_metri...
docs/speed/diagnostic_metrics.md:17: Summations enable us to answer these
questions. In a Summation diagnostic, the diagnostic metrics sum up to the
higher level metric. For example, a Summation diagnostic for First Meaningful
Paint (FMP) might be the durations the main thread spent doing various tasks,
such as Style, Layout, V8, Idle, etc before FMP fired. These diagnostics often
to lead to hierarchies, where the top level metric, such as FMP, has a
diagnostic metric, such as time spent in V8 before FMP, which has further
diagnostic metrics, such as the time spent parsing, compiling, or executing JS.
On 2017/07/13 17:35:38, benjhayden wrote:
> "often to lead to"

Done.

https://codereview.chromium.org/2973213002/diff/1/docs/speed/good_toplevel_me...
File docs/speed/good_toplevel_metrics.md (right):

https://codereview.chromium.org/2973213002/diff/1/docs/speed/good_toplevel_me...
docs/speed/good_toplevel_metrics.md:26: * Measure the error between the
implementation results and the hand picked results. Ideally, our error
measurement would be more forgiving in cases where humans were unsure of the
correct point in time. We don’t have a concrete plan here yet.
On 2017/07/13 13:12:27, sullivan wrote:
> I really liked the analysis ksakamoto did here:
>
https://docs.google.com/document/d/1eMA9YWTLFCJomyiKUO4Be4PeVIOBFMfXMdE8bpudL...
> 
> Maybe link it?
> 
> Actually noticed the example below, it stands on its own.

ksakamoto's results haven't been reproducible, so I hesitate to use it as an
example of what to do. Issues with that analysis resulted in a number of the
insights in this doc, such as requiring multiple people unfamiliar with the
metric to perform the hand identification.

[benjhayden, 2017-07-20 17:22:26]

lgtm

[tdresser, 2017-07-20 17:23:49]

The CQ bit was checked by tdresser@chromium.org

[tdresser, 2017-07-20 17:23:49]

The patchset sent to the CQ was uploaded after l-g-t-m from
sullivan@chromium.org
Link to the patchset: https://codereview.chromium.org/2973213002/#ps20001
(title: "Address feedback.")

[commit-bot: I haz the power, 2017-07-20 17:24:05]

CQ is trying da patch.

Follow status at:
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-07-20 17:51:58]

CQ is committing da patch.

Bot data: {"patchset_id": 20001, "attempt_start_ts": 1500571429285030,
"parent_rev": "af101146a113f5648a79f65e6e6cc48800bdc1e1", "commit_rev":
"2cda996dd6938ee9200abf28b391407ef9bfb2b3"}

[commit-bot: I haz the power, 2017-07-20 17:52:12]

Description was changed from

==========
Add documentation on authoring metrics.

BUG=None
==========

to

==========
Add documentation on authoring metrics.

BUG=None

Review-Url: https://codereview.chromium.org/2973213002
Cr-Commit-Position: refs/heads/master@{#488299}
Committed:
https://chromium.googlesource.com/chromium/src/+/2cda996dd6938ee9200abf28b391...
==========

[commit-bot: I haz the power, 2017-07-20 17:52:13]

Committed patchset #2 (id:20001) as
https://chromium.googlesource.com/chromium/src/+/2cda996dd6938ee9200abf28b391...