Add "Known unknowns" assessment criterium.

Add "Known unknowns" assessment criterium.

Severity and prevalence are often very hard to assess in practice --
many regressions follow an "iceberg" pattern where the initial minor bug
is just a hint of the true scope of the problem.  In such cases, coming
up with an objective assessment of severity and prevalence is very
difficult and it may be challenging to make the case for a blocking
label purely on the basis of observations.  However, developers often
have a good intuition whether a particular regression falls in one of
the categories of bugs prone to "icebergs", based on their past
experience with the subsystem, and it should be encouraged to mark a bug
ReleaseBlock when they harbor such a fear.

Also add a paragraph with some extra nuances about the role that
regressions play in the assessment criteria (in particular,
nonregressions have much fewer unknowns).

BUG=

[aelias_OOO_until_Jul13, 2017-06-26 23:57:40]

aelias@chromium.org changed reviewers:
+ amineer@chromium.org

[aelias_OOO_until_Jul13, 2017-06-26 23:58:32]

Hi amineer@, this is what I've been able to come up with for expressing my
concern about "iceberg" regressions.  Let me know if this makes sense to you.

[Dirk Pranke, 2017-06-30 01:20:49]

dpranke@chromium.org changed reviewers:
+ dpranke@chromium.org

[Dirk Pranke, 2017-06-30 01:20:49]

I'll leave the bulk of this review to amineer, but I'm not wild about the "known
unknowns" phrase; it has a particularly colorful set of connotations, and I'm
not actually sure that it captures what you're trying to get at, either.

What you've written sounds something more like uncertainty, confidence, or
riskiness; maybe one of those words works better?

[amineer, 2017-07-06 21:57:01]

On 2017/06/30 01:20:49, Dirk Pranke wrote:
> I'll leave the bulk of this review to amineer, but I'm not wild about the
"known
> unknowns" phrase; it has a particularly colorful set of connotations, and I'm
> not actually sure that it captures what you're trying to get at, either.
> 
> What you've written sounds something more like uncertainty, confidence, or
> riskiness; maybe one of those words works better?

+1 to Dirk's comment and rephrasing it to "Uncertainty"

[amineer, 2017-07-06 21:57:16]

https://codereview.chromium.org/2959883002/diff/1/docs/process/release_blocke...
File docs/process/release_blockers.md (right):

https://codereview.chromium.org/2959883002/diff/1/docs/process/release_blocke...
docs/process/release_blockers.md:90: ### Known unknowns
"Addressing Uncertainty"

https://codereview.chromium.org/2959883002/diff/1/docs/process/release_blocke...
docs/process/release_blockers.md:94: *   **High** - A recently introduced race
condition causing tabs to never load
I don't know that adding dimensions here (e.g. breaking things out to high /
medium / low) help too much, esp. since they may confuse users who try to slot
these definitions into the explicit table above (which does not mention this
factor).

Would you be OK to change this to a paragraph which basically states that the
more recently a bug has been introduced, the higher uncertainty we have in
assessing severity / prevalence accurately, which means it should likely be
labeled more conservatively (e.g. high instead of medium sev, some prevalence
instead of few, etc)?  We can point out the converse too, that anything that's
existed for a while likely has a lower severity / prevalence?

https://codereview.chromium.org/2959883002/diff/1/docs/process/release_blocke...
docs/process/release_blockers.md:166: the greater the odds that
severity/prevalence/unknowns are actually low.)
s/unknowns/uncertainty

[aelias_OOO_until_Jul13, 2017-07-20 21:28:17]

I closed this and reuploaded it as
https://chromium-review.googlesource.com/c/580293/