Add "Known unknowns" assessment criterium.

docs/process/release_blockers.md

 # Release Blockers
 
 [TOC]
 
 ## tl;dr
 
 *   Only mark bugs as blockers if the product **must not** be shipped with the
     bug present.
 *   **Everyone** on the team can add or remove blocking labels.
 *   Evaluate bugs as potential blockers based on their **severity** and
@@ skipping 69 matching lines @@
     downloading files, etc.
 *   **All (75% - 100%)**: The bug affects core product functionality, e.g.
     scrolling a page.
 
 Note that prevalence should be evaluated based on the population of users they
 affect - e.g. a bug affecting all Android users (but not Windows users)
 would still be considered to affect all users, and a bug affecting all
 Enterprise Windows users (but not all consumer Windows users) could also be
 considered to affect all users.
 
+### Known unknowns

[amineer, 2017/07/06 21:57:16]

"Addressing Uncertainty"

+
+Known unknowns are defined as the assessable likelihood that the actual true
+severity and prevalence may be higher than initially observed.  Examples:
+*   **High** - A recently introduced race condition causing tabs to never load

[amineer, 2017/07/06 21:57:16]

I don't know that adding dimensions here (e.g. breaking things out to high /
medium / low) help too much, esp. since they may confuse users who try to slot
these definitions into the explicit table above (which does not mention this
factor).

Would you be OK to change this to a paragraph which basically states that the
more recently a bug has been introduced, the higher uncertainty we have in
assessing severity / prevalence accurately, which means it should likely be
labeled more conservatively (e.g. high instead of medium sev, some prevalence
instead of few, etc)?  We can point out the converse too, that anything that's
existed for a while likely has a lower severity / prevalence?

+    (without crashing, and thus without generating a report for our metrics).
+    There are few user complaints so far, but there's reason to believe it might
+    show up more frequently with Arabic language builds, where we tend to
+    receive feedback more slowly due to the language barrier.
+*   **Medium** - A recently introduced CSS bug causing slightly misaligned
+    layout.  But there's a slim possibility that on a different site we haven't
+    tested yet, the content might be misaligned all the way off the screen.
+*   **None** - A longstanding null pointer crash with a simple local repro and
+    trustworthy prevalence metrics from Stable channel users.
+*   **N/A** - The person adding the releaseblock label doesn't have enough
+    information to assess this.
+
 ### Customization
 
 The definitions provided above are examples; teams are encouraged to customize
 where it makes sense, e.g. the web platform team may consider developer impact
 for severity and feature usage for prevalence.
 
 ## Blocker Management
 
 **Everyone should feel free to add, modify, or remove release blocking labels
 where appropriate, so long as you follow the guidelines below.**  If a TPM or
@@ skipping 34 matching lines @@
 
 ### Regressions
 
 Regressions should follow the same guidelines as listed above; an issue should
 not be tagged as a release blocker simply because it is a regression.  While
 we'd like to prevent regressions in general, there is a large backlog of bugs we
 need to address, and we should focus on the most important.  To ensure we
 maintain a high bar for product quality, we should track the number of
 introduced versus escaped regressions, and follow up if the number starts to
 rise.
+
+In practice, it is still expected that the majority of release blockers filed
+will be recent regressions, because on average they have higher severity,
+prevalence and unknowns than longstanding bugs.  (A rule of thumb is that the
+longer a bug has been present on stable channel without intense pressure to fix,
+the greater the odds that severity/prevalence/unknowns are actually low.)

[amineer, 2017/07/06 21:57:16]

s/unknowns/uncertainty