Clarifications to network bug triage process.

net/docs/bug-triage-suggested-workflow.txt

+Look for new crashers:
+* Go to go/chromecrash.
+* Choose the platforms and releases for which you will be

[mmenke, 2015/03/05 15:49:28]

Think this would be clearer as "For each platform, look through the releases for
which..." And then you can remove the last sentence of the paragraph

[Randy Smith (Not in Mondays), 2015/03/06 00:21:35]

Done.

+  investigating crashers.  As per bug-triage.txt, this should be the
+  most recent canary, the previous canary (if the most recent is less
+  than a day old), and any of dev/beta/stable that were released in the
+  last couple of days.  These releases should be investigated for all
+  platforms. 
+* For each release, in the "Process Type" frame, click on "browser".
+* At the bottom of the "Magic Signature" frame,  click "limit 1000".
+  Reported crashers are sorted in decreasing order of the number of reports for
+  that crash signature.
+* Search the page for "net::".  
+* For each found signature:
+  * If there is a bug already filed, make sure it is correctly
+    describing the current bug (e.g. not closed, or not describing a
+    long-past issue), and make sure that if it is a net:: bug, that
+    it is labeled as such (and in such a way that it will be caught
+    by other parts of the triage process or the owners of the more
+    narrowly labeled network buckets).  

[mmenke, 2015/03/05 15:49:28]

I don't think the "(and in such a way..." part of this description adds
anything.

[Randy Smith (Not in Mondays), 2015/03/06 00:21:35]

Done.

+  * Ignore signatures that only occur once, as memory corruption can
+    easily cause one-off failures when the sample size is large
+    enough.
+  * Ignore signatures that only come from a single client ID, as
+    individual machine malware and breakage can also easily cause
+    one-off failures.  
+  * Click on the number of reports field to see details of
+    crash. Ignore it if it doesn't appear to be a network bug. 
+  * Otherwise, file a new bug directly from chromecrash.  Note that
+    this may result in filing bugs for low- and very-low- frequency
+    crashes.  That's ok; the bug tracker is a better tool to figure
+    out whether or not we put resources into those crashes than a snap
+    judgement when filing bugs.
+* For each bug you file, include the following information:
+  * The backtrace.  Note that the backtrace should not be added to the
+    bug if Restrict-View-Google isn't set on the bug as it may contain
+    PII.  Filing the bug from the crash reporter should do this
+    automatically, but check.
+  * The channel in which the bug is seen (canary/dev/beta/stable),
+    its frequency in that channel, and its rank among crashers in the channel.
+  * The frequency of this signature in recent releases.  This
+    information is available by:
+    * Clicking on the signature in the "Magic Signature" list
+    * Clicking "Edit" on the dremel query at the top of the page
+    * Removing the "product.version='X.Y.Z.W' AND" string and clicking
+      "Update".
+    * Clicking "Limit 1000" in the Product Version list in the
+      resulting page (without this, the listing will be restricted to
+      the releases in which the signature is most common, which will
+      often not include the canary/dev release being investigated).
+    * Choose some subset of that list, or all of it, to include in the
+      bug.  Make sure to indicate if there is a defined point in the
+      past before which the signature is not present.
+
 Identifying unlabeled network bugs on the tracker:
 * Look at new uncomfirmed bugs since noon PST on the last triager's rotation:
     https://code.google.com/p/chromium/issues/list?can=2&q=status%3Aunconfirmed&sort=-id&num=1000
 * Press "h" to bring up a preview of the bug text.
 * Use "j" and "k" to advance through bugs.
 * If a bug looks like it might be network/download/safe-browsing related, middle
     click [or command-click on OSX] to open in new tab.
 * If a user provides a crash ID for a crasher for a bug that could be
     net-related, look at the crash stack at go/crash, and see if it looks to be
     network related.  Be sure to check if other bug reports have that stack
     trace, and mark as a dupe if so.  Even if the bug isn't network related,
     paste the stack trace in the bug, so no one else has to look up the crash
     stack from the ID.
   * If there's no other information than the crash ID, ask for more details and
       add the Needs-Feedback label.
 * If network causes are possible, ask for a net-internals log (If it's not a
     browser crash) and attach the most specific internals-network label that's
     applicable.  If there isn't an applicable narrower label, a clear owner for
     the issue, or there are multiple possibilities, attach the internals-network
     label and proceed with further investigation.
 * If non-network causes also seem possible, attach those labels as well.
 
+Request data about recent unassigned Cr-Internals-Network bugs from reporters. 
+* For all bugs in this category marked "Needs-Feedback", go through
+  them and remove that label if the reporter has responded with the
+  requested information.
+* For all bugs in this category not marked "Needs-Feedback", go
+  through those bugs and determine if more information is needed from
+  the reporter to make forward progress on the bug.  If it is, request
+  that information and mark the bug Needs-Feedback.  

[mmenke, 2015/03/05 15:49:28]

The first bullet seems redundant with the "Dealing with old bugs" section at the
bottom, and the second bullet is covered in "Investigating Cr-Internals-Network
bugs".  Open to reorganizing, but don't want to be too redundant, since that
inevitably leads to one copy getting outdated.

[Randy Smith (Not in Mondays), 2015/03/06 00:21:35]

Hmmm.  This brings up a tricky set of questions, that I"m not sure how to
resolve:
* I don't personally believe that people will ever get to the "Dealing with old
bugs" section (or rarely) since it's best effort, and there's a fair amount of
stuff in the active responsibilities section.  
* I'd like to have sections in "suggested-workflow" that map pretty directly
from the responsibilities listed in bug-triage.txt; this section was absent.  

Proposal: Both here and in bug-triage.txt, combine "Request data about recent
unassigned Cr-Internals-Network bugs" in with "Investigate each recent
Cr-Internals-Network issue", but make sure that the information in both bullet
points above is included in that (some redundancy with "old bugs" as you note). 
I'll do that in a separate PS so that it can be examined and reverted easily;
let me know what you think.

[mmenke, 2015/03/09 18:49:10]

SGTM

+
 Investigating Cr-Internals-Network bugs:
 * It's recommended that while on triage duty, you subscribe to the
     Cr-Internals-Network label.  To do this, go to
     https://code.google.com/p/chromium/issues/ and click on "Subscriptions".
     Enter Cr-Internals-Network and click submit.
 * Look through uncomfirmed and untriaged Cr-Internals-Network bugs, prioritizing
     those updated within the last week:
     https://code.google.com/p/chromium/issues/list?can=2&q=Cr%3DInternals-Network+-status%3AAssigned+-status%3AStarted+-status%3AAvailable+&sort=-modified
 * While investigating a new issue, change the status to Untriaged.
 * If a bug is a potential security issue (Allows for code execution from remote
@@ skipping 35 matching lines @@
     strongly suspect CLs.
 * If you are having trouble with an issue, particularly for help understanding
     net-internals logs, email the public net-dev@chromium.org list for help
     debugging.  If it's a crasher, or for some other reason discussion needs to
     be done in private, use chrome-network-debugging@google.com.
     TODO(mmenke): Write up a net-internals tips and tricks docs.
 * If it appears to be a bug in the unowned core of the network stack (i.e. no
     sublabel applies, or only the Cr-Internals-Network-HTTP sublabel applies,
     and there's no clear owner), try to figure out the exact cause.
 
-Look for new crashers:
-* Go to go/chromecrash.
-* For each platform, go to the latest canary.
-* In the "Process Type" frame, click on "browser".
-* At the bottom of the "Magic Signature" frame,  click "limit 1000".
-  Reported crashers are sorted in decreasing order of the number of reports for
-  that crash signature.
-* Search the page for "net::".  Ignore crashes that only occur once, as
-    memory corruption can easily cause one-off failures when the sample size is
-    large enough.
-* Click on the number of reports field to see details of crash. Look at the
-    stack trace to confirm it's a network bug.
-  * If it is, and there's no associated bug filed, file a new bug directly from
-      chromecrash, looking at earlier canaries to determine if it's a recent
-      regression.  Use the most specific label possible.
-* The most recent Canary may not yet have a full day of crashes, so it may be
-    worth looking at more than one version.
-* If there's been a dev, beta, or stable release in the last couple days, should
-    also look at those.
+Monitor UMA histograms and gasper alerts.  For each Gasper alert that
+fires, determine if it's a real alert and file a bug if so.  
+* Don't file if the alert is coincident with a major volume change.
+  The volume at a particular date can be determined by hovering the
+  mouse over the appropriate location on the alert line.
+* Don't file if the alert is on a graph with very low volume (< ~200
+  data points); it's probably noise, and we probably don't care even
+  if it isn't.
+* Don't file if the graph is really noisy (but eyeball it to decide if
+  there is an underlying important shift under the noise).
+* Don't file if the alert is in the "Known Ignorable" list:
+  * SimpleCache on Windows
+  * DiskCache on Android.
+For each Gasper alert, respond to chrome-network-debugging@ with a
+summary of the action you've taken and why, including issue link if an
+issue was filed.  
 
 Investigating crashers:
 * Only investigate crashers that are still occurring, as identified by above
     section.  If a search on go/crash indicates a crasher is no longer
     occurring, mark it as WontFix.
 * Particularly for Windows, look for weird dlls associated with the crashes.
     If there are some, it may be caused by malware.  You can often figure out if
     a dll is malware by a search, though it's harder to figure out if a dll is
     definitively not malware.
 * See if the same users are repeatedly running into the same issue.  This can be
@@ skipping 17 matching lines @@
   * If a bug is over 2 months old, and the underlying problem was never
     reproduced or really understood:
     * If it's over a year old, go ahead and mark the issue as Archived.
     * Otherwise, ask reporters if the issue is still present, and attach the
         Needs-Feedback label.
 * Old unconfirmed or untriaged Cr-Internals-Network issues can be investigated
     just like newer ones.  Crashers should generally be given higher priority,
     since we can verify if they still occur, and then newer issues, as they're
     more likely to still be present, and more likely to have a still responsive
     bug reporter.