| OLD | NEW |
| (Empty) |
| 1 Look for new crashers: | |
| 2 * Go to go/chromecrash. | |
| 3 * For each platform, look through the releases for which releases to | |
| 4 investigate. As per bug-triage.txt, this should be the | |
| 5 most recent canary, the previous canary (if the most recent is less | |
| 6 than a day old), and any of dev/beta/stable that were released in the | |
| 7 last couple of days. | |
| 8 * For each release, in the "Process Type" frame, click on "browser". | |
| 9 * At the bottom of the "Magic Signature" frame, click "limit 1000". | |
| 10 Reported crashers are sorted in decreasing order of the number of reports for | |
| 11 that crash signature. | |
| 12 * Search the page for "net::". | |
| 13 * For each found signature: | |
| 14 * If there is a bug already filed, make sure it is correctly | |
| 15 describing the current bug (e.g. not closed, or not describing a | |
| 16 long-past issue), and make sure that if it is a net:: bug, that | |
| 17 it is labeled as such. | |
| 18 * Ignore signatures that only occur once, as memory corruption can | |
| 19 easily cause one-off failures when the sample size is large | |
| 20 enough. | |
| 21 * Ignore signatures that only come from a single client ID, as | |
| 22 individual machine malware and breakage can also easily cause | |
| 23 one-off failures. | |
| 24 * Click on the number of reports field to see details of | |
| 25 crash. Ignore it if it doesn't appear to be a network bug. | |
| 26 * Otherwise, file a new bug directly from chromecrash. Note that | |
| 27 this may result in filing bugs for low- and very-low- frequency | |
| 28 crashes. That's ok; the bug tracker is a better tool to figure | |
| 29 out whether or not we put resources into those crashes than a snap | |
| 30 judgement when filing bugs. | |
| 31 * For each bug you file, include the following information: | |
| 32 * The backtrace. Note that the backtrace should not be added to the | |
| 33 bug if Restrict-View-Google isn't set on the bug as it may contain | |
| 34 PII. Filing the bug from the crash reporter should do this | |
| 35 automatically, but check. | |
| 36 * The channel in which the bug is seen (canary/dev/beta/stable), | |
| 37 its frequency in that channel, and its rank among crashers in the channel. | |
| 38 * The frequency of this signature in recent releases. This | |
| 39 information is available by: | |
| 40 * Clicking on the signature in the "Magic Signature" list | |
| 41 * Clicking "Edit" on the dremel query at the top of the page | |
| 42 * Removing the "product.version='X.Y.Z.W' AND" string and clicking | |
| 43 "Update". | |
| 44 * Clicking "Limit 1000" in the Product Version list in the | |
| 45 resulting page (without this, the listing will be restricted to | |
| 46 the releases in which the signature is most common, which will | |
| 47 often not include the canary/dev release being investigated). | |
| 48 * Choose some subset of that list, or all of it, to include in the | |
| 49 bug. Make sure to indicate if there is a defined point in the | |
| 50 past before which the signature is not present. | |
| 51 | |
| 52 Identifying unlabeled network bugs on the tracker: | |
| 53 * Look at new uncomfirmed bugs since noon PST on the last triager's rotation: | |
| 54 https://code.google.com/p/chromium/issues/list?can=2&q=status%3Aunconfirmed&
sort=-id&num=1000 | |
| 55 * Press "h" to bring up a preview of the bug text. | |
| 56 * Use "j" and "k" to advance through bugs. | |
| 57 * If a bug looks like it might be network/download/safe-browsing related, middle | |
| 58 click [or command-click on OSX] to open in new tab. | |
| 59 * If a user provides a crash ID for a crasher for a bug that could be | |
| 60 net-related, look at the crash stack at go/crash, and see if it looks to be | |
| 61 network related. Be sure to check if other bug reports have that stack | |
| 62 trace, and mark as a dupe if so. Even if the bug isn't network related, | |
| 63 paste the stack trace in the bug, so no one else has to look up the crash | |
| 64 stack from the ID. | |
| 65 * If there's no other information than the crash ID, ask for more details and | |
| 66 add the Needs-Feedback label. | |
| 67 * If network causes are possible, ask for a net-internals log (If it's not a | |
| 68 browser crash) and attach the most specific internals-network label that's | |
| 69 applicable. If there isn't an applicable narrower label, a clear owner for | |
| 70 the issue, or there are multiple possibilities, attach the internals-network | |
| 71 label and proceed with further investigation. | |
| 72 * If non-network causes also seem possible, attach those labels as well. | |
| 73 | |
| 74 Investigating Cr-Internals-Network bugs: | |
| 75 * It's recommended that while on triage duty, you subscribe to the | |
| 76 Cr-Internals-Network label. To do this, go to | |
| 77 https://code.google.com/p/chromium/issues/ and click on "Subscriptions". | |
| 78 Enter Cr-Internals-Network and click submit. | |
| 79 * Look through uncomfirmed and untriaged Cr-Internals-Network bugs, prioritizing | |
| 80 those updated within the last week: | |
| 81 https://code.google.com/p/chromium/issues/list?can=2&q=Cr%3DInternals-Networ
k+-status%3AAssigned+-status%3AStarted+-status%3AAvailable+&sort=-modified | |
| 82 * If more information is needed from the reporter, ask for it and | |
| 83 add the 'Needs-Feedback' label. If the reporter has answered an | |
| 84 earlier request for information, remove that label. | |
| 85 * While investigating a new issue, change the status to Untriaged. | |
| 86 * If a bug is a potential security issue (Allows for code execution from remote | |
| 87 site, allows crossing security boundaries, unchecked array bounds, etc) mark | |
| 88 it Type-Bug-Security. If it has privacy implication (History, cookies | |
| 89 discoverable by an entity that shouldn't be able to do so, incognito state | |
| 90 being saved in memory or on disk beyond the lifetime of incognito tabs, | |
| 91 etc), mark it Cr-Privacy. | |
| 92 * For bugs that already have a more specific network label, go ahead and remove | |
| 93 the Cr-Internals-Network label and move on. | |
| 94 * Try to figure out if it's really a network bug. See common non-network labels | |
| 95 section for description of common labels needed for issues incorrectly | |
| 96 tagged as Cr-Internals-Network. | |
| 97 * If it's not, attach appropriate labels and go no further. | |
| 98 * If it may be a network bug, attach additional possibly relevant labels if any, | |
| 99 and continue investigating. Once you either determine it's a non-network | |
| 100 bug, or figure out accurate more specific network labels, your job is done, | |
| 101 though you should still ask for a net-internals dump if it seems likely to | |
| 102 be useful. | |
| 103 * Note that ChromeOS-specific network-related code (Captive portal detection, | |
| 104 connectivity detection, login, etc) may not all have appropriate more | |
| 105 specific labels, but are not in areas handled by the network stack team. | |
| 106 Just make sure those have the OS-Chrome label, and any more specific labels | |
| 107 if applicable, and then move on. | |
| 108 * Gather data and investigate. | |
| 109 * Remember to add the Needs-Feedback label whenever waiting for the user to | |
| 110 respond with more information, and remove it when not waiting on the user. | |
| 111 * Try to reproduce locally. If you can, and it's a regression, use | |
| 112 src/tools/bisect-builds.py to figure out when it regressed. | |
| 113 * Ask more data from the user as needed (net-internals dumps, repro case, | |
| 114 crash ID from about:crashes, run tests, etc). | |
| 115 * If asking for an about:net-internals dump, provide this link: | |
| 116 https://sites.google.com/a/chromium.org/dev/for-testers/providing-network-
details. | |
| 117 Can just grab the link from about:net-internals, as needed. | |
| 118 * Try to figure out what's going on, and which more specific network label is | |
| 119 most appropriate. | |
| 120 * If it's a regression, browse through the git history of relevant files to try | |
| 121 and figure out when it regressed. CC authors / primary reviewers of any | |
| 122 strongly suspect CLs. | |
| 123 * If you are having trouble with an issue, particularly for help understanding | |
| 124 net-internals logs, email the public net-dev@chromium.org list for help | |
| 125 debugging. If it's a crasher, or for some other reason discussion needs to | |
| 126 be done in private, use chrome-network-debugging@google.com. | |
| 127 TODO(mmenke): Write up a net-internals tips and tricks docs. | |
| 128 * If it appears to be a bug in the unowned core of the network stack (i.e. no | |
| 129 sublabel applies, or only the Cr-Internals-Network-HTTP sublabel applies, | |
| 130 and there's no clear owner), try to figure out the exact cause. | |
| 131 | |
| 132 Monitor UMA histograms and gasper alerts. For each Gasper alert that | |
| 133 fires, determine if it's a real alert and file a bug if so. | |
| 134 * Don't file if the alert is coincident with a major volume change. | |
| 135 The volume at a particular date can be determined by hovering the | |
| 136 mouse over the appropriate location on the alert line. | |
| 137 * Don't file if the alert is on a graph with very low volume (< ~200 | |
| 138 data points); it's probably noise, and we probably don't care even | |
| 139 if it isn't. | |
| 140 * Don't file if the graph is really noisy (but eyeball it to decide if | |
| 141 there is an underlying important shift under the noise). | |
| 142 * Don't file if the alert is in the "Known Ignorable" list: | |
| 143 * SimpleCache on Windows | |
| 144 * DiskCache on Android. | |
| 145 For each Gasper alert, respond to chrome-network-debugging@ with a | |
| 146 summary of the action you've taken and why, including issue link if an | |
| 147 issue was filed. | |
| 148 | |
| 149 Investigating crashers: | |
| 150 * Only investigate crashers that are still occurring, as identified by above | |
| 151 section. If a search on go/crash indicates a crasher is no longer | |
| 152 occurring, mark it as WontFix. | |
| 153 * Particularly for Windows, look for weird dlls associated with the crashes. | |
| 154 If there are some, it may be caused by malware. You can often figure out if | |
| 155 a dll is malware by a search, though it's harder to figure out if a dll is | |
| 156 definitively not malware. | |
| 157 * See if the same users are repeatedly running into the same issue. This can be | |
| 158 accomplished by search for (Or clicking on) the client ID associated with a | |
| 159 crash report, and seeing if there are multiple reports for the same crash. | |
| 160 If this is the case, it may be also be malware, or an issue with an unusual | |
| 161 system/chrome/network config. | |
| 162 * Dig through crash reports to figure out when the crash first appeared, and dig | |
| 163 through revision history in related files to try and locate a suspect CL. | |
| 164 TODO(mmenke): Add more detail here. | |
| 165 * Load crash dumps, try to figure out a cause. | |
| 166 See http://www.chromium.org/developers/crash-reports for more information | |
| 167 | |
| 168 Dealing with old bugs: | |
| 169 * For all network issues (Even those with owners, or a more specific labels): | |
| 170 * If the issue has had the Needs-Feedback label for over a month, verify it | |
| 171 is waiting on feedback from the user. If not, remove the label. | |
| 172 Otherwise, go ahead and mark the issue WontFix due to lack of response and | |
| 173 suggest the user file a new bug if the issue is still present. | |
| 174 Old Needs-Feedback issues: https://code.google.com/p/chromium/issues/list
?can=2&q=Cr%3AInternals-Network%20Needs=Feedback+modified-before%3Atoday-30&sort
=-modified | |
| 175 * If a bug is over 2 months old, and the underlying problem was never | |
| 176 reproduced or really understood: | |
| 177 * If it's over a year old, go ahead and mark the issue as Archived. | |
| 178 * Otherwise, ask reporters if the issue is still present, and attach the | |
| 179 Needs-Feedback label. | |
| 180 * Old unconfirmed or untriaged Cr-Internals-Network issues can be investigated | |
| 181 just like newer ones. Crashers should generally be given higher priority, | |
| 182 since we can verify if they still occur, and then newer issues, as they're | |
| 183 more likely to still be present, and more likely to have a still responsive | |
| 184 bug reporter. | |
| OLD | NEW |