| OLD | NEW |
| 1 # Histogram Guidelines | 1 # Histogram Guidelines |
| 2 | 2 |
| 3 This document gives the best practices on how to use histograms in code and how | 3 This document gives the best practices on how to use histograms in code and how |
| 4 to document the histograms for the dashboards. There are three general types | 4 to document the histograms for the dashboards. There are three general types |
| 5 of histograms: enumerated histograms (appropriate for enums), count histograms | 5 of histograms: enumerated histograms, count histograms (for arbitrary numbers), |
| 6 (appropriate for arbitrary numbers), and sparse histogram (appropriate for | 6 and sparse histograms (for anything when the precision is important over a wide |
| 7 anything when the precision is important over a wide range is large and/or the | 7 range and/or the range is not possible to specify a priori). |
| 8 range is not possible to specify a priori). | |
| 9 | 8 |
| 10 [TOC] | 9 [TOC] |
| 11 | 10 |
| 12 ## Emitting to Histograms | 11 ## Emitting to Histograms |
| 13 | 12 |
| 14 ### Directly Measure What You Want | 13 ### Directly Measure What You Want |
| 15 | 14 |
| 16 Measure exactly what you want, whether that's time used for a function call, | 15 Measure exactly what you want, whether that's time used for a function call, |
| 17 number of bytes transmitted to fetch a page, number of items in a list, etc. | 16 number of bytes transmitted to fetch a page, number of items in a list, etc. |
| 18 Do not assume you can calculate what you want from other histograms. Most of | 17 Do not assume you can calculate what you want from other histograms. Most of |
| (...skipping 28 matching lines...) Expand all Loading... |
| 47 You may append to your enum if the possible states/actions grows. However, you | 46 You may append to your enum if the possible states/actions grows. However, you |
| 48 should not reorder, renumber, or otherwise reuse existing values. As such, | 47 should not reorder, renumber, or otherwise reuse existing values. As such, |
| 49 please put this warning by the enum definition: | 48 please put this warning by the enum definition: |
| 50 ``` | 49 ``` |
| 51 // These values are written to logs. New enum values can be added, but existing | 50 // These values are written to logs. New enum values can be added, but existing |
| 52 // enums must never be renumbered or deleted and reused. | 51 // enums must never be renumbered or deleted and reused. |
| 53 ``` | 52 ``` |
| 54 | 53 |
| 55 Also, please explicitly set enum values `= 0`, `= 1`, `= 2`, etc. This makes | 54 Also, please explicitly set enum values `= 0`, `= 1`, `= 2`, etc. This makes |
| 56 clearer that the actual values are important. In addition, it helps confirm | 55 clearer that the actual values are important. In addition, it helps confirm |
| 57 the values align between the enum definition and histograms.xml. | 56 the values align between the enum definition and |
| 57 [histograms.xml](./histograms.xml). |
| 58 | 58 |
| 59 ### Count Histograms | 59 ### Count Histograms |
| 60 | 60 |
| 61 [histogram_macros.h](https://cs.chromium.org/chromium/src/base/metrics/histogram
_macros.h) | 61 [histogram_macros.h](https://cs.chromium.org/chromium/src/base/metrics/histogram
_macros.h) |
| 62 provides macros for some common count types such as memory or elapsed time, in | 62 provides macros for some common count types such as memory or elapsed time, in |
| 63 addition to general count macros. These have reasonable default values; you | 63 addition to general count macros. These have reasonable default values; you |
| 64 will not often need to choose number of buckets or histogram min. You still | 64 will not often need to choose number of buckets or histogram min. You still |
| 65 will need to choose the histogram max (use the advice below). | 65 will need to choose the histogram max (use the advice below). |
| 66 | 66 |
| 67 If none of the default macros work well for you, please thoughtfully choose | 67 If none of the default macros work well for you, please thoughtfully choose |
| 68 a min, max, and bucket count for your histogram using the advice below. | 68 a min, max, and bucket count for your histogram using the advice below. |
| 69 | 69 |
| 70 #### Count Histograms: Choosing Min and Max | 70 #### Count Histograms: Choosing Min and Max |
| 71 | 71 |
| 72 For histogram max, choose a value so that very few emission to the histogram | 72 For histogram max, choose a value so that very few emission to the histogram |
| 73 will exceed the max. If many emissions hit the max, it can be difficult to | 73 will exceed the max. If many emissions hit the max, it can be difficult to |
| 74 compute statistics such as average. One rule of thumb is at most 1% of samples | 74 compute statistics such as average. One rule of thumb is at most 1% of samples |
| 75 should be in the overflow bucket. This allows analysis of the 99th percentile. | 75 should be in the overflow bucket. This allows analysis of the 99th percentile. |
| 76 Err on the side of too large a range versus too short a range. (Remember that i
f you choose poorly, you'll have to wait for another release cycle to fix it.) | 76 Err on the side of too large a range versus too short a range. (Remember that |
| 77 if you choose poorly, you'll have to wait for another release cycle to fix it.) |
| 77 | 78 |
| 78 For histogram min, if you care about all possible values (zero and above), | 79 For histogram min, if you care about all possible values (zero and above), |
| 79 choose a min of 1. (All histograms have an underflow bucket; emitted zeros | 80 choose a min of 1. (All histograms have an underflow bucket; emitted zeros |
| 80 will go there. That's why a min of 1 is appropriate.) Otherwise, choose the | 81 will go there. That's why a min of 1 is appropriate.) Otherwise, choose the |
| 81 min appropriate for your particular situation. | 82 min appropriate for your particular situation. |
| 82 | 83 |
| 83 #### Count Histograms: Choosing Number of Buckets | 84 #### Count Histograms: Choosing Number of Buckets |
| 84 | 85 |
| 85 Choose the smallest number of buckets that will get you the granularity you | 86 Choose the smallest number of buckets that will get you the granularity you |
| 86 need. By default count histograms bucket sizes scale exponentially so you can | 87 need. By default count histograms bucket sizes scale exponentially so you can |
| (...skipping 21 matching lines...) Expand all Loading... |
| 108 state A or B. In this case, they want to know the count of X under state A, | 109 state A or B. In this case, they want to know the count of X under state A, |
| 109 as well as the other three permutations. | 110 as well as the other three permutations. |
| 110 | 111 |
| 111 There is no general purpose solution for this type of analysis. We suggest | 112 There is no general purpose solution for this type of analysis. We suggest |
| 112 using the workaround of using an enum of length MxN, where you log each unique | 113 using the workaround of using an enum of length MxN, where you log each unique |
| 113 pair {state, feature} as a separate entry in the same enum. If this causes a | 114 pair {state, feature} as a separate entry in the same enum. If this causes a |
| 114 large explosion in data (i.e. >100 enum entries), a [sparse histogram](#When-To-
Use-Sparse-Histograms) may be appropriate. If you are unsure of the best way to
proceed, please contact someone from the OWNERS file. | 115 large explosion in data (i.e. >100 enum entries), a [sparse histogram](#When-To-
Use-Sparse-Histograms) may be appropriate. If you are unsure of the best way to
proceed, please contact someone from the OWNERS file. |
| 115 | 116 |
| 116 ### Testing | 117 ### Testing |
| 117 | 118 |
| 118 Test your histograms using *chrome://histograms*. Make sure they're being | 119 Test your histograms using `chrome://histograms`. Make sure they're being |
| 119 emitted to when you expect and not emitted to at other times. Also check that | 120 emitted to when you expect and not emitted to at other times. Also check that |
| 120 the values emitted to are correct. Finally, for count histograms, make sure | 121 the values emitted to are correct. Finally, for count histograms, make sure |
| 121 that buckets capture enough precision for your needs over the range. | 122 that buckets capture enough precision for your needs over the range. |
| 122 | 123 |
| 123 ### Revising Histograms | 124 ### Revising Histograms |
| 124 | 125 |
| 125 If you're changing the semantics of a histogram (when it's emitted, what | 126 If you're changing the semantics of a histogram (when it's emitted, what |
| 126 buckets mean, etc.), make it into a new histogram with a new name. Otherwise | 127 buckets mean, etc.), make it into a new histogram with a new name. Otherwise |
| 127 the "Everything" view on the dashboard will be mixing two different | 128 the "Everything" view on the dashboard will be mixing two different |
| 128 interpretations of the data and make no sense. | 129 interpretations of the data and make no sense. |
| 129 | 130 |
| 130 ### Deleting Histograms | 131 ### Deleting Histograms |
| 131 | 132 |
| 132 Please delete the code that emits to histograms that are no longer needed. | 133 Please delete the code that emits to histograms that are no longer needed. |
| 133 Histograms take up memory. Cleaning up histograms that you no longer care about | 134 Histograms take up memory. Cleaning up histograms that you no longer care about |
| 134 is good! But see the note below on [Deleting Histogram Entries](#Deleting-Histo
gram-Entries). | 135 is good! But see the note below on [Deleting Histogram Entries](#Deleting-Histo
gram-Entries). |
| 135 | 136 |
| 136 ## Documenting Histograms | 137 ## Documenting Histograms |
| 137 | 138 |
| 138 ### Add Histogram and Documentation in the Same Changelist | 139 ### Add Histogram and Documentation in the Same Changelist |
| 139 | 140 |
| 140 If possible, please add the histograms.xml description in the same changelist | 141 If possible, please add the [histograms.xml](./histograms.xml) description in |
| 141 in which you add the histogram-emitting code. This has several benefits. One, | 142 the same changelist in which you add the histogram-emitting code. This has |
| 142 it sometimes happens that the histograms.xml reviewer has questions or concerns | 143 several benefits. One, it sometimes happens that the |
| 143 about the histogram description that reveal problems with interpretation of the | 144 [histograms.xml](./histograms.xml) reviewer has questions or concerns about the |
| 144 data and call for a different recording strategy. Two, it allows the histogram | 145 histogram description that reveal problems with interpretation of the data and |
| 145 reviewer to easily review the emission code to see if it comports with these | 146 call for a different recording strategy. Two, it allows the histogram reviewer |
| 146 best practices, and to look for other errors. | 147 to easily review the emission code to see if it comports with these best |
| 148 practices, and to look for other errors. |
| 147 | 149 |
| 148 ### Understandable to Everyone | 150 ### Understandable to Everyone |
| 149 | 151 |
| 150 Histogram descriptions should be roughly understandable to someone not familiar | 152 Histogram descriptions should be roughly understandable to someone not familiar |
| 151 with your feature. Please add a sentence or two of background if necessary. | 153 with your feature. Please add a sentence or two of background if necessary. |
| 152 | 154 |
| 153 It is good practice to note caveats associated with your histogram in this | 155 It is good practice to note caveats associated with your histogram in this |
| 154 section, such as which platforms are supported (if the set of supported | 156 section, such as which platforms are supported (if the set of supported |
| 155 platforms is surprising). E.g., a desktop feature that happens not to be logged | 157 platforms is surprising). E.g., a desktop feature that happens not to be logged |
| 156 on Mac. | 158 on Mac. |
| 157 | 159 |
| 158 ### State When It Is Recorded | 160 ### State When It Is Recorded |
| 159 | 161 |
| 160 Histogram descriptions should clearly state when the histogram is emitted | 162 Histogram descriptions should clearly state when the histogram is emitted |
| 161 (profile open? network request received? etc.). | 163 (profile open? network request received? etc.). |
| 162 | 164 |
| 163 ### Owners | 165 ### Owners |
| 164 | 166 |
| 165 Histograms need to be owned by a person or set of people. These indicate who | 167 Histograms need to be owned by a person or set of people. These indicate who |
| 166 the current experts on this metric are. Being the owner means you are | 168 the current experts on this metric are. Being the owner means you are |
| 167 responsible for answering questions about the metric, handling the maintenance | 169 responsible for answering questions about the metric, handling the maintenance |
| 168 if there are functional changes, and deprecating the metric if it outlives its | 170 if there are functional changes, and deprecating the metric if it outlives its |
| 169 usefulness. The owners should be added in the original histogram description. | 171 usefulness. The owners should be added in the original histogram description. |
| 170 If you are using a metric heavily and understand it intimately, feel free to | 172 If you are using a metric heavily and understand it intimately, feel free to |
| 171 add yourself as an owner. @chromium.org email addresses are preferred. | 173 add yourself as an owner. @chromium.org email addresses are preferred. |
| 172 | 174 |
| 173 | 175 |
| 174 ### Deleting Histogram Entries | 176 ### Deleting Histogram Entries |
| 175 | 177 |
| 176 Do not delete histograms from histograms.xml. Instead, mark unused histograms | 178 Do not delete histograms from [histograms.xml](./histograms.xml). Instead, mark |
| 177 as obsolete, annotating them with the associated date or milestone in the | 179 unused histograms as obsolete, annotating them with the associated date or |
| 178 obsolete tag entry. If your histogram is being replaced by a new version, we | 180 milestone in the obsolete tag entry. If your histogram is being replaced by a |
| 179 suggest noting that in the previous histogram's description. | 181 new version, we suggest noting that in the previous histogram's description. |
| 180 | 182 |
| 181 Deleting histogram entries would be bad if someone to accidentally reused your | 183 Deleting histogram entries would be bad if someone to accidentally reused your |
| 182 old histogram name and thereby corrupts new data with whatever old data is still | 184 old histogram name and thereby corrupts new data with whatever old data is still |
| 183 coming in. It's also useful to keep obsolete histogram descriptions in | 185 coming in. It's also useful to keep obsolete histogram descriptions in |
| 184 histograms.xml--that way, if someone is searching for a histogram to answer | 186 [histograms.xml](./histograms.xml) -- that way, if someone is searching for a |
| 185 a particular question, they can learn if there was a histogram at some point | 187 histogram to answer a particular question, they can learn if there was a |
| 186 that did so even if it isn't active now. | 188 histogram at some point that did so even if it isn't active now. |
| 187 | 189 |
| 188 ## When To Use Sparse Histograms | 190 ## When To Use Sparse Histograms |
| 189 | 191 |
| 190 Sparse histograms are well suited for recording counts of exact sample values | 192 Sparse histograms are well suited for recording counts of exact sample values |
| 191 that are sparsely distributed over a large range. | 193 that are sparsely distributed over a large range. |
| 192 | 194 |
| 193 The implementation uses a lock and a map, whereas other histogram types use a | 195 The implementation uses a lock and a map, whereas other histogram types use a |
| 194 vector and no lock. It is thus more costly to add values to, and each value | 196 vector and no lock. It is thus more costly to add values to, and each value |
| 195 stored has more overhead, compared to the other histogram types. However it | 197 stored has more overhead, compared to the other histogram types. However it |
| 196 may be more efficient in memory if the total number of sample values is small | 198 may be more efficient in memory if the total number of sample values is small |
| 197 compared to the range of their values. | 199 compared to the range of their values. |
| 198 | 200 |
| 199 For more information, see [sparse_histograms.h](https://cs.chromium.org/chromium
/src/base/metrics/sparse_histogram.h). | 201 For more information, see [sparse_histograms.h](https://cs.chromium.org/chromium
/src/base/metrics/sparse_histogram.h). |
| OLD | NEW |