Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(239)

Side by Side Diff: tools/metrics/histograms/README.md

Issue 2574533003: metrics: improve histogram docs (Closed)
Patch Set: address feedback Created 4 years ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
« no previous file with comments | « no previous file | no next file » | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 # Histogram Guidelines 1 # Histogram Guidelines
2 2
3 This document gives the best practices on how to use histograms in code and how 3 This document gives the best practices on how to use histograms in code and how
4 to document the histograms for the dashboards. There are three general types 4 to document the histograms for the dashboards. There are three general types
5 of histograms: enumerated histograms (appropriate for enums), count histograms 5 of histograms: enumerated histograms, count histograms (for arbitrary numbers),
6 (appropriate for arbitrary numbers), and sparse histogram (appropriate for 6 and sparse histograms (for anything when the precision is important over a wide
7 anything when the precision is important over a wide range is large and/or the 7 range and/or the range is not possible to specify a priori).
8 range is not possible to specify a priori).
9 8
10 [TOC] 9 [TOC]
11 10
12 ## Emitting to Histograms 11 ## Emitting to Histograms
13 12
14 ### Directly Measure What You Want 13 ### Directly Measure What You Want
15 14
16 Measure exactly what you want, whether that's time used for a function call, 15 Measure exactly what you want, whether that's time used for a function call,
17 number of bytes transmitted to fetch a page, number of items in a list, etc. 16 number of bytes transmitted to fetch a page, number of items in a list, etc.
18 Do not assume you can calculate what you want from other histograms. Most of 17 Do not assume you can calculate what you want from other histograms. Most of
(...skipping 28 matching lines...) Expand all
47 You may append to your enum if the possible states/actions grows. However, you 46 You may append to your enum if the possible states/actions grows. However, you
48 should not reorder, renumber, or otherwise reuse existing values. As such, 47 should not reorder, renumber, or otherwise reuse existing values. As such,
49 please put this warning by the enum definition: 48 please put this warning by the enum definition:
50 ``` 49 ```
51 // These values are written to logs. New enum values can be added, but existing 50 // These values are written to logs. New enum values can be added, but existing
52 // enums must never be renumbered or deleted and reused. 51 // enums must never be renumbered or deleted and reused.
53 ``` 52 ```
54 53
55 Also, please explicitly set enum values `= 0`, `= 1`, `= 2`, etc. This makes 54 Also, please explicitly set enum values `= 0`, `= 1`, `= 2`, etc. This makes
56 clearer that the actual values are important. In addition, it helps confirm 55 clearer that the actual values are important. In addition, it helps confirm
57 the values align between the enum definition and histograms.xml. 56 the values align between the enum definition and
57 [histograms.xml](./histograms.xml).
58 58
59 ### Count Histograms 59 ### Count Histograms
60 60
61 [histogram_macros.h](https://cs.chromium.org/chromium/src/base/metrics/histogram _macros.h) 61 [histogram_macros.h](https://cs.chromium.org/chromium/src/base/metrics/histogram _macros.h)
62 provides macros for some common count types such as memory or elapsed time, in 62 provides macros for some common count types such as memory or elapsed time, in
63 addition to general count macros. These have reasonable default values; you 63 addition to general count macros. These have reasonable default values; you
64 will not often need to choose number of buckets or histogram min. You still 64 will not often need to choose number of buckets or histogram min. You still
65 will need to choose the histogram max (use the advice below). 65 will need to choose the histogram max (use the advice below).
66 66
67 If none of the default macros work well for you, please thoughtfully choose 67 If none of the default macros work well for you, please thoughtfully choose
68 a min, max, and bucket count for your histogram using the advice below. 68 a min, max, and bucket count for your histogram using the advice below.
69 69
70 #### Count Histograms: Choosing Min and Max 70 #### Count Histograms: Choosing Min and Max
71 71
72 For histogram max, choose a value so that very few emission to the histogram 72 For histogram max, choose a value so that very few emission to the histogram
73 will exceed the max. If many emissions hit the max, it can be difficult to 73 will exceed the max. If many emissions hit the max, it can be difficult to
74 compute statistics such as average. One rule of thumb is at most 1% of samples 74 compute statistics such as average. One rule of thumb is at most 1% of samples
75 should be in the overflow bucket. This allows analysis of the 99th percentile. 75 should be in the overflow bucket. This allows analysis of the 99th percentile.
76 Err on the side of too large a range versus too short a range. (Remember that i f you choose poorly, you'll have to wait for another release cycle to fix it.) 76 Err on the side of too large a range versus too short a range. (Remember that
77 if you choose poorly, you'll have to wait for another release cycle to fix it.)
77 78
78 For histogram min, if you care about all possible values (zero and above), 79 For histogram min, if you care about all possible values (zero and above),
79 choose a min of 1. (All histograms have an underflow bucket; emitted zeros 80 choose a min of 1. (All histograms have an underflow bucket; emitted zeros
80 will go there. That's why a min of 1 is appropriate.) Otherwise, choose the 81 will go there. That's why a min of 1 is appropriate.) Otherwise, choose the
81 min appropriate for your particular situation. 82 min appropriate for your particular situation.
82 83
83 #### Count Histograms: Choosing Number of Buckets 84 #### Count Histograms: Choosing Number of Buckets
84 85
85 Choose the smallest number of buckets that will get you the granularity you 86 Choose the smallest number of buckets that will get you the granularity you
86 need. By default count histograms bucket sizes scale exponentially so you can 87 need. By default count histograms bucket sizes scale exponentially so you can
(...skipping 21 matching lines...) Expand all
108 state A or B. In this case, they want to know the count of X under state A, 109 state A or B. In this case, they want to know the count of X under state A,
109 as well as the other three permutations. 110 as well as the other three permutations.
110 111
111 There is no general purpose solution for this type of analysis. We suggest 112 There is no general purpose solution for this type of analysis. We suggest
112 using the workaround of using an enum of length MxN, where you log each unique 113 using the workaround of using an enum of length MxN, where you log each unique
113 pair {state, feature} as a separate entry in the same enum. If this causes a 114 pair {state, feature} as a separate entry in the same enum. If this causes a
114 large explosion in data (i.e. >100 enum entries), a [sparse histogram](#When-To- Use-Sparse-Histograms) may be appropriate. If you are unsure of the best way to proceed, please contact someone from the OWNERS file. 115 large explosion in data (i.e. >100 enum entries), a [sparse histogram](#When-To- Use-Sparse-Histograms) may be appropriate. If you are unsure of the best way to proceed, please contact someone from the OWNERS file.
115 116
116 ### Testing 117 ### Testing
117 118
118 Test your histograms using *chrome://histograms*. Make sure they're being 119 Test your histograms using `chrome://histograms`. Make sure they're being
119 emitted to when you expect and not emitted to at other times. Also check that 120 emitted to when you expect and not emitted to at other times. Also check that
120 the values emitted to are correct. Finally, for count histograms, make sure 121 the values emitted to are correct. Finally, for count histograms, make sure
121 that buckets capture enough precision for your needs over the range. 122 that buckets capture enough precision for your needs over the range.
122 123
123 ### Revising Histograms 124 ### Revising Histograms
124 125
125 If you're changing the semantics of a histogram (when it's emitted, what 126 If you're changing the semantics of a histogram (when it's emitted, what
126 buckets mean, etc.), make it into a new histogram with a new name. Otherwise 127 buckets mean, etc.), make it into a new histogram with a new name. Otherwise
127 the "Everything" view on the dashboard will be mixing two different 128 the "Everything" view on the dashboard will be mixing two different
128 interpretations of the data and make no sense. 129 interpretations of the data and make no sense.
129 130
130 ### Deleting Histograms 131 ### Deleting Histograms
131 132
132 Please delete the code that emits to histograms that are no longer needed. 133 Please delete the code that emits to histograms that are no longer needed.
133 Histograms take up memory. Cleaning up histograms that you no longer care about 134 Histograms take up memory. Cleaning up histograms that you no longer care about
134 is good! But see the note below on [Deleting Histogram Entries](#Deleting-Histo gram-Entries). 135 is good! But see the note below on [Deleting Histogram Entries](#Deleting-Histo gram-Entries).
135 136
136 ## Documenting Histograms 137 ## Documenting Histograms
137 138
138 ### Add Histogram and Documentation in the Same Changelist 139 ### Add Histogram and Documentation in the Same Changelist
139 140
140 If possible, please add the histograms.xml description in the same changelist 141 If possible, please add the [histograms.xml](./histograms.xml) description in
141 in which you add the histogram-emitting code. This has several benefits. One, 142 the same changelist in which you add the histogram-emitting code. This has
142 it sometimes happens that the histograms.xml reviewer has questions or concerns 143 several benefits. One, it sometimes happens that the
143 about the histogram description that reveal problems with interpretation of the 144 [histograms.xml](./histograms.xml) reviewer has questions or concerns about the
144 data and call for a different recording strategy. Two, it allows the histogram 145 histogram description that reveal problems with interpretation of the data and
145 reviewer to easily review the emission code to see if it comports with these 146 call for a different recording strategy. Two, it allows the histogram reviewer
146 best practices, and to look for other errors. 147 to easily review the emission code to see if it comports with these best
148 practices, and to look for other errors.
147 149
148 ### Understandable to Everyone 150 ### Understandable to Everyone
149 151
150 Histogram descriptions should be roughly understandable to someone not familiar 152 Histogram descriptions should be roughly understandable to someone not familiar
151 with your feature. Please add a sentence or two of background if necessary. 153 with your feature. Please add a sentence or two of background if necessary.
152 154
153 It is good practice to note caveats associated with your histogram in this 155 It is good practice to note caveats associated with your histogram in this
154 section, such as which platforms are supported (if the set of supported 156 section, such as which platforms are supported (if the set of supported
155 platforms is surprising). E.g., a desktop feature that happens not to be logged 157 platforms is surprising). E.g., a desktop feature that happens not to be logged
156 on Mac. 158 on Mac.
157 159
158 ### State When It Is Recorded 160 ### State When It Is Recorded
159 161
160 Histogram descriptions should clearly state when the histogram is emitted 162 Histogram descriptions should clearly state when the histogram is emitted
161 (profile open? network request received? etc.). 163 (profile open? network request received? etc.).
162 164
163 ### Owners 165 ### Owners
164 166
165 Histograms need to be owned by a person or set of people. These indicate who 167 Histograms need to be owned by a person or set of people. These indicate who
166 the current experts on this metric are. Being the owner means you are 168 the current experts on this metric are. Being the owner means you are
167 responsible for answering questions about the metric, handling the maintenance 169 responsible for answering questions about the metric, handling the maintenance
168 if there are functional changes, and deprecating the metric if it outlives its 170 if there are functional changes, and deprecating the metric if it outlives its
169 usefulness. The owners should be added in the original histogram description. 171 usefulness. The owners should be added in the original histogram description.
170 If you are using a metric heavily and understand it intimately, feel free to 172 If you are using a metric heavily and understand it intimately, feel free to
171 add yourself as an owner. @chromium.org email addresses are preferred. 173 add yourself as an owner. @chromium.org email addresses are preferred.
172 174
173 175
174 ### Deleting Histogram Entries 176 ### Deleting Histogram Entries
175 177
176 Do not delete histograms from histograms.xml. Instead, mark unused histograms 178 Do not delete histograms from [histograms.xml](./histograms.xml). Instead, mark
177 as obsolete, annotating them with the associated date or milestone in the 179 unused histograms as obsolete, annotating them with the associated date or
178 obsolete tag entry. If your histogram is being replaced by a new version, we 180 milestone in the obsolete tag entry. If your histogram is being replaced by a
179 suggest noting that in the previous histogram's description. 181 new version, we suggest noting that in the previous histogram's description.
180 182
181 Deleting histogram entries would be bad if someone to accidentally reused your 183 Deleting histogram entries would be bad if someone to accidentally reused your
182 old histogram name and thereby corrupts new data with whatever old data is still 184 old histogram name and thereby corrupts new data with whatever old data is still
183 coming in. It's also useful to keep obsolete histogram descriptions in 185 coming in. It's also useful to keep obsolete histogram descriptions in
184 histograms.xml--that way, if someone is searching for a histogram to answer 186 [histograms.xml](./histograms.xml) -- that way, if someone is searching for a
185 a particular question, they can learn if there was a histogram at some point 187 histogram to answer a particular question, they can learn if there was a
186 that did so even if it isn't active now. 188 histogram at some point that did so even if it isn't active now.
187 189
188 ## When To Use Sparse Histograms 190 ## When To Use Sparse Histograms
189 191
190 Sparse histograms are well suited for recording counts of exact sample values 192 Sparse histograms are well suited for recording counts of exact sample values
191 that are sparsely distributed over a large range. 193 that are sparsely distributed over a large range.
192 194
193 The implementation uses a lock and a map, whereas other histogram types use a 195 The implementation uses a lock and a map, whereas other histogram types use a
194 vector and no lock. It is thus more costly to add values to, and each value 196 vector and no lock. It is thus more costly to add values to, and each value
195 stored has more overhead, compared to the other histogram types. However it 197 stored has more overhead, compared to the other histogram types. However it
196 may be more efficient in memory if the total number of sample values is small 198 may be more efficient in memory if the total number of sample values is small
197 compared to the range of their values. 199 compared to the range of their values.
198 200
199 For more information, see [sparse_histograms.h](https://cs.chromium.org/chromium /src/base/metrics/sparse_histogram.h). 201 For more information, see [sparse_histograms.h](https://cs.chromium.org/chromium /src/base/metrics/sparse_histogram.h).
OLDNEW
« no previous file with comments | « no previous file | no next file » | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698