Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(1)

Side by Side Diff: tools/metrics/histograms/README.md

Issue 2574533003: metrics: improve histogram docs (Closed)
Patch Set: metrics: improve histogram docs Created 4 years ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
« no previous file with comments | « no previous file | no next file » | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 # Histogram Guidelines 1 # Histogram Guidelines
2 2
3 This document gives the best practices on how to use histograms in code and how 3 This document gives the best practices on how to use histograms in code and how
4 to document the histograms for the dashboards. There are three general types 4 to document the histograms for the dashboards. There are three general types
5 of histograms: enumerated histograms (appropriate for enums), count histograms 5 of histograms:
Mark P 2016/12/13 19:09:05 This table of contents above the actual table of c
vapier 2016/12/13 22:15:29 ok
6 (appropriate for arbitrary numbers), and sparse histogram (appropriate for 6
7 anything when the precision is important over a wide range is large and/or the 7 * [enumerated histograms](#Enum-Histograms): for enums
8 range is not possible to specify a priori). 8 * [count histograms](#Count-Histograms): for arbitrary numbers
9 * [sparse histogram](#When-To-Use-Sparse-Histograms): for anything when the
10 precision is important over a wide range is large and/or the range is not
Mark P 2016/12/13 19:09:05 omit: "is large"
vapier 2016/12/13 22:15:29 done
11 possible to specify a priori
9 12
10 [TOC] 13 [TOC]
11 14
12 ## Emitting to Histograms 15 ## Emitting to Histograms
13 16
14 ### Directly Measure What You Want 17 ### Directly Measure What You Want
15 18
16 Measure exactly what you want, whether that's time used for a function call, 19 Measure exactly what you want, whether that's time used for a function call,
17 number of bytes transmitted to fetch a page, number of items in a list, etc. 20 number of bytes transmitted to fetch a page, number of items in a list, etc.
18 Do not assume you can calculate what you want from other histograms. Most of 21 Do not assume you can calculate what you want from other histograms. Most of
(...skipping 28 matching lines...) Expand all
47 You may append to your enum if the possible states/actions grows. However, you 50 You may append to your enum if the possible states/actions grows. However, you
48 should not reorder, renumber, or otherwise reuse existing values. As such, 51 should not reorder, renumber, or otherwise reuse existing values. As such,
49 please put this warning by the enum definition: 52 please put this warning by the enum definition:
50 ``` 53 ```
51 // These values are written to logs. New enum values can be added, but existing 54 // These values are written to logs. New enum values can be added, but existing
52 // enums must never be renumbered or deleted and reused. 55 // enums must never be renumbered or deleted and reused.
53 ``` 56 ```
54 57
55 Also, please explicitly set enum values `= 0`, `= 1`, `= 2`, etc. This makes 58 Also, please explicitly set enum values `= 0`, `= 1`, `= 2`, etc. This makes
56 clearer that the actual values are important. In addition, it helps confirm 59 clearer that the actual values are important. In addition, it helps confirm
57 the values align between the enum definition and histograms.xml. 60 the values align between the enum definition and
61 [histograms.xml](./histograms.xml).
58 62
59 ### Count Histograms 63 ### Count Histograms
60 64
61 [histogram_macros.h](https://cs.chromium.org/chromium/src/base/metrics/histogram _macros.h) 65 [histogram_macros.h](https://cs.chromium.org/chromium/src/base/metrics/histogram _macros.h)
62 provides macros for some common count types such as memory or elapsed time, in 66 provides macros for some common count types such as memory or elapsed time, in
63 addition to general count macros. These have reasonable default values; you 67 addition to general count macros. These have reasonable default values; you
64 will not often need to choose number of buckets or histogram min. You still 68 will not often need to choose number of buckets or histogram min. You still
65 will need to choose the histogram max (use the advice below). 69 will need to choose the histogram max (use the advice below).
66 70
67 If none of the default macros work well for you, please thoughtfully choose 71 If none of the default macros work well for you, please thoughtfully choose
68 a min, max, and bucket count for your histogram using the advice below. 72 a min, max, and bucket count for your histogram using the advice below.
69 73
70 #### Count Histograms: Choosing Min and Max 74 #### Count Histograms: Choosing Min and Max
71 75
72 For histogram max, choose a value so that very few emission to the histogram 76 For histogram max, choose a value so that very few emission to the histogram
73 will exceed the max. If many emissions hit the max, it can be difficult to 77 will exceed the max. If many emissions hit the max, it can be difficult to
74 compute statistics such as average. One rule of thumb is at most 1% of samples 78 compute statistics such as average. One rule of thumb is at most 1% of samples
75 should be in the overflow bucket. This allows analysis of the 99th percentile. 79 should be in the overflow bucket. This allows analysis of the 99th percentile.
76 Err on the side of too large a range versus too short a range. (Remember that i f you choose poorly, you'll have to wait for another release cycle to fix it.) 80 Err on the side of too large a range versus too short a range. (Remember that
81 if you choose poorly, you'll have to wait for another release cycle to fix it.)
77 82
78 For histogram min, if you care about all possible values (zero and above), 83 For histogram min, if you care about all possible values (zero and above),
79 choose a min of 1. (All histograms have an underflow bucket; emitted zeros 84 choose a min of 1. (All histograms have an underflow bucket; emitted zeros
80 will go there. That's why a min of 1 is appropriate.) Otherwise, choose the 85 will go there. That's why a min of 1 is appropriate.) Otherwise, choose the
81 min appropriate for your particular situation. 86 min appropriate for your particular situation.
82 87
83 #### Count Histograms: Choosing Number of Buckets 88 #### Count Histograms: Choosing Number of Buckets
84 89
85 Choose the smallest number of buckets that will get you the granularity you 90 Choose the smallest number of buckets that will get you the granularity you
86 need. By default count histograms bucket sizes scale exponentially so you can 91 need. By default count histograms bucket sizes scale exponentially so you can
(...skipping 21 matching lines...) Expand all
108 state A or B. In this case, they want to know the count of X under state A, 113 state A or B. In this case, they want to know the count of X under state A,
109 as well as the other three permutations. 114 as well as the other three permutations.
110 115
111 There is no general purpose solution for this type of analysis. We suggest 116 There is no general purpose solution for this type of analysis. We suggest
112 using the workaround of using an enum of length MxN, where you log each unique 117 using the workaround of using an enum of length MxN, where you log each unique
113 pair {state, feature} as a separate entry in the same enum. If this causes a 118 pair {state, feature} as a separate entry in the same enum. If this causes a
114 large explosion in data (i.e. >100 enum entries), a [sparse histogram](#When-To- Use-Sparse-Histograms) may be appropriate. If you are unsure of the best way to proceed, please contact someone from the OWNERS file. 119 large explosion in data (i.e. >100 enum entries), a [sparse histogram](#When-To- Use-Sparse-Histograms) may be appropriate. If you are unsure of the best way to proceed, please contact someone from the OWNERS file.
115 120
116 ### Testing 121 ### Testing
117 122
118 Test your histograms using *chrome://histograms*. Make sure they're being 123 Test your histograms using `chrome://histograms`. Make sure they're being
119 emitted to when you expect and not emitted to at other times. Also check that 124 emitted to when you expect and not emitted to at other times. Also check that
120 the values emitted to are correct. Finally, for count histograms, make sure 125 the values emitted to are correct. Finally, for count histograms, make sure
121 that buckets capture enough precision for your needs over the range. 126 that buckets capture enough precision for your needs over the range.
122 127
123 ### Revising Histograms 128 ### Revising Histograms
124 129
125 If you're changing the semantics of a histogram (when it's emitted, what 130 If you're changing the semantics of a histogram (when it's emitted, what
126 buckets mean, etc.), make it into a new histogram with a new name. Otherwise 131 buckets mean, etc.), make it into a new histogram with a new name. Otherwise
127 the "Everything" view on the dashboard will be mixing two different 132 the "Everything" view on the dashboard will be mixing two different
128 interpretations of the data and make no sense. 133 interpretations of the data and make no sense.
129 134
130 ### Deleting Histograms 135 ### Deleting Histograms
131 136
132 Please delete the code that emits to histograms that are no longer needed. 137 Please delete the code that emits to histograms that are no longer needed.
133 Histograms take up memory. Cleaning up histograms that you no longer care about 138 Histograms take up memory. Cleaning up histograms that you no longer care about
134 is good! But see the note below on [Deleting Histogram Entries](#Deleting-Histo gram-Entries). 139 is good! But see the note below on [Deleting Histogram Entries](#Deleting-Histo gram-Entries).
135 140
141 ## Adding New Histograms
Mark P 2016/12/13 19:09:05 I'm not sure this section adds anything. The firs
vapier 2016/12/13 22:15:29 what got me started here is that i needed to add a
Mark P 2016/12/13 23:04:52 Okay.
142
143 Once you declare the new histogram in the [histograms.xml](./histograms.xml)
144 database, it's ready to be used immediately by emitting code.
145
146 Consult that file for details on the format, and review the documentation
147 section below for best practices.
148
136 ## Documenting Histograms 149 ## Documenting Histograms
137 150
138 ### Add Histogram and Documentation in the Same Changelist 151 ### Add Histogram and Documentation in the Same Changelist
139 152
140 If possible, please add the histograms.xml description in the same changelist 153 If possible, please add the [histograms.xml](./histograms.xml) description in
141 in which you add the histogram-emitting code. This has several benefits. One, 154 the same changelist in which you add the histogram-emitting code. This has
142 it sometimes happens that the histograms.xml reviewer has questions or concerns 155 several benefits. One, it sometimes happens that the
143 about the histogram description that reveal problems with interpretation of the 156 [histograms.xml](./histograms.xml) reviewer has questions or concerns about the
144 data and call for a different recording strategy. Two, it allows the histogram 157 histogram description that reveal problems with interpretation of the data and
145 reviewer to easily review the emission code to see if it comports with these 158 call for a different recording strategy. Two, it allows the histogram reviewer
146 best practices, and to look for other errors. 159 to easily review the emission code to see if it comports with these best
160 practices, and to look for other errors.
147 161
148 ### Understandable to Everyone 162 ### Understandable to Everyone
149 163
150 Histogram descriptions should be roughly understandable to someone not familiar 164 Histogram descriptions should be roughly understandable to someone not familiar
151 with your feature. Please add a sentence or two of background if necessary. 165 with your feature. Please add a sentence or two of background if necessary.
152 166
153 It is good practice to note caveats associated with your histogram in this 167 It is good practice to note caveats associated with your histogram in this
154 section, such as which platforms are supported (if the set of supported 168 section, such as which platforms are supported (if the set of supported
155 platforms is surprising). E.g., a desktop feature that happens not to be logged 169 platforms is surprising). E.g., a desktop feature that happens not to be logged
156 on Mac. 170 on Mac.
157 171
158 ### State When It Is Recorded 172 ### State When It Is Recorded
159 173
160 Histogram descriptions should clearly state when the histogram is emitted 174 Histogram descriptions should clearly state when the histogram is emitted
161 (profile open? network request received? etc.). 175 (profile open? network request received? etc.).
162 176
163 ### Owners 177 ### Owners
164 178
165 Histograms need to be owned by a person or set of people. These indicate who 179 Histograms need to be owned by a person or set of people. These indicate who
166 the current experts on this metric are. Being the owner means you are 180 the current experts on this metric are. Being the owner means you are
167 responsible for answering questions about the metric, handling the maintenance 181 responsible for answering questions about the metric, handling the maintenance
168 if there are functional changes, and deprecating the metric if it outlives its 182 if there are functional changes, and deprecating the metric if it outlives its
169 usefulness. The owners should be added in the original histogram description. 183 usefulness. The owners should be added in the original histogram description.
170 If you are using a metric heavily and understand it intimately, feel free to 184 If you are using a metric heavily and understand it intimately, feel free to
171 add yourself as an owner. @chromium.org email addresses are preferred. 185 add yourself as an owner. @chromium.org email addresses are preferred.
172 186
173 187
174 ### Deleting Histogram Entries 188 ### Deleting Histogram Entries
175 189
176 Do not delete histograms from histograms.xml. Instead, mark unused histograms 190 Do not delete histograms from [histograms.xml](./histograms.xml). Instead, mark
177 as obsolete, annotating them with the associated date or milestone in the 191 unused histograms as obsolete, annotating them with the associated date or
178 obsolete tag entry. If your histogram is being replaced by a new version, we 192 milestone in the obsolete tag entry. If your histogram is being replaced by a
179 suggest noting that in the previous histogram's description. 193 new version, we suggest noting that in the previous histogram's description.
180 194
181 Deleting histogram entries would be bad if someone to accidentally reused your 195 Deleting histogram entries would be bad if someone to accidentally reused your
182 old histogram name and thereby corrupts new data with whatever old data is still 196 old histogram name and thereby corrupts new data with whatever old data is still
183 coming in. It's also useful to keep obsolete histogram descriptions in 197 coming in. It's also useful to keep obsolete histogram descriptions in
184 histograms.xml--that way, if someone is searching for a histogram to answer 198 [histograms.xml](./histograms.xml) -- that way, if someone is searching for a
185 a particular question, they can learn if there was a histogram at some point 199 histogram to answer a particular question, they can learn if there was a
186 that did so even if it isn't active now. 200 histogram at some point that did so even if it isn't active now.
187 201
188 ## When To Use Sparse Histograms 202 ## When To Use Sparse Histograms
189 203
190 Sparse histograms are well suited for recording counts of exact sample values 204 Sparse histograms are well suited for recording counts of exact sample values
191 that are sparsely distributed over a large range. 205 that are sparsely distributed over a large range.
192 206
193 The implementation uses a lock and a map, whereas other histogram types use a 207 The implementation uses a lock and a map, whereas other histogram types use a
194 vector and no lock. It is thus more costly to add values to, and each value 208 vector and no lock. It is thus more costly to add values to, and each value
195 stored has more overhead, compared to the other histogram types. However it 209 stored has more overhead, compared to the other histogram types. However it
196 may be more efficient in memory if the total number of sample values is small 210 may be more efficient in memory if the total number of sample values is small
197 compared to the range of their values. 211 compared to the range of their values.
198 212
199 For more information, see [sparse_histograms.h](https://cs.chromium.org/chromium /src/base/metrics/sparse_histogram.h). 213 For more information, see [sparse_histograms.h](https://cs.chromium.org/chromium /src/base/metrics/sparse_histogram.h).
OLDNEW
« no previous file with comments | « no previous file | no next file » | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698