Use lock-free lazy initialization for static histogram references

base/metrics/histogram.h

 // Copyright (c) 2011 The Chromium Authors. All rights reserved.
 // Use of this source code is governed by a BSD-style license that can be
 // found in the LICENSE file.
 
 // Histogram is an object that aggregates statistics, and can summarize them in
 // various forms, including ASCII graphical, HTML, and numerically (as a
 // vector of numbers corresponding to each of the aggregating buckets).
 
 // It supports calls to accumulate either time intervals (which are processed
 // as integral number of milliseconds), or arbitrary integral units.
@@ skipping 10 matching lines @@
 // The above example has an exponential ratio of 2 (doubling the bucket width
 // in each consecutive bucket.  The Histogram class automatically calculates
 // the smallest ratio that it can use to construct the number of buckets
 // selected in the constructor.  An another example, if you had 50 buckets,
 // and millisecond time values from 1 to 10000, then the ratio between
 // consecutive bucket widths will be approximately somewhere around the 50th
 // root of 10000.  This approach provides very fine grain (narrow) buckets
 // at the low end of the histogram scale, but allows the histogram to cover a
 // gigantic range with the addition of very few buckets.
 
+// Histograms use a pattern involving a function static variable, that is a
+// pointer to a histogram.  This static is explicitly initialized on any thread
+// that detects a uninitialized (NULL) pointer.  The potentially racy
+// initialization is not a problem as it is always set to point to the same
+// value (i.e., the FactoryGet always returns the same value).  FactoryGet
+// is also completely thread safe, which results in a completely thread safe,
+// and relatively fast, set of counters.  To avoid races at shutdown, the static
+// pointer is NOT deleted, and we leak the histograms at process termination.

[rvargas (doing something else), 2011/04/01 19:20:52]

Have you considered using LazyInstance<> instead? It was designed to provide
exactly this functionality, and it doesn't leak.

[jar (doing other things), 2011/04/01 21:50:27]

As per discussion, one major point is that this code runs in any thread,
including worker threads (that are not joined at shutdown).  In addition, the
"statics" can often refer to a single "shared" (and registered) histogram.  This
precludes use of a singular static for a single named histogram (and is part of
why we call a factory, rather than new).

Bottom line: I couldn't see a way to reuse LazyInstance code.


On 2011/04/01 19:20:52, rvargas wrote:

[willchan no longer on Chromium, 2011/04/02 07:43:31]

I think you misunderstand LazyInstance. Both LazyInstance and Singleton provide
"leaky" traits (LeakyLazyInstanceTraits) that don't delete on shutdown. This
means they are not vulnerable to shutdown races/crashes as you imply.

Secondly, just like the "statics" can often refer to a single "shared" (and
registered) histogram, LazyInstance can do exactly the same thing. Singleton is
globally scoped (hence being called a Singleton), but LayInstance is simply a
lazily constructed (using a spinlock) instance. So, it's somewhat equivalent to
a threadsafe initialized function static which may register itself to the
AtExitManager (DefaultLazyInstanceTraits) for destruction on shutdown, or may
just leak (LeakyLazyInstanceTraits).

The static is cheaper because it doesn't have to pay for the spinlock on each
access.

+
 #ifndef BASE_METRICS_HISTOGRAM_H_
 #define BASE_METRICS_HISTOGRAM_H_
 #pragma once
 
 #include <map>
 #include <string>
 #include <vector>
 
 #include "base/base_api.h"
 #include "base/gtest_prod_util.h"
 #include "base/logging.h"
-#include "base/memory/ref_counted.h"
 #include "base/time.h"
 
 class Pickle;
 
 namespace base {
 
 class Lock;
 
 //------------------------------------------------------------------------------
 // Provide easy general purpose histogram in a macro, just like stats counters.
 // The first four macros use 50 buckets.
 
 #define HISTOGRAM_TIMES(name, sample) HISTOGRAM_CUSTOM_TIMES( \
     name, sample, base::TimeDelta::FromMilliseconds(1), \
     base::TimeDelta::FromSeconds(10), 50)
 
 #define HISTOGRAM_COUNTS(name, sample) HISTOGRAM_CUSTOM_COUNTS( \
     name, sample, 1, 1000000, 50)
 
 #define HISTOGRAM_COUNTS_100(name, sample) HISTOGRAM_CUSTOM_COUNTS( \
     name, sample, 1, 100, 50)
 
 #define HISTOGRAM_COUNTS_10000(name, sample) HISTOGRAM_CUSTOM_COUNTS( \
     name, sample, 1, 10000, 50)
 
 #define HISTOGRAM_CUSTOM_COUNTS(name, sample, min, max, bucket_count) do { \
-    scoped_refptr<base::Histogram> counter = \
-        base::Histogram::FactoryGet(name, min, max, bucket_count, \
-                                    base::Histogram::kNoFlags); \
+    static base::Histogram* counter(NULL); \
+    if (!counter) \
+      counter = base::Histogram::FactoryGet(name, min, max, bucket_count, \
+                                            base::Histogram::kNoFlags); \
     DCHECK_EQ(name, counter->histogram_name()); \
-    if (counter.get()) counter->Add(sample); \
+    counter->Add(sample); \
   } while (0)
 
 #define HISTOGRAM_PERCENTAGE(name, under_one_hundred) \
     HISTOGRAM_ENUMERATION(name, under_one_hundred, 101)
 
 // For folks that need real specific times, use this to select a precise range
 // of times you want plotted, and the number of buckets you want used.
 #define HISTOGRAM_CUSTOM_TIMES(name, sample, min, max, bucket_count) do { \
-    scoped_refptr<base::Histogram> counter = \
-        base::Histogram::FactoryTimeGet(name, min, max, bucket_count, \
-                                        base::Histogram::kNoFlags); \
+    static base::Histogram* counter(NULL); \
+    if (!counter) \
+      counter = base::Histogram::FactoryTimeGet(name, min, max, bucket_count, \
+                                                base::Histogram::kNoFlags); \
     DCHECK_EQ(name, counter->histogram_name()); \
-    if (counter.get()) counter->AddTime(sample); \
+    counter->AddTime(sample); \
   } while (0)
 
-// DO NOT USE THIS.  It is being phased out, in favor of HISTOGRAM_CUSTOM_TIMES.
+// DO NOT USE THIS.  It is being phased out, in favor of HISTOGRAM_CUSTOM_TIMES. ....

[ramant (doing other things), 2011/04/01 20:05:38]

More than 80 characters. Did you want to delete this macro?

[jar (doing other things), 2011/04/01 21:50:27]

Done.

 #define HISTOGRAM_CLIPPED_TIMES(name, sample, min, max, bucket_count) do { \
-    scoped_refptr<base::Histogram> counter = \
-        base::Histogram::FactoryTimeGet(name, min, max, bucket_count, \
-                                        base::Histogram::kNoFlags); \
+    static base::Histogram* counter(NULL); \
+    if (!counter) \
+      counter = base::Histogram::FactoryTimeGet(name, min, max, bucket_count, \
+                                                base::Histogram::kNoFlags); \
     DCHECK_EQ(name, counter->histogram_name()); \
-    if ((sample) < (max) && counter.get()) counter->AddTime(sample); \
+    if ((sample) < (max)) counter->AddTime(sample); \
   } while (0)
 
 // Support histograming of an enumerated value.  The samples should always be
 // less than boundary_value.
 
 #define HISTOGRAM_ENUMERATION(name, sample, boundary_value) do { \
-    scoped_refptr<base::Histogram> counter = \
-        base::LinearHistogram::FactoryGet(name, 1, boundary_value, \
-                                          boundary_value + 1, \
-                                          base::Histogram::kNoFlags); \
+    static base::Histogram* counter(NULL); \
+    if (!counter) \
+      counter = base::LinearHistogram::FactoryGet(name, 1, boundary_value, \
+          boundary_value + 1, base::Histogram::kNoFlags); \
     DCHECK_EQ(name, counter->histogram_name()); \
-    if (counter.get()) counter->Add(sample); \
+    counter->Add(sample); \
   } while (0)
 
 #define HISTOGRAM_CUSTOM_ENUMERATION(name, sample, custom_ranges) do { \
-    scoped_refptr<base::Histogram> counter = \
-        base::CustomHistogram::FactoryGet(name, custom_ranges, \
-                                          base::Histogram::kNoFlags); \
+    static base::Histogram* counter(NULL); \
+    if (!counter) \
+      counter = base::CustomHistogram::FactoryGet(name, custom_ranges, \
+                                                  base::Histogram::kNoFlags); \
     DCHECK_EQ(name, counter->histogram_name()); \
-    if (counter.get()) counter->Add(sample); \
+    counter->Add(sample); \
   } while (0)
 
 
 //------------------------------------------------------------------------------
 // Define Debug vs non-debug flavors of macros.
 #ifndef NDEBUG
 
 #define DHISTOGRAM_TIMES(name, sample) HISTOGRAM_TIMES(name, sample)
 #define DHISTOGRAM_COUNTS(name, sample) HISTOGRAM_COUNTS(name, sample)
 #define DHISTOGRAM_PERCENTAGE(name, under_one_hundred) HISTOGRAM_PERCENTAGE(\
@@ skipping 39 matching lines @@
 #define UMA_HISTOGRAM_MEDIUM_TIMES(name, sample) UMA_HISTOGRAM_CUSTOM_TIMES( \
     name, sample, base::TimeDelta::FromMilliseconds(10), \
     base::TimeDelta::FromMinutes(3), 50)
 
 // Use this macro when times can routinely be much longer than 10 seconds.
 #define UMA_HISTOGRAM_LONG_TIMES(name, sample) UMA_HISTOGRAM_CUSTOM_TIMES( \
     name, sample, base::TimeDelta::FromMilliseconds(1), \
     base::TimeDelta::FromHours(1), 50)
 
 #define UMA_HISTOGRAM_CUSTOM_TIMES(name, sample, min, max, bucket_count) do { \
-    scoped_refptr<base::Histogram> counter = \
-        base::Histogram::FactoryTimeGet(name, min, max, bucket_count, \
+    static base::Histogram* counter(NULL); \
+    if (!counter) \
+      counter = base::Histogram::FactoryTimeGet(name, min, max, bucket_count, \
             base::Histogram::kUmaTargetedHistogramFlag); \
     DCHECK_EQ(name, counter->histogram_name()); \
-    if (counter.get()) counter->AddTime(sample); \
+    counter->AddTime(sample); \
   } while (0)
 
 // DO NOT USE THIS.  It is being phased out, in favor of HISTOGRAM_CUSTOM_TIMES.
 #define UMA_HISTOGRAM_CLIPPED_TIMES(name, sample, min, max, bucket_count) do { \
-    scoped_refptr<base::Histogram> counter = \
-        base::Histogram::FactoryTimeGet(name, min, max, bucket_count, \
-            base::Histogram::kUmaTargetedHistogramFlag); \
+    static base::Histogram* counter(NULL); \
+    if (!counter) \
+      counter = base::Histogram::FactoryTimeGet(name, min, max, bucket_count, \
+           base::Histogram::kUmaTargetedHistogramFlag); \
     DCHECK_EQ(name, counter->histogram_name()); \
-    if ((sample) < (max) && counter.get()) counter->AddTime(sample); \
+    if ((sample) < (max)) counter->AddTime(sample); \
   } while (0)
 
 #define UMA_HISTOGRAM_COUNTS(name, sample) UMA_HISTOGRAM_CUSTOM_COUNTS( \
     name, sample, 1, 1000000, 50)
 
 #define UMA_HISTOGRAM_COUNTS_100(name, sample) UMA_HISTOGRAM_CUSTOM_COUNTS( \
     name, sample, 1, 100, 50)
 
 #define UMA_HISTOGRAM_COUNTS_10000(name, sample) UMA_HISTOGRAM_CUSTOM_COUNTS( \
     name, sample, 1, 10000, 50)
 
 #define UMA_HISTOGRAM_CUSTOM_COUNTS(name, sample, min, max, bucket_count) do { \
-    scoped_refptr<base::Histogram> counter = \
-       base::Histogram::FactoryGet(name, min, max, bucket_count, \
-           base::Histogram::kUmaTargetedHistogramFlag); \
+    static base::Histogram* counter(NULL); \
+    if (!counter) \
+      counter = base::Histogram::FactoryGet(name, min, max, bucket_count, \
+          base::Histogram::kUmaTargetedHistogramFlag); \
     DCHECK_EQ(name, counter->histogram_name()); \
-    if (counter.get()) counter->Add(sample); \
+    counter->Add(sample); \
   } while (0)
 
 #define UMA_HISTOGRAM_MEMORY_KB(name, sample) UMA_HISTOGRAM_CUSTOM_COUNTS( \
     name, sample, 1000, 500000, 50)
 
 #define UMA_HISTOGRAM_MEMORY_MB(name, sample) UMA_HISTOGRAM_CUSTOM_COUNTS( \
     name, sample, 1, 1000, 50)
 
 #define UMA_HISTOGRAM_PERCENTAGE(name, under_one_hundred) \
     UMA_HISTOGRAM_ENUMERATION(name, under_one_hundred, 101)
 
 #define UMA_HISTOGRAM_ENUMERATION(name, sample, boundary_value) do { \
-    scoped_refptr<base::Histogram> counter = \
-        base::LinearHistogram::FactoryGet(name, 1, boundary_value, \
-            boundary_value + 1, base::Histogram::kUmaTargetedHistogramFlag); \
+    static base::Histogram* counter(NULL); \
+    if (!counter) \
+      counter = base::LinearHistogram::FactoryGet(name, 1, boundary_value, \
+          boundary_value + 1, base::Histogram::kUmaTargetedHistogramFlag); \
     DCHECK_EQ(name, counter->histogram_name()); \
-    if (counter.get()) counter->Add(sample); \
+    counter->Add(sample); \
   } while (0)
 
 #define UMA_HISTOGRAM_CUSTOM_ENUMERATION(name, sample, custom_ranges) do { \
-    scoped_refptr<base::Histogram> counter = \
-        base::CustomHistogram::FactoryGet(name, custom_ranges, \
-            base::Histogram::kUmaTargetedHistogramFlag); \
+    static base::Histogram* counter(NULL); \
+    if (!counter) \
+      counter = base::CustomHistogram::FactoryGet(name, custom_ranges, \
+          base::Histogram::kUmaTargetedHistogramFlag); \
     DCHECK_EQ(name, counter->histogram_name()); \
-    if (counter.get()) counter->Add(sample); \
+    counter->Add(sample); \
   } while (0)
 
 //------------------------------------------------------------------------------
 
 class BooleanHistogram;
 class CustomHistogram;
 class Histogram;
 class LinearHistogram;
 
-class BASE_API Histogram : public base::RefCountedThreadSafe<Histogram> {
+class BASE_API Histogram {
  public:
   typedef int Sample;  // Used for samples (and ranges of samples).
   typedef int Count;  // Used to count samples in a bucket.
   static const Sample kSampleType_MAX = INT_MAX;
   // Initialize maximum number of buckets in histograms as 16,384.
   static const size_t kBucketCount_MAX;
 
   typedef std::vector<Count> Counts;
   typedef std::vector<Sample> Ranges;
 
@@ skipping 89 matching lines @@
     // that due to races in histogram accumulation (if a histogram is indeed
     // updated on several threads simultaneously), the tallies might mismatch,
     // and also the snapshotting code may asynchronously get a mismatch (though
     // generally either race based mismatch cause is VERY rare).
     int64 redundant_count_;
   };
 
   //----------------------------------------------------------------------------
   // minimum should start from 1. 0 is invalid as a minimum. 0 is an implicit
   // default underflow bucket.
-  static scoped_refptr<Histogram> FactoryGet(const std::string& name,
-      Sample minimum, Sample maximum, size_t bucket_count, Flags flags);
-  static scoped_refptr<Histogram> FactoryTimeGet(const std::string& name,
-      base::TimeDelta minimum, base::TimeDelta maximum, size_t bucket_count,
-      Flags flags);
+  static Histogram* FactoryGet(const std::string& name,
+                               Sample minimum,
+                               Sample maximum,
+                               size_t bucket_count,
+                               Flags flags);
+  static Histogram* FactoryTimeGet(const std::string& name,
+                                   base::TimeDelta minimum,
+                                   base::TimeDelta maximum,
+                                   size_t bucket_count,
+                                   Flags flags);
 
   void Add(int value);
 
   // This method is an interface, used only by BooleanHistogram.
   virtual void AddBoolean(bool value);
 
   // Accept a TimeDelta to increment.
   void AddTime(TimeDelta time) {
     Add(static_cast<int>(time.InMilliseconds()));
   }
@@ skipping 54 matching lines @@
   virtual bool HasConstructorArguments(Sample minimum, Sample maximum,
                                        size_t bucket_count);
 
   virtual bool HasConstructorTimeDeltaArguments(TimeDelta minimum,
                                                 TimeDelta maximum,
                                                 size_t bucket_count);
   // Return true iff the range_checksum_ matches current ranges_ vector.
   bool HasValidRangeChecksum() const;
 
  protected:
-  friend class base::RefCountedThreadSafe<Histogram>;
   Histogram(const std::string& name, Sample minimum,
             Sample maximum, size_t bucket_count);
   Histogram(const std::string& name, TimeDelta minimum,
             TimeDelta maximum, size_t bucket_count);
 
   virtual ~Histogram();
 
   // Initialize ranges_ mapping.
   void InitializeBucketRange();
 
@@ skipping 33 matching lines @@
 
   virtual uint32 CalculateRangeChecksum() const;
 
  private:
   // Allow tests to corrupt our innards for testing purposes.
   FRIEND_TEST(HistogramTest, CorruptBucketBounds);
   FRIEND_TEST(HistogramTest, CorruptSampleCounts);
   FRIEND_TEST(HistogramTest, Crc32SampleHash);
   FRIEND_TEST(HistogramTest, Crc32TableTest);
 
+  friend class StatisticsRecorder;  // To allow it to delete duplicates.
+
   // Post constructor initialization.
   void Initialize();
 
   // Checksum function for accumulating range values into a checksum.
   static uint32 Crc32(uint32 sum, Sample range);
 
   //----------------------------------------------------------------------------
   // Helpers for emitting Ascii graphic.  Each method appends data to output.
 
   // Find out how large the (graphically) the largest bucket will appear to be.
@@ skipping 55 matching lines @@
 //------------------------------------------------------------------------------
 
 // LinearHistogram is a more traditional histogram, with evenly spaced
 // buckets.
 class BASE_API LinearHistogram : public Histogram {
  public:
   virtual ~LinearHistogram();
 
   /* minimum should start from 1. 0 is as minimum is invalid. 0 is an implicit
      default underflow bucket. */
-  static scoped_refptr<Histogram> FactoryGet(const std::string& name,
-      Sample minimum, Sample maximum, size_t bucket_count, Flags flags);
-  static scoped_refptr<Histogram> FactoryTimeGet(const std::string& name,
-      TimeDelta minimum, TimeDelta maximum, size_t bucket_count,
-      Flags flags);
+  static Histogram* FactoryGet(const std::string& name,
+                               Sample minimum,
+                               Sample maximum,
+                               size_t bucket_count,
+                               Flags flags);
+  static Histogram* FactoryTimeGet(const std::string& name,
+                                   TimeDelta minimum,
+                                   TimeDelta maximum,
+                                   size_t bucket_count,
+                                   Flags flags);
 
   // Overridden from Histogram:
   virtual ClassType histogram_type() const;
 
   // Store a list of number/text values for use in rendering the histogram.
   // The last element in the array has a null in its "description" slot.
   virtual void SetRangeDescriptions(const DescriptionPair descriptions[]);
 
  protected:
   LinearHistogram(const std::string& name, Sample minimum,
@@ skipping 22 matching lines @@
   BucketDescriptionMap bucket_description_;
 
   DISALLOW_COPY_AND_ASSIGN(LinearHistogram);
 };
 
 //------------------------------------------------------------------------------
 
 // BooleanHistogram is a histogram for booleans.
 class BASE_API BooleanHistogram : public LinearHistogram {
  public:
-  static scoped_refptr<Histogram> FactoryGet(const std::string& name,
-      Flags flags);
+  static Histogram* FactoryGet(const std::string& name, Flags flags);
 
   virtual ClassType histogram_type() const;
 
   virtual void AddBoolean(bool value);
 
  private:
   explicit BooleanHistogram(const std::string& name);
 
   DISALLOW_COPY_AND_ASSIGN(BooleanHistogram);
 };
 
 //------------------------------------------------------------------------------
 
 // CustomHistogram is a histogram for a set of custom integers.
 class BASE_API CustomHistogram : public Histogram {
  public:
 
-  static scoped_refptr<Histogram> FactoryGet(const std::string& name,
-      const std::vector<Sample>& custom_ranges, Flags flags);
+  static Histogram* FactoryGet(const std::string& name,
+                               const std::vector<Sample>& custom_ranges,
+                               Flags flags);
 
   // Overridden from Histogram:
   virtual ClassType histogram_type() const;
 
  protected:
   CustomHistogram(const std::string& name,
                   const std::vector<Sample>& custom_ranges);
 
   // Initialize ranges_ mapping.
   void InitializedCustomBucketRange(const std::vector<Sample>& custom_ranges);
   virtual double GetBucketSize(Count current, size_t i) const;
 
   DISALLOW_COPY_AND_ASSIGN(CustomHistogram);
 };
 
 //------------------------------------------------------------------------------
 // StatisticsRecorder handles all histograms in the system.  It provides a
 // general place for histograms to register, and supports a global API for
 // accessing (i.e., dumping, or graphing) the data in all the histograms.
 
 class BASE_API StatisticsRecorder {
  public:
-  typedef std::vector<scoped_refptr<Histogram> > Histograms;
+  typedef std::vector<Histogram*> Histograms;
 
   StatisticsRecorder();
 
   ~StatisticsRecorder();
 
   // Find out if histograms can now be registered into our list.
   static bool IsActive();
 
   // Register, or add a new histogram to the collection of statistics. If an
   // identically named histogram is already registered, then the argument
-  // |histogram| will be replaced by the previously registered value, discarding
-  // the referenced argument.
-  static void RegisterOrDiscardDuplicate(scoped_refptr<Histogram>* histogram);
+  // |histogram| will deleted.  The returns value is always the registered

[rvargas (doing something else), 2011/04/01 19:20:52]

nit: the returned value

[jar (doing other things), 2011/04/01 21:50:27]

Done.

+  // histogram (either the argument, or the pre-existing registered histogram).
+  static Histogram* RegisterOrDeleteDuplicate(Histogram* histogram);
 
   // Methods for printing histograms.  Only histograms which have query as
   // a substring are written to output (an empty string will process all
   // registered histograms).
   static void WriteHTMLGraph(const std::string& query, std::string* output);
   static void WriteGraph(const std::string& query, std::string* output);
 
   // Method for extracting histograms which were marked for use by UMA.
   static void GetHistograms(Histograms* output);
 
   // Find a histogram by name. It matches the exact name. This method is thread
   // safe.  If a matching histogram is not found, then the |histogram| is
   // not changed.
-  static bool FindHistogram(const std::string& query,
-                            scoped_refptr<Histogram>* histogram);
+  static bool FindHistogram(const std::string& query, Histogram** histogram);
 
   static bool dump_on_exit() { return dump_on_exit_; }
 
   static void set_dump_on_exit(bool enable) { dump_on_exit_ = enable; }
 
   // GetSnapshot copies some of the pointers to registered histograms into the
   // caller supplied vector (Histograms).  Only histograms with names matching
   // query are returned. The query must be a substring of histogram name for its
   // pointer to be copied.
   static void GetSnapshot(const std::string& query, Histograms* snapshot);
 
 
  private:
   // We keep all registered histograms in a map, from name to histogram.
-  typedef std::map<std::string, scoped_refptr<Histogram> > HistogramMap;
+  typedef std::map<std::string, Histogram*> HistogramMap;
 
   static HistogramMap* histograms_;
 
   // lock protects access to the above map.
   static base::Lock* lock_;
 
   // Dump all known histograms to log.
   static bool dump_on_exit_;
 
   DISALLOW_COPY_AND_ASSIGN(StatisticsRecorder);
 };
 
 }  // namespace base
 
 #endif  // BASE_METRICS_HISTOGRAM_H_