add classes trace_analyzer::Query and TraceAnalyzer to make it easy to search through trace data

base/debug/trace_event_analyzer.h

+// Copyright (c) 2011 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+// Use base::debug::trace::Query and base::debug::TraceAnalyzer to search for
+// specific trace events that were generated by the trace_event.h API.
+//
+// The basic procedure:
+// - Get trace events JSON string from base::debug::TraceLog.

[nduca, 2011/10/13 01:23:58]

The more I think about it, a cleanup here is
- remove clearAssociations. if you want to change associations, make anotehr
analyzer?
- make setDefaultAssociations private
- once you findEvents, you can't call AssociateEvents?

+// - Pass the events string to TraceAnalyzer.

[nduca, 2011/10/13 01:23:58]

Isn't there an optional step here before FindEvents where you can call
AssociateEvents to add additional associations?

+// - Call TraceAnalyzer::FindEvents with queries to find specific events.
+//
+// A Query is a boolean expression tree that evaluates to true or false for a
+// given trace event. Queries can be combined into a tree using boolean,
+// arithmetic and comparison operators that refer to data of an individual trace
+// event.
+//
+// The events are returned as base::debug::TestTraceEvent objects.
+// TestTraceEvent contains a single trace event's data, as well as a pointer to
+// a related trace event. The related trace event is typically the matching end
+// of a begin event or the matching begin of an end event.
+//
+// The following examples use this basic setup code to construct TraceAnalyzer
+// with the json trace string retrieved from TraceLog and construct an event
+// vector for retrieving events:
+//
+// TraceAnalyzer analyzer(json_events);
+// ConstEventPtrVector events;

[nduca, 2011/10/13 01:23:58]

When I see this sort of thing, its a sign something is a little odd with the
API. Here, I get that you're trying to return a vector to pointers that don't
have to be managed.

You could call this a TraceEventVector.

Or, since this isn't perf code, you can make this a lot cleaner:
    vector<TraceEvent> FindEvents(...);

Yes, yes, you'd get in trouble currently because other_event.
BUT, you could do:
  struct TraceEvent {
    const TraceEventInternal* ptr;
    // accessors that delegate to ptr; 
  }

You're welcome to argue "API later" if you want.

+//
+// During construction, TraceAnalyzer::SetDefaultAssociations is called to
+// associate all matching begin/end pairs similar to how they are shown in
+// about:tracing.
+//
+// EXAMPLE 1: Find events named "my_event".
+//
+// analyzer.FindEvents(Query(EVENT_NAME) == "my_event", &events);
+//
+// EXAMPLE 2: Find begin events named "my_event" with duration > 1 second.
+//
+// Query q = (Query(EVENT_NAME) == "my_event" &&
+//            Query(EVENT_PHASE) == TRACE_EVENT_PHASE_BEGIN &&
+//            Query(EVENT_DURATION) > 1000000.0);
+// analyzer.FindEvents(q, &events);
+//
+// EXAMPLE 3: Matching begin/end events across threads.
+//
+// By default, only begin/end events on the same thread will ever be associated.

[nduca, 2011/10/13 01:23:58]

I dont think begin or end events should *ever* be allowed to get an other_event
on another thread.

+// Begin/end events on the same thread will ever be associated.
+// If the test needs to analyze something that starts and ends on different
threads, the test needs to use INSTANT events.
+// The typical procedure is to include a
+// unique ID as one of the TRACE_EVENT arguments that only matches a single
+// INSTANT event across all Chrome processes and threads.
+//

+// If the test needs to analyze events that begin and end on different threads,
+// it can specify custom assocations. The typical procedure is to include a
+// unique ID as one of the TRACE_EVENT arguments that only matches a single
+// begin/end pair across all Chrome processes and threads.
+//
+// Step 1: instrument code with custom begin/end trace events.
+//   [Thread 1 tracing code]
+//   TRACE_EVENT_INSTANT1("test_latency", "timing1_begin", "id", 3);
+//   [Thread 2 tracing code]
+//   TRACE_EVENT_INSTANT1("test_latency", "timing1_end", "id", 3);
+//
+// Step 2: associate these custom begin/end pairs.
+//   Query begin(Query(EVENT_NAME) == "timing1_begin");
+//   Query end(Query(EVENT_NAME) == "timing1_end");
+//   Query match(Query(EVENT_ARG, "id") == Query(OTHER_ARG, "id"));
+//   analyzer.AssociateEvents(begin, end, match);
+//
+// Step 3: search for "timing1_begin" events with existing other event.
+//   Query q = (Query(EVENT_NAME) == "timing1_begin" && Query(EVENT_HAS_OTHER));
+//   analyzer.FindEvents(q, &events);
+//
+// Step 4: analyze events, such as checking durations.
+//   for (size_t i = 0; i < events.size(); ++i) {
+//     double duration;
+//     EXPECT_TRUE(events[i].GetDuration(&duration));
+//     EXPECT_LT(duration, 1000000.0/60.0); // expect less than 1/60 second.
+//   }
+
+
+#ifndef BASE_DEBUG_TRACE_EVENT_ANALYZER_H_
+#define BASE_DEBUG_TRACE_EVENT_ANALYZER_H_
+#pragma once
+
+#include <map>
+
+#include "base/debug/trace_event.h"
+
+namespace base {
+class Value;
+
+namespace debug {
+
+namespace trace {
+class QueryNode;
+}
+
+// TestTraceEvent is a more convenient form of the TraceEvent class to make
+// tracing-based tests easier to write.
+struct TestTraceEvent {

[nduca, 2011/10/13 01:23:58]

I think I mentioned this somewhere else but just call this TraceEvent?

+  // PidTid contains a Process ID and Thread ID.
+  struct PidTid {
+    PidTid() : pid(0), tid(0) {}
+    PidTid(int pid, int tid) : pid(pid), tid(tid) {}
+    bool operator< (PidTid rhs) const {
+      if (pid != rhs.pid)
+        return pid < rhs.pid;
+      return tid < rhs.tid;
+    }
+    int pid;
+    int tid;
+  };
+
+  TestTraceEvent();
+  TestTraceEvent(const base::Value* event_value);
+  ~TestTraceEvent();
+
+  // Convert JSON string to array of TestTraceEvent.
+  // |output| is appended with the parsed events.
+  static bool ParseEventsFromJson(const std::string& json,
+                                  std::vector<TestTraceEvent>* output);
+
+  bool operator< (const TestTraceEvent& rhs) const {
+    return timestamp < rhs.timestamp;
+  }
+
+  bool has_other_event() const { return other_event; }
+
+  // Returns duration if has_other_event() is true.

[nduca, 2011/10/13 01:23:58]

Might make a note about the unit of the duration.

+  bool GetDuration(double* duration) const;
+
+  // Return the argument value if it exists and it is a string.
+  bool GetArgAsString(const std::string& name, std::string* arg) const;
+  // Return the argument value if it exists and it is a number.
+  bool GetArgAsNumber(const std::string& name, double* arg) const;
+
+  // Process ID and Thread ID.
+  PidTid pid_tid;
+  // Time since epoch in microseconds.
+  // Stored as double to match its JSON representation.
+  double timestamp;
+  TraceEventPhase phase;
+  std::string category;
+  std::string name;
+  // All numbers and bool values from TraceEvent args are cast to double.
+  // bool becomes 1.0 (true) or 0.0 (false).
+  std::map<std::string, double> arg_numbers;
+  std::map<std::string, std::string> arg_strings;
+  // The other event associated with this event (or NULL).
+  const TestTraceEvent* other_event;
+};
+
+// By using the trace namespace, tests can use short terms like "Query".

[nduca, 2011/10/13 01:23:58]

Presumably this will change with the new namespace plans...

+namespace trace {
+
+// Pass these values to Query to compare with the corresponding member of a
+// TestTraceEvent.
+enum TraceEventMember {
+  EVENT_INVALID,
+  // Use these to access the event members:
+  EVENT_PID,
+  EVENT_TID,
+  // Return the timestamp of the event in microseconds since epoch.
+  EVENT_TIME,
+  // Return the duration of an event in seconds.
+  // Only works for events with associated BEGIN/END: Query(EVENT_HAS_OTHER).
+  EVENT_DURATION,
+  EVENT_PHASE,
+  EVENT_CATEGORY,
+  EVENT_NAME,
+  EVENT_HAS_ARG,
+  EVENT_ARG,
+  // Return true if associated event exists.
+  // (Typically BEGIN for END or END for BEGIN).
+  EVENT_HAS_OTHER,
+  // Use these to access the associated event's members:
+  OTHER_PID,
+  OTHER_TID,
+  OTHER_TIME,
+  OTHER_PHASE,
+  OTHER_CATEGORY,
+  OTHER_NAME,
+  OTHER_HAS_ARG,
+  OTHER_ARG
+};
+
+class Query {
+ public:
+  // Compare with the given member.
+  Query(TraceEventMember member);
+
+  // Compare with the given member argument value.
+  Query(TraceEventMember member, const std::string& arg_name);
+
+  // Compare with the given string.
+  Query(const std::string& str);
+  Query(const char* str);
+
+  // Compare with the given number.
+  Query(double num);
+  Query(float num);
+  Query(int num);
+  Query(uint32 num);
+
+  // Compare with the given bool.
+  Query(bool boolean);
+
+  // Compare with the given phase.
+  Query(TraceEventPhase phase);
+
+  Query(const Query& query);
+
+  ~Query();
+
+  // Compare with the given string pattern. Only works with == and != operators.
+  // Example: Query(EVENT_NAME) == Query::Pattern("bla_*")
+  static Query Pattern(const std::string& pattern);
+
+  // Common queries:
+
+  // Find BEGIN events that have a corresponding END event.
+  static Query MatchBeginWithEnd() {
+    return (Query(EVENT_PHASE) == TRACE_EVENT_PHASE_BEGIN) &&
+           Query(EVENT_HAS_OTHER);
+  }
+
+  // Find END events that have a corresponding BEGIN event.
+  static Query MatchEndWithBegin() {
+    return (Query(EVENT_PHASE) == TRACE_EVENT_PHASE_END) &&
+           Query(EVENT_HAS_OTHER);
+  }
+
+  // Find BEGIN events of given |name| which also have associated END events.
+  static Query MatchBeginName(const std::string& name) {
+    return (Query(EVENT_NAME) == name) && MatchBeginWithEnd();
+  }
+
+  // Match given Process ID and Thread ID.
+  static Query MatchPidTid(base::debug::TestTraceEvent::PidTid pid_tid) {
+    return (Query(EVENT_PID) == pid_tid.pid) &&
+           (Query(EVENT_TID) == pid_tid.tid);
+  }
+
+  // Match event pair that spans multiple threads.
+  static Query MatchCrossPidTid() {
+    return (Query(EVENT_PID) != Query(OTHER_PID)) ||
+           (Query(EVENT_TID) != Query(OTHER_TID));
+  }
+
+  // Boolean operators:
+  Query operator==(const Query& rhs) const;
+  Query operator!=(const Query& rhs) const;
+  Query operator< (const Query& rhs) const;
+  Query operator<=(const Query& rhs) const;
+  Query operator> (const Query& rhs) const;
+  Query operator>=(const Query& rhs) const;
+  Query operator&&(const Query& rhs) const;
+  Query operator||(const Query& rhs) const;
+  Query operator! () const;
+
+  // Arithmetic operators:
+  // Following operators are applied to double arguments:
+  Query operator+(const Query& rhs) const;
+  Query operator-(const Query& rhs) const;
+  Query operator*(const Query& rhs) const;
+  Query operator/(const Query& rhs) const;
+  Query operator-() const;
+  // Mod operates on int64 args (doubles are casted to int64 beforehand):
+  Query operator%(const Query& rhs) const;
+
+  // Return true if the given event matches this query tree.
+  // This is a recursive method that walks the query tree.
+  bool Evaluate(const TestTraceEvent& event) const;
+
+ private:
+  enum Operator {
+    OP_INVALID,
+    // Boolean operators:
+    OP_EQ,
+    OP_NE,
+    OP_LT,
+    OP_LE,
+    OP_GT,
+    OP_GE,
+    OP_AND,
+    OP_OR,
+    OP_NOT,
+    // Arithmetic operators:
+    OP_ADD,
+    OP_SUB,
+    OP_MUL,
+    OP_DIV,
+    OP_MOD,
+    OP_NEGATE
+  };
+
+  enum QueryType {
+    QUERY_BOOLEAN_OPERATOR,
+    QUERY_ARITHMETIC_OPERATOR,
+    QUERY_EVENT_MEMBER,
+    QUERY_NUMBER,
+    QUERY_STRING
+  };
+
+  // Construct a boolean Query that returns (left <binary_op> right).
+  Query(const Query& left, const Query& right, Operator binary_op);
+
+  // Construct a boolean Query that returns (<binary_op> left).
+  Query(const Query& left, Operator unary_op);
+
+  // Try to compare left_ against right_ based on operator_.
+  // If either left or right does not convert to double, false is returned.
+  // Otherwise, true is returned and |result| is set to the comparison result.
+  bool CompareAsDouble(const TestTraceEvent& event, bool* result) const;
+
+  // Try to compare left_ against right_ based on operator_.
+  // If either left or right does not convert to string, false is returned.
+  // Otherwise, true is returned and |result| is set to the comparison result.
+  bool CompareAsString(const TestTraceEvent& event, bool* result) const;
+
+  // Attempt to convert this Query to a double. On success, true is returned
+  // and the double value is stored in |num|.
+  bool GetAsDouble(const TestTraceEvent& event, double* num) const;
+
+  // Attempt to convert this Query to a string. On success, true is returned
+  // and the string value is stored in |str|.
+  bool GetAsString(const TestTraceEvent& event, std::string* str) const;
+
+  // Evaluate this Query as an arithmetic operator on left_ and right_.
+  bool EvaluateArithmeticOperator(const TestTraceEvent& event,
+                                  double* num) const;
+
+  // For QUERY_EVENT_MEMBER Query: attempt to get the value of the Query.
+  // The TraceValue will either be TRACE_TYPE_DOUBLE, TRACE_TYPE_STRING,
+  // or if requested member does not exist, it will be TRACE_TYPE_UNDEFINED.
+  TraceValue GetMemberValue(const TestTraceEvent& event) const;
+
+  // Does this Query represent a value?
+  bool is_value() const { return type_ != QUERY_BOOLEAN_OPERATOR; }
+
+  bool is_unary_operator() const {
+    return operator_ == OP_NOT || operator_ == OP_NEGATE;
+  }
+
+  const Query& left() const;
+  const Query& right() const;
+
+  QueryType type_;
+  Operator operator_;
+  scoped_refptr<QueryNode> left_;
+  scoped_refptr<QueryNode> right_;
+  TraceEventMember member_;
+  double number_;
+  std::string string_;
+  bool is_pattern_;
+};
+
+// Implementation detail:
+// QueryNode allows Query to store a ref-counted query tree.
+class QueryNode : public RefCounted<QueryNode> {
+ public:
+  QueryNode(const Query& query);
+  const Query& query() const { return query_; }
+
+ private:
+  friend class RefCounted<QueryNode>;
+  ~QueryNode();
+
+  Query query_;
+};
+
+}  // namespace trace
+
+// TraceAnalyzer is designed to make tracing-based tests easier to write.
+class TraceAnalyzer {
+ public:
+  typedef std::vector<base::debug::TestTraceEvent> EventVector;
+  typedef std::vector<const base::debug::TestTraceEvent*> ConstEventPtrVector;
+
+  TraceAnalyzer();
+  TraceAnalyzer(const std::string& json_events);
+  TraceAnalyzer(const EventVector& events);

[nduca, 2011/10/13 01:23:58]

what uses this?

+  ~TraceAnalyzer();
+
+  // Replace all events with |json_events|.
+  void SetEvents(const std::string& json_events);
+  // Replace all events with |events|.
+  void SetEvents(const EventVector& events);
+
+  // SetEvents calls this internally to match up typical begin/end pairs of
+  // events. This allows Query(OTHER_*) to access the associated event and
+  // enables Query(EVENT_DURATION).
+  // By default, an end event will match the most recent begin event with the
+  // same name, category, process ID and thread ID.
+  void SetDefaultAssociations();
+
+  // Clear existing event associations.
+  void ClearAssociations();
+
+  // By default, matching begin and end events are associated with each other as
+  // described in SetDefaultAssociations.
+  // AssociateEvents can be used to customize begin/end event associations.
+  // The assumptions are:
+  // - end events occur after begin events.
+  // - the closest matching end event is the best match.
+  //
+  // |begin| - Eligible begin events match this query.
+  // |end|   - Eligible end events match this query.
+  // |match| - This query is run on the begin event. The OTHER event members
+  //           will point to an eligible end event. The query should evaluate to
+  //           true if the begin/end pair is a match.
+  //
+  // When a match is found, the pair will be associated by having their
+  // other_event member point to each other. Non-matching events are left with
+  // their existing assocations, so you may also want to call ClearAssociations.
+  void AssociateEvents(const trace::Query& begin,
+                       const trace::Query& end,
+                       const trace::Query& match);
+
+  const EventVector& events() { return raw_events_; }

[nduca, 2011/10/13 01:23:58]

Why expose this? Couldn't one do FindEvents(Query(1))?

+
+  const std::string& GetThreadName(const base::debug::TestTraceEvent::PidTid&
+                                   pid_tid);
+
+  // Find all events that match query and replace output vector.
+  size_t FindEvents(const trace::Query& query,
+                    ConstEventPtrVector* output) const;
+
+  // Helper method: find first event that matches query
+  const base::debug::TestTraceEvent* FindOneEvent(
+      const trace::Query& query) const;
+
+ private:
+  // Read metadata (thread names, etc) from events.
+  void ParseMetadata();
+
+  std::map<base::debug::TestTraceEvent::PidTid, std::string> thread_names_;
+  EventVector raw_events_;
+};
+
+}  // namespace debug
+}  // namespace base
+
+#endif  // BASE_DEBUG_TRACE_EVENT_ANALYZER_H_