[wrapper-tracing] Add heap snapshot generator infrastructure

third_party/WebKit/Source/bindings/core/v8/V8GCController.cpp

Index: third_party/WebKit/Source/bindings/core/v8/V8GCController.cpp
diff --git a/third_party/WebKit/Source/bindings/core/v8/V8GCController.cpp b/third_party/WebKit/Source/bindings/core/v8/V8GCController.cpp
index 6ded783a981de097db3e6563388ac6f197bae47a..1820b624cb8a17dd015ff43dfc6009b71b65422d 100644
--- a/third_party/WebKit/Source/bindings/core/v8/V8GCController.cpp
+++ b/third_party/WebKit/Source/bindings/core/v8/V8GCController.cpp
@@ -51,6 +51,8 @@
 #include "wtf/Vector.h"
 #include "wtf/allocator/Partitions.h"
 #include <algorithm>
+#include <unordered_map>
+#include <unordered_set>
 
 namespace blink {
 
@@ -148,6 +150,140 @@ class MinorGCUnmodifiedWrapperVisitor : public v8::PersistentHandleVisitor {
   v8::Isolate* m_isolate;
 };
 
+class HeapSnaphotWrapperVisitor : public ScriptWrappableVisitor,
+                                  public v8::PersistentHandleVisitor {
+ public:
+  explicit HeapSnaphotWrapperVisitor(v8::Isolate* isolate)
+      : ScriptWrappableVisitor(isolate),
+        m_currentParent(nullptr),
+        m_onlyTraceSingleLevel(false),
+        m_firstTracedScriptWrappable(false) {
+    m_nodesRequiringTracing.clear();
+  }
+
+  void VisitPersistentHandle(v8::Persistent<v8::Value>* value,
+                             uint16_t classId) override {
+    if (classId != WrapperTypeInfo::NodeClassId)
+      return;
+
+    DCHECK(!value->IsIndependent());
+    v8::Local<v8::Object> wrapper = v8::Local<v8::Object>::New(
+        m_isolate, v8::Persistent<v8::Object>::Cast(*value));
+    DCHECK(V8Node::hasInstance(wrapper, m_isolate));
+    Node* node = V8Node::toImpl(wrapper);
+    Node* root = V8GCController::opaqueRootForGC(m_isolate, node);
+    m_nodesRequiringTracing[root].push_back(node);

[haraken, 2017/01/16 02:46:56]

Why don't you need to push Nodes that are found during tracePendingActivities?
i.e., why do you need to push only Nodes that are found during
VisitPersistentHandle?

[Michael Lippautz, 2017/01/16 10:45:06]

VisitPersistentHandle visits all persistent handles V8 knows about. If a "Node"
would be   a pending activity, we would also catch it here. 

This mirrors the groups that are currently collected for devtools, which
basically groups DOM nodes by DOM tree and puts pending activities in a separate
group.

[haraken, 2017/01/16 12:48:13]

What happens if there is a pending activity on a DOM object X (which is not a
Node) and there are Nodes reachable from X?

(But I agree with you that this CL keeps the existing behavior.)

[Michael Lippautz, 2017/01/16 13:15:38]

Then tracePendingActivities will catch it.

+  }
+
+  // Trace through the blink heap to find all V8 wrappers reachable from
+  // ActiveScriptWrappables. Also collect retainer edges on the way.
+  void tracePendingActivities() {
+    CHECK(m_tempFoundV8Wrappers.empty());
+    m_currentParent = nullptr;
+
+    TracePrologue();
+    ActiveScriptWrappableBase::traceActiveScriptWrappables(m_isolate, this);
+    AdvanceTracing(
+        0,
+        v8::EmbedderHeapTracer::AdvanceTracingActions(
+            v8::EmbedderHeapTracer::ForceCompletionAction::FORCE_COMPLETION));
+    // Abort instead of Epilogue as we want to finish synchronously.
+    AbortTracing();

[haraken, 2017/01/16 02:46:56]

What's a difference between AbortTracing and Epilogue? Does AbortTracing finish
the current tracing forcibly? (I was assuming that AbortTracing gives up the
ongoing incremental tracing, throwing away various states.)

[Michael Lippautz, 2017/01/16 10:45:06]

Since we finished the line before with FORE_COMPLETION, the only difference is
that we synchronously do performCleanup(). The is no idle time here, so a
asynchronous cleanup using tasks doesn't make sense.

We might want to add this possibility to the tracer API, e.g. have
TraceEpilogue(EmbedderHeapTracer::SYNCHRONOUS);

[haraken, 2017/01/16 12:48:12]

Yeah, that looks tidier. (You can address it in a follow-up though.)

[Michael Lippautz, 2017/01/16 13:15:38]

Acknowledged.

+
+    m_groups.push_back(
+        std::make_pair(new SuspendableObjectsInfo(m_tempFoundV8Wrappers.size()),
+                       std::move(m_tempFoundV8Wrappers)));
+  }
+
+  // Trace through the blink heap to find all V8 wrappers reachable from
+  // the nodes that require tracing. Also collect retainer edges on the way.
+  void traceNodesThatRequireTracing() {

[haraken, 2017/01/16 02:46:56]

I'm not sure why we need the complexity of traceNodesThatRequireTracing. Would
it be possible to directly call findV8WrappersDirectlyReachableFrom() when you
find a Node object during VisitPersistentHandle? Then we won't need
m_nodesRequiringTracing.

[Michael Lippautz, 2017/01/16 10:45:06]

We cannot do that, as we need a mapping from nodes to RetainedDOMInfo. If we
would trace in VisitPersistentHandle, we would need a side map from
root->RetainedDOMInfo, as we would need to keep track of that mapping. 

I think the 2-pass approach is slightly easier to understand.

+    for (auto& groupPair : m_nodesRequiringTracing) {
+      v8::HeapProfiler::RetainerChildren groupChildren;
+      for (auto& node : groupPair.second) {
+        auto wrappers = findV8WrappersDirectlyReachableFrom(node);
+        groupChildren.insert(wrappers.begin(), wrappers.end());
+      }
+      m_groups.push_back(std::make_pair(new RetainedDOMInfo(groupPair.first),
+                                        std::move(groupChildren)));
+    }
+  }
+
+  v8::HeapProfiler::RetainerEdges edges() { return std::move(m_edges); }
+  v8::HeapProfiler::RetainerGroups groups() { return std::move(m_groups); }
+
+  void markWrapper(const v8::PersistentBase<v8::Value>* value) const override {
+    if (!m_tracingInProgress)

[haraken, 2017/01/16 02:46:56]

Can this happen?

[Michael Lippautz, 2017/01/16 10:45:06]

Removed as it cannot happen. This was copied over from the regular tracer.

+      return;
+    if (m_currentParent && m_currentParent != value)
+      m_edges.push_back(std::make_pair(m_currentParent, value));
+    m_tempFoundV8Wrappers.insert(value);
+  }
+
+  void dispatchTraceWrappers(const TraceWrapperBase* traceable) const override {
+    if (!m_onlyTraceSingleLevel || !traceable->isScriptWrappable() ||
+        m_firstTracedScriptWrappable) {
+      m_firstTracedScriptWrappable = false;
+      traceable->traceWrappers(this);
+    }

[haraken, 2017/01/16 02:46:56]

Don't we need to reset m_firstTracedScriptWrappable to true after calling
traceWrappers?

Otherwise, findV8WrappersDirectlyReachableFrom will always return only one
ScriptWrappable, right?

[Michael Lippautz, 2017/01/16 10:45:06]

Yes, that's the intention. It should find wrappers reachable from this exact
ScriptWrappable.

It will trace through non-scriptwrappables (e.g. NodeRareData and friends)
though.

[haraken, 2017/01/16 12:48:13]

But what happens if there are multiple ScriptWrappables (at the first level)
reachable from the Node?

[Michael Lippautz, 2017/01/16 13:15:38]

They are traced and we insert edges from the v8 wrapper represented by the
current ScriptWrappable to all v8 wrappers of the children ScriptWrappables.

The details:
- We add m_currentParent (=ScriptWrappable) to the deque through the call
traceable->wrapperTypeInfo()->traceWrappers(this, traceable) on line 241.
- Call to AdvanceTracing. There's only a single object on the marking deque.
- m_firstTracedScriptWrappableTraced is false, so we can trace this object.
- Tracing enqueues all children (ScriptWrappables, and special cases) onto the
deque.
- m_firstTracedScriptWrappableTraced is set to true, so only special cases will
be further traced (effectively hiding the intermediate links from the snapshot)
- All found V8 wrappers from this one level of tracing will be assigned an edge
starting at the initial wrapper.

[haraken, 2017/01/17 00:31:25]

Imagine the following object graph:

  Node --> TraceWrappable1 --> ScriptWrappable1
   |
   ----> TraceWrappable2 --> TraceWrappable3 --> ScriptWrappable2

In this case, ScriptWrappable1 will be added to m_foundV8Wrappers but
ScriptWrappable2 won't be added, right? (because m_onlyTraceSingleLevel prevents
us from tracing from TraceWrappable3 to ScriptWrappable2)

[Michael Lippautz, 2017/01/17 09:15:32]

Works, and here is why :)

My assumptions:
- Node is also ScriptWrappable
- TraceWrappable 1-3 are TraceWrapperBase
- ScriptWrappable 1+2 are ScriptWrappable

The key is that we always trace through non-ScriptWrappables and establish edges
to their children.

What happens in details:
- Marking Deque (MD): <||> (empty)
- We will dispatchTraceWrappers for Node on line 241
(traceable->wrapperTypeInfo()->traceWrappers(this, traceable)).
- After Node m_firstScriptWrappableTraced will be true and the marking deque
(MD) will be <|TraceWrappable1|TraceWrappable2|>
- We then traceWrappers for TraceWrappable1. This will work since it's not a
ScriptWrappable. After this one MD: <|TraceWrappable2|ScriptWrappabel1|>
- Same for TraceWrappable2. MD: <|ScriptWrappable1|TraceWrappable3|>.
- When we try to process ScriptWrappable1, we will not trace anymore, but do
markWrappersInAllWorls, so we find the edge. MD: <|TraceWrappable3|>.
- TraceWrappable3 is not a ScriptWrappable, so we just trace. MD:
<|ScriptWrappable2|>
- We don't trace ScriptWrappable2, but add the edge because we find its wrapper
through markWrappersInAllWorlds.

Let me know if this is not clear!

[haraken, 2017/01/17 09:22:41]

Ah, I was missing this point.

+  }
+
+ private:
+  v8::HeapProfiler::RetainerChildren findV8WrappersDirectlyReachableFrom(
+      Node* traceable) {
+    CHECK(m_tempFoundV8Wrappers.empty());
+    WTF::AutoReset<bool>(&m_onlyTraceSingleLevel, true);
+    m_firstTracedScriptWrappable = true;

[haraken, 2017/01/16 02:46:56]

Maybe should we flip true/false?

Initially the first ScriptWrappable is NOT yet traced.

[Michael Lippautz, 2017/01/16 10:45:06]

Done.

+    m_currentParent =
+        &v8::Persistent<v8::Value>::Cast(*traceable->rawMainWorldWrapper());
+
+    TracePrologue();
+    traceable->wrapperTypeInfo()->traceWrappers(this, traceable);
+    AdvanceTracing(
+        0,
+        v8::EmbedderHeapTracer::AdvanceTracingActions(
+            v8::EmbedderHeapTracer::ForceCompletionAction::FORCE_COMPLETION));
+    // Abort instead of Epilogue as we want to finish synchronously.
+    AbortTracing();
+
+    return std::move(m_tempFoundV8Wrappers);
+  }
+
+  // Input obtained from |VisitPersistentHandle|.
+  std::unordered_map<Node*, std::vector<Node*>> m_nodesRequiringTracing;
+
+  // Temporaries used for tracing a single Node.
+  const v8::PersistentBase<v8::Value>* m_currentParent;
+  bool m_onlyTraceSingleLevel;
+  mutable bool m_firstTracedScriptWrappable;
+  mutable v8::HeapProfiler::RetainerChildren m_tempFoundV8Wrappers;

[haraken, 2017/01/16 02:46:56]

m_tempFoundV8Wrappers => m_foundV8Wrappers

[Michael Lippautz, 2017/01/16 10:45:06]

Done.

+
+  // Out variables
+  mutable v8::HeapProfiler::RetainerEdges m_edges;
+  mutable v8::HeapProfiler::RetainerGroups m_groups;
+};
+
+// The function |getRetainerInfos| processing all handles on the blink heap,
+// logically grouping together DOM trees (attached and detached) and pending
+// activities, while at the same time finding parent to child relationships
+// for non-Node wrappers. Since we are processing *all* handles there is no
+// way we miss out on a handle. V8 will figure out the liveness information
+// for the provided information itself.
+v8::HeapProfiler::RetainerInfos V8GCController::getRetainerInfos(
+    v8::Isolate* isolate) {
+  HeapSnaphotWrapperVisitor tracer(isolate);

[haraken, 2017/01/16 02:46:56]

I guess we should use:

  std::unique_ptr<HeapSnaphotWrapperVisitor> tracer = new ...;

See my comment in V8PerIsolateData.

[Michael Lippautz, 2017/01/16 10:45:06]

Done.

+  V8PerIsolateData::TemporaryScriptWrappableVisitorScope scope(isolate,
+                                                               &tracer);
+
+  // Collect which handles need to be processed.
+  isolate->VisitHandlesWithClassIds(&tracer);

[haraken, 2017/01/16 02:46:56]

Nit: I'd prefer writing this as tracer->traceV8Roots().

[Michael Lippautz, 2017/01/16 10:51:01]

Done: collectV8Roots.

+
+  tracer.tracePendingActivities();
+  tracer.traceNodesThatRequireTracing();

[haraken, 2017/01/16 02:46:56]

Nit: tracer->traceBlinkRoots().

[Michael Lippautz, 2017/01/16 10:51:01]

This is now traceV8Roots, as it traces exactly what we collected above.

+
+  return v8::HeapProfiler::RetainerInfos{tracer.groups(), tracer.edges()};
+}
+
 class MajorGCWrapperVisitor : public v8::PersistentHandleVisitor {
  public:
   explicit MajorGCWrapperVisitor(v8::Isolate* isolate,