[Interpreter] Add Ignition profile visualization tool.

tools/ignition/linux_perf_report.py

+#! /usr/bin/python2
+#
+# Copyright 2016 the V8 project authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+#
+
+import argparse
+import collections
+import re
+import subprocess
+import sys
+
+
+__DESCRIPTION = """
+Processes a perf.data sample file and reports the hottest Ignition bytecodes,
+or write an input file for flamegraph.pl.
+"""
+
+
+__HELP_EPILOGUE = """
+examples:
+  # Get a flamegraph for Ignition bytecode handlers on Octane benchmark,
+  # without considering the time spent compiling JS code, misattributed samples

[rmcilroy, 2016/04/15 14:58:41]

nit - I wouldn't mention misattributed samples here since we want that to go
away once there is support for DWARF-style unwinding.

[Stefano Sanfilippo, 2016/04/15 16:13:18]

Done.

+  # entry trampoline samples and other non-Ignition samples.
+  #
+  $ tools/run-perf.sh out/x64.release/d8 \\
+        --ignition --noturbo --nocrankshaft run.js
+  $ tools/ignition/linux_perf_report.py --flamegraph -o out.collapsed
+  $ flamegraph.pl --colors js out.collapsed > out.svg
+
+  # Same as above, but show all samples, including time spent compiling JS code,
+  # misattributed samples, entry trampoline samples and other samples.
+  $ # ...
+  $ tools/ignition/linux_perf_report.py \\
+      --flamegraph --show-other -o out.collapsed
+  $ # ...
+
+  # Same as above, but show full function signatures in the flamegraph.
+  $ # ...
+  $ tools/ignition/linux_perf_report.py \\
+      --flamegraph --show-full-signatures -o out.collapsed
+  $ # ...
+
+  # See the hottest bytecodes on Octane benchmark, by number of samples.
+  #
+  $ tools/run-perf.sh out/x64.release/d8 \\
+        --ignition --noturbo --nocrankshaft octane/run.js
+  $ tools/ignition/linux_perf_report.py
+"""
+
+
+COMPILER_SYMBOLS_RE = re.compile(
+  r"v8::internal::(?:\(anonymous namespace\)::)?Compile|v8::internal::Parser")
+
+
+def get_function_name(symbol):
+  if symbol[-1] != ')': return symbol
+  pos = 1
+  parenthesis_count = 0
+  for c in reversed(symbol):
+    if c == ')':
+      parenthesis_count += 1
+    elif c == '(':
+      parenthesis_count -= 1
+    if parenthesis_count == 0:
+      break
+    else:
+      pos += 1
+  return symbol[:-pos]
+
+
+def collapsed_callchains_generator(perf_stream, show_other=False,
+                                   show_full_signatures=False):
+  current_chain = []
+  skip_until_end_of_chain = False
+  compiler_symbol_in_chain = False
+
+  for line in perf_stream:
+    # Lines starting with a "#" are comments, skip them.
+    if line[0] == "#":
+      continue
+
+    line = line.strip()
+
+    # Empty line signals the end of the callchain.
+    if not line:
+      if not skip_until_end_of_chain and current_chain and show_other:
+        current_chain.append("[other]")
+        yield current_chain
+      # Reset parser status.
+      current_chain = []
+      skip_until_end_of_chain = False
+      compiler_symbol_in_chain = False
+      continue
+
+    if skip_until_end_of_chain:
+      continue
+
+    symbol = line.split(" ", 1)[1]
+    if not show_full_signatures:
+      symbol = get_function_name(symbol)

[rmcilroy, 2016/04/15 14:58:41]

nit - s/get_function_name/strip_function_parameters/

[Stefano Sanfilippo, 2016/04/15 16:13:19]

Done.

+    current_chain.append(symbol)
+
+    if symbol.startswith("BytecodeHandler:"):
+      yield current_chain
+      skip_until_end_of_chain = True
+    elif symbol == "Stub:CEntryStub" and compiler_symbol_in_chain:
+      if show_other:
+        current_chain[-1] = "[compiler]"
+        yield current_chain
+      skip_until_end_of_chain = True
+    elif COMPILER_SYMBOLS_RE.match(symbol):
+      compiler_symbol_in_chain = True
+    elif symbol == "Builtin:InterpreterEntryTrampoline":
+      # The chain cannot be empty at this point

[rmcilroy, 2016/04/15 14:58:41]

Not sure what this comment means. I think what you want is:

# If we see an InterpreterEntryTrampoline which is not at the top of the chain,
and doesn't have a BytecodeHandler above it, then we have skipped the top
BytecodeHandler due to the top-level stub not building a frame. File the chain
in a misattributed bucket.

Also put it in the "elif show_other"

[Stefano Sanfilippo, 2016/04/15 16:13:18]

The comment was because the else branch assumes the chain has at least one
element. Since the case of a single element is handled in the if branch, we can
enter the else with many elements or none, and the latter case should not happen
because the chain can never be empty. This is because it necessarily contains at
least the current symbol.

Anwyay, I pasted your comment in the else branch. Should we add an assert as
well for the condition above?

[rmcilroy, 2016/04/18 09:22:24]

I don't think it is necessary, you unconditinally insert an element into the
chain at the top of the whole if block.

[Stefano Sanfilippo, 2016/04/18 10:37:23]

Acknowledged.

+      if len(current_chain) == 1:
+        yield ["[entry trampoline]"]
+      elif show_other:
+        current_chain[-1] = "[misattributed]"
+        yield current_chain
+      skip_until_end_of_chain = True
+
+
+def calculate_samples_count_per_callchain(callchains):
+  chain_counters = collections.defaultdict(int)
+  for callchain in callchains:
+    # flamegraph.pl appears to have problems when no ; is in the chain,
+    # so we add a dummy one at the end if there is only one entry in the chain.
+    if len(callchain) == 1:
+      key = callchain[0] + ";"

[rmcilroy, 2016/04/15 14:58:42]

Could you just always add a ";" to the end rather than just if len == 1?

[Stefano Sanfilippo, 2016/04/15 16:13:19]

This is not the format shown in flamegraph examples (and the one produced by the
various converter scripts), but it works anyway.

The trailing ; was originally supposed to be a workaround for split blocks, for
me is the same though.

[rmcilroy, 2016/04/18 09:22:24]

I think it would be better to make things consistent (i.e., always insert the
dummy ';' at the end of the chain). If so, please update the comment above too.

[Stefano Sanfilippo, 2016/04/18 10:37:23]

No problem, done.

+    else:
+      key = ";".join(reversed(callchain))
+    chain_counters[key] += 1
+  return chain_counters.items()
+
+
+def calculate_samples_count_per_handler(callchains):
+  def strip_handler_prefix_if_any(handler):
+    return handler if handler[0] == "[" else handler.split(":", 1)[1]
+
+  handler_counters = collections.defaultdict(int)
+  for callchain in callchains:
+    handler = strip_handler_prefix_if_any(callchain[-1])
+    handler_counters[handler] += 1
+  return handler_counters.items()
+
+
+def write_flamegraph_input_file(output_stream, callchains):
+  for callchain, count in calculate_samples_count_per_callchain(callchains):
+    output_stream.write("{} {}\n".format(callchain, count))
+
+
+def write_handlers_report(output_stream, callchains):
+  handler_counters = calculate_samples_count_per_handler(callchains)
+  samples_num = sum(counter for _, counter in handler_counters)
+  # Sort by decreasing number of samples
+  handler_counters.sort(key=lambda entry: entry[1], reverse=True)
+  for bytecode_name, count in handler_counters:
+    output_stream.write(
+      "{}\t{}\t{:.3f}%\n".format(bytecode_name, count,
+                                 100. * count / samples_num))
+
+
+def parse_command_line():
+  command_line_parser = argparse.ArgumentParser(
+    formatter_class=argparse.RawDescriptionHelpFormatter,
+    description=__DESCRIPTION,
+    epilog=__HELP_EPILOGUE)
+
+  command_line_parser.add_argument(
+    "perf_filename",
+    help="perf sample file to process (default: perf.data)",
+    nargs="?",
+    default="perf.data",
+    metavar="<perf filename>"
+  )
+  command_line_parser.add_argument(
+    "--flamegraph", "-f",
+    help="output an input file for flamegraph.pl, not a report",
+    action="store_true",
+    dest="output_flamegraph"
+  )
+  command_line_parser.add_argument(
+    "--show-other", "-r",

[rmcilroy, 2016/04/15 14:58:41]

-r is a strange shortening. How about "--show-all" and "-a" ?

[Stefano Sanfilippo, 2016/04/15 16:13:18]

Done.

+    help="show samples outside Ignition interpreter",
+    action="store_true"
+  )
+  command_line_parser.add_argument(
+    "--show-full-signatures", "-s",
+    help="show full signatures instead of function names",
+    action="store_true"
+  )
+  command_line_parser.add_argument(
+    "--output", "-o",
+    help="output file name (stdout if omitted)",
+    type=argparse.FileType('wt'),
+    default=sys.stdout,
+    metavar="<output filename>",
+    dest="output_stream"
+  )
+
+  return command_line_parser.parse_args()
+
+
+def main():
+  program_options = parse_command_line()
+
+  perf = subprocess.Popen(["perf", "script", "-f", "ip,sym",
+                           "-i", program_options.perf_filename],
+                          stdout=subprocess.PIPE)
+
+  callchains = collapsed_callchains_generator(
+    perf.stdout, program_options.show_other,
+    program_options.show_full_signatures)
+
+  if program_options.output_flamegraph:
+    write_flamegraph_input_file(program_options.output_stream, callchains)
+  else:
+    write_handlers_report(program_options.output_stream, callchains)
+
+  perf.communicate()

[rmcilroy, 2016/04/15 14:58:42]

Why is this needed? Shouldn't it be before reading stdout if needed?

[Stefano Sanfilippo, 2016/04/15 16:13:18]

This was intended to work as a wait() to ensure we exit only after our
subprocess. I suppose directly using wait() should not cause any deadlock
because we are reading the stdout of the subprocess till EOF.

[rmcilroy, 2016/04/18 09:22:24]

I don't see why this is necessary at all. You are already waiting for EOF, which
should only be sent if the subprocess exits normally (unless it does something
strange).

[Stefano Sanfilippo, 2016/04/18 10:37:23]

Agreed, removed.

+
+
+if __name__ == "__main__":
+  main()