[Findit] findit algorithms and a cluster-fuzz implementation.

tools/findit/findit_for_clusterfuzz.py

Index: tools/findit/findit_for_clusterfuzz.py
diff --git a/tools/findit/findit_for_clusterfuzz.py b/tools/findit/findit_for_clusterfuzz.py
new file mode 100644
index 0000000000000000000000000000000000000000..70c89c540d27842718e3e15b2f136cecf3433710
--- /dev/null
+++ b/tools/findit/findit_for_clusterfuzz.py
@@ -0,0 +1,145 @@
+# Copyright (c) 2014 The Chromium Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+import logging
+
+import chromium_deps
+import crash_utils
+import findit_for_crash as findit
+import stacktrace
+
+
+def SplitStacktrace(stacktrace_string):
+  """Preprocesses stacktrace string into two parts, release and debug.
+
+  Args:
+    stacktrace_string: A string representation of stacktrace,
+                       in clusterfuzz format.
+
+  Returns:
+    A tuple of list of strings, release build stacktrace and
+    debug build stacktrace.
+  """
+  # Make sure we only parse release/debug build stacktrace, and ignore
+  # unsymbolised stacktrace.
+  in_release_or_debug_stacktrace = False
+  release_build_stacktrace_lines = None
+  debug_build_stacktrace_lines = None
+  current_stacktrace_lines = []
+
+  # Iterate through all lines in stacktrace.
+  for line in stacktrace_string.splitlines():
+    line = line.strip()
+
+    # If the line starts with +, it signifies the start of new stacktrace.
+    if line.startswith('+'):
+      if 'Release Build Stacktrace' in line:
+        in_release_or_debug_stacktrace = True
+        current_stacktrace_lines = []
+        release_build_stacktrace_lines = current_stacktrace_lines
+
+      elif 'Debug Build Stacktrace' in line:
+        in_release_or_debug_stacktrace = True
+        current_stacktrace_lines = []
+        debug_build_stacktrace_lines = current_stacktrace_lines
+
+      # If the stacktrace is neither release/debug build stacktrace, ignore
+      # all lines after it until we encounter release/debug build stacktrace.
+      else:
+        in_release_or_debug_stacktrace = False
+
+    # This case, it must be that the line is an actual stack frame, so add to
+    # the current stacktrace.
+    elif in_release_or_debug_stacktrace:
+      current_stacktrace_lines.append(line)
+
+  return (release_build_stacktrace_lines, debug_build_stacktrace_lines)
+
+
+def FindCulpritCLs(stacktrace_string,
+                   build_type,
+                   chrome_regression=None,
+                   component_regression=None,
+                   chrome_crash_revision=None,
+                   component_crash_revision=None,
+                   crashing_component=None):
+  """Returns json format of the result, a list of result.Result objects.

[stgao, 2014/08/18 22:37:07]

Seems json format is wrong now?

[jeun, 2014/08/18 23:52:13]

Done.

+
+  Args:
+    stacktrace_string: A string representing stacktrace.
+    build_type: The type of the job.
+    chrome_regression: A string, chrome regression from clusterfuzz, in format
+                       '123456:123457'
+    component_regression: A string, component regression in the same format.
+    chrome_crash_revision: A crash revision of chrome, in string.
+    component_crash_revision: A crash revision of the component,
+                              if component build.
+    crashing_component: Yet to be decided.
+
+  Returns:
+    A list of result objects, along with the short description on where the
+    result is from.
+  """
+  build_type = build_type.lower()
+  if 'android' in build_type or 'syzyasan' in build_type:

[Martin Barbella, 2014/08/19 02:51:35]

Why are these job types being skipped explicitly?

[jeun, 2014/08/19 18:30:06]

The stacktrace parsing for them is not supported. I will add them.

+    return ('This build type is currently not supported.', [])
+
+  logging.basicConfig(filename='errors.log', level=logging.WARNING,

[stgao, 2014/08/18 22:37:07]

Findit should not write any log.

It is the responsibility of the caller to setup the logging if they want.
Please also include this in the function documentation above.

[jeun, 2014/08/18 23:52:13]

Does this mean anything other than this file should not use logging module in
it? If so, how would I be able to log error messages?

+                      filemode='w+')
+
+  crash_revision_dict = {}
+  regression_dict = {}
+
+  # TODO(jeun): Come up with a good way to connect crashing component name to

[Martin Barbella, 2014/08/19 02:51:35]

What work is still needed to be done here? I thought the limitation was just
that we didn't have the chromium revision for component builds yet. If called
with all of the required information, it's important that this works.

[jeun, 2014/08/19 18:30:05]

Since we started parsing DEPS file for regression range/crash revisions (and not
clusterfuzz website like
https://cluster-fuzz.appspot.com/revisions?job=windows_syzyasan_content_shell...),
we do not have an access to the "name" of the component, like v8, blink,
chromium etc. Instead, we use the "path" of the component (for example,
'src/third_party/Source' for blink). For DEPS file parsing, path is unique while
parsed component name is not. We were thinking about what would be a good way of
passing in the name of the crashing component. If you are fine with it, you can
just pass in the path as defined in DEPS file, or we can create a config file
mapping a component's path to its component name. We prefer first one, what is
your opinion on this?

+  # its path.
+  if component_regression or component_crash_revision:
+    return ('This feature is not supported.', [])

[stgao, 2014/08/18 22:37:07]

"Component builds are not supported yet."

[jeun, 2014/08/18 23:52:13]

Done.

+
+  # If chrome regression is available, parse DEPS file.
+  chrome_regression = crash_utils.SplitRange(chrome_regression)
+  if chrome_regression:
+    chrome_regression_start = chrome_regression[0]
+    chrome_regression_end = chrome_regression[1]
+
+    # If the regression is not reliable, do not parse regression information.

[Martin Barbella, 2014/08/19 02:51:35]

A regression range starting with 0 has nothing to do with its reliability. In
fact, they are probably the most reliable regression ranges we have since it
means the crash reproduced in the earliest build for a job type.

This should just say something like "Do not parse regression information for
crashes introduced before the first archived build for a job type" (or some
shorter variation of that).

[jeun, 2014/08/19 18:30:05]

Done.

+    if chrome_regression_start != '0':
+      regression_dict = chromium_deps.GetChromiumComponentRange(
+          chrome_regression_start, chrome_regression_end)
+
+  # Parse crash revision.
+  if chrome_crash_revision:
+    crash_revision_dict = chromium_deps.GetChromiumComponents(

[stgao, 2014/08/18 22:37:06]

Do we need to verify |chrome_crash_revision| is not None?

[jeun, 2014/08/18 23:52:13]

Yes, in case we only have component crash revision

+        chrome_crash_revision)
+
+  # Parsed DEPS is used to normalize the stacktrace. Since parsed regression
+  # and parsed crash state essentially contain same information, use either.
+  if regression_dict:
+    parsed_deps = regression_dict
+  elif crash_revision_dict:
+    parsed_deps = crash_revision_dict
+  else:
+    return ('No regression and crash revision information available.', [])

[Martin Barbella, 2014/08/19 02:51:35]

If someone runs into an error, they probably don't really care what went wrong
but rather what needs to be done about it. If we run into this case, it's
probably the caller's fault. Try to make how they can fix this clear.

Maybe something like "Identifying culprit CLs requires a crash revision or
regression range."

[jeun, 2014/08/19 18:30:05]

Done.

+
+  # Split stacktrace into release build/debug build and parse them.
+  (release_build_stacktrace, debug_build_stacktrace) = SplitStacktrace(
+      stacktrace_string)
+  parsed_release_build_stacktrace = stacktrace.Stacktrace(
+      release_build_stacktrace, build_type, parsed_deps)
+  parsed_debug_build_stacktrace = stacktrace.Stacktrace(
+      debug_build_stacktrace, build_type, parsed_deps)
+
+  # Get a highest priority callstack from stacktrace, with release build
+  # stacktrace in higher priority than debug build stacktace.
+  if parsed_release_build_stacktrace.stack_list:
+    main_stack = parsed_release_build_stacktrace.GetCrashStack()

[stgao, 2014/08/18 22:37:06]

What's main stack? What it is for? Why only handle it?
Add an explanation for this.

[jeun, 2014/08/18 23:52:13]

Done.

+  elif parsed_debug_build_stacktrace.stack_list:
+    main_stack = parsed_debug_build_stacktrace.GetCrashStack()
+  else:
+    return ('Stacktrace is malformed.', [])
+
+  # Run the algorithm on the parsed stacktrace, and return the JSON.

[stgao, 2014/08/18 22:37:07]

We are not returning JSON any more, right?

[jeun, 2014/08/18 23:52:13]

Done.

+  stacktrace_list = [parsed_release_build_stacktrace,
+                     parsed_debug_build_stacktrace]
+  return findit.FindItForCrash(
+      stacktrace_list, main_stack, regression_dict, crash_revision_dict)