[Findit] Enable project classifier and component classifier

appengine/findit/crash/classifier.py

Index: appengine/findit/crash/classifier.py
diff --git a/appengine/findit/crash/classifier.py b/appengine/findit/crash/classifier.py
new file mode 100644
index 0000000000000000000000000000000000000000..bd02785f40a08094b8a7b4a25db29379174ddd5a
--- /dev/null
+++ b/appengine/findit/crash/classifier.py
@@ -0,0 +1,100 @@
+# Copyright 2016 The Chromium Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+from collections import namedtuple
+from collections import OrderedDict
+
+from common import constants
+from crash.type_enums import CallStackLanguageType
+from model.crash.crash_config import CrashConfig
+
+
+class ClassOccurrenceInfo(object):

[stgao, 2016/05/17 21:40:57]

Not sure if "class" would be the best term here, but it seems hard to understand
at first glance.

Adding examples of possible "class"es would help.

[Sharu Jiang, 2016/05/20 23:16:33]

Done.

+  """Represents information of a class in results or crash_stack.
+
+  Class information includes the name of the class, a list of indice (index in
+  results list or in the crash_stack) of occurrences."""
+
+  def __init__(self, name, occurrences):
+    self.name = name
+    self.occurrences = occurrences
+
+
+def DefaultRankFunction(class_info):
+  """Default rank function to rank classes.
+
+  Note: The default behavior works for component classifier and for
+  project classifier, it works for cpp callstack class ranking.
+  """
+  # If the top 2 frames are in the same class, give this class highest
+  # priority.
+  if 0 in class_info.occurrences and 1 in class_info.occurrences:

[stgao, 2016/05/17 21:40:57]

It is not clear what 0 and 1 are for at first read.

[Sharu Jiang, 2016/05/20 23:16:33]

I cannot think of a way to make it more clear other than a comment explaining
this...

+    return -float('inf'), class_info.occurrences[0]
+
+  return -len(class_info.occurrences), class_info.occurrences[0]
+
+
+class Classifier(object):  # pragma: no cover.

[stgao, 2016/05/17 21:40:57]

Why 'no cover'?

[Sharu Jiang, 2016/05/20 23:16:33]

I thought it would be tested in project and component classifiers, and it is a
interface, but I can add test for this.

+  """Classifys results or crash stack into a class or a list of classes."""
+
+  def GetClassFromStackFrame(self, frame):
+    raise NotImplementedError()
+
+  def GetClassFromResult(self, result):
+    raise NotImplementedError()
+
+  def _Classify(self, results, crash_stack, top_n, max_classes,
+                rank_function=DefaultRankFunction):
+    """Classifies a crash to a list of classes, ranked by rank_function.
+
+    Extracts a list of classes from results or crash_stack, rank the classes and
+    returns max_classes number of classes on the top.
+
+    Args:
+      results (list of Result): Culprit results.
+      crash_stack (CallStack): The callstack that caused the crash.
+      top_n (int): Number of top frames to be considered when classifying.
+      max_classes (int): Maximal number of classes to return.
+      rank_function (function): Used to rank classes based on
+        ClassOccurrenceInfos.
+
+    Returns:
+      A list of classes of this crash.
+    """
+    # Extracts the class list from culprit results if possible since it's more
+    # reliable.
+    if results:
+      class_list = map(self.GetClassFromResult, results[:top_n])
+    else:
+      class_list = map(self.GetClassFromStackFrame, crash_stack[:top_n])

[stgao, 2016/05/17 21:40:57]

Could this be handled in the sub classes?

[Sharu Jiang, 2016/05/20 23:16:33]

would it be duplicate if I handle the same thing in both project classifier and
component classifier?

+
+    def _GetClassOccurrenceInfos():
+      """Gets ClassOccurrenceInfo list from class_list.
+
+      The ClassOccurrenceInfo list is ordered by the first occurences of each
+      class.
+      """
+      if not class_list:
+        return class_list
+
+      infos = OrderedDict()

[stgao, 2016/05/17 21:40:57]

We sort at #93 below, why we also need ordered dict here?

[Sharu Jiang, 2016/05/20 23:16:33]

Done.

+
+      # Get occurences information of each class.
+      for index, class_name in enumerate(class_list):
+        if class_name not in infos:
+          infos[class_name] = ClassOccurrenceInfo(class_name, [index])
+        else:
+          infos[class_name].occurrences.append(index)
+
+      return infos.values()
+
+    class_infos = _GetClassOccurrenceInfos()
+    class_infos = sorted(class_infos, key=rank_function)
+
+    classes = [info.name for info in class_infos]
+
+    return classes[:max_classes]
+
+  def Classify(self, results, crash_stack):
+    raise NotImplementedError()