Findit] Flake Checker: Pipeline to trigger try jobs to identify flake culprits

appengine/findit/model/flake/master_flake_analysis.py

 # Copyright 2016 The Chromium Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
 # found in the LICENSE file.
 
 import base64
 
 from google.appengine.ext import ndb
 
 from gae_libs.model.versioned_model import VersionedModel
 from model import result_status
 from model import triage_status
 from model.base_analysis import BaseAnalysis
 from model.base_build_model import BaseBuildModel
 from model.base_triaged_model import TriagedModel
 from model.flake.flake_culprit import FlakeCulprit
 from model.flake.flake_swarming_task import FlakeSwarmingTaskData
 
 
 class DataPoint(ndb.Model):
+  # The build number corresponding to this data point. Only relevant if this
+  # data point is generated as the result of a flake swarming task.

[chanli, 2017/01/13 00:47:32]

Should we mention this is relevant for build level or something? And for
try_job_id and try_job_url we can say they are only relevant for revision level.

[lijeffrey, 2017/01/13 01:39:23]

Done.

   build_number = ndb.IntegerProperty(indexed=False)
+
+  # The pass rate of the test when run against this commit.
   pass_rate = ndb.FloatProperty(indexed=False)
+
+  # The ID of the swarming task responsible for generating this data.
   task_id = ndb.StringProperty(indexed=False)
+
+  # The git position of this data point.

[chanli, 2017/01/13 00:47:33]

I don't think git position is the right way to call it... I think commit
position is just OK as is.

[lijeffrey, 2017/01/13 01:39:23]

Done.

   commit_position = ndb.IntegerProperty(indexed=False)
+
+  # The git hash of this data point.
   git_hash = ndb.StringProperty(indexed=False)
+
+  # The commit position of the build preceding this one. Only relevant if this
+  # data point is generated as the result of a flake swarming taask.

[chanli, 2017/01/13 00:47:33]

Nit: taask -> task

[lijeffrey, 2017/01/13 01:39:23]

Done.

   previous_build_commit_position = ndb.IntegerProperty(indexed=False)
+
+  # The git hash of the data point 1 build before this one. Only relevant if
+  # this data point is generated as the result of a flake swarming task.
   previous_build_git_hash = ndb.StringProperty(indexed=False)

[chanli, 2017/01/13 00:47:33]

Maybe now is a little late.... But why do we need both commit_position and
git_hash of previous build?

[lijeffrey, 2017/01/13 01:39:23]

git hash is the useful one, previous build commit position is for UI/display
purposes.

+
+  # The list of revisions between this build and the previous build. Only
+  # relevant if this data point is generated as the result of a flake swarming
+  # task.
   blame_list = ndb.StringProperty(repeated=True)
 
+  # The ID of the try job that generated this data point, if any.
+  try_job_id = ndb.StringProperty(indexed=False)

[chanli, 2017/01/13 00:47:33]

As we discussed offline, maybe remove try_job_id for now

[lijeffrey, 2017/01/13 01:39:23]

Done.

+
+  # The URL to the try job that generated this data point, if any.
+  try_job_url = ndb.StringProperty(indexed=False)
+
+  def GetCommitPosition(self, revision):

[chanli, 2017/01/13 00:47:32]

I think this function is a little hacky... Although we have confirmed
commit_position should be continuous.

BTW, why do we need this?

[lijeffrey, 2017/01/13 01:39:23]

Unfortunately blame_list only contains revisions and doesn't have
commit_position, so we have to calculate it. In the end it's is only used for
display purposes and is effectively treated as an index in the blame list for
which to run the next try job, so even if it's off the try job's lookback
algorithm is unaffected.

+    """Gets the commit position of a revision within blame_list.
+
+    Args:
+      revision (str): The revision to search for.
+
+    Returns:
+      commit_position (int): The calculated commit position of revision.
+    """
+    assert revision in self.blame_list
+
+    for i in range(0, len(self.blame_list)):  # pragma: no branch
+      if revision == self.blame_list[i]:
+        return i + self.previous_build_commit_position + 1
+
+  def GetRevisionAtCommitPosition(self, commit_position):
+    """Gets the corresponding revision to commit_position.
+
+    Args:
+      commit_position (int): The commit position for which to find the
+          corresponding revision within self.blame_list.
+
+    Returns:
+      revision (str): The git revision corresponding to commit_position.
+    """
+    length = len(self.blame_list)
+    assert (commit_position > self.commit_position - length and
+            commit_position <= self.commit_position)
+    return self.blame_list[
+        length - (self.commit_position - commit_position) - 1]
+
 
 class MasterFlakeAnalysis(
     BaseAnalysis, BaseBuildModel, VersionedModel, TriagedModel):
   """Represents an analysis of a flaky test on a Waterfall test cycle."""
 
   @ndb.ComputedProperty
   def step_name(self):
     return self.key.pairs()[0][1].split('/')[3]
 
   @ndb.ComputedProperty
@@ skipping 130 matching lines @@
 
   # The suspected build number to have introduced the flakiness.
   suspected_flake_build_number = ndb.IntegerProperty()
 
   # The culprit CL associated with the try job results, if any.
   culprit = ndb.LocalStructuredProperty(FlakeCulprit)
 
   # The status of try jobs, if any. None if try jobs have not been triggered.
   # Status should be PENDING or STARTED when the first try job is triggered,
   # and COMPLETED when the last one finishes. If any try job ends in error,
-  # status will be ERROR. 
+  # status will be ERROR.
   try_job_status = ndb.IntegerProperty(indexed=False)
 
   # The data points used to plot the flakiness graph build over build.
   data_points = ndb.LocalStructuredProperty(
       DataPoint, repeated=True, compressed=True)
 
   # Whether the analysis was triggered by a manual request through check flake,
   # Findit's automatic analysis upon detection, or Findit API.
   triggering_source = ndb.IntegerProperty(default=None, indexed=True)
 
   # Who triggered the analysis. Used for differentiating between manual and
   # automatic runs, and determining the most active users to gather feedback.
   triggering_user_email = ndb.StringProperty(default=None, indexed=False)
 
   # Overall conclusion of analysis result for the flake. Found untriaged, Found
   # Correct, etc. used to filter what is displayed on the check flake dashboard.
   result_status = ndb.IntegerProperty(indexed=True)