Always identify test names that lead to failures

recipe_engine/test_result.proto

 // Copyright 2017 The LUCI Authors. All rights reserved.
 // Use of this source code is governed under the Apache License, Version 2.0
 // that can be found in the LICENSE file.
 
 syntax = "proto3";
 
 package recipe_engine;
 
 // Result of running recipe tests.
 message TestResult {
@@ skipping 33 matching lines @@
     // Keyword arguments to the function.
     map<string, string> kwargs = 4;
 
     // Filename where the check was added.
     string filename = 5;
 
     // Line number where the check was added.
     int64 lineno = 6;
   }
 
+  message InternalFailure {
+  }
+
   message TestFailure {
     oneof failures {
       DiffFailure diff_failure = 1;
 
       CheckFailure check_failure = 2;
+
+      InternalFailure internal_failure = 3;

[iannucci, 2017/04/04 20:24:51]

I suspect 'internal_failure' isn't quite right; most of the time this just means
that the recipe code needs modification. Maybe 'incompatibility_failure'?

followup: I would also add some comments to this proto to document what the
different failure types mean, exactly.

[Paweł Hajdan Jr., 2017/04/04 20:32:03]

Added comment to the proto.

As indicated by the comment, incompatibility in user recipe code is not the only
case which can trigger this.

     }
   }
 
   message TestFailures {
     repeated TestFailure failures = 1;
   }
   // Maps test names to failure details.
   map<string, TestFailures> test_failures = 4;
 
   // Module names completely lacking test coverage.
   repeated string uncovered_modules = 5;
 
   // Absolute native paths of unused recipe expectation files.
   repeated string unused_expectations = 6;
 }