🎯 Fail if an instrumentation test is missing size annotation

build/android/pylib/instrumentation/instrumentation_test_instance.py

Index: build/android/pylib/instrumentation/instrumentation_test_instance.py
diff --git a/build/android/pylib/instrumentation/instrumentation_test_instance.py b/build/android/pylib/instrumentation/instrumentation_test_instance.py
index cd8617cfc15f7bb84fa79a60caa1a71261dd2de9..4ea374d669a9779330f14e232e2b50dc8a32f32e 100644
--- a/build/android/pylib/instrumentation/instrumentation_test_instance.py
+++ b/build/android/pylib/instrumentation/instrumentation_test_instance.py
@@ -547,7 +547,7 @@ class InstrumentationTestInstance(test_instance.TestInstance):
               or unittest_util.FilterTestNames(t, self._test_filter))
 
     def annotation_filter(all_annotations):
-      if not self._annotations:
+      if not self._annotations or not all_annotations:

[jbudorick, 2016/04/07 02:30:11]

I don't think this is logically consistent. A test with no annotations
_shouldn't_ match on a non-empty set of desired annotations.

The appropriate fixes would be to either: 
 - support WebRTC providing a completely empty set of desired annotations, or
 - have WebRTC add size annotations to their tests.

[jbudorick, 2016/04/07 02:31:26]

(or 3, remove the default annotations entirely, but that's a change with a large
potential for side effects)

[agrieve, 2016/04/07 02:41:12]

Should we add an exception when we detect a test method with no annotation?
Seems a bit weird that we can detect test methods, and then have no way for them
to not get filtered out. This change special-cases them so that no annotation
means always included it, but I think if we don't want that logic, we should
throw an error if we find an unannotated test (since there is no way to run it).
wdyt?

[jbudorick, 2016/04/07 02:50:18]

Honestly, I think the right answer for this is probably either to have WebRTC
add annotations or to remove the default set and only do annotation-based
filtering if a user explicitly passes one, and I'm leaning toward the latter.

In order to confidently do that, we'd have to figure out which tests in each of
the four chromium suites aren't annotated with any of the default annotations or
the default excluded annotations.

         return True
       return any_annotation_matches(self._annotations, all_annotations)