[Android] Support conditional test disabling based on android.os.Build values.

chrome/android/javatests/src/org/chromium/chrome/browser/NavigateTest.java

 // Copyright 2015 The Chromium Authors. All rights reserved.
 // Use of this source code is governed by a BSD-style license that can be
 // found in the LICENSE file.
 
 package org.chromium.chrome.browser;
 
 import static org.chromium.base.test.util.Restriction.RESTRICTION_TYPE_TABLET;
 
 import android.test.FlakyTest;
 import android.test.suitebuilder.annotation.MediumTest;
 import android.test.suitebuilder.annotation.Smoke;
 import android.text.TextUtils;
 import android.view.KeyEvent;
 
 import org.chromium.base.ThreadUtils;
-import org.chromium.base.test.util.DisabledTest;
+import org.chromium.base.test.util.DisableIf;
 import org.chromium.base.test.util.Feature;
 import org.chromium.base.test.util.Restriction;
 import org.chromium.base.test.util.UrlUtils;
 import org.chromium.chrome.R;
 import org.chromium.chrome.browser.omnibox.LocationBarLayout;
 import org.chromium.chrome.browser.omnibox.UrlBar;
 import org.chromium.chrome.browser.tab.EmptyTabObserver;
 import org.chromium.chrome.browser.tab.Tab;
 import org.chromium.chrome.browser.tab.TabObserver;
 import org.chromium.chrome.browser.tabmodel.TabModel;
@@ skipping 321 matching lines @@
                     urls[1], getActivity().getActivityTab().getUrl());
 
             singleClickView(getActivity().findViewById(R.id.forward_button));
             UiUtils.settleDownUI(getInstrumentation());
             assertEquals(String.format(Locale.US,
                     "URL mismatch after pressing fwd button for the 2nd time in repetition %d.", i),
                     urls[2], getActivity().getActivityTab().getUrl());
         }
     }
 
-    @DisabledTest // Fails on android-one: crbug.com/540723
+    @DisableIf.Build(hardware_is = "sprout") // Fails on android-one: crbug.com/540723

[jbudorick, 2015/12/10 15:23:15]

There are, unsurprisingly, other tests where this can be used, but I just wanted
to include one as a sample.

I'm a bit concerned here about how this could facilitate accidental leaks, but
I'm not sure how to resolve or mitigate that. Maybe a PRESUBMIT check of some
kind?

[Yaron, 2015/12/11 02:22:16]

:(
Should we include a message= attribute which can link to the bug (so you can see
in the test runner that it's skipping some tsets?)
Ya, I could definitely see that happen. You could have a PRESUBMIT of
whitelisted devices based on what we run on waterfalls. Of course that can still
fail if it's on the internal waterfall. Another option is to wait to introduce
this until more of the tests are moved up (I know, not the greatest)

[jbudorick, 2015/12/17 02:08:13]

I like this idea, but I'll address how the test runner handles this annotation
in general (and message= specifically) in a separate CL if at all.
I'm less worried about the tests and more worried about the possibility of
wanting to disable an upstream test on prerelease hardware.

[Yaron, 2015/12/17 15:12:37]

Right. So what options do we provide for sheriffs now? The SDK check <= (which I
complained about) seemed likes the easiest solution. I think in general it's
expected that we're testing with unreleased software and there's always a new
android coming - but hardware codenames are more sensitive. I guess I take it
back - feel free to add back sdk_is_less_than and we'll see to what extent it
gets (ab)used. It seems like the easiest way to support this restriction and
minimize leaks.

Relatedly - is there a significant difference between DisableIf and
Restrictions? DisableIf means it's broken and Restriction means it's never
intended to work?

[jbudorick, 2015/12/18 17:43:06]

Hm, ok.
That's what I had in mind, but perhaps that's a more nuanced split than we want.

     @MediumTest
     @Feature({"Navigation"})
     public void testWindowOpenUrlSpoof() throws Exception {
         TestWebServer webServer = TestWebServer.start();
         try {
             // Make sure that we start with one tab.
             final TabModel model = getActivity().getTabModelSelector().getModel(false);
 
             final Semaphore urlServedSemaphore = new Semaphore(0);
             Runnable checkAction = new Runnable() {
@@ skipping 84 matching lines @@
             assert false : "Unexpected TimeoutException";
         }
         return null;
     }
 
     @Override
     public void startMainActivity() throws InterruptedException {
         startMainActivityFromLauncher();
     }
 }