[Android] Add partial md documentation for instrumentation test features.

testing/android/docs/instrumentation.md

Index: testing/android/docs/instrumentation.md
diff --git a/testing/android/docs/instrumentation.md b/testing/android/docs/instrumentation.md
new file mode 100644
index 0000000000000000000000000000000000000000..649816711c0c53f58414f691db1da55813cb1968
--- /dev/null
+++ b/testing/android/docs/instrumentation.md
@@ -0,0 +1,262 @@
+# Android Instrumentation Tests
+
+Instrumentation tests are Java tests based on
+[`android.app.Instrumentation`](https://developer.android.com/reference/android/app/Instrumentation.html).
+They run on a device.
+
+[TOC]
+
+## Writing an instrumentation test
+
+Currently, an instrumentation test is just a JUnit3-style test based on
+[android.test.InstrumentationTestCase](https://developer.android.com/reference/android/test/InstrumentationTestCase.html).
+(This will change starting in [Android N](https://en.wikipedia.org/wiki/Android_Nougat).)
+
+Writing an instrumentation test case can be simple, e.g.
+
+```java
+package org.chromium.sample.test;
+
+import android.test.InstrumentationTestCase;
+
+public class MyInstrumentationTest extends InstrumentationTestCase {
+
+    // Note that, because it's a JUnit3-style test, the test method *must*
+    // start with "test".
+    public void testTheFirst() {
+        bool writingInstrumentationTestsCanBeEasy = true;
+
+        // InstrumentationTestCase inherits the assert* methods through
+        // junit.framework.TestCase.
+        assertTrue(writingInstrumentationTestsCanBeEasy);
+    }
+
+    public void testTheSecond() {
+        bool writingInstrumentationTestsIsAlwaysEasy = false;
+        assertFalse(writingInstrumentationTestsIsAlwaysEasy);
+    }
+}
+```
+
+After writing a test, you can run it by:
+
+ - Adding the file to the relevant gn target if the entire file is new.
+Typically, the "relevant gn target" is simply the target containing the
+other files in the same directory.
+ - Rebuild.
+ - Run the test using the process described [here](/testing/android/docs/todo.md).
+
+## Instrumentation test features
+
+In many cases, Chromium has extended the instrumentation test framework
+classes to implement additional features.
+
+### Test runners
+
+[todo](/testing/android/docs/todo.md)
+
+### Test cases
+
+[todo](/testing/android/docs/todo.md)
+
+### Annotations
+
+Instrumentation tests in Chromium use a wide variety of annotations to control
+and manipulate test execution. Some of these are implemented in Chromium, while
+others are pulled in from outside. They include:
+
+#### Size annotations
+
+Size annotations are used primarily by the test runner to determine the length
+of time to wait before considering a test hung (i.e., its timeout duration).
+
+Several of the annotations are Android APIs from
+[android.test.suitebuilder.annotation](https://developer.android.com/reference/android/test/suitebuilder/annotation/package-summary.html)
+(prior to [Android N](https://en.wikipedia.org/wiki/Android_Nougat)) or
+[android.support.test.filters](https://developer.android.com/reference/android/support/test/filters/package-summary.html)
+(starting in Android N). These are all fairly self-explanatory:
+
+ - [`@SmallTest`](https://developer.android.com/reference/android/support/test/filters/SmallTest.html) (timeout: **1 minute**)
+ - [`@MediumTest`](https://developer.android.com/reference/android/support/test/filters/MediumTest.html) (timeout: **3 minutes**)
+ - [`@LargeTest`](https://developer.android.com/reference/android/support/test/filters/LargeTest.html) (timeout: **5 minutes**)
+
+A few additional size annotations are provided in
+[//base](https://chromium.googlesource.com/chromium/src/+/master/base):
+
+ - [`@EnormousTest`](https://chromium.googlesource.com/chromium/src/+/master/base/test/android/javatests/src/org/chromium/base/test/util/EnormousTest.java)
+(timeout: **10 minutes**) Typically used for tests that require WiFi.
+ - [`@IntegrationTest`](https://chromium.googlesource.com/chromium/src/+/master/base/test/android/javatests/src/org/chromium/base/test/util/IntegrationTest.java)
+(timeout: **30 minutes**) Used for tests that run against real services.
+ - [`@Manual`](https://chromium.googlesource.com/chromium/src/+/master/base/test/android/javatests/src/org/chromium/base/test/util/Manual.java)
+(timeout: **10 hours**) Used for manual tests.
+
+Beware that the timeout duration the corresponds to each of these annotations
+is subject to change.
+
+#### Annotations that disable tests
+
+There are several annotations that control whether or not a test runs.
+Some are conditional, others are not.
+
+##### Unconditional disabling
+
+Two annotations disable a test unless a user specifically asks for tests
+annotated with that annotation to be run
+(e.g., via passing `-A DisabledTest` to the test runner).
+Note that these are both implemented in Chromium.
+There are versions of @DisabledTest and @FlakyTest in Android that do not allow users to provide a message field.
+
+ - [`@DisabledTest`](https://chromium.googlesource.com/chromium/src/+/master/base/test/android/javatests/src/org/chromium/base/test/util/DisabledTest.java)`(msg = "")`

[mikecase (-- gone --), 2016/08/16 23:08:58]

s/msg/message

[jbudorick, 2016/08/16 23:54:23]

Done.

+ - [`@FlakyTest`](https://chromium.googlesource.com/chromium/src/+/master/base/test/android/javatests/src/org/chromium/base/test/util/FlakyTest.java)`(msg = "")`

[mikecase (-- gone --), 2016/08/16 23:08:58]

s/msg/message

[jbudorick, 2016/08/16 23:54:23]

Done.

+
+In both cases, `msg` is a message that should describe why the test is

[mikecase (-- gone --), 2016/08/16 23:08:57]

s/msg/message

[jbudorick, 2016/08/16 23:54:23]

Done.

+disabled or marked as flaky. It should include a crbug link.
+
+##### Conditional disabling
+
+There are two primary annotation categories that conditionally disable tests:
+**@DisableIf** and **Restriction**. The **@DisableIf** annotations are intended

[mikecase (-- gone --), 2016/08/16 23:08:57]

nit: change Restriction to @Restriction? To be consistent with everything else.

[jbudorick, 2016/08/16 23:54:23]

Done.

+to temporarily disable a test in certain scenarios where it *should* work but
+doesn't. In contrast, the **@Restriction** annotation is intended to
+permanently limit a test to specific configurations. It signifies that the test
+was not, is not, and will not be intended to run beyond those configurations.
+In both cases, conditional disabling manifests as a skipped test.
+
+**@DisableIf.Build** allows for conditional test disabling based on values in
+[`android.os.Build`](https://developer.android.com/reference/android/os/Build.html):
+
+```java
+@DisableIf.Build(
+
+    // Describes why the test is disabled.
+    message = "",
+
+    // Disables the test on SDK levels that match the given conditions.
+    // Checks against Build.VERSION.SDK_INT.
+    sdk_is_greater_than = 0,
+    sdk_is_less_than = Integer.MAX_VALUE,
+
+    // Disables the test on devices that support the given ABI
+    // (e.g. "arm64-v8a"). Checks against:
+    //  - Build.SUPPORTED_ABIS on L+
+    //  - Build.CPU_ABI and Build.CPU_ABI2 otherwise
+    supported_abis_includes = "",
+
+    // Disables the test on devices with hardware that matches the given
+    // value. Checks against Build.HARDWARE.
+    hardware_is = "",
+
+    // Disables the test on devices with product names that contain the
+    // given value. Checks against Build.PRODUCT.
+    product_name_includes = "",
+
+)
+```
+
+**@DisableIf.Device** allows for conditional test disabling based on whether
+a device is a phone, a tablet, or a "large tablet" as determined by
+[org.chromium.ui.base.DeviceFormFactor](https://chromium.googlesource.com/chromium/src/+/master/ui/android/java/src/org/chromium/ui/base/DeviceFormFactor.java).
+Note that this is currently only available to tests in
+[//chrome](https://chromium.googlesource.com/chromium/src/+/master/chrome/)
+or code that uses //chrome.
+
+```java
+@DisableIf.Device(
+    // Disables the test on devices that match the given type(s) as described
+    // above.
+    type = {}
+)
+```
+
+**@Restriction** currently allows for conditional test disabling based on device
+type, device performance, internet connectivity, whether Google Play Services is
+up to date, and whether the build was an official one.
+
+```java
+@Restriction(
+    // Possible values include:
+    //
+    // base: 
+    //  - Restriction.RESTRICTION_TYPE_LOW_END_DEVICE
+    //    Restricts the test to low-end devices as determined by SysUtils.isLowEndDevice().
+    //
+    //  - Restriction.RESTRICTION_TYPE_NON_LOW_END_DEVICE
+    //    Restricts the test to non-low-end devices as determined by SysUtils.isLowEndDevice().
+    //
+    //  - Restriction.RESTRICTION_TYPE_INTERNET
+    //    Restricts the test to devices that have an internet connection.
+    //
+    // chrome:
+    //  - ChromeRestriction.RESTRICTION_TYPE_GOOGLE_PLAY_SERVICES
+    //    Restricts the test to devices with up-to-date versions of Google Play Services.
+    //
+    //  - ChromeRestriction.RESTRICTION_TYPE_PHONE
+    //    Restricts the test to phones as determined by DeviceFormFactor.
+    //
+    //  - ChromeRestriction.RESTRICTION_TYPE_TABLET
+    //    Restricts the test to tablets as determined by DeviceFormFactor.
+    //
+    //  - ChromeRestriction.RESTRICTION_TYPE_OFFICIAL_BUILD
+    //    Restricts the test to official builds as determined by ChromeVersionInfo.isOfficialBuild().
+    value = {}
+)
+```
+
+**@MinAndroidSdkLevel** is similar to @Restriction in purpose in that it's
+intended to permanently limit a test to only recent versions of Android.
+
+```java
+@MinAndroidSdkLevel(
+    // The minimum SDK level at which this test should be executed. Checks
+    // against Build.VERSION.SDK_INT.
+    value = 0
+)
+```
+
+#### Annotations that affect how a test is run

[mikecase (-- gone --), 2016/08/16 23:08:57]

super nit:

idk if it was just the markdown viewer you set up, but header4 is almost
indistinguishable from just bold **<text>**. So it makes some of these
paragraphs hard to break up, since a lot of lines start with a few bolded words.

example...

**@MinAndroidSdkLevel**
#### Annotations that affect how a test is run

^ both of these look like headers

[jbudorick, 2016/08/16 23:54:23]

//tools/md_browser/md_browser.py (which I used locally) is definitely renders
differently than the gitiles markdown renderer. I'm going to land this as-is,
see how it looks in gitiles, and tweak it from there.

[jbudorick, 2016/08/16 23:55:15]

s/is definitely renders/definitely renders/

used up my proofreading quota for the day.

+
+Several annotations affect how a test is run in interesting or nontrivial ways.
+
+**@CommandLineFlags.Add** and **@CommandLineFlags.Remove** manipulate Chrome's
+command-line flags on a per-test basis (i.e., the flags handled by
+[`org.chromium.base.CommandLine`](https://chromium.googlesource.com/chromium/src/+/master/base/android/java/src/org/chromium/base/CommandLine.java) and
+[`base::CommandLine`](https://chromium.googlesource.com/chromium/src/+/master/base/command_line.h)).
+
+```java
+@CommandLineFlags.Add(
+    // The flags to add to the command line for this test. These can be
+    // anything and typically should include the leading dashes (e.g. "--foo").
+    value = {}
+)
+
+@CommandLineFlags.Remove(
+    // The flags to remove from the command line for this test. These can only
+    // be flags added via @CommandLineFlags.Add. Flags already present in the
+    // command-line file on the device are only present in the native
+    // CommandLine object and cannot be manipulated.
+    value = {}
+)
+```
+
+#### Feature annotations
+
+**@Feature** has been used inconsistently in Chromium to group tests across
+test cases according to the feature they're testing.
+
+```java
+@Feature(
+    // The features associated with this test. These can be anything.
+    value = {}
+)
+```
+
+@Feature doesn't have an inherent function, but it can be used to filter tests
+via the test runner's
+[-A/--annotation](/testing/android/docs/todo.md) and
+[-E/--exclude-annotation](/testing/android/docs/todo.md) flags. For example,
+this would run only the tests with @Feature annotations containing at least
+"Sync" in `chrome_public_test_apk`:
+
+```bash
+out/Debug/bin/run_chrome_public_test_apk -A Feature=Sync
+```