OLD | NEW |
(Empty) | |
| 1 # Android Test Instructions |
| 2 |
| 3 Device Setup Tests are runnable on physical devices or emulators. See the |
| 4 instructions below for setting up either a physical device or an emulator. |
| 5 |
| 6 [TOC] |
| 7 |
| 8 ## Physical Device Setup **ADB Debugging** |
| 9 |
| 10 In order to allow the ADB to connect to the device, you must enable USB |
| 11 debugging: |
| 12 * Before Android 4.1 (Jelly Bean): |
| 13 * Go to "System Settings" |
| 14 * Go to "Developer options" |
| 15 * Check "USB debugging". |
| 16 * Un-check "Verify apps over USB". |
| 17 * On Jelly Bean, developer options are hidden by default. To unhide them: |
| 18 * Go to "About phone" |
| 19 * Tap 10 times on "Build number" |
| 20 * The "Developer options" menu will now be available. |
| 21 * Check "USB debugging". |
| 22 * Un-check "Verify apps over USB". |
| 23 |
| 24 ### Screen |
| 25 |
| 26 You MUST ensure that the screen stays on while testing: `adb shell svc power |
| 27 stayon usb` Or do this manually on the device: Settings -> Developer options |
| 28 -> Stay Awake. |
| 29 |
| 30 If this option is greyed out, stay awake is probably disabled by policy. In that |
| 31 case, get another device or log in with a normal, unmanaged account (because the |
| 32 tests will break in exciting ways if stay awake is off). |
| 33 |
| 34 ### Enable Asserts! |
| 35 |
| 36 `adb shell setprop debug.assert 1` |
| 37 |
| 38 ### Disable Verify Apps |
| 39 |
| 40 You may see a dialog like |
| 41 [this one](http://www.samsungmobileusa.com/simulators/ATT_GalaxyMega/mobile/scre
ens/06-02_12.jpg), |
| 42 which states, _Google may regularly check installed apps for potentially harmful |
| 43 behavior._ This can interfere with the test runner. To disable this dialog, run: |
| 44 `adb shell settings put global package_verifier_enable 0` |
| 45 |
| 46 ## Emulator Setup |
| 47 |
| 48 ### Option 1: |
| 49 |
| 50 Use an emulator (i.e. Android Virtual Device, AVD): Enabling Intel's |
| 51 Virtualizaton support provides the fastest, most reliable emulator configuration |
| 52 available (i.e. x86 emulator with GPU acceleration and KVM support). |
| 53 |
| 54 1. Enable Intel Virtualization support in the BIOS. |
| 55 |
| 56 2. Set up your environment: |
| 57 |
| 58 ```shell |
| 59 . build/android/envsetup.sh |
| 60 ``` |
| 61 |
| 62 3. Install emulator deps: |
| 63 |
| 64 ```shell |
| 65 build/android/install_emulator_deps.py --api-level=19 |
| 66 ``` |
| 67 |
| 68 This script will download Android SDK and place it a directory called |
| 69 android\_tools in the same parent directory as your chromium checkout. It |
| 70 will also download the system-images for the emulators (i.e. arm and x86). |
| 71 Note that this is a different SDK download than the Android SDK in the |
| 72 chromium source checkout (i.e. src/third\_party/android\_emulator\_sdk). |
| 73 |
| 74 4. Run the avd.py script. To start up _num_ emulators use -n. For non-x86 use |
| 75 --abi. |
| 76 |
| 77 ```shell |
| 78 build/android/avd.py --api-level=19 |
| 79 ``` |
| 80 |
| 81 This script will attempt to use GPU emulation, so you must be running the |
| 82 emulators in an environment with hardware rendering available. See |
| 83 `avd.py --help` for more details. |
| 84 |
| 85 ### Option 2: |
| 86 |
| 87 Alternatively, you can create an run your own emulator using the tools provided |
| 88 by the Android SDK. When doing so, be sure to enable GPU emulation in hardware |
| 89 settings, since Chromium requires it to render. |
| 90 |
| 91 ## Building Tests |
| 92 |
| 93 It may not be immediately obvious where your test code gets compiled to, so here |
| 94 are some general rules: |
| 95 |
| 96 * If your test code lives under /content, it will probably be built as part of |
| 97 the content\_shell\_test\_apk * If your test code lives under /chrome (or |
| 98 higher), it will probably be built as part of the chrome\_shell\_test\_apk * |
| 99 (Please fill in more details here if you know them). |
| 100 |
| 101 NB: We used to call the chrome\_shell\_test\_apk the |
| 102 chromium\_shell\_test\_apk. There may still be references to this kicking |
| 103 around, but wherever you see chromium\_shell\_test you should replace with |
| 104 chrome\_shell\_test. |
| 105 |
| 106 Once you know what to build, just do it like you normally would build anything |
| 107 else, e.g.: `ninja -C out/Release chrome_shell_test_apk` |
| 108 |
| 109 ## Running Tests |
| 110 |
| 111 All functional tests are run using `build/android/test_runner.py`. |
| 112 Tests are sharded across all attached devices. In order to run tests, call: |
| 113 `build/android/test_runner.py <test_type> [options]` |
| 114 For a list of valid test types, see `test_runner.py --help`. For |
| 115 help on a specific test type, run `test_runner.py <test_type> --help`. |
| 116 |
| 117 The commands used by the buildbots are printed in the logs. Look at |
| 118 http://build.chromium.org/ to duplicate the same test command as a particular |
| 119 builder. |
| 120 |
| 121 If you build in an output directory other than "out", you may have to tell |
| 122 test\_runner.py where you place it. Say you build your android code in |
| 123 out\_android, then do `export CHROMIUM_OUT_DIR=out_android` before running the |
| 124 command below. |
| 125 |
| 126 ## INSTALL\_FAILED\_CONTAINER\_ERROR or INSTALL\_FAILED\_INSUFFICIENT\_STORAGE |
| 127 |
| 128 If you see this error when test\_runner.py is attempting to deploy the test |
| 129 binaries to the AVD emulator, you may need to resize your userdata partition |
| 130 with the following commands: |
| 131 |
| 132 ```shell |
| 133 # Resize userdata partition to be 1G resize2fs |
| 134 android_emulator_sdk/sdk/system-images/android-19/x86/userdata.img 1G |
| 135 |
| 136 # Set filesystem parameter to continue on errors; Android doesn't like some |
| 137 # things e2fsprogs does. |
| 138 tune2fs -e continue |
| 139 android_emulator_sdk/sdk/system-images/android-19/x86/userdata.img |
| 140 ``` |
| 141 |
| 142 ## Symbolizing Crashes |
| 143 |
| 144 Crash stacks are logged and can be viewed using adb logcat. To symbolize the |
| 145 traces, pipe the output through |
| 146 `third_party/android_platform/development/scripts/stack`. If you build in an |
| 147 output directory other than "out", pass |
| 148 `--chrome-symbols-dir=out_directory/{Debug,Release}/lib` to the script as well. |
| 149 |
| 150 ## Gtests |
| 151 |
| 152 ```shell |
| 153 # Build a test suite |
| 154 ninja -C out/Release content_unittests_apk |
| 155 |
| 156 # Run a test suite |
| 157 build/android/test_runner.py gtest -s content_unittests --release -vvv |
| 158 |
| 159 # Run a subset of tests |
| 160 build/android/test_runner.py gtest -s content_unittests --release -vvv \ |
| 161 --gtest-filter ByteStreamTest.* |
| 162 ``` |
| 163 |
| 164 ## Instrumentation Tests |
| 165 |
| 166 In order to run instrumentation tests, you must leave your device screen ON and |
| 167 UNLOCKED. Otherwise, the test will timeout trying to launch an intent. |
| 168 Optionally you can disable screen lock under Settings -> Security -> Screen Lock |
| 169 -> None. |
| 170 |
| 171 Next, you need to build the app, build your tests, install the application APK, |
| 172 and then run your tests (which will install the test APK automatically). |
| 173 |
| 174 Examples: |
| 175 |
| 176 ContentShell tests: |
| 177 |
| 178 ```shell |
| 179 # Build the code under test |
| 180 ninja -C out/Release content_shell_apk |
| 181 |
| 182 # Build the tests themselves |
| 183 ninja -C out/Release content_shell_test_apk |
| 184 |
| 185 # Install the code under test |
| 186 build/android/adb_install_apk.py out/Release/apks/ContentShell.apk |
| 187 |
| 188 # Run the test (will automagically install the test APK) |
| 189 build/android/test_runner.py instrumentation --test-apk=ContentShellTest \ |
| 190 --isolate-file-path content/content_shell_test_apk.isolate --release -vv |
| 191 ``` |
| 192 |
| 193 ChromeShell tests: |
| 194 |
| 195 ```shell |
| 196 # Build the code under test |
| 197 ninja -C out/Release chrome_shell_apk |
| 198 |
| 199 # Build the tests themselves |
| 200 ninja -C out/Release chrome_shell_test_apk |
| 201 |
| 202 # Install the code under test |
| 203 build/android/adb_install_apk.py out/Release/apks/ChromeShell.apk |
| 204 |
| 205 # Run the test (will automagically install the test APK) |
| 206 build/android/test_runner.py instrumentation --test-apk=ChromeShellTest \ |
| 207 --isolate-file-path chrome/chrome_shell_test_apk.isolate --release -vv |
| 208 ``` |
| 209 |
| 210 AndroidWebView tests: |
| 211 |
| 212 ```shell |
| 213 ninja -C out/Release android_webview_apk |
| 214 ninja -C out/Release android_webview_test_apk |
| 215 build/android/adb_install_apk.py out/Release/apks/AndroidWebView.apk \ |
| 216 build/android/test_runner.py instrumentation --test-apk=AndroidWebViewTest \ |
| 217 --test_data webview:android_webview/test/data/device_files --release -vvv |
| 218 ``` |
| 219 |
| 220 Use adb\_install\_apk.py to install the app under test, then run the test |
| 221 command. In order to run a subset of tests, use -f to filter based on test |
| 222 class/method or -A/-E to filter using annotations. |
| 223 |
| 224 Filtering examples: |
| 225 |
| 226 ```shell |
| 227 # Run a test suite |
| 228 build/android/test_runner.py instrumentation --test-apk=ContentShellTest |
| 229 |
| 230 # Run a specific test class |
| 231 build/android/test_runner.py instrumentation --test-apk=ContentShellTest -f \ |
| 232 AddressDetectionTest |
| 233 |
| 234 # Run a specific test method |
| 235 build/android/test_runner.py instrumentation --test-apk=ContentShellTest -f \ |
| 236 AddressDetectionTest#testAddressLimits |
| 237 |
| 238 # Run a subset of tests by size (Smoke, SmallTest, MediumTest, LargeTest, |
| 239 # EnormousTest) |
| 240 build/android/test_runner.py instrumentation --test-apk=ContentShellTest -A \ |
| 241 Smoke |
| 242 |
| 243 # Run a subset of tests by annotation, such as filtering by Feature |
| 244 build/android/test_runner.py instrumentation --test-apk=ContentShellTest -A \ |
| 245 Feature=Navigation |
| 246 ``` |
| 247 |
| 248 You might want to add stars `*` to each as a regular expression, e.g. |
| 249 `*`AddressDetectionTest`*` |
| 250 |
| 251 ## Running Blink Layout Tests |
| 252 |
| 253 See |
| 254 https://sites.google.com/a/chromium.org/dev/developers/testing/webkit-layout-tes
ts |
| 255 |
| 256 ## Running GPU tests |
| 257 |
| 258 (e.g. the "Android Debug (Nexus 7)" bot on the chromium.gpu waterfall) |
| 259 |
| 260 See http://www.chromium.org/developers/testing/gpu-testing for details. Use |
| 261 --browser=android-content-shell. Examine the stdio from the test invocation on |
| 262 the bots to see arguments to pass to src/content/test/gpu/run\_gpu\_test.py. |
OLD | NEW |