Chromium Code Reviews| OLD | NEW |
|---|---|
| 1 # Android Test Instructions | 1 # Android Test Instructions |
| 2 | 2 |
| 3 Device Setup Tests are runnable on physical devices or emulators. See the | |
| 4 instructions below for setting up either a physical device or an emulator. | |
| 5 | |
| 6 [TOC] | 3 [TOC] |
| 7 | 4 |
| 8 ## Physical Device Setup **ADB Debugging** | 5 ## Device Setup |
| 6 | |
| 7 ### Physical Device Setup | |
| 8 | |
| 9 #### ADB Debugging | |
| 9 | 10 |
| 10 In order to allow the ADB to connect to the device, you must enable USB | 11 In order to allow the ADB to connect to the device, you must enable USB |
| 11 debugging: | 12 debugging: |
| 12 | 13 |
| 13 * Before Android 4.1 (Jelly Bean): | 14 * Before Android 4.1 (Jelly Bean): |
| 14 * Go to "System Settings" | 15 * Go to "System Settings" |
| 15 * Go to "Developer options" | 16 * Go to "Developer options" |
| 16 * Check "USB debugging". | 17 * Check "USB debugging". |
| 17 * Un-check "Verify apps over USB". | 18 * Un-check "Verify apps over USB". |
| 18 * On Jelly Bean, developer options are hidden by default. To unhide them: | 19 * On Jelly Bean and above, developer options are hidden by default. To unhide them: |
| 19 * Go to "About phone" | 20 * Go to "About phone" |
| 20 * Tap 10 times on "Build number" | 21 * Tap 10 times on "Build number" |
| 21 * The "Developer options" menu will now be available. | 22 * The "Developer options" menu will now be available. |
| 22 * Check "USB debugging". | 23 * Check "USB debugging". |
| 23 * Un-check "Verify apps over USB". | 24 * Un-check "Verify apps over USB". |
| 24 | 25 |
| 25 ### Screen | 26 #### Screen |
| 26 | 27 |
| 27 You MUST ensure that the screen stays on while testing: `adb shell svc power | 28 You MUST ensure that the screen stays on while testing: `adb shell svc power |
| 28 stayon usb` Or do this manually on the device: Settings -> Developer options | 29 stayon usb` Or do this manually on the device: Settings -> Developer options |
| 29 -> Stay Awake. | 30 -> Stay Awake. |
| 30 | 31 |
| 31 If this option is greyed out, stay awake is probably disabled by policy. In that | 32 If this option is greyed out, stay awake is probably disabled by policy. In that |
| 32 case, get another device or log in with a normal, unmanaged account (because the | 33 case, get another device or log in with a normal, unmanaged account (because the |
| 33 tests will break in exciting ways if stay awake is off). | 34 tests will break in exciting ways if stay awake is off). |
| 34 | 35 |
| 35 ### Enable Asserts! | 36 #### Enable Asserts! |
|
rnephew (Reviews Here)
2016/08/03 18:08:39
Not that it really matters, but the ! seems out of
jbudorick
2016/08/03 18:12:40
Done.
| |
| 36 | 37 |
| 37 adb shell setprop debug.assert 1 | 38 adb shell setprop debug.assert 1 |
| 38 | 39 |
| 39 ### Disable Verify Apps | 40 #### Disable Verify Apps |
| 40 | 41 |
| 41 You may see a dialog like | 42 You may see a dialog like |
| 42 [this one](http://www.samsungmobileusa.com/simulators/ATT_GalaxyMega/mobile/scre ens/06-02_12.jpg), | 43 [this one](http://www.samsungmobileusa.com/simulators/ATT_GalaxyMega/mobile/scre ens/06-02_12.jpg), |
| 43 which states, _Google may regularly check installed apps for potentially harmful | 44 which states, _Google may regularly check installed apps for potentially harmful |
| 44 behavior._ This can interfere with the test runner. To disable this dialog, run: | 45 behavior._ This can interfere with the test runner. To disable this dialog, run: |
| 45 `adb shell settings put global package_verifier_enable 0` | 46 `adb shell settings put global package_verifier_enable 0` |
| 46 | 47 |
| 47 ## Emulator Setup | 48 ### Emulator Setup |
| 48 | 49 |
| 49 ### Option 1: | 50 #### Option 1 |
| 50 | 51 |
| 51 Use an emulator (i.e. Android Virtual Device, AVD): Enabling Intel's | 52 Use an emulator (i.e. Android Virtual Device, AVD): Enabling Intel's |
| 52 Virtualizaton support provides the fastest, most reliable emulator configuration | 53 Virtualizaton support provides the fastest, most reliable emulator configuration |
| 53 available (i.e. x86 emulator with GPU acceleration and KVM support). | 54 available (i.e. x86 emulator with GPU acceleration and KVM support). |
| 54 Remember to build with `target_arch=ia32` for x86. Otherwise installing the APKs | 55 Remember to build with `target_arch=ia32` for x86. Otherwise installing the APKs |
| 55 will fail with `INSTALL_FAILED_NO_MATCHING_ABIS`. | 56 will fail with `INSTALL_FAILED_NO_MATCHING_ABIS`. |
| 56 | 57 |
| 57 1. Enable Intel Virtualization support in the BIOS. | 58 1. Enable Intel Virtualization support in the BIOS. |
| 58 | 59 |
| 59 2. Set up your environment: | 60 2. Set up your environment: |
| 60 | 61 |
| 61 ```shell | 62 ```shell |
| 62 . build/android/envsetup.sh | 63 . build/android/envsetup.sh |
| 63 ``` | 64 ``` |
| 64 | 65 |
| 65 3. Install emulator deps: | 66 3. Install emulator deps: |
| 66 | 67 |
| 67 ```shell | 68 ```shell |
| 68 build/android/install_emulator_deps.py --api-level=23 | 69 build/android/install_emulator_deps.py --api-level=23 |
| 69 ``` | 70 ``` |
| 70 | 71 |
| 71 This script will download Android SDK and place it a directory called | 72 This script will download Android SDK and place it a directory called |
| 72 android\_tools in the same parent directory as your chromium checkout. It | 73 android\_tools in the same parent directory as your chromium checkout. It |
| 73 will also download the system-images for the emulators (i.e. arm and x86). | 74 will also download the system-images for the emulators (i.e. arm and x86). |
| 74 Note that this is a different SDK download than the Android SDK in the | 75 Note that this is a different SDK download than the Android SDK in the |
| 75 chromium source checkout (i.e. src/third\_party/android\_emulator\_sdk). | 76 chromium source checkout (i.e. `src/third_party/android_emulator_sdk`). |
| 76 | 77 |
| 77 4. Run the avd.py script. To start up _num_ emulators use -n. For non-x86 use | 78 4. Run the avd.py script. To start up _num_ emulators use -n. For non-x86 use |
| 78 --abi. | 79 --abi. |
| 79 | 80 |
| 80 ```shell | 81 ```shell |
| 81 build/android/avd.py --api-level=23 | 82 build/android/avd.py --api-level=23 |
| 82 ``` | 83 ``` |
| 83 | 84 |
| 84 This script will attempt to use GPU emulation, so you must be running the | 85 This script will attempt to use GPU emulation, so you must be running the |
| 85 emulators in an environment with hardware rendering available. See | 86 emulators in an environment with hardware rendering available. See |
| 86 `avd.py --help` for more details. | 87 `avd.py --help` for more details. |
| 87 | 88 |
| 88 ### Option 2: | 89 #### Option 2 |
| 89 | 90 |
| 90 Alternatively, you can create an run your own emulator using the tools provided | 91 Alternatively, you can create an run your own emulator using the tools provided |
|
rnephew (Reviews Here)
2016/08/03 18:08:39
create a run*
jbudorick
2016/08/03 18:12:40
create and run, but done.
| |
| 91 by the Android SDK. When doing so, be sure to enable GPU emulation in hardware | 92 by the Android SDK. When doing so, be sure to enable GPU emulation in hardware |
| 92 settings, since Chromium requires it to render. | 93 settings, since Chromium requires it to render. |
| 93 | 94 |
| 94 ## Building Tests | 95 ## Building Tests |
| 95 | 96 |
| 96 It may not be immediately obvious where your test code gets compiled to, so here | 97 If you're adding a new test file, you'll need to explicitly add it to a gn |
| 97 are some general rules: | 98 target. If you're adding a test to an existing file, you won't to make gn |
| 99 changes, but you may be interested in where your test winds up. In either case, | |
| 100 here are some guidelines for where a test belongs: | |
| 98 | 101 |
| 99 * If your test code lives under /base, it will be built as part of the | 102 ### C++ |
| 100 base_unittests_apk. | |
| 101 * If your test code lives under /content, it will probably be built as part of | |
| 102 the content_shell_test_apk | |
| 103 * If your test code lives under /chrome (or higher), it will probably be built | |
| 104 as part of the chrome_public_test_apk | |
| 105 * (Please fill in more details here if you know them). | |
| 106 | 103 |
| 107 NB: We used to call the chrome_public_test_apk the | 104 C++ test files typically belong in `<top-level directory>_unittests` (e.g. |
| 108 chromium_shell_test_apk. There may still be references to this kicking | 105 `base_unittests` for `//base`). There are a few exceptions -- browser tests are |
| 109 around, but wherever you see chromium_shell_test you should replace with | 106 typically their own target (e.g. `content_browsertests` for `//content`, or |
| 110 chrome_public_test. | 107 `browser_tests` for `//chrome`), and some unit test suites are broken at the |
| 108 second directory rather than the top-level one. | |
| 109 | |
| 110 ### Java | |
| 111 | |
| 112 Java test files vary a bit more widely than their C++ counterparts: | |
| 113 | |
| 114 - Instrumentation test files -- i.e., tests that will run on a device -- | |
| 115 typically belong in either `<top-level directory>_javatests` or | |
| 116 `<top-level directory>_test_java`. Regardless, they'll wind up getting packaged | |
| 117 into one of a few test APKs: | |
| 118 - `android_webview_test_apk` for anything in `//android_webview` | |
| 119 - `content_shell_test_apk` for anything in `//content` or below | |
| 120 - `chrome_public_test_apk` for most things in `//chrome` | |
| 121 - `chrome_sync_shell_test_apk` in a few exceptional cases | |
| 122 - JUnit or Robolectric test files -- i.e., tests that will run on the host -- | |
| 123 typically belong in `<top-level directory>_junit_tests` (e.g. `base_junit_tests` | |
| 124 for `//base`), though here again there are cases (particularly in | |
| 125 `//components`) where suites are split at the second directory rather than the | |
| 126 top-level one. | |
| 111 | 127 |
| 112 Once you know what to build, just do it like you normally would build anything | 128 Once you know what to build, just do it like you normally would build anything |
| 113 else, e.g.: `ninja -C out/Release chrome_public_test_apk` | 129 else, e.g.: `ninja -C out/Release chrome_public_test_apk` |
| 114 | 130 |
| 115 ## Running Tests | 131 ## Running Tests |
| 116 | 132 |
| 117 All functional tests are run using `build/android/test_runner.py`, either | 133 All functional tests should be runnable via the wrapper scripts generated at |
| 118 directly or via a generated wrapper script in `<output directory>/bin/`. | 134 build time: |
| 119 Tests are sharded across all attached devices. In order to run tests, call: | 135 |
| 120 `build/android/test_runner.py <test_type> [options]` | 136 ```sh |
| 121 (or `<generated wrapper script> [options]`). | 137 <output directory>/bin/run_<target_name> [options] |
| 122 For a list of valid test types, see `test_runner.py --help`. For | 138 ``` |
| 123 help on a specific test type, run `test_runner.py <test_type> --help`. | 139 |
| 140 Note that tests are sharded across all attached devices unless explicitly told | |
| 141 to do otherwise by `-d/--device`. | |
| 124 | 142 |
| 125 The commands used by the buildbots are printed in the logs. Look at | 143 The commands used by the buildbots are printed in the logs. Look at |
| 126 http://build.chromium.org/ to duplicate the same test command as a particular | 144 http://build.chromium.org/ to duplicate the same test command as a particular |
| 127 builder. | 145 builder. |
| 128 | 146 |
| 129 If you build in an output directory other than "out", you may have to tell | 147 ### INSTALL\_FAILED\_CONTAINER\_ERROR or INSTALL\_FAILED\_INSUFFICIENT\_STORAGE |
| 130 test\_runner.py where you place it. Say you build your android code in | |
| 131 out\_android, then do `export CHROMIUM_OUT_DIR=out_android` before running the | |
| 132 command below. You have to do this even if your "out" directory is a symlink | |
| 133 pointing to "out_android". You can also use `--output-directory` to point to the | |
| 134 path of your output directory, for example, | |
| 135 `--output-directory=out_android/Debug`. (The generated wrapper scripts handle | |
| 136 this automatically.) | |
| 137 | 148 |
| 138 ## INSTALL\_FAILED\_CONTAINER\_ERROR or INSTALL\_FAILED\_INSUFFICIENT\_STORAGE | 149 If you see this error when the test runner is attempting to deploy the test |
| 139 | |
| 140 If you see this error when test\_runner.py is attempting to deploy the test | |
| 141 binaries to the AVD emulator, you may need to resize your userdata partition | 150 binaries to the AVD emulator, you may need to resize your userdata partition |
| 142 with the following commands: | 151 with the following commands: |
| 143 | 152 |
| 144 ```shell | 153 ```shell |
| 145 # Resize userdata partition to be 1G | 154 # Resize userdata partition to be 1G |
| 146 resize2fs android_emulator_sdk/sdk/system-images/android-23/x86/userdata.img 1G | 155 resize2fs android_emulator_sdk/sdk/system-images/android-23/x86/userdata.img 1G |
| 147 | 156 |
| 148 # Set filesystem parameter to continue on errors; Android doesn't like some | 157 # Set filesystem parameter to continue on errors; Android doesn't like some |
| 149 # things e2fsprogs does. | 158 # things e2fsprogs does. |
| 150 tune2fs -e continue android_emulator_sdk/sdk/system-images/android-23/x86/userda ta.img | 159 tune2fs -e continue android_emulator_sdk/sdk/system-images/android-23/x86/userda ta.img |
| (...skipping 47 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... | |
| 198 | 207 |
| 199 # Run a subset of tests. You might need to pass the package name for some tests. | 208 # Run a subset of tests. You might need to pass the package name for some tests. |
| 200 build/android/test_runner.py junit -s chrome_junit_tests --release -vvv | 209 build/android/test_runner.py junit -s chrome_junit_tests --release -vvv |
| 201 -f "org.chromium.chrome.browser.media.*" | 210 -f "org.chromium.chrome.browser.media.*" |
| 202 ``` | 211 ``` |
| 203 | 212 |
| 204 ## Gtests | 213 ## Gtests |
| 205 | 214 |
| 206 ```shell | 215 ```shell |
| 207 # Build a test suite | 216 # Build a test suite |
| 208 ninja -C out/Release content_unittests_apk | 217 ninja -C out/Release content_unittests |
| 209 | 218 |
| 210 # Run a test suite | 219 # Run a test suite |
| 211 out/Release/bin/run_content_unittests [-vv] | 220 out/Release/bin/run_content_unittests [-vv] |
| 212 | 221 |
| 213 # Run a subset of tests | 222 # Run a subset of tests |
| 214 out/Release/bin/run_content_unittests [-vv] --gtest-filter ByteStreamTest.* | 223 out/Release/bin/run_content_unittests [-vv] --gtest-filter ByteStreamTest.* |
| 215 ``` | 224 ``` |
| 216 | 225 |
| 217 ## Instrumentation Tests | 226 ## Instrumentation Tests |
| 218 | 227 |
| (...skipping 71 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... | |
| 290 ## Running Blink Layout Tests | 299 ## Running Blink Layout Tests |
| 291 | 300 |
| 292 See | 301 See |
| 293 https://sites.google.com/a/chromium.org/dev/developers/testing/webkit-layout-tes ts | 302 https://sites.google.com/a/chromium.org/dev/developers/testing/webkit-layout-tes ts |
| 294 | 303 |
| 295 ## Running GPU tests | 304 ## Running GPU tests |
| 296 | 305 |
| 297 (e.g. the "Android Debug (Nexus 7)" bot on the chromium.gpu waterfall) | 306 (e.g. the "Android Debug (Nexus 7)" bot on the chromium.gpu waterfall) |
| 298 | 307 |
| 299 See http://www.chromium.org/developers/testing/gpu-testing for details. Use | 308 See http://www.chromium.org/developers/testing/gpu-testing for details. Use |
| 300 --browser=android-content-shell. Examine the stdio from the test invocation on | 309 `--browser=android-content-shell`. Examine the stdio from the test invocation on |
| 301 the bots to see arguments to pass to src/content/test/gpu/run\_gpu\_test.py. | 310 the bots to see arguments to pass to `src/content/test/gpu/run_gpu_test.py`. |
| OLD | NEW |