| OLD | NEW |
| 1 # Android Test Instructions | 1 # Android Test Instructions |
| 2 | 2 |
| 3 Device Setup Tests are runnable on physical devices or emulators. See the | 3 Device Setup Tests are runnable on physical devices or emulators. See the |
| 4 instructions below for setting up either a physical device or an emulator. | 4 instructions below for setting up either a physical device or an emulator. |
| 5 | 5 |
| 6 [TOC] | 6 [TOC] |
| 7 | 7 |
| 8 ## Physical Device Setup **ADB Debugging** | 8 ## Physical Device Setup **ADB Debugging** |
| 9 | 9 |
| 10 In order to allow the ADB to connect to the device, you must enable USB | 10 In order to allow the ADB to connect to the device, you must enable USB |
| (...skipping 96 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
| 107 NB: We used to call the chrome_public_test_apk the | 107 NB: We used to call the chrome_public_test_apk the |
| 108 chromium_shell_test_apk. There may still be references to this kicking | 108 chromium_shell_test_apk. There may still be references to this kicking |
| 109 around, but wherever you see chromium_shell_test you should replace with | 109 around, but wherever you see chromium_shell_test you should replace with |
| 110 chrome_public_test. | 110 chrome_public_test. |
| 111 | 111 |
| 112 Once you know what to build, just do it like you normally would build anything | 112 Once you know what to build, just do it like you normally would build anything |
| 113 else, e.g.: `ninja -C out/Release chrome_public_test_apk` | 113 else, e.g.: `ninja -C out/Release chrome_public_test_apk` |
| 114 | 114 |
| 115 ## Running Tests | 115 ## Running Tests |
| 116 | 116 |
| 117 All functional tests are run using `build/android/test_runner.py`. | 117 All functional tests are run using `build/android/test_runner.py`, either |
| 118 directly or via a generated wrapper script in `<output directory>/bin/`. |
| 118 Tests are sharded across all attached devices. In order to run tests, call: | 119 Tests are sharded across all attached devices. In order to run tests, call: |
| 119 `build/android/test_runner.py <test_type> [options]` | 120 `build/android/test_runner.py <test_type> [options]` |
| 121 (or `<generated wrapper script> [options]`). |
| 120 For a list of valid test types, see `test_runner.py --help`. For | 122 For a list of valid test types, see `test_runner.py --help`. For |
| 121 help on a specific test type, run `test_runner.py <test_type> --help`. | 123 help on a specific test type, run `test_runner.py <test_type> --help`. |
| 122 | 124 |
| 123 The commands used by the buildbots are printed in the logs. Look at | 125 The commands used by the buildbots are printed in the logs. Look at |
| 124 http://build.chromium.org/ to duplicate the same test command as a particular | 126 http://build.chromium.org/ to duplicate the same test command as a particular |
| 125 builder. | 127 builder. |
| 126 | 128 |
| 127 If you build in an output directory other than "out", you may have to tell | 129 If you build in an output directory other than "out", you may have to tell |
| 128 test\_runner.py where you place it. Say you build your android code in | 130 test\_runner.py where you place it. Say you build your android code in |
| 129 out\_android, then do `export CHROMIUM_OUT_DIR=out_android` before running the | 131 out\_android, then do `export CHROMIUM_OUT_DIR=out_android` before running the |
| 130 command below. You have to do this even if your "out" directory is a symlink | 132 command below. You have to do this even if your "out" directory is a symlink |
| 131 pointing to "out_android". You can also use `--output-directory` to point to the | 133 pointing to "out_android". You can also use `--output-directory` to point to the |
| 132 path of your output directory, for example, | 134 path of your output directory, for example, |
| 133 `--output-directory=out_android/Debug`. | 135 `--output-directory=out_android/Debug`. (The generated wrapper scripts handle |
| 136 this automatically.) |
| 134 | 137 |
| 135 ## INSTALL\_FAILED\_CONTAINER\_ERROR or INSTALL\_FAILED\_INSUFFICIENT\_STORAGE | 138 ## INSTALL\_FAILED\_CONTAINER\_ERROR or INSTALL\_FAILED\_INSUFFICIENT\_STORAGE |
| 136 | 139 |
| 137 If you see this error when test\_runner.py is attempting to deploy the test | 140 If you see this error when test\_runner.py is attempting to deploy the test |
| 138 binaries to the AVD emulator, you may need to resize your userdata partition | 141 binaries to the AVD emulator, you may need to resize your userdata partition |
| 139 with the following commands: | 142 with the following commands: |
| 140 | 143 |
| 141 ```shell | 144 ```shell |
| 142 # Resize userdata partition to be 1G | 145 # Resize userdata partition to be 1G |
| 143 resize2fs android_emulator_sdk/sdk/system-images/android-23/x86/userdata.img 1G | 146 resize2fs android_emulator_sdk/sdk/system-images/android-23/x86/userdata.img 1G |
| (...skipping 54 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
| 198 -f "org.chromium.chrome.browser.media.*" | 201 -f "org.chromium.chrome.browser.media.*" |
| 199 ``` | 202 ``` |
| 200 | 203 |
| 201 ## Gtests | 204 ## Gtests |
| 202 | 205 |
| 203 ```shell | 206 ```shell |
| 204 # Build a test suite | 207 # Build a test suite |
| 205 ninja -C out/Release content_unittests_apk | 208 ninja -C out/Release content_unittests_apk |
| 206 | 209 |
| 207 # Run a test suite | 210 # Run a test suite |
| 208 build/android/test_runner.py gtest -s content_unittests --release -vvv | 211 out/Release/bin/run_content_unittests [-vv] |
| 209 | 212 |
| 210 # Run a subset of tests | 213 # Run a subset of tests |
| 211 build/android/test_runner.py gtest -s content_unittests --release -vvv \ | 214 out/Release/bin/run_content_unittests [-vv] --gtest-filter ByteStreamTest.* |
| 212 --gtest-filter ByteStreamTest.* | |
| 213 ``` | 215 ``` |
| 214 | 216 |
| 215 ## Instrumentation Tests | 217 ## Instrumentation Tests |
| 216 | 218 |
| 217 In order to run instrumentation tests, you must leave your device screen ON and | 219 In order to run instrumentation tests, you must leave your device screen ON and |
| 218 UNLOCKED. Otherwise, the test will timeout trying to launch an intent. | 220 UNLOCKED. Otherwise, the test will timeout trying to launch an intent. |
| 219 Optionally you can disable screen lock under Settings -> Security -> Screen Lock | 221 Optionally you can disable screen lock under Settings -> Security -> Screen Lock |
| 220 -> None. | 222 -> None. |
| 221 | 223 |
| 222 Next, you need to build the app, build your tests, install the application APK, | 224 Next, you need to build the app, build your tests, and then run your tests |
| 223 and then run your tests (which will install the test APK automatically). | 225 (which will install the APK under test and the test APK automatically). |
| 224 | 226 |
| 225 Examples: | 227 Examples: |
| 226 | 228 |
| 227 ContentShell tests: | 229 ContentShell tests: |
| 228 | 230 |
| 229 ```shell | 231 ```shell |
| 230 # Build the code under test | 232 # Build the code under test |
| 231 ninja -C out/Release content_shell_apk | 233 ninja -C out/Release content_shell_apk |
| 232 | 234 |
| 233 # Build the tests themselves | 235 # Build the tests themselves |
| 234 ninja -C out/Release content_shell_test_apk | 236 ninja -C out/Release content_shell_test_apk |
| 235 | 237 |
| 236 # Install the code under test | 238 # Run the test (will automagically install the APK under test and the test APK) |
| 237 build/android/adb_install_apk.py out/Release/apks/ContentShell.apk | 239 out/Release/bin/run_content_shell_test_apk [-vv] |
| 238 | |
| 239 # Run the test (will automagically install the test APK) | |
| 240 build/android/test_runner.py instrumentation --test-apk=ContentShellTest \ | |
| 241 --isolate-file-path content/content_shell_test_apk.isolate --release -vv | |
| 242 ``` | 240 ``` |
| 243 | 241 |
| 244 ChromePublic tests: | 242 ChromePublic tests: |
| 245 | 243 |
| 246 ```shell | 244 ```shell |
| 247 # Build the code under test | 245 # Build the code under test |
| 248 ninja -C out/Release chrome_public_apk | 246 ninja -C out/Release chrome_public_apk |
| 249 | 247 |
| 250 # Build the tests themselves | 248 # Build the tests themselves |
| 251 ninja -C out/Release chrome_public_test_apk | 249 ninja -C out/Release chrome_public_test_apk |
| 252 | 250 |
| 253 # Install the code under test | 251 # Run the test (will automagically install the APK under test and the test APK) |
| 254 build/android/adb_install_apk.py out/Release/apks/ChromePublic.apk | 252 out/Release/bin/run_chrome_public_test_apk [-vv] |
| 255 | |
| 256 # Run the test (will automagically install the test APK) | |
| 257 build/android/test_runner.py instrumentation --test-apk=ChromePublicTest \ | |
| 258 --isolate-file-path chrome/chrome_public_test_apk.isolate --release -vv | |
| 259 ``` | 253 ``` |
| 260 | 254 |
| 261 AndroidWebView tests: | 255 AndroidWebView tests: |
| 262 | 256 |
| 263 ```shell | 257 ```shell |
| 264 ninja -C out/Release android_webview_apk | 258 ninja -C out/Release android_webview_apk |
| 265 ninja -C out/Release android_webview_test_apk | 259 ninja -C out/Release android_webview_test_apk |
| 266 build/android/adb_install_apk.py out/Release/apks/AndroidWebView.apk \ | 260 out/Release/bin/run_android_webview_test_apk [-vv] |
| 267 build/android/test_runner.py instrumentation --test-apk=AndroidWebViewTest \ | |
| 268 --test_data webview:android_webview/test/data/device_files --release -vvv | |
| 269 ``` | 261 ``` |
| 270 | 262 |
| 271 Use adb\_install\_apk.py to install the app under test, then run the test | 263 In order to run a subset of tests, use -f to filter based on test |
| 272 command. In order to run a subset of tests, use -f to filter based on test | |
| 273 class/method or -A/-E to filter using annotations. | 264 class/method or -A/-E to filter using annotations. |
| 274 | 265 |
| 275 Filtering examples: | 266 Filtering examples: |
| 276 | 267 |
| 277 ```shell | 268 ```shell |
| 278 # Run a test suite | 269 # Run a test suite |
| 279 build/android/test_runner.py instrumentation --test-apk=ContentShellTest | 270 out/Debug/bin/run_content_shell_test_apk |
| 280 | 271 |
| 281 # Run a specific test class | 272 # Run a specific test class |
| 282 build/android/test_runner.py instrumentation --test-apk=ContentShellTest -f \ | 273 out/Debug/bin/run_content_shell_test_apk -f AddressDetectionTest.* |
| 283 AddressDetectionTest | |
| 284 | 274 |
| 285 # Run a specific test method | 275 # Run a specific test method |
| 286 build/android/test_runner.py instrumentation --test-apk=ContentShellTest -f \ | 276 out/Debug/bin/run_content_shell_test_apk -f \ |
| 287 AddressDetectionTest#testAddressLimits | 277 AddressDetectionTest#testAddressLimits |
| 288 | 278 |
| 289 # Run a subset of tests by size (Smoke, SmallTest, MediumTest, LargeTest, | 279 # Run a subset of tests by size (Smoke, SmallTest, MediumTest, LargeTest, |
| 290 # EnormousTest) | 280 # EnormousTest) |
| 291 build/android/test_runner.py instrumentation --test-apk=ContentShellTest -A \ | 281 out/Debug/bin/run_content_shell_test_apk -A Smoke |
| 292 Smoke | |
| 293 | 282 |
| 294 # Run a subset of tests by annotation, such as filtering by Feature | 283 # Run a subset of tests by annotation, such as filtering by Feature |
| 295 build/android/test_runner.py instrumentation --test-apk=ContentShellTest -A \ | 284 out/Debug/bin/run_content_shell_test_apk -A Feature=Navigation |
| 296 Feature=Navigation | |
| 297 ``` | 285 ``` |
| 298 | 286 |
| 299 You might want to add stars `*` to each as a regular expression, e.g. | 287 You might want to add stars `*` to each as a regular expression, e.g. |
| 300 `*`AddressDetectionTest`*` | 288 `*`AddressDetectionTest`*` |
| 301 | 289 |
| 302 ## Running Blink Layout Tests | 290 ## Running Blink Layout Tests |
| 303 | 291 |
| 304 See | 292 See |
| 305 https://sites.google.com/a/chromium.org/dev/developers/testing/webkit-layout-tes
ts | 293 https://sites.google.com/a/chromium.org/dev/developers/testing/webkit-layout-tes
ts |
| 306 | 294 |
| 307 ## Running GPU tests | 295 ## Running GPU tests |
| 308 | 296 |
| 309 (e.g. the "Android Debug (Nexus 7)" bot on the chromium.gpu waterfall) | 297 (e.g. the "Android Debug (Nexus 7)" bot on the chromium.gpu waterfall) |
| 310 | 298 |
| 311 See http://www.chromium.org/developers/testing/gpu-testing for details. Use | 299 See http://www.chromium.org/developers/testing/gpu-testing for details. Use |
| 312 --browser=android-content-shell. Examine the stdio from the test invocation on | 300 --browser=android-content-shell. Examine the stdio from the test invocation on |
| 313 the bots to see arguments to pass to src/content/test/gpu/run\_gpu\_test.py. | 301 the bots to see arguments to pass to src/content/test/gpu/run\_gpu\_test.py. |
| OLD | NEW |