OLD | NEW |
1 # Android Test Instructions | 1 # Android Test Instructions |
2 | 2 |
3 Device Setup Tests are runnable on physical devices or emulators. See the | 3 Device Setup Tests are runnable on physical devices or emulators. See the |
4 instructions below for setting up either a physical device or an emulator. | 4 instructions below for setting up either a physical device or an emulator. |
5 | 5 |
6 [TOC] | 6 [TOC] |
7 | 7 |
8 ## Physical Device Setup **ADB Debugging** | 8 ## Physical Device Setup **ADB Debugging** |
9 | 9 |
10 In order to allow the ADB to connect to the device, you must enable USB | 10 In order to allow the ADB to connect to the device, you must enable USB |
(...skipping 96 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
107 NB: We used to call the chrome_public_test_apk the | 107 NB: We used to call the chrome_public_test_apk the |
108 chromium_shell_test_apk. There may still be references to this kicking | 108 chromium_shell_test_apk. There may still be references to this kicking |
109 around, but wherever you see chromium_shell_test you should replace with | 109 around, but wherever you see chromium_shell_test you should replace with |
110 chrome_public_test. | 110 chrome_public_test. |
111 | 111 |
112 Once you know what to build, just do it like you normally would build anything | 112 Once you know what to build, just do it like you normally would build anything |
113 else, e.g.: `ninja -C out/Release chrome_public_test_apk` | 113 else, e.g.: `ninja -C out/Release chrome_public_test_apk` |
114 | 114 |
115 ## Running Tests | 115 ## Running Tests |
116 | 116 |
117 All functional tests are run using `build/android/test_runner.py`. | 117 All functional tests are run using `build/android/test_runner.py`, either |
| 118 directly or via a generated wrapper script in `<output directory>/bin/`. |
118 Tests are sharded across all attached devices. In order to run tests, call: | 119 Tests are sharded across all attached devices. In order to run tests, call: |
119 `build/android/test_runner.py <test_type> [options]` | 120 `build/android/test_runner.py <test_type> [options]` |
| 121 (or `<generated wrapper script> [options]`). |
120 For a list of valid test types, see `test_runner.py --help`. For | 122 For a list of valid test types, see `test_runner.py --help`. For |
121 help on a specific test type, run `test_runner.py <test_type> --help`. | 123 help on a specific test type, run `test_runner.py <test_type> --help`. |
122 | 124 |
123 The commands used by the buildbots are printed in the logs. Look at | 125 The commands used by the buildbots are printed in the logs. Look at |
124 http://build.chromium.org/ to duplicate the same test command as a particular | 126 http://build.chromium.org/ to duplicate the same test command as a particular |
125 builder. | 127 builder. |
126 | 128 |
127 If you build in an output directory other than "out", you may have to tell | 129 If you build in an output directory other than "out", you may have to tell |
128 test\_runner.py where you place it. Say you build your android code in | 130 test\_runner.py where you place it. Say you build your android code in |
129 out\_android, then do `export CHROMIUM_OUT_DIR=out_android` before running the | 131 out\_android, then do `export CHROMIUM_OUT_DIR=out_android` before running the |
130 command below. You have to do this even if your "out" directory is a symlink | 132 command below. You have to do this even if your "out" directory is a symlink |
131 pointing to "out_android". You can also use `--output-directory` to point to the | 133 pointing to "out_android". You can also use `--output-directory` to point to the |
132 path of your output directory, for example, | 134 path of your output directory, for example, |
133 `--output-directory=out_android/Debug`. | 135 `--output-directory=out_android/Debug`. (The generated wrapper scripts handle |
| 136 this automatically.) |
134 | 137 |
135 ## INSTALL\_FAILED\_CONTAINER\_ERROR or INSTALL\_FAILED\_INSUFFICIENT\_STORAGE | 138 ## INSTALL\_FAILED\_CONTAINER\_ERROR or INSTALL\_FAILED\_INSUFFICIENT\_STORAGE |
136 | 139 |
137 If you see this error when test\_runner.py is attempting to deploy the test | 140 If you see this error when test\_runner.py is attempting to deploy the test |
138 binaries to the AVD emulator, you may need to resize your userdata partition | 141 binaries to the AVD emulator, you may need to resize your userdata partition |
139 with the following commands: | 142 with the following commands: |
140 | 143 |
141 ```shell | 144 ```shell |
142 # Resize userdata partition to be 1G | 145 # Resize userdata partition to be 1G |
143 resize2fs android_emulator_sdk/sdk/system-images/android-23/x86/userdata.img 1G | 146 resize2fs android_emulator_sdk/sdk/system-images/android-23/x86/userdata.img 1G |
(...skipping 54 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
198 -f "org.chromium.chrome.browser.media.*" | 201 -f "org.chromium.chrome.browser.media.*" |
199 ``` | 202 ``` |
200 | 203 |
201 ## Gtests | 204 ## Gtests |
202 | 205 |
203 ```shell | 206 ```shell |
204 # Build a test suite | 207 # Build a test suite |
205 ninja -C out/Release content_unittests_apk | 208 ninja -C out/Release content_unittests_apk |
206 | 209 |
207 # Run a test suite | 210 # Run a test suite |
208 build/android/test_runner.py gtest -s content_unittests --release -vvv | 211 out/Release/bin/run_content_unittests [-vv] |
209 | 212 |
210 # Run a subset of tests | 213 # Run a subset of tests |
211 build/android/test_runner.py gtest -s content_unittests --release -vvv \ | 214 out/Release/bin/run_content_unittests [-vv] --gtest-filter ByteStreamTest.* |
212 --gtest-filter ByteStreamTest.* | |
213 ``` | 215 ``` |
214 | 216 |
215 ## Instrumentation Tests | 217 ## Instrumentation Tests |
216 | 218 |
217 In order to run instrumentation tests, you must leave your device screen ON and | 219 In order to run instrumentation tests, you must leave your device screen ON and |
218 UNLOCKED. Otherwise, the test will timeout trying to launch an intent. | 220 UNLOCKED. Otherwise, the test will timeout trying to launch an intent. |
219 Optionally you can disable screen lock under Settings -> Security -> Screen Lock | 221 Optionally you can disable screen lock under Settings -> Security -> Screen Lock |
220 -> None. | 222 -> None. |
221 | 223 |
222 Next, you need to build the app, build your tests, install the application APK, | 224 Next, you need to build the app, build your tests, and then run your tests |
223 and then run your tests (which will install the test APK automatically). | 225 (which will install the APK under test and the test APK automatically). |
224 | 226 |
225 Examples: | 227 Examples: |
226 | 228 |
227 ContentShell tests: | 229 ContentShell tests: |
228 | 230 |
229 ```shell | 231 ```shell |
230 # Build the code under test | 232 # Build the code under test |
231 ninja -C out/Release content_shell_apk | 233 ninja -C out/Release content_shell_apk |
232 | 234 |
233 # Build the tests themselves | 235 # Build the tests themselves |
234 ninja -C out/Release content_shell_test_apk | 236 ninja -C out/Release content_shell_test_apk |
235 | 237 |
236 # Install the code under test | 238 # Run the test (will automagically install the APK under test and the test APK) |
237 build/android/adb_install_apk.py out/Release/apks/ContentShell.apk | 239 out/Release/bin/run_content_shell_test_apk [-vv] |
238 | |
239 # Run the test (will automagically install the test APK) | |
240 build/android/test_runner.py instrumentation --test-apk=ContentShellTest \ | |
241 --isolate-file-path content/content_shell_test_apk.isolate --release -vv | |
242 ``` | 240 ``` |
243 | 241 |
244 ChromePublic tests: | 242 ChromePublic tests: |
245 | 243 |
246 ```shell | 244 ```shell |
247 # Build the code under test | 245 # Build the code under test |
248 ninja -C out/Release chrome_public_apk | 246 ninja -C out/Release chrome_public_apk |
249 | 247 |
250 # Build the tests themselves | 248 # Build the tests themselves |
251 ninja -C out/Release chrome_public_test_apk | 249 ninja -C out/Release chrome_public_test_apk |
252 | 250 |
253 # Install the code under test | 251 # Run the test (will automagically install the APK under test and the test APK) |
254 build/android/adb_install_apk.py out/Release/apks/ChromePublic.apk | 252 out/Release/bin/run_chrome_public_test_apk [-vv] |
255 | |
256 # Run the test (will automagically install the test APK) | |
257 build/android/test_runner.py instrumentation --test-apk=ChromePublicTest \ | |
258 --isolate-file-path chrome/chrome_public_test_apk.isolate --release -vv | |
259 ``` | 253 ``` |
260 | 254 |
261 AndroidWebView tests: | 255 AndroidWebView tests: |
262 | 256 |
263 ```shell | 257 ```shell |
264 ninja -C out/Release android_webview_apk | 258 ninja -C out/Release android_webview_apk |
265 ninja -C out/Release android_webview_test_apk | 259 ninja -C out/Release android_webview_test_apk |
266 build/android/adb_install_apk.py out/Release/apks/AndroidWebView.apk \ | 260 out/Release/bin/run_android_webview_test_apk [-vv] |
267 build/android/test_runner.py instrumentation --test-apk=AndroidWebViewTest \ | |
268 --test_data webview:android_webview/test/data/device_files --release -vvv | |
269 ``` | 261 ``` |
270 | 262 |
271 Use adb\_install\_apk.py to install the app under test, then run the test | 263 In order to run a subset of tests, use -f to filter based on test |
272 command. In order to run a subset of tests, use -f to filter based on test | |
273 class/method or -A/-E to filter using annotations. | 264 class/method or -A/-E to filter using annotations. |
274 | 265 |
275 Filtering examples: | 266 Filtering examples: |
276 | 267 |
277 ```shell | 268 ```shell |
278 # Run a test suite | 269 # Run a test suite |
279 build/android/test_runner.py instrumentation --test-apk=ContentShellTest | 270 out/Debug/bin/run_content_shell_test_apk |
280 | 271 |
281 # Run a specific test class | 272 # Run a specific test class |
282 build/android/test_runner.py instrumentation --test-apk=ContentShellTest -f \ | 273 out/Debug/bin/run_content_shell_test_apk -f AddressDetectionTest.* |
283 AddressDetectionTest | |
284 | 274 |
285 # Run a specific test method | 275 # Run a specific test method |
286 build/android/test_runner.py instrumentation --test-apk=ContentShellTest -f \ | 276 out/Debug/bin/run_content_shell_test_apk -f \ |
287 AddressDetectionTest#testAddressLimits | 277 AddressDetectionTest#testAddressLimits |
288 | 278 |
289 # Run a subset of tests by size (Smoke, SmallTest, MediumTest, LargeTest, | 279 # Run a subset of tests by size (Smoke, SmallTest, MediumTest, LargeTest, |
290 # EnormousTest) | 280 # EnormousTest) |
291 build/android/test_runner.py instrumentation --test-apk=ContentShellTest -A \ | 281 out/Debug/bin/run_content_shell_test_apk -A Smoke |
292 Smoke | |
293 | 282 |
294 # Run a subset of tests by annotation, such as filtering by Feature | 283 # Run a subset of tests by annotation, such as filtering by Feature |
295 build/android/test_runner.py instrumentation --test-apk=ContentShellTest -A \ | 284 out/Debug/bin/run_content_shell_test_apk -A Feature=Navigation |
296 Feature=Navigation | |
297 ``` | 285 ``` |
298 | 286 |
299 You might want to add stars `*` to each as a regular expression, e.g. | 287 You might want to add stars `*` to each as a regular expression, e.g. |
300 `*`AddressDetectionTest`*` | 288 `*`AddressDetectionTest`*` |
301 | 289 |
302 ## Running Blink Layout Tests | 290 ## Running Blink Layout Tests |
303 | 291 |
304 See | 292 See |
305 https://sites.google.com/a/chromium.org/dev/developers/testing/webkit-layout-tes
ts | 293 https://sites.google.com/a/chromium.org/dev/developers/testing/webkit-layout-tes
ts |
306 | 294 |
307 ## Running GPU tests | 295 ## Running GPU tests |
308 | 296 |
309 (e.g. the "Android Debug (Nexus 7)" bot on the chromium.gpu waterfall) | 297 (e.g. the "Android Debug (Nexus 7)" bot on the chromium.gpu waterfall) |
310 | 298 |
311 See http://www.chromium.org/developers/testing/gpu-testing for details. Use | 299 See http://www.chromium.org/developers/testing/gpu-testing for details. Use |
312 --browser=android-content-shell. Examine the stdio from the test invocation on | 300 --browser=android-content-shell. Examine the stdio from the test invocation on |
313 the bots to see arguments to pass to src/content/test/gpu/run\_gpu\_test.py. | 301 the bots to see arguments to pass to src/content/test/gpu/run\_gpu\_test.py. |
OLD | NEW |