Chromium Code Reviews Chromium Code Reviews
Issue 2543883002:
Add documentation to the testing/buildbot directory. (Closed)
| OLD | NEW |
|---|---|
| (Empty) | |
| 1 # Buildbot Testing Configuration Files | |
| 2 | |
| 3 The files in this directory control how tests are run on the | |
| 4 [Chromium buildbots](https://www.chromium.org/developers/testing/chromium-build- infrastructure/tour-of-the-chromium-buildbot). | |
| 5 In addition to specifying what tests run on which builders, they also specify | |
| 6 special arguments and constraints for the tests. | |
| 7 | |
| 8 ## A tour of the directory | |
| 9 * <master_name\>.json -- buildbot configuration json files. These are used to | |
|
ghost stip (do not use)
2016/12/01 02:27:57
don't ask me why < should be unescaped but > can't
| |
| 10 configure what tests are run on what builders, in addition to specifying | |
| 11 builder-specific arguments and parameters. | |
| 12 * [gn_isolate_map.pyl](./gn_isolate_map.pyl) -- maps Ninja build target names | |
| 13 to GN labels. Allows for certain overrides to get certain tests targets to work | |
| 14 with GN (and properly run when isolated). | |
| 15 * [trybot_analyze_config.json](./trybot_analyze_config.json) -- used to provide | |
| 16 exclusions to | |
| 17 [the analyze step](https://www.chromium.org/developers/testing/commit-queue/chro mium_trybot-json) | |
| 18 on trybots. | |
| 19 * [filters/](./filters/) -- filters out tests that shouldn't be | |
| 20 run in a particular mode. | |
| 21 * [timeouts.py](./timeouts.py) -- calculates acceptable timeouts for tests by | |
| 22 analyzing their execution on | |
| 23 [swarming](https://github.com/luci/luci-py/tree/master/appengine/swarming). | |
| 24 * [manage.py](./manage.py) -- makes sure the buildbot configuration json is in | |
| 25 a standardized format. | |
| 26 | |
| 27 ## How the files are consumed | |
| 28 ### Buildbot configuration json | |
| 29 Logic in the | |
| 30 [Chromium recipe](https://chromium.googlesource.com/chromium/tools/build/+/refs/ heads/master/scripts/slave/recipes/chromium.py) | |
| 31 looks up each builder for each master and test generators in | |
| 32 [chromium_tests/steps.py](https://chromium.googlesource.com/chromium/tools/build /+/refs/heads/master/scripts/slave/recipe_modules/chromium_tests/steps.py) | |
| 33 parse the data. For example, as of | |
| 34 [a6e11220](https://chromium.googlesource.com/chromium/tools/build/+/a6e11220d97d 578d6ba091abd68beba28a004722) | |
| 35 [generate_gtest](https://chromium.googlesource.com/chromium/tools/build/+/a6e112 20d97d578d6ba091abd68beba28a004722/scripts/slave/recipe_modules/chromium_tests/s teps.py#416) | |
| 36 parses any entry in a builder's | |
| 37 ['gtest_tests'](https://chromium.googlesource.com/chromium/src/+/5750756522296b2 a9a08009d8d2cc90db3b88f56/testing/buildbot/chromium.android.json#1243) | |
| 38 entry. | |
| 39 | |
| 40 ## How to edit | |
| 41 ### Making the changes | |
| 42 #### Buildbot configuration json | |
| 43 After editing any buildbot json, run `./manage.py -w` to load and write in the | |
| 44 canonical format. Then commit as normal. | |
| 45 | |
| 46 Note that trybots mirror regular waterfall bots, with the mapping defined in | |
| 47 [trybots.py](https://chromium.googlesource.com/chromium/tools/build/+/refs/heads /master/scripts/slave/recipe_modules/chromium_tests/trybots.py). | |
| 48 This means that, as of | |
| 49 [81fcc4bc](https://chromium.googlesource.com/chromium/src/+/81fcc4bc6123ace8dd37 db74fd2592e3e15ea46a/testing/buildbot/), | |
| 50 if you want to edit | |
| 51 [linux_android_rel_ng](https://chromium.googlesource.com/chromium/tools/build/+/ 59a2653d5f143213f4f166714657808b0c646bd7/scripts/slave/recipe_modules/chromium_t ests/trybots.py#142), | |
| 52 you actually need to edit | |
| 53 [Android Tests](https://chromium.googlesource.com/chromium/src/+/81fcc4bc6123ace 8dd37db74fd2592e3e15ea46a/testing/buildbot/chromium.linux.json#23). | |
| 54 | |
| 55 ### Trying the changes on trybots | |
| 56 You should be able to try build changes that affect the trybots directly (for | |
| 57 example, adding a test to linux_android_rel_ng should show up immediately in | |
| 58 your tryjob). Non-trybot changes have to be landed manually :(. | |
| 59 | |
| 60 ## Capacity considerations when editing the buildbot configuration json | |
| 61 When adding tests or bumping timeouts in the buildbot configuration json, care | |
| 62 must be taken to ensure the infrastructure has capacity to handle the extra | |
| 63 load. This is especially true for the established | |
| 64 [Chromium CQ builders](https://chromium.googlesource.com/chromium/src/+/master/i nfra/config/cq.cfg), | |
| 65 as they operate under strict execution requirements. Make sure to get an | |
| 66 infrastructure engineer on the Crossover Team to sign off that there is both | |
| 67 buildbot and swarming capacity available. | |
| OLD | NEW |
