Chromium Code Reviews| OLD | NEW |
|---|---|
| (Empty) | |
| 1 # Buildbot Testing Configuration Files | |
| 2 | |
| 3 The files in this directory control how tests are run on the | |
| 4 [Chromium buildbots](https://www.chromium.org/developers/testing/chromium-build- infrastructure/tour-of-the-chromium-buildbot). | |
| 5 In addition to specifying what tests run on which builders, they also specify | |
| 6 special arguments and constraints for the tests. | |
| 7 | |
| 8 ## A tour of the directory | |
| 9 * <master_name>.json -- buildbot configuration json files. These are used to | |
| 10 configure what tests are run on what builders, in addition to specifying | |
| 11 builder-specific arguments and parameters. | |
| 12 * [gn_isolate_map.pyl](../gn_isolate_map.pyl) -- maps Ninja build target names | |
| 13 to GN labels. Allows for certain overrides to get certain tests targets to work | |
| 14 with GN (and properly run when isolated). | |
| 15 * [trybot_analyze_config.json](../trybot_analyze_config.json) -- used to provide | |
| 16 exclusions to | |
| 17 [the analyze step](https://www.chromium.org/developers/testing/commit-queue/chro mium_trybot-json) | |
| 18 on trybots. | |
| 19 * [filters/](../filters/) -- filters out tests that shouldn't be | |
| 20 run in a particular mode. | |
| 21 * [timeouts.py](../timeouts.py) -- calculates acceptable timeouts for tests by | |
| 22 analyzing their execution on | |
| 23 [swarming](https://github.com/luci/luci-py/tree/master/appengine/swarming). | |
| 24 * [manage.py](../manage.py) -- makes sure the buildbot configuration json is in | |
| 25 a standardized format. | |
| 26 | |
| 27 ## How the files are consumed | |
| 28 ### Buildbot configuration json | |
| 29 Logic in the | |
| 30 [Chromium recipe](https://chromium.googlesource.com/chromium/tools/build/+/refs/ heads/master/scripts/slave/recipes/chromium.py) | |
| 31 looks up each builder for each master and test generators in | |
| 32 [chromium_tests/steps.py](https://chromium.googlesource.com/chromium/tools/build /+/refs/heads/master/scripts/slave/recipe_modules/chromium_tests/steps.py) | |
| 33 parse the data. For example, as of | |
| 34 [a6e11220](https://chromium.googlesource.com/chromium/tools/build/+/a6e11220d97d 578d6ba091abd68beba28a004722) | |
|
agable
2016/12/01 01:31:54
...4722),
ghost stip (do not use)
2016/12/01 02:26:50
ish, but sure
| |
| 35 [generate_gtest](https://chromium.googlesource.com/chromium/tools/build/+/a6e112 20d97d578d6ba091abd68beba28a004722/scripts/slave/recipe_modules/chromium_tests/s teps.py#416) | |
| 36 parses any entry in a builder's | |
| 37 ['gtest_tests'](https://chromium.googlesource.com/chromium/src/+/5750756522296b2 a9a08009d8d2cc90db3b88f56/testing/buildbot/chromium.android.json#1243) | |
| 38 entry. | |
| 39 | |
| 40 ## How to edit ### Making the changes #### Buildbot configuration json After | |
|
agable
2016/12/01 01:31:54
Looks like these got accidentally hoisted onto the
ghost stip (do not use)
2016/12/01 02:26:50
ugh, vim's "gq" isn't markdown aware and I thought
| |
| 41 editing any buildbot json, run ``./manage.py -w`` to load and write in the | |
|
agable
2016/12/01 01:31:53
I think you only need single backticks
| |
| 42 canonical format. Then commit as normal. | |
| 43 | |
| 44 Note that trybots mirror regular waterfall bots, with the mapping defined in | |
| 45 [trybots.py](https://chromium.googlesource.com/chromium/tools/build/+/refs/heads /master/scripts/slave/recipe_modules/chromium_tests/trybots.py). | |
| 46 This means that, as of | |
| 47 [81fcc4bc](https://chromium.googlesource.com/chromium/src/+/81fcc4bc6123ace8dd37 db74fd2592e3e15ea46a/testing/buildbot/), | |
| 48 if you want to edit | |
| 49 [linux_android_rel_ng](https://chromium.googlesource.com/chromium/tools/build/+/ 59a2653d5f143213f4f166714657808b0c646bd7/scripts/slave/recipe_modules/chromium_t ests/trybots.py#142), | |
| 50 you actually need to edit | |
| 51 [Android Tests](https://chromium.googlesource.com/chromium/src/+/81fcc4bc6123ace 8dd37db74fd2592e3e15ea46a/testing/buildbot/chromium.linux.json#23). | |
| 52 | |
| 53 ### Trying the changes on trybots You should be able to try build changes that | |
|
agable
2016/12/01 01:31:54
Some hoisting here too
| |
| 54 affect the trybots directly (for example, adding a test to linux_android_rel_ng | |
| 55 should show up immediately in your tryjob). Non-trybot changes have to be | |
| 56 landed manually :(. | |
| 57 | |
| 58 ## Capacity considerations when editing the buildbot configuration json When | |
|
agable
2016/12/01 01:31:54
And here
| |
| 59 adding tests or bumping timeouts in the buildbot configuration json, care must | |
| 60 be taken to ensure the infrastructure has capacity to handle the extra load. | |
| 61 This is especially true for the established | |
| 62 [Chromium CQ builders](https://chromium.googlesource.com/chromium/src/+/master/i nfra/config/cq.cfg), | |
| 63 as they operate under strict execution requirements. Make sure to get an | |
| 64 infrastructure engineer on the Crossover Team to sign off that there is both | |
| 65 buildbot and swarming capacity available. | |
| OLD | NEW |