Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(498)

Side by Side Diff: testing/buildbot/README.md

Issue 2543883002: Add documentation to the testing/buildbot directory. (Closed)
Patch Set: Created 4 years ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
« no previous file with comments | « no previous file | no next file » | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
(Empty)
1 # Buildbot Testing Configuration Files
2
3 The files in this directory control how tests are run on the
4 [Chromium buildbots](https://www.chromium.org/developers/testing/chromium-build- infrastructure/tour-of-the-chromium-buildbot).
5 In addition to specifying what tests run on which builders, they also specify
6 special arguments and constraints for the tests.
7
8 ## A tour of the directory
9 * <master_name>.json -- buildbot configuration json files. These are used to
10 configure what tests are run on what builders, in addition to specifying
11 builder-specific arguments and parameters.
12 * [gn_isolate_map.pyl](../gn_isolate_map.pyl) -- maps Ninja build target names
13 to GN labels. Allows for certain overrides to get certain tests targets to work
14 with GN (and properly run when isolated).
15 * [trybot_analyze_config.json](../trybot_analyze_config.json) -- used to provide
16 exclusions to
17 [the analyze step](https://www.chromium.org/developers/testing/commit-queue/chro mium_trybot-json)
18 on trybots.
19 * [filters/](../filters/) -- filters out tests that shouldn't be
20 run in a particular mode.
21 * [timeouts.py](../timeouts.py) -- calculates acceptable timeouts for tests by
22 analyzing their execution on
23 [swarming](https://github.com/luci/luci-py/tree/master/appengine/swarming).
24 * [manage.py](../manage.py) -- makes sure the buildbot configuration json is in
25 a standardized format.
26
27 ## How the files are consumed
28 ### Buildbot configuration json
29 Logic in the
30 [Chromium recipe](https://chromium.googlesource.com/chromium/tools/build/+/refs/ heads/master/scripts/slave/recipes/chromium.py)
31 looks up each builder for each master and test generators in
32 [chromium_tests/steps.py](https://chromium.googlesource.com/chromium/tools/build /+/refs/heads/master/scripts/slave/recipe_modules/chromium_tests/steps.py)
33 parse the data. For example, as of
34 [a6e11220](https://chromium.googlesource.com/chromium/tools/build/+/a6e11220d97d 578d6ba091abd68beba28a004722)
agable 2016/12/01 01:31:54 ...4722),
ghost stip (do not use) 2016/12/01 02:26:50 ish, but sure
35 [generate_gtest](https://chromium.googlesource.com/chromium/tools/build/+/a6e112 20d97d578d6ba091abd68beba28a004722/scripts/slave/recipe_modules/chromium_tests/s teps.py#416)
36 parses any entry in a builder's
37 ['gtest_tests'](https://chromium.googlesource.com/chromium/src/+/5750756522296b2 a9a08009d8d2cc90db3b88f56/testing/buildbot/chromium.android.json#1243)
38 entry.
39
40 ## How to edit ### Making the changes #### Buildbot configuration json After
agable 2016/12/01 01:31:54 Looks like these got accidentally hoisted onto the
ghost stip (do not use) 2016/12/01 02:26:50 ugh, vim's "gq" isn't markdown aware and I thought
41 editing any buildbot json, run ``./manage.py -w`` to load and write in the
agable 2016/12/01 01:31:53 I think you only need single backticks
42 canonical format. Then commit as normal.
43
44 Note that trybots mirror regular waterfall bots, with the mapping defined in
45 [trybots.py](https://chromium.googlesource.com/chromium/tools/build/+/refs/heads /master/scripts/slave/recipe_modules/chromium_tests/trybots.py).
46 This means that, as of
47 [81fcc4bc](https://chromium.googlesource.com/chromium/src/+/81fcc4bc6123ace8dd37 db74fd2592e3e15ea46a/testing/buildbot/),
48 if you want to edit
49 [linux_android_rel_ng](https://chromium.googlesource.com/chromium/tools/build/+/ 59a2653d5f143213f4f166714657808b0c646bd7/scripts/slave/recipe_modules/chromium_t ests/trybots.py#142),
50 you actually need to edit
51 [Android Tests](https://chromium.googlesource.com/chromium/src/+/81fcc4bc6123ace 8dd37db74fd2592e3e15ea46a/testing/buildbot/chromium.linux.json#23).
52
53 ### Trying the changes on trybots You should be able to try build changes that
agable 2016/12/01 01:31:54 Some hoisting here too
54 affect the trybots directly (for example, adding a test to linux_android_rel_ng
55 should show up immediately in your tryjob). Non-trybot changes have to be
56 landed manually :(.
57
58 ## Capacity considerations when editing the buildbot configuration json When
agable 2016/12/01 01:31:54 And here
59 adding tests or bumping timeouts in the buildbot configuration json, care must
60 be taken to ensure the infrastructure has capacity to handle the extra load.
61 This is especially true for the established
62 [Chromium CQ builders](https://chromium.googlesource.com/chromium/src/+/master/i nfra/config/cq.cfg),
63 as they operate under strict execution requirements. Make sure to get an
64 infrastructure engineer on the Crossover Team to sign off that there is both
65 buildbot and swarming capacity available.
OLDNEW
« no previous file with comments | « no previous file | no next file » | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698