OLD | NEW |
1 # The MB (Meta-Build wrapper) user guide | 1 # The MB (Meta-Build wrapper) user guide |
2 | 2 |
3 [TOC] | 3 [TOC] |
4 | 4 |
5 ## Introduction | 5 ## Introduction |
6 | 6 |
7 `mb` is a simple python wrapper around the GYP and GN meta-build tools to | 7 `mb` is a simple python wrapper around the GYP and GN meta-build tools to |
8 be used as part of the GYP->GN migration. | 8 be used as part of the GYP->GN migration. |
9 | 9 |
10 It is intended to be used by bots to make it easier to manage the configuration | 10 It is intended to be used by bots to make it easier to manage the configuration |
11 each bot builds (i.e., the configurations can be changed from chromium | 11 each bot builds (i.e., the configurations can be changed from chromium |
12 commits), and to consolidate the list of all of the various configurations | 12 commits), and to consolidate the list of all of the various configurations |
13 that Chromium is built in. | 13 that Chromium is built in. |
14 | 14 |
15 Ideally this tool will no longer be needed after the migration is complete. | 15 Ideally this tool will no longer be needed after the migration is complete. |
16 | 16 |
17 ## `mb gen` | 17 For more discussion of MB, see also [the design spec](design_spec.md). |
18 | 18 |
19 `mb gen` is responsible for generating the Ninja files by invoking either GYP | 19 ## MB subcommands |
20 or GN as appropriate. It takes arguments to specify a build config and | |
21 a directory, then runs GYP or GN as appropriate: | |
22 | 20 |
23 ``` | 21 ### `mb analyze` |
24 % mb gen -m tryserver.chromium.linux -b linux_rel //out/Release | |
25 % mb gen -c linux_rel_trybot //out/Release | |
26 ``` | |
27 | |
28 Either the `-c/--config` flag or the `-m/--master` and `-b/--builder` flags | |
29 must be specified so that `mb` can figure out which config to use. | |
30 | |
31 The path must be a GN-style "source-absolute" path (as above). | |
32 | |
33 If gen ends up using GYP, the path must have a valid GYP configuration as the | |
34 last component of the path (i.e., specify `//out/Release_x64`, not `//out`). | |
35 | |
36 ## `mb analyze` | |
37 | 22 |
38 `mb analyze` is reponsible for determining what targets are affected by | 23 `mb analyze` is reponsible for determining what targets are affected by |
39 a list of files (e.g., the list of files in a patch on a trybot): | 24 a list of files (e.g., the list of files in a patch on a trybot): |
40 | 25 |
41 ``` | 26 ``` |
42 mb analyze -c chromium_linux_rel //out/Release input.json output.json | 27 mb analyze -c chromium_linux_rel //out/Release input.json output.json |
43 ``` | 28 ``` |
44 | 29 |
45 Either the `-c/--config` flag or the `-m/--master` and `-b/--builder` flags | 30 Either the `-c/--config` flag or the `-m/--master` and `-b/--builder` flags |
46 must be specified so that `mb` can figure out which config to use. | 31 must be specified so that `mb` can figure out which config to use. |
47 | 32 |
48 | |
49 The first positional argument must be a GN-style "source-absolute" path | 33 The first positional argument must be a GN-style "source-absolute" path |
50 to the build directory. | 34 to the build directory. |
51 | 35 |
52 The second positional argument is a (normal) path to a JSON file containing | 36 The second positional argument is a (normal) path to a JSON file containing |
53 a single object with two fields: | 37 a single object with two fields: |
54 | 38 |
55 * `files`: an array of the modified filenames to check (as | 39 * `files`: an array of the modified filenames to check (as |
56 paths relative to the checkout root). | 40 paths relative to the checkout root). |
57 * `targets`: an array of the unqualified target names to check. | 41 * `targets`: an array of the unqualified target names to check. |
58 | 42 |
59 The third positional argument is a (normal) path to where mb will write | 43 The third positional argument is a (normal) path to where mb will write |
60 the result, also as a JSON object. This object may contain the following | 44 the result, also as a JSON object. This object may contain the following |
61 fields: | 45 fields: |
62 | 46 |
63 * `error`: this should only be present if something failed. | 47 * `error`: this should only be present if something failed. |
64 * `targets`: the subset of the input `targets` that depend on the | 48 * `targets`: the subset of the input `targets` that depend on the |
65 input `files`. | 49 input `files`. |
66 * `build_targets`: the minimal subset of targets needed to build all | 50 * `build_targets`: the minimal subset of targets needed to build all |
67 of `targets` that were affected. | 51 of `targets` that were affected. |
68 * `status`: one of three strings: | 52 * `status`: one of three strings: |
69 * `"Found dependency"` (build the `build_targets`) | 53 * `"Found dependency"` (build the `build_targets`) |
70 * `"No dependency"` (i.e., no build needed) | 54 * `"No dependency"` (i.e., no build needed) |
71 * `"Found dependency (all)"` (build everything, in which case | 55 * `"Found dependency (all)"` (build everything, in which case |
72 `targets` and `build_targets` are not returned). | 56 `targets` and `build_targets` are not returned). |
73 | 57 |
74 ## `mb help` | 58 The `-b/--builder`, `-c/--config`, `-f/--config-file`, `-m/--master`, |
| 59 `-q/--quiet`, and `-v/--verbose` flags work as documented for `mb gen`. |
| 60 |
| 61 |
| 62 ### `mb gen` |
| 63 |
| 64 `mb gen` is responsible for generating the Ninja files by invoking either GYP |
| 65 or GN as appropriate. It takes arguments to specify a build config and |
| 66 a directory, then runs GYP or GN as appropriate: |
| 67 |
| 68 ``` |
| 69 % mb gen -m tryserver.chromium.linux -b linux_rel //out/Release |
| 70 % mb gen -c linux_rel_trybot //out/Release |
| 71 ``` |
| 72 |
| 73 Either the `-c/--config` flag or the `-m/--master` and `-b/--builder` flags |
| 74 must be specified so that `mb` can figure out which config to use. |
| 75 |
| 76 By default, MB will look in `//tools/mb/mb_config.pyl` to look up the config |
| 77 information, but you can specify a custom config file using the |
| 78 `-f/--config-file` flag. |
| 79 |
| 80 The path must be a GN-style "source-absolute" path (as above). |
| 81 |
| 82 You can pass the `-n/--dryrun` flag to mb gen to see what will happen without |
| 83 actually writing anything. |
| 84 |
| 85 You can pass the `-q/--quiet` flag to get mb to be silent unless there is an |
| 86 error, and pass the `-v/--verbose` flag to get mb to log all of the files |
| 87 that are read and written, and all the commands that are run. |
| 88 |
| 89 If the build config will use the Goma distributed-build system, you can pass |
| 90 the path to your Goma client in the `-g/--goma-dir` flag, and it will be |
| 91 incorporated into the appropriate flags for GYP or GN as needed. |
| 92 |
| 93 If gen ends up using GYP, the path must have a valid GYP configuration as the |
| 94 last component of the path (i.e., specify `//out/Release_x64`, not `//out`). |
| 95 |
| 96 ### `mb help` |
75 | 97 |
76 Produces help output on the other subcommands | 98 Produces help output on the other subcommands |
77 | 99 |
78 ## `mb lookup` | 100 ### `mb lookup` |
79 | 101 |
80 Prints what command will be run by `mb gen` (like `mb gen -n` but does | 102 Prints what command will be run by `mb gen` (like `mb gen -n` but does |
81 not require you to specify a path). | 103 not require you to specify a path). |
82 | 104 |
83 ## `mb validate` | 105 The `-b/--builder`, `-c/--config`, `-f/--config-file`, `-m/--master`, |
| 106 `-q/--quiet`, and `-v/--verbose` flags work as documented for `mb gen`. |
| 107 |
| 108 ### `mb validate` |
84 | 109 |
85 Does internal checking to make sure the config file is syntactically | 110 Does internal checking to make sure the config file is syntactically |
86 valid and that all of the entries are used properly. It does not validate | 111 valid and that all of the entries are used properly. It does not validate |
87 that the flags make sense, or that the builder names are legal or | 112 that the flags make sense, or that the builder names are legal or |
88 comprehensive, but it does complain about configs and mixins that aren't | 113 comprehensive, but it does complain about configs and mixins that aren't |
89 used. | 114 used. |
90 | 115 |
| 116 The `-f/--config-file` and `-q/--quiet` flags work as documented for |
| 117 `mb gen`. |
| 118 |
91 This is mostly useful as a presubmit check and for verifying changes to | 119 This is mostly useful as a presubmit check and for verifying changes to |
92 the config file. | 120 the config file. |
93 | 121 |
94 ## mb_config.pyl | 122 ## Isolates and Swarming |
| 123 |
| 124 `mb gen` is also responsible for generating the `.isolate` and |
| 125 `.isolated.gen.json` files needed to run test executables through swarming |
| 126 in a GN build (in a GYP build, this is done as part of the compile step). |
| 127 |
| 128 If you wish to generate the isolate files, pass `mb gen` the |
| 129 `--swarming-targets-file` command line argument; that arg should be a path |
| 130 to a file containing a list of ninja build targets to compute the runtime |
| 131 dependencies for (on Windows, use the ninja target name, not the file, so |
| 132 `base_unittests`, not `base_unittests.exe`). |
| 133 |
| 134 MB will take this file, translate each build target to the matching GN |
| 135 label (e.g., `base_unittests` -> `//base:base_unittests`, write that list |
| 136 to a file called `runtime_deps` in the build directory, and pass that to |
| 137 `gn gen $BUILD ... --runtime-deps-list-file=$BUILD/runtime_deps`. |
| 138 |
| 139 Once GN has computed the lists of runtime dependencies, MB will then |
| 140 look up the command line for each target (currently this is hard-coded |
| 141 in [mb.py](https://code.google.com/p/chromium/codesearch?q=mb.py#chromium/src/to
ols/mb/mb.py&q=mb.py%20GetIsolateCommand&sq=package:chromium&type=cs)), and writ
e out the |
| 142 matching `.isolate` and `.isolated.gen.json` files. |
| 143 |
| 144 ## The `mb_config.pyl` config file |
95 | 145 |
96 The `mb_config.pyl` config file is intended to enumerate all of the | 146 The `mb_config.pyl` config file is intended to enumerate all of the |
97 supported build configurations for Chromium. Generally speaking, you | 147 supported build configurations for Chromium. Generally speaking, you |
98 should never need to (or want to) build a configuration that isn't | 148 should never need to (or want to) build a configuration that isn't |
99 listed here, and so by using the configs in this file you can avoid | 149 listed here, and so by using the configs in this file you can avoid |
100 having to juggle long lists of GYP_DEFINES and gn args by hand. | 150 having to juggle long lists of GYP_DEFINES and gn args by hand. |
101 | 151 |
102 `mb_config.pyl` is structured as a file containing a single PYthon Literal | 152 `mb_config.pyl` is structured as a file containing a single PYthon Literal |
103 expression: a dictionary with three main keys, `masters`, `configs` and | 153 expression: a dictionary with three main keys, `masters`, `configs` and |
104 `mixins`. | 154 `mixins`. |
(...skipping 66 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
171 } | 221 } |
172 } | 222 } |
173 | 223 |
174 and you ran `mb gen -c linux_release_trybot //out/Release`, it would | 224 and you ran `mb gen -c linux_release_trybot //out/Release`, it would |
175 translate into a call to `gyp_chromium -G Release` with `GYP_DEFINES` set to | 225 translate into a call to `gyp_chromium -G Release` with `GYP_DEFINES` set to |
176 `"use_goma=true dcheck_always_on=false dcheck_always_on=true"`. | 226 `"use_goma=true dcheck_always_on=false dcheck_always_on=true"`. |
177 | 227 |
178 (From that you can see that mb is intentionally dumb and does not | 228 (From that you can see that mb is intentionally dumb and does not |
179 attempt to de-dup the flags, it lets gyp do that). | 229 attempt to de-dup the flags, it lets gyp do that). |
180 | 230 |
181 ## Debugging (-v/--verbose and -n/--dry-run) | 231 ## Debugging MB |
182 | 232 |
183 By design, MB should be simple enough that very little can go wrong. | 233 By design, MB should be simple enough that very little can go wrong. |
184 | 234 |
185 The most obvious issue is that you might see different commands being | 235 The most obvious issue is that you might see different commands being |
186 run than you expect; running `'mb -v'` will print what it's doing and | 236 run than you expect; running `'mb -v'` will print what it's doing and |
187 run the commands; `'mb -n'` will print what it will do but *not* run | 237 run the commands; `'mb -n'` will print what it will do but *not* run |
188 the commands. | 238 the commands. |
189 | 239 |
190 If you hit weirder things than that, add some print statements to the | 240 If you hit weirder things than that, add some print statements to the |
191 python script, send a question to gn-dev@chromium.org, or | 241 python script, send a question to gn-dev@chromium.org, or |
192 [file a bug](https://crbug.com/new) with the label | 242 [file a bug](https://crbug.com/new) with the label |
193 'mb' and cc: dpranke@chromium.org. | 243 'mb' and cc: dpranke@chromium.org. |
194 | 244 |
195 | 245 |
OLD | NEW |