| OLD | NEW |
| 1 # The MB (Meta-Build wrapper) user guide | 1 # The MB (Meta-Build wrapper) user guide |
| 2 | 2 |
| 3 [TOC] | 3 [TOC] |
| 4 | 4 |
| 5 ## Introduction | 5 ## Introduction |
| 6 | 6 |
| 7 `mb` is a simple python wrapper around the GYP and GN meta-build tools to | 7 `mb` is a simple python wrapper around the GYP and GN meta-build tools to |
| 8 be used as part of the GYP->GN migration. | 8 be used as part of the GYP->GN migration. |
| 9 | 9 |
| 10 It is intended to be used by bots to make it easier to manage the configuration | 10 It is intended to be used by bots to make it easier to manage the configuration |
| 11 each bot builds (i.e., the configurations can be changed from chromium | 11 each bot builds (i.e., the configurations can be changed from chromium |
| 12 commits), and to consolidate the list of all of the various configurations | 12 commits), and to consolidate the list of all of the various configurations |
| 13 that Chromium is built in. | 13 that Chromium is built in. |
| 14 | 14 |
| 15 Ideally this tool will no longer be needed after the migration is complete. | 15 Ideally this tool will no longer be needed after the migration is complete. |
| 16 | 16 |
| 17 ## `mb gen` | 17 For more discussion of MB, see also [the design spec](design_spec.md). |
| 18 | 18 |
| 19 `mb gen` is responsible for generating the Ninja files by invoking either GYP | 19 ## MB subcommands |
| 20 or GN as appropriate. It takes arguments to specify a build config and | |
| 21 a directory, then runs GYP or GN as appropriate: | |
| 22 | 20 |
| 23 ``` | 21 ### `mb analyze` |
| 24 % mb gen -m tryserver.chromium.linux -b linux_rel //out/Release | |
| 25 % mb gen -c linux_rel_trybot //out/Release | |
| 26 ``` | |
| 27 | |
| 28 Either the `-c/--config` flag or the `-m/--master` and `-b/--builder` flags | |
| 29 must be specified so that `mb` can figure out which config to use. | |
| 30 | |
| 31 The path must be a GN-style "source-absolute" path (as above). | |
| 32 | |
| 33 If gen ends up using GYP, the path must have a valid GYP configuration as the | |
| 34 last component of the path (i.e., specify `//out/Release_x64`, not `//out`). | |
| 35 | |
| 36 ## `mb analyze` | |
| 37 | 22 |
| 38 `mb analyze` is reponsible for determining what targets are affected by | 23 `mb analyze` is reponsible for determining what targets are affected by |
| 39 a list of files (e.g., the list of files in a patch on a trybot): | 24 a list of files (e.g., the list of files in a patch on a trybot): |
| 40 | 25 |
| 41 ``` | 26 ``` |
| 42 mb analyze -c chromium_linux_rel //out/Release input.json output.json | 27 mb analyze -c chromium_linux_rel //out/Release input.json output.json |
| 43 ``` | 28 ``` |
| 44 | 29 |
| 45 Either the `-c/--config` flag or the `-m/--master` and `-b/--builder` flags | 30 Either the `-c/--config` flag or the `-m/--master` and `-b/--builder` flags |
| 46 must be specified so that `mb` can figure out which config to use. | 31 must be specified so that `mb` can figure out which config to use. |
| 47 | 32 |
| 48 | |
| 49 The first positional argument must be a GN-style "source-absolute" path | 33 The first positional argument must be a GN-style "source-absolute" path |
| 50 to the build directory. | 34 to the build directory. |
| 51 | 35 |
| 52 The second positional argument is a (normal) path to a JSON file containing | 36 The second positional argument is a (normal) path to a JSON file containing |
| 53 a single object with two fields: | 37 a single object with two fields: |
| 54 | 38 |
| 55 * `files`: an array of the modified filenames to check (as | 39 * `files`: an array of the modified filenames to check (as |
| 56 paths relative to the checkout root). | 40 paths relative to the checkout root). |
| 57 * `targets`: an array of the unqualified target names to check. | 41 * `targets`: an array of the unqualified target names to check. |
| 58 | 42 |
| 59 The third positional argument is a (normal) path to where mb will write | 43 The third positional argument is a (normal) path to where mb will write |
| 60 the result, also as a JSON object. This object may contain the following | 44 the result, also as a JSON object. This object may contain the following |
| 61 fields: | 45 fields: |
| 62 | 46 |
| 63 * `error`: this should only be present if something failed. | 47 * `error`: this should only be present if something failed. |
| 64 * `targets`: the subset of the input `targets` that depend on the | 48 * `targets`: the subset of the input `targets` that depend on the |
| 65 input `files`. | 49 input `files`. |
| 66 * `build_targets`: the minimal subset of targets needed to build all | 50 * `build_targets`: the minimal subset of targets needed to build all |
| 67 of `targets` that were affected. | 51 of `targets` that were affected. |
| 68 * `status`: one of three strings: | 52 * `status`: one of three strings: |
| 69 * `"Found dependency"` (build the `build_targets`) | 53 * `"Found dependency"` (build the `build_targets`) |
| 70 * `"No dependency"` (i.e., no build needed) | 54 * `"No dependency"` (i.e., no build needed) |
| 71 * `"Found dependency (all)"` (build everything, in which case | 55 * `"Found dependency (all)"` (build everything, in which case |
| 72 `targets` and `build_targets` are not returned). | 56 `targets` and `build_targets` are not returned). |
| 73 | 57 |
| 74 ## `mb help` | 58 The `-b/--builder`, `-c/--config`, `-f/--config-file`, `-m/--master`, |
| 59 `-q/--quiet`, and `-v/--verbose` flags work as documented for `mb gen`. |
| 60 |
| 61 |
| 62 ### `mb gen` |
| 63 |
| 64 `mb gen` is responsible for generating the Ninja files by invoking either GYP |
| 65 or GN as appropriate. It takes arguments to specify a build config and |
| 66 a directory, then runs GYP or GN as appropriate: |
| 67 |
| 68 ``` |
| 69 % mb gen -m tryserver.chromium.linux -b linux_rel //out/Release |
| 70 % mb gen -c linux_rel_trybot //out/Release |
| 71 ``` |
| 72 |
| 73 Either the `-c/--config` flag or the `-m/--master` and `-b/--builder` flags |
| 74 must be specified so that `mb` can figure out which config to use. |
| 75 |
| 76 By default, MB will look in `//tools/mb/mb_config.pyl` to look up the config |
| 77 information, but you can specify a custom config file using the |
| 78 `-f/--config-file` flag. |
| 79 |
| 80 The path must be a GN-style "source-absolute" path (as above). |
| 81 |
| 82 You can pass the `-n/--dryrun` flag to mb gen to see what will happen without |
| 83 actually writing anything. |
| 84 |
| 85 You can pass the `-q/--quiet` flag to get mb to be silent unless there is an |
| 86 error, and pass the `-v/--verbose` flag to get mb to log all of the files |
| 87 that are read and written, and all the commands that are run. |
| 88 |
| 89 If the build config will use the Goma distributed-build system, you can pass |
| 90 the path to your Goma client in the `-g/--goma-dir` flag, and it will be |
| 91 incorporated into the appropriate flags for GYP or GN as needed. |
| 92 |
| 93 If gen ends up using GYP, the path must have a valid GYP configuration as the |
| 94 last component of the path (i.e., specify `//out/Release_x64`, not `//out`). |
| 95 |
| 96 ### `mb help` |
| 75 | 97 |
| 76 Produces help output on the other subcommands | 98 Produces help output on the other subcommands |
| 77 | 99 |
| 78 ## `mb lookup` | 100 ### `mb lookup` |
| 79 | 101 |
| 80 Prints what command will be run by `mb gen` (like `mb gen -n` but does | 102 Prints what command will be run by `mb gen` (like `mb gen -n` but does |
| 81 not require you to specify a path). | 103 not require you to specify a path). |
| 82 | 104 |
| 83 ## `mb validate` | 105 The `-b/--builder`, `-c/--config`, `-f/--config-file`, `-m/--master`, |
| 106 `-q/--quiet`, and `-v/--verbose` flags work as documented for `mb gen`. |
| 107 |
| 108 ### `mb validate` |
| 84 | 109 |
| 85 Does internal checking to make sure the config file is syntactically | 110 Does internal checking to make sure the config file is syntactically |
| 86 valid and that all of the entries are used properly. It does not validate | 111 valid and that all of the entries are used properly. It does not validate |
| 87 that the flags make sense, or that the builder names are legal or | 112 that the flags make sense, or that the builder names are legal or |
| 88 comprehensive, but it does complain about configs and mixins that aren't | 113 comprehensive, but it does complain about configs and mixins that aren't |
| 89 used. | 114 used. |
| 90 | 115 |
| 116 The `-f/--config-file` and `-q/--quiet` flags work as documented for |
| 117 `mb gen`. |
| 118 |
| 91 This is mostly useful as a presubmit check and for verifying changes to | 119 This is mostly useful as a presubmit check and for verifying changes to |
| 92 the config file. | 120 the config file. |
| 93 | 121 |
| 94 ## mb_config.pyl | 122 ## Isolates and Swarming |
| 123 |
| 124 `mb gen` is also responsible for generating the `.isolate` and |
| 125 `.isolated.gen.json` files needed to run test executables through swarming |
| 126 in a GN build (in a GYP build, this is done as part of the compile step). |
| 127 |
| 128 If you wish to generate the isolate files, pass `mb gen` the |
| 129 `--swarming-targets-file` command line argument; that arg should be a path |
| 130 to a file containing a list of ninja build targets to compute the runtime |
| 131 dependencies for (on Windows, use the ninja target name, not the file, so |
| 132 `base_unittests`, not `base_unittests.exe`). |
| 133 |
| 134 MB will take this file, translate each build target to the matching GN |
| 135 label (e.g., `base_unittests` -> `//base:base_unittests`, write that list |
| 136 to a file called `runtime_deps` in the build directory, and pass that to |
| 137 `gn gen $BUILD ... --runtime-deps-list-file=$BUILD/runtime_deps`. |
| 138 |
| 139 Once GN has computed the lists of runtime dependencies, MB will then |
| 140 look up the command line for each target (currently this is hard-coded |
| 141 in [mb.py](https://code.google.com/p/chromium/codesearch?q=mb.py#chromium/src/to
ols/mb/mb.py&q=mb.py%20GetIsolateCommand&sq=package:chromium&type=cs)), and writ
e out the |
| 142 matching `.isolate` and `.isolated.gen.json` files. |
| 143 |
| 144 ## The `mb_config.pyl` config file |
| 95 | 145 |
| 96 The `mb_config.pyl` config file is intended to enumerate all of the | 146 The `mb_config.pyl` config file is intended to enumerate all of the |
| 97 supported build configurations for Chromium. Generally speaking, you | 147 supported build configurations for Chromium. Generally speaking, you |
| 98 should never need to (or want to) build a configuration that isn't | 148 should never need to (or want to) build a configuration that isn't |
| 99 listed here, and so by using the configs in this file you can avoid | 149 listed here, and so by using the configs in this file you can avoid |
| 100 having to juggle long lists of GYP_DEFINES and gn args by hand. | 150 having to juggle long lists of GYP_DEFINES and gn args by hand. |
| 101 | 151 |
| 102 `mb_config.pyl` is structured as a file containing a single PYthon Literal | 152 `mb_config.pyl` is structured as a file containing a single PYthon Literal |
| 103 expression: a dictionary with three main keys, `masters`, `configs` and | 153 expression: a dictionary with three main keys, `masters`, `configs` and |
| 104 `mixins`. | 154 `mixins`. |
| (...skipping 66 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
| 171 } | 221 } |
| 172 } | 222 } |
| 173 | 223 |
| 174 and you ran `mb gen -c linux_release_trybot //out/Release`, it would | 224 and you ran `mb gen -c linux_release_trybot //out/Release`, it would |
| 175 translate into a call to `gyp_chromium -G Release` with `GYP_DEFINES` set to | 225 translate into a call to `gyp_chromium -G Release` with `GYP_DEFINES` set to |
| 176 `"use_goma=true dcheck_always_on=false dcheck_always_on=true"`. | 226 `"use_goma=true dcheck_always_on=false dcheck_always_on=true"`. |
| 177 | 227 |
| 178 (From that you can see that mb is intentionally dumb and does not | 228 (From that you can see that mb is intentionally dumb and does not |
| 179 attempt to de-dup the flags, it lets gyp do that). | 229 attempt to de-dup the flags, it lets gyp do that). |
| 180 | 230 |
| 181 ## Debugging (-v/--verbose and -n/--dry-run) | 231 ## Debugging MB |
| 182 | 232 |
| 183 By design, MB should be simple enough that very little can go wrong. | 233 By design, MB should be simple enough that very little can go wrong. |
| 184 | 234 |
| 185 The most obvious issue is that you might see different commands being | 235 The most obvious issue is that you might see different commands being |
| 186 run than you expect; running `'mb -v'` will print what it's doing and | 236 run than you expect; running `'mb -v'` will print what it's doing and |
| 187 run the commands; `'mb -n'` will print what it will do but *not* run | 237 run the commands; `'mb -n'` will print what it will do but *not* run |
| 188 the commands. | 238 the commands. |
| 189 | 239 |
| 190 If you hit weirder things than that, add some print statements to the | 240 If you hit weirder things than that, add some print statements to the |
| 191 python script, send a question to gn-dev@chromium.org, or | 241 python script, send a question to gn-dev@chromium.org, or |
| 192 [file a bug](https://crbug.com/new) with the label | 242 [file a bug](https://crbug.com/new) with the label |
| 193 'mb' and cc: dpranke@chromium.org. | 243 'mb' and cc: dpranke@chromium.org. |
| 194 | 244 |
| 195 | 245 |
| OLD | NEW |