| OLD | NEW |
| 1 # The MB (Meta-Build wrapper) user guide | 1 # The MB (Meta-Build wrapper) user guide |
| 2 | 2 |
| 3 [TOC] | 3 [TOC] |
| 4 | 4 |
| 5 ## Introduction | 5 ## Introduction |
| 6 | 6 |
| 7 `mb` is a simple python wrapper around the GYP and GN meta-build tools to | 7 `mb` is a simple python wrapper around the GYP and GN meta-build tools to |
| 8 be used as part of the GYP->GN migration. | 8 be used as part of the GYP->GN migration. |
| 9 | 9 |
| 10 It is intended to be used by bots to make it easier to manage the configuration | 10 It is intended to be used by bots to make it easier to manage the configuration |
| 11 each bot builds (i.e., the configurations can be changed from chromium | 11 each bot builds (i.e., the configurations can be changed from chromium |
| 12 commits), and to consolidate the list of all of the various configurations | 12 commits), and to consolidate the list of all of the various configurations |
| 13 that Chromium is built in. | 13 that Chromium is built in. |
| 14 | 14 |
| 15 Ideally this tool will no longer be needed after the migration is complete. | 15 Ideally this tool will no longer be needed after the migration is complete. |
| 16 | 16 |
| 17 For more discussion of MB, see also [the design spec](design_spec.md). | 17 ## `mb gen` |
| 18 | 18 |
| 19 ## MB subcommands | 19 `mb gen` is responsible for generating the Ninja files by invoking either GYP |
| 20 or GN as appropriate. It takes arguments to specify a build config and |
| 21 a directory, then runs GYP or GN as appropriate: |
| 20 | 22 |
| 21 ### `mb analyze` | 23 ``` |
| 24 % mb gen -m tryserver.chromium.linux -b linux_rel //out/Release |
| 25 % mb gen -c linux_rel_trybot //out/Release |
| 26 ``` |
| 27 |
| 28 Either the `-c/--config` flag or the `-m/--master` and `-b/--builder` flags |
| 29 must be specified so that `mb` can figure out which config to use. |
| 30 |
| 31 The path must be a GN-style "source-absolute" path (as above). |
| 32 |
| 33 If gen ends up using GYP, the path must have a valid GYP configuration as the |
| 34 last component of the path (i.e., specify `//out/Release_x64`, not `//out`). |
| 35 |
| 36 ## `mb analyze` |
| 22 | 37 |
| 23 `mb analyze` is reponsible for determining what targets are affected by | 38 `mb analyze` is reponsible for determining what targets are affected by |
| 24 a list of files (e.g., the list of files in a patch on a trybot): | 39 a list of files (e.g., the list of files in a patch on a trybot): |
| 25 | 40 |
| 26 ``` | 41 ``` |
| 27 mb analyze -c chromium_linux_rel //out/Release input.json output.json | 42 mb analyze -c chromium_linux_rel //out/Release input.json output.json |
| 28 ``` | 43 ``` |
| 29 | 44 |
| 30 Either the `-c/--config` flag or the `-m/--master` and `-b/--builder` flags | 45 Either the `-c/--config` flag or the `-m/--master` and `-b/--builder` flags |
| 31 must be specified so that `mb` can figure out which config to use. | 46 must be specified so that `mb` can figure out which config to use. |
| 32 | 47 |
| 48 |
| 33 The first positional argument must be a GN-style "source-absolute" path | 49 The first positional argument must be a GN-style "source-absolute" path |
| 34 to the build directory. | 50 to the build directory. |
| 35 | 51 |
| 36 The second positional argument is a (normal) path to a JSON file containing | 52 The second positional argument is a (normal) path to a JSON file containing |
| 37 a single object with two fields: | 53 a single object with two fields: |
| 38 | 54 |
| 39 * `files`: an array of the modified filenames to check (as | 55 * `files`: an array of the modified filenames to check (as |
| 40 paths relative to the checkout root). | 56 paths relative to the checkout root). |
| 41 * `targets`: an array of the unqualified target names to check. | 57 * `targets`: an array of the unqualified target names to check. |
| 42 | 58 |
| 43 The third positional argument is a (normal) path to where mb will write | 59 The third positional argument is a (normal) path to where mb will write |
| 44 the result, also as a JSON object. This object may contain the following | 60 the result, also as a JSON object. This object may contain the following |
| 45 fields: | 61 fields: |
| 46 | 62 |
| 47 * `error`: this should only be present if something failed. | 63 * `error`: this should only be present if something failed. |
| 48 * `targets`: the subset of the input `targets` that depend on the | 64 * `targets`: the subset of the input `targets` that depend on the |
| 49 input `files`. | 65 input `files`. |
| 50 * `build_targets`: the minimal subset of targets needed to build all | 66 * `build_targets`: the minimal subset of targets needed to build all |
| 51 of `targets` that were affected. | 67 of `targets` that were affected. |
| 52 * `status`: one of three strings: | 68 * `status`: one of three strings: |
| 53 * `"Found dependency"` (build the `build_targets`) | 69 * `"Found dependency"` (build the `build_targets`) |
| 54 * `"No dependency"` (i.e., no build needed) | 70 * `"No dependency"` (i.e., no build needed) |
| 55 * `"Found dependency (all)"` (build everything, in which case | 71 * `"Found dependency (all)"` (build everything, in which case |
| 56 `targets` and `build_targets` are not returned). | 72 `targets` and `build_targets` are not returned). |
| 57 | 73 |
| 58 The `-b/--builder`, `-c/--config`, `-f/--config-file`, `-m/--master`, | 74 ## `mb help` |
| 59 `-q/--quiet`, and `-v/--verbose` flags work as documented for `mb gen`. | |
| 60 | |
| 61 | |
| 62 ### `mb gen` | |
| 63 | |
| 64 `mb gen` is responsible for generating the Ninja files by invoking either GYP | |
| 65 or GN as appropriate. It takes arguments to specify a build config and | |
| 66 a directory, then runs GYP or GN as appropriate: | |
| 67 | |
| 68 ``` | |
| 69 % mb gen -m tryserver.chromium.linux -b linux_rel //out/Release | |
| 70 % mb gen -c linux_rel_trybot //out/Release | |
| 71 ``` | |
| 72 | |
| 73 Either the `-c/--config` flag or the `-m/--master` and `-b/--builder` flags | |
| 74 must be specified so that `mb` can figure out which config to use. | |
| 75 | |
| 76 By default, MB will look in `//tools/mb/mb_config.pyl` to look up the config | |
| 77 information, but you can specify a custom config file using the | |
| 78 `-f/--config-file` flag. | |
| 79 | |
| 80 The path must be a GN-style "source-absolute" path (as above). | |
| 81 | |
| 82 You can pass the `-n/--dryrun` flag to mb gen to see what will happen without | |
| 83 actually writing anything. | |
| 84 | |
| 85 You can pass the `-q/--quiet` flag to get mb to be silent unless there is an | |
| 86 error, and pass the `-v/--verbose` flag to get mb to log all of the files | |
| 87 that are read and written, and all the commands that are run. | |
| 88 | |
| 89 If the build config will use the Goma distributed-build system, you can pass | |
| 90 the path to your Goma client in the `-g/--goma-dir` flag, and it will be | |
| 91 incorporated into the appropriate flags for GYP or GN as needed. | |
| 92 | |
| 93 If gen ends up using GYP, the path must have a valid GYP configuration as the | |
| 94 last component of the path (i.e., specify `//out/Release_x64`, not `//out`). | |
| 95 | |
| 96 ### `mb help` | |
| 97 | 75 |
| 98 Produces help output on the other subcommands | 76 Produces help output on the other subcommands |
| 99 | 77 |
| 100 ### `mb lookup` | 78 ## `mb lookup` |
| 101 | 79 |
| 102 Prints what command will be run by `mb gen` (like `mb gen -n` but does | 80 Prints what command will be run by `mb gen` (like `mb gen -n` but does |
| 103 not require you to specify a path). | 81 not require you to specify a path). |
| 104 | 82 |
| 105 The `-b/--builder`, `-c/--config`, `-f/--config-file`, `-m/--master`, | 83 ## `mb validate` |
| 106 `-q/--quiet`, and `-v/--verbose` flags work as documented for `mb gen`. | |
| 107 | |
| 108 ### `mb validate` | |
| 109 | 84 |
| 110 Does internal checking to make sure the config file is syntactically | 85 Does internal checking to make sure the config file is syntactically |
| 111 valid and that all of the entries are used properly. It does not validate | 86 valid and that all of the entries are used properly. It does not validate |
| 112 that the flags make sense, or that the builder names are legal or | 87 that the flags make sense, or that the builder names are legal or |
| 113 comprehensive, but it does complain about configs and mixins that aren't | 88 comprehensive, but it does complain about configs and mixins that aren't |
| 114 used. | 89 used. |
| 115 | 90 |
| 116 The `-f/--config-file` and `-q/--quiet` flags work as documented for | |
| 117 `mb gen`. | |
| 118 | |
| 119 This is mostly useful as a presubmit check and for verifying changes to | 91 This is mostly useful as a presubmit check and for verifying changes to |
| 120 the config file. | 92 the config file. |
| 121 | 93 |
| 122 ## Isolates and Swarming | 94 ## mb_config.pyl |
| 123 | |
| 124 `mb gen` is also responsible for generating the `.isolate` and | |
| 125 `.isolated.gen.json` files needed to run test executables through swarming | |
| 126 in a GN build (in a GYP build, this is done as part of the compile step). | |
| 127 | |
| 128 If you wish to generate the isolate files, pass `mb gen` the | |
| 129 `--swarming-targets-file` command line argument; that arg should be a path | |
| 130 to a file containing a list of ninja build targets to compute the runtime | |
| 131 dependencies for (on Windows, use the ninja target name, not the file, so | |
| 132 `base_unittests`, not `base_unittests.exe`). | |
| 133 | |
| 134 MB will take this file, translate each build target to the matching GN | |
| 135 label (e.g., `base_unittests` -> `//base:base_unittests`, write that list | |
| 136 to a file called `runtime_deps` in the build directory, and pass that to | |
| 137 `gn gen $BUILD ... --runtime-deps-list-file=$BUILD/runtime_deps`. | |
| 138 | |
| 139 Once GN has computed the lists of runtime dependencies, MB will then | |
| 140 look up the command line for each target (currently this is hard-coded | |
| 141 in [mb.py](https://code.google.com/p/chromium/codesearch?q=mb.py#chromium/src/to
ols/mb/mb.py&q=mb.py%20GetIsolateCommand&sq=package:chromium&type=cs)), and writ
e out the | |
| 142 matching `.isolate` and `.isolated.gen.json` files. | |
| 143 | |
| 144 ## The `mb_config.pyl` config file | |
| 145 | 95 |
| 146 The `mb_config.pyl` config file is intended to enumerate all of the | 96 The `mb_config.pyl` config file is intended to enumerate all of the |
| 147 supported build configurations for Chromium. Generally speaking, you | 97 supported build configurations for Chromium. Generally speaking, you |
| 148 should never need to (or want to) build a configuration that isn't | 98 should never need to (or want to) build a configuration that isn't |
| 149 listed here, and so by using the configs in this file you can avoid | 99 listed here, and so by using the configs in this file you can avoid |
| 150 having to juggle long lists of GYP_DEFINES and gn args by hand. | 100 having to juggle long lists of GYP_DEFINES and gn args by hand. |
| 151 | 101 |
| 152 `mb_config.pyl` is structured as a file containing a single PYthon Literal | 102 `mb_config.pyl` is structured as a file containing a single PYthon Literal |
| 153 expression: a dictionary with three main keys, `masters`, `configs` and | 103 expression: a dictionary with three main keys, `masters`, `configs` and |
| 154 `mixins`. | 104 `mixins`. |
| (...skipping 66 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
| 221 } | 171 } |
| 222 } | 172 } |
| 223 | 173 |
| 224 and you ran `mb gen -c linux_release_trybot //out/Release`, it would | 174 and you ran `mb gen -c linux_release_trybot //out/Release`, it would |
| 225 translate into a call to `gyp_chromium -G Release` with `GYP_DEFINES` set to | 175 translate into a call to `gyp_chromium -G Release` with `GYP_DEFINES` set to |
| 226 `"use_goma=true dcheck_always_on=false dcheck_always_on=true"`. | 176 `"use_goma=true dcheck_always_on=false dcheck_always_on=true"`. |
| 227 | 177 |
| 228 (From that you can see that mb is intentionally dumb and does not | 178 (From that you can see that mb is intentionally dumb and does not |
| 229 attempt to de-dup the flags, it lets gyp do that). | 179 attempt to de-dup the flags, it lets gyp do that). |
| 230 | 180 |
| 231 ## Debugging MB | 181 ## Debugging (-v/--verbose and -n/--dry-run) |
| 232 | 182 |
| 233 By design, MB should be simple enough that very little can go wrong. | 183 By design, MB should be simple enough that very little can go wrong. |
| 234 | 184 |
| 235 The most obvious issue is that you might see different commands being | 185 The most obvious issue is that you might see different commands being |
| 236 run than you expect; running `'mb -v'` will print what it's doing and | 186 run than you expect; running `'mb -v'` will print what it's doing and |
| 237 run the commands; `'mb -n'` will print what it will do but *not* run | 187 run the commands; `'mb -n'` will print what it will do but *not* run |
| 238 the commands. | 188 the commands. |
| 239 | 189 |
| 240 If you hit weirder things than that, add some print statements to the | 190 If you hit weirder things than that, add some print statements to the |
| 241 python script, send a question to gn-dev@chromium.org, or | 191 python script, send a question to gn-dev@chromium.org, or |
| 242 [file a bug](https://crbug.com/new) with the label | 192 [file a bug](https://crbug.com/new) with the label |
| 243 'mb' and cc: dpranke@chromium.org. | 193 'mb' and cc: dpranke@chromium.org. |
| 244 | 194 |
| 245 | 195 |
| OLD | NEW |