Implement mb - a meta-build wrapper for bots to use in the GYP->GN migration.

Implement mb - a meta-build wrapper for bots to use in the GYP->GN migration.

This CL implements a new tool called mb (//tools/mb) that wraps both GYP
and GN to implement the 'gen' and 'analyze' steps on the bots. It
serves three purposes:

1. To move the decision to use GYP or GN to generate the build files to
   a configuration stored in the repo (and to hide it from the bots).
2. To move which flags are used to a configuration stored in the repo,
   so that we can version the flags alongside the code, and change the
   flags without needing to change things in the build repo.
3. To consolidate a list of all of the different build configs we need
   to support and verify for the GYP->GN migration.

For more details on the above, see //tools/md/README.md and the linked
docs.

R=scottmg@chromium.org, brettw@chromium.org
BUG=466436
Committed: https://crrev.com/fe4602310d2a93867c24dbd28447a26fb991540f
Cr-Commit-Position: refs/heads/master@{#324231}

[Dirk Pranke, 2015-04-04 02:14:42]

Okay, here's a revised version of the meta-build wrapper, with all of the
user-facing, build-multiple-things-at-once, logic removed.

I've also implemented the stuff that was missing, worked out a few last kinks,
and added some tests.

I think this is good to be reviewed and landed as a first pass (though it can
use more tests, particularly for analyze) and we can start to try it out on the
GN bots.

For posterity, the initial work on the more full-featured mb was done in
https://codereview.chromium.org/1005723003/

[brettw, 2015-04-06 20:03:06]

lgtm

https://codereview.chromium.org/1062613004/diff/1/tools/mb/docs/design_spec.md
File tools/mb/docs/design_spec.md (right):

https://codereview.chromium.org/1062613004/diff/1/tools/mb/docs/design_spec.m...
tools/mb/docs/design_spec.md:130: * MB is not (yet?) intended to replace direct
invocation of GN or GYP for
I'd just remove "yet". I don't want people to think about this and get confused.

https://codereview.chromium.org/1062613004/diff/1/tools/mb/mb_config.pyl
File tools/mb/mb_config.pyl (right):

https://codereview.chromium.org/1062613004/diff/1/tools/mb/mb_config.pyl#newc...
tools/mb/mb_config.pyl:8: 'common_dev_configs': [
I like everything being in one file.

https://codereview.chromium.org/1062613004/diff/1/tools/mb/mb_config.pyl#newc...
tools/mb/mb_config.pyl:13: 'configs': {
It would be nice to have a comment saying what this is a map from and to.

https://codereview.chromium.org/1062613004/diff/1/tools/mb/mb_config.pyl#newc...
tools/mb/mb_config.pyl:29: # TODO(dpranke): add in remaining bots on the
waterfalls.
Ditto, these are groups of bots with friendly-name-to-bot-name mappings?

[Dirk Pranke, 2015-04-06 20:09:40]

https://codereview.chromium.org/1062613004/diff/1/tools/mb/docs/design_spec.md
File tools/mb/docs/design_spec.md (right):

https://codereview.chromium.org/1062613004/diff/1/tools/mb/docs/design_spec.m...
tools/mb/docs/design_spec.md:130: * MB is not (yet?) intended to replace direct
invocation of GN or GYP for
On 2015/04/06 20:03:05, brettw wrote:
> I'd just remove "yet". I don't want people to think about this and get
confused.

True. Will do.

https://codereview.chromium.org/1062613004/diff/1/tools/mb/mb_config.pyl
File tools/mb/mb_config.pyl (right):

https://codereview.chromium.org/1062613004/diff/1/tools/mb/mb_config.pyl#newc...
tools/mb/mb_config.pyl:8: 'common_dev_configs': [
On 2015/04/06 20:03:05, brettw wrote:
> I like everything being in one file.

okey doke.

https://codereview.chromium.org/1062613004/diff/1/tools/mb/mb_config.pyl#newc...
tools/mb/mb_config.pyl:13: 'configs': {
On 2015/04/06 20:03:05, brettw wrote:
> It would be nice to have a comment saying what this is a map from and to.

Will add.

https://codereview.chromium.org/1062613004/diff/1/tools/mb/mb_config.pyl#newc...
tools/mb/mb_config.pyl:29: # TODO(dpranke): add in remaining bots on the
waterfalls.
On 2015/04/06 20:03:05, brettw wrote:
> Ditto, these are groups of bots with friendly-name-to-bot-name mappings?

Not exactly, these are maps from master name/builder name to configs. The config
is *not* necessarilly a bot name, since we do not encode the platform into the
config names.

[Dirk Pranke, 2015-04-08 15:37:29]

The CQ bit was checked by dpranke@chromium.org

[Dirk Pranke, 2015-04-08 15:37:29]

The patchset sent to the CQ was uploaded after l-g-t-m from brettw@chromium.org
Link to the patchset: https://codereview.chromium.org/1062613004/#ps60001
(title: "bug fixes and tests, reorganize mb_config.pyl")

[commit-bot: I haz the power, 2015-04-08 15:38:20]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1062613004/60001

[commit-bot: I haz the power, 2015-04-08 16:20:51]

Committed patchset #4 (id:60001)

[commit-bot: I haz the power, 2015-04-08 16:22:01]

Patchset 4 (id:??) landed as
https://crrev.com/fe4602310d2a93867c24dbd28447a26fb991540f
Cr-Commit-Position: refs/heads/master@{#324231}