Clean up mb_config.pyl now that we're off GYP.

Clean up mb_config.pyl now that we're off GYP.

This change cleans up a bunch of things now that we don't need to
support GYP:

'gn' becomes the default build type, so we no longer need 'gn_' in
the config name or need to specify the 'gn' mixin.

'swarming' and 'noswarming' are no longer needed in the config
names since the swarming flag was only needed for GYP.
'archive_gpu_tests' is also gone, for the same reason.

The 'none' builder type is gone; builders that don't build simply
aren't listed.

The 'configs' and 'mixins' sections are sorted and de-duped.

Any non-clang win builder that was specifying both goma and
minimal_symbols is now just goma, since goma + win + !clang
implies minimal symbols. A follow-on CL should make goma
imply minimal symbols across the board ...

'x64' no longer needs to be specified explicitly on windows
since it is the default.

The 80-col line limit in mb_config.pyl was dropped so that
configs can be listed on one line.

The 'symbolized' config is being removed; it was never
implemented in GN since we dropped those builders;
the one remaining v8 builder has been reporting
warnings since :).

R=thakis@chromium.org, machenbach@chromium.org
BUG=634184
Committed: https://crrev.com/f37aebb917de5c93bf44f93370694b813e2f8ecc
Cr-Commit-Position: refs/heads/master@{#420529}

[Dirk Pranke, 2016-09-20 23:44:23]

Patchset #1 (id:1) has been deleted

[Dirk Pranke, 2016-09-20 23:44:30]

Patchset #1 (id:20001) has been deleted

[Dirk Pranke, 2016-09-20 23:45:36]

Description was changed from

==========
Clean up mb_config.pyl now that we're off GYP.

This change cleans up a bunch of things now that we don't need to support GYP:

'gn' becomes the default build type, so we no longer need 'gn_' in the config
name or need to specify the 'gn' mixin.

'swarming' and 'noswarming' are no longer needed in the config names since
the swarming flag was only needed for GYP.

'archive_gpu_tests' is also gone accordingly.

The 'none' builder type is gone; builders that don't build simply aren't
listed.

The 'configs' and 'mixins' sections are sorted and de-duped.

Any non-clang win builder that was specifying both goma and minimal_symbols
is now just goma, since goma + win + !clang implies minimal symbols. A
follow-on CL should make goma imply minimal symbols across the board ...

'x64' no longer needs to be specified explicitly on windows since it is
the default.

The 80-col line limit in mb_config.pyl was dropped so that configs can be
listed on one line.

R=thakis@chromium.org
BUG=634184
==========

to

==========
Clean up mb_config.pyl now that we're off GYP.

This change cleans up a bunch of things now that we don't need to support GYP:

'gn' becomes the default build type, so we no longer need 'gn_' in the config
name or need to specify the 'gn' mixin.

'swarming' and 'noswarming' are no longer needed in the config names since
the swarming flag was only needed for GYP.

'archive_gpu_tests' is also gone accordingly.

The 'none' builder type is gone; builders that don't build simply aren't
listed.

The 'configs' and 'mixins' sections are sorted and de-duped.

Any non-clang win builder that was specifying both goma and minimal_symbols
is now just goma, since goma + win + !clang implies minimal symbols. A
follow-on CL should make goma imply minimal symbols across the board ...

'x64' no longer needs to be specified explicitly on windows since it is
the default.

The 80-col line limit in mb_config.pyl was dropped so that configs can be
listed on one line.

The 'symbolized' config is being removed; it was never implemented in GN
since we dropped those builders, and the one remaining v8 builder has 
been reporting warnings since :).

R=thakis@chromium.org, machenbach@chromium.org
BUG=634184
==========

[Dirk Pranke, 2016-09-20 23:45:36]

dpranke@chromium.org changed reviewers:
+ machenbach@chromium.org

[Dirk Pranke, 2016-09-20 23:53:01]

Patchset #1 (id:40001) has been deleted

[Dirk Pranke, 2016-09-20 23:59:02]

Patchset #1 (id:60001) has been deleted

[Dirk Pranke, 2016-09-21 00:03:04]

Please take a look, and apologies in advance for the un-reviewable-ness of this
CL.

Once you decide you're going to delete the 'swarming' and 'gn' things from the
config names (which needed to be done) and dedup/sort things, the CL became
more-or-less unreviewable. At that point I decided to add in all of the other
cleanup that wouldn't result in a functional change just so I could do this all
at once.

There is one functional change (sort of): @machenbach - the "symbolized" config
was never implemented in GN, because we decided to kill off that configuration
on the bots, but I guess I missed the v8 bot. At any rate, the 'Chromium ASAN
(symbolized)' builder on 'client.v8.fyi' has never been doing an actual
symbolized build, and you should just shut it down; this change will break it
otherwise.

Also, @thakis - please take a look and make sure I didn't mistype something in
one of the clang builders, since they were exempt from some of the other changes
related to minimal_symbols usage.

[Michael Achenbach, 2016-09-21 07:50:40]

> There is one functional change (sort of): @machenbach - the "symbolized"
config
> was never implemented in GN, because we decided to kill off that configuration
> on the bots, but I guess I missed the v8 bot. At any rate, the 'Chromium ASAN
> (symbolized)' builder on 'client.v8.fyi' has never been doing an actual
> symbolized build, and you should just shut it down; this change will break it
> otherwise.

I don't see how, we have our own copy of mb :)

Btw, we patched our configs to match our old gyp bot:
https://cs.chromium.org/chromium/src/v8/infra/mb/mb_config.pyl?q=mb_confi&sq=...
https://cs.chromium.org/chromium/src/v8/BUILD.gn?q=v8_no_inline&sq=package:ch...

The name is totally misleading, I know. This was just to smoothly transition
from gyp without changing much.

[Michael Achenbach, 2016-09-21 08:00:39]

As for reviewing, I didn't find a good strategy how to start and don't have the
time for a (good) manual review of this ATM.

Could you maybe provide a simple script that serves as a reviewer?

E.g. loop over all bots, call mb and enforce equal args.gn files with and
without this CL. (ignoring bots that have no configs at all)

I'd be happy to review and run such a script and don't care so much for the
exact ways how the mb_config.pyl file changes.

[Dirk Pranke, 2016-09-21 17:41:16]

On 2016/09/21 07:50:40, machenbach (slow) wrote:
> > There is one functional change (sort of): @machenbach - the "symbolized"
> config
> > was never implemented in GN, because we decided to kill off that
configuration
> > on the bots, but I guess I missed the v8 bot. At any rate, the 'Chromium
ASAN
> > (symbolized)' builder on 'client.v8.fyi' has never been doing an actual
> > symbolized build, and you should just shut it down; this change will break
it
> > otherwise.
> 
> I don't see how, we have our own copy of mb :)

It looks like on the client.v8.fyi waterfall you're still running the chromium
version
of mb.py and using chromium's mb_config.pyl.

[Dirk Pranke, 2016-09-21 17:41:56]

On 2016/09/21 08:00:39, machenbach (slow) wrote:
> As for reviewing, I didn't find a good strategy how to start and don't have
the
> time for a (good) manual review of this ATM.
> 
> Could you maybe provide a simple script that serves as a reviewer?
> 
> E.g. loop over all bots, call mb and enforce equal args.gn files with and
> without this CL. (ignoring bots that have no configs at all)

Okay, I had thought about doing this as well but was too lazy. It shouldn't
take much work.

[Dirk Pranke, 2016-09-21 23:02:02]

Patchset #2 (id:100001) has been deleted

[Dirk Pranke, 2016-09-21 23:02:08]

Patchset #2 (id:120001) has been deleted

[Dirk Pranke, 2016-09-21 23:42:35]

Patchset #3 (id:160001) has been deleted

[Dirk Pranke, 2016-09-21 23:54:50]

Patchset #3 (id:180001) has been deleted

[Dirk Pranke, 2016-09-21 23:55:01]

Patchset #3 (id:200001) has been deleted

[Dirk Pranke, 2016-09-21 23:57:27]

Okay, doing the export/validation exercise turned out to be quite useful (I
found a number of bugs and minor nits), so I'll probably leave the `export`
command in the final patch (but I won't commit the exported file).

Please take a look.

If you want to reproduce this locally, you can just run `python tools/mb/mb.py
export > mb_config.pyl.export`.

https://codereview.chromium.org/2357483002/diff/140001/tools/mb/mb_config.pyl...
File tools/mb/mb_config.pyl.export (right):

https://codereview.chromium.org/2357483002/diff/140001/tools/mb/mb_config.pyl...
tools/mb/mb_config.pyl.export:320: "Chromium ASAN (symbolized)": "is_asan=true
is_lsan=true sanitizer_coverage_flags=\"edge\" v8_enable_verify_heap=true
symbolized=true is_debug=false is_component_build=false use_goma=true",
This is the aforementioned intentional change for the symbolized bot.

https://codereview.chromium.org/2357483002/diff/220001/tools/mb/mb_config.pyl...
File tools/mb/mb_config.pyl.export (right):

https://codereview.chromium.org/2357483002/diff/220001/tools/mb/mb_config.pyl...
tools/mb/mb_config.pyl.export:113: "CrWinClang(shared)": "is_clang=true
symbol_level=1 is_component_build=true is_debug=false use_goma=true
target_cpu=\"x86\"",
This change is intentional, but it has no effect; now the args match the
ordering in the config name.

https://codereview.chromium.org/2357483002/diff/220001/tools/mb/mb_config.pyl...
tools/mb/mb_config.pyl.export:427: "linux_chromium_compile_rel_ng":
"is_debug=false is_component_build=false use_goma=true symbol_level=1
dcheck_always_on=true",
This is a fix for a bug I discovered while doing this; the builder was not
correctly configured as a trybot before.

[Michael Achenbach, 2016-09-22 08:09:24]

lgtm with comments:

Thanks for the export function and the way you uploaded the file - was very
helpful for review.

Please run the export once more after rebase and final tidying of new bots not
incorporated yet. Maybe also upload (but not commit) the final export file - for
a post commit review.

https://codereview.chromium.org/2357483002/diff/140001/tools/mb/mb_config.pyl...
File tools/mb/mb_config.pyl.export (right):

https://codereview.chromium.org/2357483002/diff/140001/tools/mb/mb_config.pyl...
tools/mb/mb_config.pyl.export:108: "ClangToTWin": "llvm_force_head_revision=true
clang_use_chrome_plugins=false is_clang=true is_chrome_branded=true
is_official_build=true is_debug=false symbol_level=1 is_component_build=false
is_debug=false target_cpu=\"x86\"",
Is this intentional? Wouldn't this turn the bot into using x64?

https://codereview.chromium.org/2357483002/diff/220001/tools/mb/mb_config.pyl
File tools/mb/mb_config.pyl (left):

https://codereview.chromium.org/2357483002/diff/220001/tools/mb/mb_config.pyl...
tools/mb/mb_config.pyl:637: 'Chromium ASAN (symbolized)':
Could you maybe keep the bot (with the same name), but turn the config into a
valid normal asan release?

https://codereview.chromium.org/2357483002/diff/220001/tools/mb/mb_config.pyl...
File tools/mb/mb_config.pyl.export (right):

https://codereview.chromium.org/2357483002/diff/220001/tools/mb/mb_config.pyl...
tools/mb/mb_config.pyl.export:427: "linux_chromium_compile_rel_ng":
"is_debug=false is_component_build=false use_goma=true symbol_level=1
dcheck_always_on=true",
On 2016/09/21 23:57:27, Dirk Pranke wrote:
> This is a fix for a bug I discovered while doing this; the builder was not
> correctly configured as a trybot before.

Good catch!

[Michael Achenbach, 2016-09-22 08:13:13]

https://codereview.chromium.org/2357483002/diff/140001/tools/mb/mb_config.pyl...
File tools/mb/mb_config.pyl.export (right):

https://codereview.chromium.org/2357483002/diff/140001/tools/mb/mb_config.pyl...
tools/mb/mb_config.pyl.export:108: "ClangToTWin": "llvm_force_head_revision=true
clang_use_chrome_plugins=false is_clang=true is_chrome_branded=true
is_official_build=true is_debug=false symbol_level=1 is_component_build=false
is_debug=false target_cpu=\"x86\"",
On 2016/09/22 08:09:24, machenbach (slow) wrote:
> Is this intentional? Wouldn't this turn the bot into using x64?

Maybe the location where I commented is confusing. It's in the patchset where
the x86 is still there. In the next one it's gone.

[Dirk Pranke, 2016-09-22 21:54:00]

https://codereview.chromium.org/2357483002/diff/140001/tools/mb/mb_config.pyl...
File tools/mb/mb_config.pyl.export (right):

https://codereview.chromium.org/2357483002/diff/140001/tools/mb/mb_config.pyl...
tools/mb/mb_config.pyl.export:108: "ClangToTWin": "llvm_force_head_revision=true
clang_use_chrome_plugins=false is_clang=true is_chrome_branded=true
is_official_build=true is_debug=false symbol_level=1 is_component_build=false
is_debug=false target_cpu=\"x86\"",
On 2016/09/22 08:13:12, machenbach (slow) wrote:
> On 2016/09/22 08:09:24, machenbach (slow) wrote:
> > Is this intentional? Wouldn't this turn the bot into using x64?
> 
> Maybe the location where I commented is confusing. It's in the patchset where
> the x86 is still there. In the next one it's gone.

Hm. The _x86 part got dropped somehow. Good catch, fixed!

https://codereview.chromium.org/2357483002/diff/220001/tools/mb/mb_config.pyl
File tools/mb/mb_config.pyl (left):

https://codereview.chromium.org/2357483002/diff/220001/tools/mb/mb_config.pyl...
tools/mb/mb_config.pyl:637: 'Chromium ASAN (symbolized)':
On 2016/09/22 08:09:24, machenbach (slow) wrote:
> Could you maybe keep the bot (with the same name), but turn the config into a
> valid normal asan release?

Sure, I'll change it to 'asan_lsan_release_bot' (the same as Linux ASAN
Builder).

[Dirk Pranke, 2016-09-22 21:55:38]

Description was changed from

==========
Clean up mb_config.pyl now that we're off GYP.

This change cleans up a bunch of things now that we don't need to support GYP:

'gn' becomes the default build type, so we no longer need 'gn_' in the config
name or need to specify the 'gn' mixin.

'swarming' and 'noswarming' are no longer needed in the config names since
the swarming flag was only needed for GYP.

'archive_gpu_tests' is also gone accordingly.

The 'none' builder type is gone; builders that don't build simply aren't
listed.

The 'configs' and 'mixins' sections are sorted and de-duped.

Any non-clang win builder that was specifying both goma and minimal_symbols
is now just goma, since goma + win + !clang implies minimal symbols. A
follow-on CL should make goma imply minimal symbols across the board ...

'x64' no longer needs to be specified explicitly on windows since it is
the default.

The 80-col line limit in mb_config.pyl was dropped so that configs can be
listed on one line.

The 'symbolized' config is being removed; it was never implemented in GN
since we dropped those builders, and the one remaining v8 builder has 
been reporting warnings since :).

R=thakis@chromium.org, machenbach@chromium.org
BUG=634184
==========

to

==========
Clean up mb_config.pyl now that we're off GYP.

This change cleans up a bunch of things now that we don't need to 
support GYP:

'gn' becomes the default build type, so we no longer need 'gn_' in 
the config name or need to specify the 'gn' mixin.

'swarming' and 'noswarming' are no longer needed in the config 
names since the swarming flag was only needed for GYP. 
'archive_gpu_tests' is also gone, for the same reason.

The 'none' builder type is gone; builders that don't build simply 
aren't listed.

The 'configs' and 'mixins' sections are sorted and de-duped.

Any non-clang win builder that was specifying both goma and 
minimal_symbols is now just goma, since goma + win + !clang 
implies minimal symbols. A follow-on CL should make goma 
imply minimal symbols across the board ...

'x64' no longer needs to be specified explicitly on windows
since it is the default.

The 80-col line limit in mb_config.pyl was dropped so that 
configs can be listed on one line.

The 'symbolized' config is being removed; it was never 
implemented in GN since we dropped those builders; 
the one remaining v8 builder has been reporting 
warnings since :).

R=thakis@chromium.org, machenbach@chromium.org
BUG=634184
==========

[Dirk Pranke, 2016-09-22 21:55:52]

The CQ bit was checked by dpranke@chromium.org

[Dirk Pranke, 2016-09-22 21:55:53]

The patchset sent to the CQ was uploaded after l-g-t-m from
machenbach@chromium.org
Link to the patchset: https://codereview.chromium.org/2357483002/#ps260001
(title: "patch for landing (.export file removed, a few missing trailing commas
added)")

[commit-bot: I haz the power, 2016-09-22 21:56:47]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-09-22 22:06:42]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-09-22 22:06:44]

Try jobs failed on following builders:
  cast_shell_android on master.tryserver.chromium.android (JOB_FAILED,
https://build.chromium.org/p/tryserver.chromium.android/builders/cast_shell_a...)
  chromeos_amd64-generic_chromium_compile_only_ng on
master.tryserver.chromium.linux (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.linux/builders/chromeos_amd64-...)

[Dirk Pranke, 2016-09-22 22:43:21]

The CQ bit was checked by dpranke@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-09-22 22:44:06]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-09-22 23:48:56]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-09-22 23:48:57]

Dry run: This issue passed the CQ dry run.

[Dirk Pranke, 2016-09-23 00:04:08]

The CQ bit was checked by dpranke@chromium.org

[Dirk Pranke, 2016-09-23 00:04:09]

The patchset sent to the CQ was uploaded after l-g-t-m from
machenbach@chromium.org
Link to the patchset: https://codereview.chromium.org/2357483002/#ps300001
(title: "patch for landing (no .export file)")

[commit-bot: I haz the power, 2016-09-23 00:05:13]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-09-23 01:15:15]

Description was changed from

==========
Clean up mb_config.pyl now that we're off GYP.

This change cleans up a bunch of things now that we don't need to 
support GYP:

'gn' becomes the default build type, so we no longer need 'gn_' in 
the config name or need to specify the 'gn' mixin.

'swarming' and 'noswarming' are no longer needed in the config 
names since the swarming flag was only needed for GYP. 
'archive_gpu_tests' is also gone, for the same reason.

The 'none' builder type is gone; builders that don't build simply 
aren't listed.

The 'configs' and 'mixins' sections are sorted and de-duped.

Any non-clang win builder that was specifying both goma and 
minimal_symbols is now just goma, since goma + win + !clang 
implies minimal symbols. A follow-on CL should make goma 
imply minimal symbols across the board ...

'x64' no longer needs to be specified explicitly on windows
since it is the default.

The 80-col line limit in mb_config.pyl was dropped so that 
configs can be listed on one line.

The 'symbolized' config is being removed; it was never 
implemented in GN since we dropped those builders; 
the one remaining v8 builder has been reporting 
warnings since :).

R=thakis@chromium.org, machenbach@chromium.org
BUG=634184
==========

to

==========
Clean up mb_config.pyl now that we're off GYP.

This change cleans up a bunch of things now that we don't need to 
support GYP:

'gn' becomes the default build type, so we no longer need 'gn_' in 
the config name or need to specify the 'gn' mixin.

'swarming' and 'noswarming' are no longer needed in the config 
names since the swarming flag was only needed for GYP. 
'archive_gpu_tests' is also gone, for the same reason.

The 'none' builder type is gone; builders that don't build simply 
aren't listed.

The 'configs' and 'mixins' sections are sorted and de-duped.

Any non-clang win builder that was specifying both goma and 
minimal_symbols is now just goma, since goma + win + !clang 
implies minimal symbols. A follow-on CL should make goma 
imply minimal symbols across the board ...

'x64' no longer needs to be specified explicitly on windows
since it is the default.

The 80-col line limit in mb_config.pyl was dropped so that 
configs can be listed on one line.

The 'symbolized' config is being removed; it was never 
implemented in GN since we dropped those builders; 
the one remaining v8 builder has been reporting 
warnings since :).

R=thakis@chromium.org, machenbach@chromium.org
BUG=634184
==========

[commit-bot: I haz the power, 2016-09-23 01:15:16]

Committed patchset #7 (id:300001)

[commit-bot: I haz the power, 2016-09-23 01:17:04]

Description was changed from

==========
Clean up mb_config.pyl now that we're off GYP.

This change cleans up a bunch of things now that we don't need to 
support GYP:

'gn' becomes the default build type, so we no longer need 'gn_' in 
the config name or need to specify the 'gn' mixin.

'swarming' and 'noswarming' are no longer needed in the config 
names since the swarming flag was only needed for GYP. 
'archive_gpu_tests' is also gone, for the same reason.

The 'none' builder type is gone; builders that don't build simply 
aren't listed.

The 'configs' and 'mixins' sections are sorted and de-duped.

Any non-clang win builder that was specifying both goma and 
minimal_symbols is now just goma, since goma + win + !clang 
implies minimal symbols. A follow-on CL should make goma 
imply minimal symbols across the board ...

'x64' no longer needs to be specified explicitly on windows
since it is the default.

The 80-col line limit in mb_config.pyl was dropped so that 
configs can be listed on one line.

The 'symbolized' config is being removed; it was never 
implemented in GN since we dropped those builders; 
the one remaining v8 builder has been reporting 
warnings since :).

R=thakis@chromium.org, machenbach@chromium.org
BUG=634184
==========

to

==========
Clean up mb_config.pyl now that we're off GYP.

This change cleans up a bunch of things now that we don't need to
support GYP:

'gn' becomes the default build type, so we no longer need 'gn_' in
the config name or need to specify the 'gn' mixin.

'swarming' and 'noswarming' are no longer needed in the config
names since the swarming flag was only needed for GYP.
'archive_gpu_tests' is also gone, for the same reason.

The 'none' builder type is gone; builders that don't build simply
aren't listed.

The 'configs' and 'mixins' sections are sorted and de-duped.

Any non-clang win builder that was specifying both goma and
minimal_symbols is now just goma, since goma + win + !clang
implies minimal symbols. A follow-on CL should make goma
imply minimal symbols across the board ...

'x64' no longer needs to be specified explicitly on windows
since it is the default.

The 80-col line limit in mb_config.pyl was dropped so that
configs can be listed on one line.

The 'symbolized' config is being removed; it was never
implemented in GN since we dropped those builders;
the one remaining v8 builder has been reporting
warnings since :).

R=thakis@chromium.org, machenbach@chromium.org
BUG=634184

Committed: https://crrev.com/f37aebb917de5c93bf44f93370694b813e2f8ecc
Cr-Commit-Position: refs/heads/master@{#420529}
==========

[commit-bot: I haz the power, 2016-09-23 01:17:05]

Patchset 7 (id:??) landed as
https://crrev.com/f37aebb917de5c93bf44f93370694b813e2f8ecc
Cr-Commit-Position: refs/heads/master@{#420529}

[Nico, 2016-09-23 15:42:39]

This broke
https://build.chromium.org/p/chromium.chromiumos/builders/Linux%20ChromiumOS%...
https://build.chromium.org/p/chromium.chromiumos/builders/Linux%20ChromiumOS%...
(two bots)

[Dirk Pranke, 2016-09-23 17:07:04]

On 2016/09/23 15:42:39, Nico wrote:
> This broke
>
https://build.chromium.org/p/chromium.chromiumos/builders/Linux%20ChromiumOS%...
>
https://build.chromium.org/p/chromium.chromiumos/builders/Linux%20ChromiumOS%...
> (two bots)

Ah, I thought those were already gone. Will remove the bots. Thanks!