Enable component builds for fuzzers

Enable component builds for fuzzers

V8 is collecting a growing amount of fuzzers, all of which take substantial
space on the bots and in chromium build archives. This CL improves that
situation by allowing component (shared library) builds for almost all fuzzers.

The parser fuzzer is handled as an exception since it would require exporting a
large number of additional functions.

A component build results in about a 50-100x improvement in file size for each
fuzzer (~50M-100M to around 1.1M).

BUG=chromium:648864
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_chromium_compile_dbg_ng;master.tryserver.chromium.android:android_clang_dbg_recipe

Committed: https://crrev.com/22606f0c29421d6a72e46595ec2995c7771ff070
Cr-Commit-Position: refs/heads/master@{#39709}

[jgruber, 2016-09-22 07:58:48]

The CQ bit was checked by jgruber@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-09-22 07:58:56]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-09-22 08:11:09]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-09-22 08:11:10]

Dry run: Try jobs failed on following builders:
  v8_win_nosnap_shared_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_win_nosnap_shared_rel_ng...)

[jgruber, 2016-09-22 08:34:25]

The CQ bit was checked by jgruber@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-09-22 08:34:34]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-09-22 08:43:23]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-09-22 08:43:23]

Dry run: Try jobs failed on following builders:
  v8_win_nosnap_shared_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_win_nosnap_shared_rel_ng...)

[jgruber, 2016-09-22 08:55:23]

The CQ bit was checked by jgruber@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-09-22 08:55:32]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-09-22 09:06:33]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-09-22 09:06:34]

Dry run: Try jobs failed on following builders:
  v8_win_nosnap_shared_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_win_nosnap_shared_rel_ng...)

[jgruber, 2016-09-22 09:19:41]

The CQ bit was checked by jgruber@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-09-22 09:19:52]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[jgruber, 2016-09-22 09:24:30]

Description was changed from

==========
Enable component builds for fuzzers

BUG=chromium:648864
==========

to

==========
Enable component builds for fuzzers

V8 is collecting a growing amount of fuzzers, all of which take substantial
space on the bots and in chromium build archives. This CL improves that
situation by allowing component (shared library) builds for almost all fuzzers.

The parser fuzzer is handled as an exception since it would require exporting a
large number of additional functions.

A component build results in about a 50-100x improvement in file size for each
fuzzer (~50M-100M to around 1.1M).

BUG=chromium:648864
==========

[jgruber, 2016-09-22 09:28:40]

Description was changed from

==========
Enable component builds for fuzzers

V8 is collecting a growing amount of fuzzers, all of which take substantial
space on the bots and in chromium build archives. This CL improves that
situation by allowing component (shared library) builds for almost all fuzzers.

The parser fuzzer is handled as an exception since it would require exporting a
large number of additional functions.

A component build results in about a 50-100x improvement in file size for each
fuzzer (~50M-100M to around 1.1M).

BUG=chromium:648864
==========

to

==========
Enable component builds for fuzzers

V8 is collecting a growing amount of fuzzers, all of which take substantial
space on the bots and in chromium build archives. This CL improves that
situation by allowing component (shared library) builds for almost all fuzzers.

The parser fuzzer is handled as an exception since it would require exporting a
large number of additional functions.

A component build results in about a 50-100x improvement in file size for each
fuzzer (~50M-100M to around 1.1M).

BUG=chromium:648864
==========

[jgruber, 2016-09-22 09:38:27]

The CQ bit was checked by jgruber@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-09-22 09:38:33]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[jgruber, 2016-09-22 09:52:23]

jgruber@chromium.org changed reviewers:
+ ahaas@chromium.org, machenbach@chromium.org

[jgruber, 2016-09-22 09:54:28]

jgruber@chromium.org changed reviewers:
+ jochen@chromium.org

[jochen (gone - plz use gerrit), 2016-09-22 09:54:46]

does this work out of the box for the gyp build? should we bother to fix this in
the gyp build at all?

[jgruber, 2016-09-22 10:16:16]

On 2016/09/22 09:54:46, jochen (slow) wrote:
> does this work out of the box for the gyp build? should we bother to fix this
in
> the gyp build at all?

Added gyp support since it was easy in this case.

[Michael Achenbach, 2016-09-22 10:26:39]

lgtm on the build files! Thanks a lot for working on this!

[jochen (gone - plz use gerrit), 2016-09-22 10:29:49]

lgtm

[Michael Achenbach, 2016-09-22 10:31:59]

The CQ bit was checked by machenbach@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-09-22 10:32:07]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[ahaas, 2016-09-22 10:42:16]

On 2016/09/22 at 10:29:49, jochen wrote:
> lgtm

lgtm, thank you very much for that!

[commit-bot: I haz the power, 2016-09-22 10:43:15]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-09-22 10:43:16]

Dry run: Try jobs failed on following builders:
  v8_win_nosnap_shared_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_win_nosnap_shared_rel_ng...)

[jgruber, 2016-09-22 10:55:46]

The CQ bit was checked by jgruber@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-09-22 10:55:56]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-09-22 11:20:55]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-09-22 11:20:56]

Dry run: This issue passed the CQ dry run.

[jgruber, 2016-09-22 11:21:09]

The CQ bit was checked by jgruber@chromium.org

[jgruber, 2016-09-22 11:21:09]

The patchset sent to the CQ was uploaded after l-g-t-m from jochen@chromium.org,
ahaas@chromium.org, machenbach@chromium.org
Link to the patchset: https://codereview.chromium.org/2360983002/#ps140001
(title: "Explicit external configs for fuzzer targets")

[commit-bot: I haz the power, 2016-09-22 11:21:18]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[Michael Achenbach, 2016-09-22 11:22:08]

Does the last patchset fix this?
https://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium...

https://codereview.chromium.org/2360983002/diff/140001/BUILD.gn
File BUILD.gn (right):

https://codereview.chromium.org/2360983002/diff/140001/BUILD.gn#newcode2336
BUILD.gn:2336: ":v8",
Instead of adding external config everywhere you probably could have moved v8 to
the public deps. But never mind.

[Michael Achenbach, 2016-09-22 11:22:33]

The CQ bit was unchecked by machenbach@chromium.org

[Michael Achenbach, 2016-09-22 11:30:29]

Description was changed from

==========
Enable component builds for fuzzers

V8 is collecting a growing amount of fuzzers, all of which take substantial
space on the bots and in chromium build archives. This CL improves that
situation by allowing component (shared library) builds for almost all fuzzers.

The parser fuzzer is handled as an exception since it would require exporting a
large number of additional functions.

A component build results in about a 50-100x improvement in file size for each
fuzzer (~50M-100M to around 1.1M).

BUG=chromium:648864
==========

to

==========
Enable component builds for fuzzers

V8 is collecting a growing amount of fuzzers, all of which take substantial
space on the bots and in chromium build archives. This CL improves that
situation by allowing component (shared library) builds for almost all fuzzers.

The parser fuzzer is handled as an exception since it would require exporting a
large number of additional functions.

A component build results in about a 50-100x improvement in file size for each
fuzzer (~50M-100M to around 1.1M).

BUG=chromium:648864
CQ_INCLUDE_TRYBOTS=tryserver.chromium.linux:linux_chromium_compile_dbg_ng;
==========

[Michael Achenbach, 2016-09-22 11:30:36]

The CQ bit was checked by machenbach@chromium.org

[commit-bot: I haz the power, 2016-09-22 11:30:46]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-09-22 11:30:47]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-09-22 11:30:49]

Your CL was about to rely on recently removed CQ feature(s):
 * Specifying master names in CQ_INCLUDE_TRYBOTS part of description without
"master." prefix is no longer supported:
     tryserver.chromium.linux
   For more details, see http://crbug.com/617627.

[Michael Achenbach, 2016-09-22 11:38:01]

Description was changed from

==========
Enable component builds for fuzzers

V8 is collecting a growing amount of fuzzers, all of which take substantial
space on the bots and in chromium build archives. This CL improves that
situation by allowing component (shared library) builds for almost all fuzzers.

The parser fuzzer is handled as an exception since it would require exporting a
large number of additional functions.

A component build results in about a 50-100x improvement in file size for each
fuzzer (~50M-100M to around 1.1M).

BUG=chromium:648864
CQ_INCLUDE_TRYBOTS=tryserver.chromium.linux:linux_chromium_compile_dbg_ng;
==========

to

==========
Enable component builds for fuzzers

V8 is collecting a growing amount of fuzzers, all of which take substantial
space on the bots and in chromium build archives. This CL improves that
situation by allowing component (shared library) builds for almost all fuzzers.

The parser fuzzer is handled as an exception since it would require exporting a
large number of additional functions.

A component build results in about a 50-100x improvement in file size for each
fuzzer (~50M-100M to around 1.1M).

BUG=chromium:648864
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_chromium_compile_dbg_ng;
==========

[Michael Achenbach, 2016-09-22 11:38:48]

The CQ bit was checked by machenbach@chromium.org

[commit-bot: I haz the power, 2016-09-22 11:38:57]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[Michael Achenbach, 2016-09-22 11:39:02]

The CQ bit was unchecked by machenbach@chromium.org

[Michael Achenbach, 2016-09-22 12:46:31]

Description was changed from

==========
Enable component builds for fuzzers

V8 is collecting a growing amount of fuzzers, all of which take substantial
space on the bots and in chromium build archives. This CL improves that
situation by allowing component (shared library) builds for almost all fuzzers.

The parser fuzzer is handled as an exception since it would require exporting a
large number of additional functions.

A component build results in about a 50-100x improvement in file size for each
fuzzer (~50M-100M to around 1.1M).

BUG=chromium:648864
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_chromium_compile_dbg_ng;
==========

to

==========
Enable component builds for fuzzers

V8 is collecting a growing amount of fuzzers, all of which take substantial
space on the bots and in chromium build archives. This CL improves that
situation by allowing component (shared library) builds for almost all fuzzers.

The parser fuzzer is handled as an exception since it would require exporting a
large number of additional functions.

A component build results in about a 50-100x improvement in file size for each
fuzzer (~50M-100M to around 1.1M).

BUG=chromium:648864
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_chromium_compile_dbg_ng;master.tryserver.chromium.android:android_clang_dbg_recipe
==========

[jgruber, 2016-09-22 12:53:50]

The CQ bit was checked by jgruber@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-09-22 12:53:58]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-09-22 13:12:00]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-09-22 13:12:01]

Dry run: Try jobs failed on following builders:
  v8_win_nosnap_shared_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_win_nosnap_shared_rel_ng...)

[jgruber, 2016-09-23 11:35:50]

The CQ bit was checked by jgruber@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-09-23 11:35:59]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-09-23 13:36:36]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-09-23 13:36:38]

Dry run: Try jobs failed on following builders:
  v8_win64_rel_ng on master.tryserver.v8 (JOB_TIMED_OUT, no build URL)
  v8_win_compile_dbg on master.tryserver.v8 (JOB_TIMED_OUT, no build URL)
  v8_win_rel_ng on master.tryserver.v8 (JOB_TIMED_OUT, no build URL)

[jgruber, 2016-09-23 14:09:27]

https://codereview.chromium.org/2360983002/diff/140001/BUILD.gn
File BUILD.gn (right):

https://codereview.chromium.org/2360983002/diff/140001/BUILD.gn#newcode2336
BUILD.gn:2336: ":v8",
On 2016/09/22 11:22:07, machenbach (slow) wrote:
> Instead of adding external config everywhere you probably could have moved v8
to
> the public deps. But never mind.

Leaving this to a cleanup commit as discussed offline - we could also possibly
merge a couple of these fuzzer targets and move internal_config_base to public
deps in fuzzer support.

[Michael Achenbach, 2016-09-23 14:18:06]

still lgtm

[jgruber, 2016-09-23 16:43:18]

The CQ bit was checked by jgruber@chromium.org

[jgruber, 2016-09-23 16:43:19]

The patchset sent to the CQ was uploaded after l-g-t-m from jochen@chromium.org,
ahaas@chromium.org
Link to the patchset: https://codereview.chromium.org/2360983002/#ps180001
(title: "Move PerThreadAssertScope exports to satisfy compiler")

[commit-bot: I haz the power, 2016-09-23 16:43:27]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-09-23 16:47:36]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-09-23 16:47:37]

Try jobs failed on following builders:
  v8_presubmit on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_presubmit/builds/24893)

[jgruber, 2016-09-26 06:47:54]

The CQ bit was checked by jgruber@chromium.org

[jgruber, 2016-09-26 06:47:54]

The patchset sent to the CQ was uploaded after l-g-t-m from jochen@chromium.org,
ahaas@chromium.org, machenbach@chromium.org
Link to the patchset: https://codereview.chromium.org/2360983002/#ps200001
(title: "Rebase")

[commit-bot: I haz the power, 2016-09-26 06:48:00]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-09-26 07:40:31]

Description was changed from

==========
Enable component builds for fuzzers

V8 is collecting a growing amount of fuzzers, all of which take substantial
space on the bots and in chromium build archives. This CL improves that
situation by allowing component (shared library) builds for almost all fuzzers.

The parser fuzzer is handled as an exception since it would require exporting a
large number of additional functions.

A component build results in about a 50-100x improvement in file size for each
fuzzer (~50M-100M to around 1.1M).

BUG=chromium:648864
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_chromium_compile_dbg_ng;master.tryserver.chromium.android:android_clang_dbg_recipe
==========

to

==========
Enable component builds for fuzzers

V8 is collecting a growing amount of fuzzers, all of which take substantial
space on the bots and in chromium build archives. This CL improves that
situation by allowing component (shared library) builds for almost all fuzzers.

The parser fuzzer is handled as an exception since it would require exporting a
large number of additional functions.

A component build results in about a 50-100x improvement in file size for each
fuzzer (~50M-100M to around 1.1M).

BUG=chromium:648864
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_chromium_compile_dbg_ng;master.tryserver.chromium.android:android_clang_dbg_recipe
==========

[commit-bot: I haz the power, 2016-09-26 07:40:32]

Committed patchset #11 (id:200001)

[commit-bot: I haz the power, 2016-09-26 07:40:56]

Description was changed from

==========
Enable component builds for fuzzers

V8 is collecting a growing amount of fuzzers, all of which take substantial
space on the bots and in chromium build archives. This CL improves that
situation by allowing component (shared library) builds for almost all fuzzers.

The parser fuzzer is handled as an exception since it would require exporting a
large number of additional functions.

A component build results in about a 50-100x improvement in file size for each
fuzzer (~50M-100M to around 1.1M).

BUG=chromium:648864
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_chromium_compile_dbg_ng;master.tryserver.chromium.android:android_clang_dbg_recipe
==========

to

==========
Enable component builds for fuzzers

V8 is collecting a growing amount of fuzzers, all of which take substantial
space on the bots and in chromium build archives. This CL improves that
situation by allowing component (shared library) builds for almost all fuzzers.

The parser fuzzer is handled as an exception since it would require exporting a
large number of additional functions.

A component build results in about a 50-100x improvement in file size for each
fuzzer (~50M-100M to around 1.1M).

BUG=chromium:648864
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_chromium_compile_dbg_ng;master.tryserver.chromium.android:android_clang_dbg_recipe

Committed: https://crrev.com/22606f0c29421d6a72e46595ec2995c7771ff070
Cr-Commit-Position: refs/heads/master@{#39709}
==========

[commit-bot: I haz the power, 2016-09-26 07:40:59]

Patchset 11 (id:??) landed as
https://crrev.com/22606f0c29421d6a72e46595ec2995c7771ff070
Cr-Commit-Position: refs/heads/master@{#39709}

[Michael Achenbach, 2016-09-27 07:36:23]

Impact so far:

- This fixed our arm Chromebooks in debug! They could indeed not cope with the
download sizes. Now green:
https://build.chromium.org/p/client.v8.ports/builders/V8%20Arm%20-%20debug/bu...

- Arm debug archive reduction 2.89 GiB down to 1.09 GiB - 63% space and traffic
saved on GS and isolate server! 
https://build.chromium.org/p/client.v8.ports/builders/V8%20Arm%20-%20debug%20...
vs.
https://build.chromium.org/p/client.v8.ports/builders/V8%20Arm%20-%20debug%20...

- Space reduction for some Chrome debug archives, e.g. 7.16 GiB down to 6.46 GiB
- 10% saved
https://build.chromium.org/p/chromium.linux/builders/Linux%20Builder%20%28dbg...
vs.
https://build.chromium.org/p/chromium.linux/builders/Linux%20Builder%20%28dbg...

Will switch more v8 debug bots to shared libs to save space, traffic and
upload/download/packing time on all of them.

[Michael Achenbach, 2016-09-27 10:41:08]

Moar impact on all v8 debug bots, around 70% space saved everywhere, examples:

- Linux debug: 681.9 MiB down to 216.78 MiB - 68% space saved!
https://build.chromium.org/p/client.v8/builders/V8%20Linux64%20-%20debug%20bu...
vs.
https://build.chromium.org/p/client.v8/builders/V8%20Linux64%20-%20debug%20bu...

- Linux arm64 sim debug: ~2.4 GB down to ~795.0 MB - 68% space saved!
https://chromium-swarm.appspot.com/user/task/318394bd5a5fdc10 vs.
https://chromium-swarm.appspot.com/user/task/3183e382a0e89510