Instrumented libraries: improve parallel build

Instrumented libraries: improve parallel build

This CL:
* Adds a --parallel option to build_and_package.py to build
  the 4 instrumented library configs concurrently.
* Removes parallel make.  Some libraries don't support parallel builds.
  Additionally, there were many overrides to set the number of jobs to
  1, and the only library that actually used parallel builds was
  libcap.
* Changes the -j argument to be the number of ninja jobs to use.
* Suppresses error output from nss.

NOTRY=true
BUG=None
R=eugenis@chromium.org

Review-Url: https://codereview.chromium.org/2735013004
Cr-Commit-Position: refs/heads/master@{#455307}
Committed: https://chromium.googlesource.com/chromium/src/+/a30808fd6d7f2e20278b303a2db92dc43076c974

[Tom (Use chromium acct), 2017-03-07 21:48:45]

Description was changed from

==========
Instrumented libraries: improve parallel build

BUG=
==========

to

==========
Instrumented libraries: improve parallel build

This CL:
* Adds a --parallel option to build_and_package.py to build
  the 4 instrumented library configs concurrently.
* Removes parallel make.  Some libraries don't support parallel builds.
  Additionally, there were many overrides to set the number of jobs to
  1, and the only library that actually used parallel builds was
  libcap.
* Changes the -j argument to be the number of ninja jobs to use.
* Suppresses error output from nss.

BUG=None
R=eugenis@chromium.org
==========

[Tom (Use chromium acct), 2017-03-07 21:48:45]

thomasanderson@google.com changed reviewers:
+ eugenis@chromium.org

[Tom (Use chromium acct), 2017-03-07 21:49:41]

eugenis ptal
build_and_package.py used to take several hours to run.
With this change, it takes only 15min when using
build_and_package.py -j 96 --parallel

[Evgeniy Stepanov, 2017-03-07 22:42:01]

eugenis@google.com changed reviewers:
+ eugenis@google.com

[Evgeniy Stepanov, 2017-03-07 22:42:02]

Nice. LGTM

https://codereview.chromium.org/2735013004/diff/1/third_party/instrumented_li...
File third_party/instrumented_libraries/scripts/download_build_install.py
(right):

https://codereview.chromium.org/2735013004/diff/1/third_party/instrumented_li...
third_party/instrumented_libraries/scripts/download_build_install.py:133:
self.shell_call('touch apt-source-lock')
Why not open('apt-source-lock', 'w') ?

[eugenis, 2017-03-07 22:48:39]

lgtm

[Tom (Use chromium acct), 2017-03-08 00:16:09]

Description was changed from

==========
Instrumented libraries: improve parallel build

This CL:
* Adds a --parallel option to build_and_package.py to build
  the 4 instrumented library configs concurrently.
* Removes parallel make.  Some libraries don't support parallel builds.
  Additionally, there were many overrides to set the number of jobs to
  1, and the only library that actually used parallel builds was
  libcap.
* Changes the -j argument to be the number of ninja jobs to use.
* Suppresses error output from nss.

BUG=None
R=eugenis@chromium.org
==========

to

==========
Instrumented libraries: improve parallel build

This CL:
* Adds a --parallel option to build_and_package.py to build
  the 4 instrumented library configs concurrently.
* Removes parallel make.  Some libraries don't support parallel builds.
  Additionally, there were many overrides to set the number of jobs to
  1, and the only library that actually used parallel builds was
  libcap.
* Changes the -j argument to be the number of ninja jobs to use.
* Suppresses error output from nss.

NOTRY=true
BUG=None
R=eugenis@chromium.org
==========

[Tom (Use chromium acct), 2017-03-08 00:16:13]

The CQ bit was checked by thomasanderson@google.com

[Tom (Use chromium acct), 2017-03-08 00:16:13]

The patchset sent to the CQ was uploaded after l-g-t-m from
eugenis@chromium.org, eugenis@google.com
Link to the patchset: https://codereview.chromium.org/2735013004/#ps20001
(title: "open('...', 'w')")

[commit-bot: I haz the power, 2017-03-08 00:17:21]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-03-08 00:23:25]

CQ is committing da patch.

Bot data: {"patchset_id": 20001, "attempt_start_ts": 1488932173514940,
"parent_rev": "1df13216e40fed4c8fa2531fc4c764eca1aae018", "commit_rev":
"a30808fd6d7f2e20278b303a2db92dc43076c974"}

[commit-bot: I haz the power, 2017-03-08 00:24:00]

Description was changed from

==========
Instrumented libraries: improve parallel build

This CL:
* Adds a --parallel option to build_and_package.py to build
  the 4 instrumented library configs concurrently.
* Removes parallel make.  Some libraries don't support parallel builds.
  Additionally, there were many overrides to set the number of jobs to
  1, and the only library that actually used parallel builds was
  libcap.
* Changes the -j argument to be the number of ninja jobs to use.
* Suppresses error output from nss.

NOTRY=true
BUG=None
R=eugenis@chromium.org
==========

to

==========
Instrumented libraries: improve parallel build

This CL:
* Adds a --parallel option to build_and_package.py to build
  the 4 instrumented library configs concurrently.
* Removes parallel make.  Some libraries don't support parallel builds.
  Additionally, there were many overrides to set the number of jobs to
  1, and the only library that actually used parallel builds was
  libcap.
* Changes the -j argument to be the number of ninja jobs to use.
* Suppresses error output from nss.

NOTRY=true
BUG=None
R=eugenis@chromium.org

Review-Url: https://codereview.chromium.org/2735013004
Cr-Commit-Position: refs/heads/master@{#455307}
Committed:
https://chromium.googlesource.com/chromium/src/+/a30808fd6d7f2e20278b303a2db9...
==========

[commit-bot: I haz the power, 2017-03-08 00:24:01]

Committed patchset #2 (id:20001) as
https://chromium.googlesource.com/chromium/src/+/a30808fd6d7f2e20278b303a2db9...

[Tom (Use chromium acct), 2017-03-08 00:24:59]

https://codereview.chromium.org/2735013004/diff/1/third_party/instrumented_li...
File third_party/instrumented_libraries/scripts/download_build_install.py
(right):

https://codereview.chromium.org/2735013004/diff/1/third_party/instrumented_li...
third_party/instrumented_libraries/scripts/download_build_install.py:133:
self.shell_call('touch apt-source-lock')
On 2017/03/07 22:42:02, Evgeniy Stepanov wrote:
> Why not open('apt-source-lock', 'w') ?

Done.