(reland) Optimize check_gn_headers.py for speed

(reland) Optimize check_gn_headers.py for speed

- Process the output of 'ninja' live through the pipe rather than
  waiting for it to finish.
- Use multiprocessing.

Benchmark results using pypy:
Before:
17.10user 14.38system 0:20.05elapsed 157%CPU (0avgtext+0avgdata 1708036maxresident)k
18.06user 15.63system 0:21.92elapsed 153%CPU (0avgtext+0avgdata 1710912maxresident)k
17.94user 13.72system 0:19.91elapsed 158%CPU (0avgtext+0avgdata 1172128maxresident)k

After:
15.28user 7.11system 0:06.25elapsed 357%CPU (0avgtext+0avgdata 319236maxresident)k
16.03user 7.74system 0:06.84elapsed 347%CPU (0avgtext+0avgdata 319396maxresident)k
15.24user 8.16system 0:06.36elapsed 367%CPU (0avgtext+0avgdata 318272maxresident)k

BUG=661774
Review-Url: https://codereview.chromium.org/2842513003
Cr-Original-Commit-Position: refs/heads/master@{#466900}
Review-Url: https://codereview.chromium.org/2846473002
Cr-Commit-Position: refs/heads/master@{#467604}
Committed: https://chromium.googlesource.com/chromium/src/+/ef74ec9973e071da2dc5813f0b93662a0f1497bf

[wychen, 2017-04-26 14:20:47]

wychen@chromium.org changed reviewers:
+ dpranke@chromium.org

[wychen, 2017-04-26 14:20:47]

PTAL

[wychen, 2017-04-26 14:21:01]

Description was changed from

==========
Optimize check_gn_headers.py for speed

- Process the output of 'ninja' live through the pipe rather than
  waiting for it to finish.
- Use multiprocessing.

Benchmark results using pypy:
Before:
17.10user 14.38system 0:20.05elapsed 157%CPU (0avgtext+0avgdata
1708036maxresident)k
18.06user 15.63system 0:21.92elapsed 153%CPU (0avgtext+0avgdata
1710912maxresident)k
17.94user 13.72system 0:19.91elapsed 158%CPU (0avgtext+0avgdata
1172128maxresident)k

After:
15.28user 7.11system 0:06.25elapsed 357%CPU (0avgtext+0avgdata
319236maxresident)k
16.03user 7.74system 0:06.84elapsed 347%CPU (0avgtext+0avgdata
319396maxresident)k
15.24user 8.16system 0:06.36elapsed 367%CPU (0avgtext+0avgdata
318272maxresident)k

BUG=661774

Review-Url: https://codereview.chromium.org/2842513003
Cr-Commit-Position: refs/heads/master@{#466900}
==========

to

==========
(reland) Optimize check_gn_headers.py for speed

- Process the output of 'ninja' live through the pipe rather than
  waiting for it to finish.
- Use multiprocessing.

Benchmark results using pypy:
Before:
17.10user 14.38system 0:20.05elapsed 157%CPU (0avgtext+0avgdata
1708036maxresident)k
18.06user 15.63system 0:21.92elapsed 153%CPU (0avgtext+0avgdata
1710912maxresident)k
17.94user 13.72system 0:19.91elapsed 158%CPU (0avgtext+0avgdata
1172128maxresident)k

After:
15.28user 7.11system 0:06.25elapsed 357%CPU (0avgtext+0avgdata
319236maxresident)k
16.03user 7.74system 0:06.84elapsed 347%CPU (0avgtext+0avgdata
319396maxresident)k
15.24user 8.16system 0:06.36elapsed 367%CPU (0avgtext+0avgdata
318272maxresident)k

BUG=661774

Review-Url: https://codereview.chromium.org/2842513003
Cr-Commit-Position: refs/heads/master@{#466900}
==========

[Dirk Pranke, 2017-04-26 19:26:38]

lgtm

[wychen, 2017-04-27 06:17:32]

The CQ bit was checked by wychen@chromium.org

[commit-bot: I haz the power, 2017-04-27 06:17:48]

CQ is trying da patch.

Follow status at:
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-04-27 06:28:28]

CQ is committing da patch.

Bot data: {"patchset_id": 40001, "attempt_start_ts": 1493273852548890,
"parent_rev": "e965dad365db87339dfcfdf7d2d98c7eff17712c", "commit_rev":
"ef74ec9973e071da2dc5813f0b93662a0f1497bf"}

[commit-bot: I haz the power, 2017-04-27 06:29:07]

Description was changed from

==========
(reland) Optimize check_gn_headers.py for speed

- Process the output of 'ninja' live through the pipe rather than
  waiting for it to finish.
- Use multiprocessing.

Benchmark results using pypy:
Before:
17.10user 14.38system 0:20.05elapsed 157%CPU (0avgtext+0avgdata
1708036maxresident)k
18.06user 15.63system 0:21.92elapsed 153%CPU (0avgtext+0avgdata
1710912maxresident)k
17.94user 13.72system 0:19.91elapsed 158%CPU (0avgtext+0avgdata
1172128maxresident)k

After:
15.28user 7.11system 0:06.25elapsed 357%CPU (0avgtext+0avgdata
319236maxresident)k
16.03user 7.74system 0:06.84elapsed 347%CPU (0avgtext+0avgdata
319396maxresident)k
15.24user 8.16system 0:06.36elapsed 367%CPU (0avgtext+0avgdata
318272maxresident)k

BUG=661774

Review-Url: https://codereview.chromium.org/2842513003
Cr-Commit-Position: refs/heads/master@{#466900}
==========

to

==========
(reland) Optimize check_gn_headers.py for speed

- Process the output of 'ninja' live through the pipe rather than
  waiting for it to finish.
- Use multiprocessing.

Benchmark results using pypy:
Before:
17.10user 14.38system 0:20.05elapsed 157%CPU (0avgtext+0avgdata
1708036maxresident)k
18.06user 15.63system 0:21.92elapsed 153%CPU (0avgtext+0avgdata
1710912maxresident)k
17.94user 13.72system 0:19.91elapsed 158%CPU (0avgtext+0avgdata
1172128maxresident)k

After:
15.28user 7.11system 0:06.25elapsed 357%CPU (0avgtext+0avgdata
319236maxresident)k
16.03user 7.74system 0:06.84elapsed 347%CPU (0avgtext+0avgdata
319396maxresident)k
15.24user 8.16system 0:06.36elapsed 367%CPU (0avgtext+0avgdata
318272maxresident)k

BUG=661774

Review-Url: https://codereview.chromium.org/2842513003
Cr-Original-Commit-Position: refs/heads/master@{#466900}
Review-Url: https://codereview.chromium.org/2846473002
Cr-Commit-Position: refs/heads/master@{#467604}
Committed:
https://chromium.googlesource.com/chromium/src/+/ef74ec9973e071da2dc5813f0b93...
==========

[commit-bot: I haz the power, 2017-04-27 06:29:08]

Committed patchset #3 (id:40001) as
https://chromium.googlesource.com/chromium/src/+/ef74ec9973e071da2dc5813f0b93...