GN: Don't import a file more than once.

GN: Don't import a file more than once.

GN caches the results of imports so we don't have to load them more than once.
But if two BUILD files load the same import at the same time, there is a race

Previously GN would resolve the winner after the import ran on the assumption
that imports are fast and collisions are rare. But some imports run expensive
scripts meaning they take a long time and collisions are likely. The result is
that a file can get imported more than once.

This patch adds extra locking around the import to block other threads and
only do the import once.

Increase the minimum thread count for low-end systems to 8. This gives the fastest running time on an older MacBook.

Both the import change and the thread change independently speed up GN on Mac Chrome by 150-300ms, so actual improvement should be 300-600ms. On beefy Windows and Linux workstations, there seems to be no change in performance with these changes, although the import change should help some worst-case situations.

BUG=599892
Committed: https://crrev.com/a5b852cfbe6b6912ee6d1af145cbe9c090e40b42
Cr-Commit-Position: refs/heads/master@{#392353}

[brettw, 2016-05-06 21:27:26]

Description was changed from

==========
GN: Don't import a file more than once.

GN caches the results of imports so we don't have to load them more than once.
But if two BUILD files load the same import at the same time, there is a race

Previously GN would resolve the winner after the import ran on the assumption
that imports are fast and collisions are rare. But some imports run expensive
scripts meaning they take a long time and collisions are likely. The result is
that a file can get imported more than once.

This patch adds extra locking around the import to block other threads and
only do the import once.

BUG=599892
==========

to

==========
GN: Don't import a file more than once.

GN caches the results of imports so we don't have to load them more than once.
But if two BUILD files load the same import at the same time, there is a race

Previously GN would resolve the winner after the import ran on the assumption
that imports are fast and collisions are rare. But some imports run expensive
scripts meaning they take a long time and collisions are likely. The result is
that a file can get imported more than once.

This patch adds extra locking around the import to block other threads and
only do the import once.

Increase the minimum thread count for low-end systems to 8. This gives the
fastest running time on an older MacBook.

Both the import change and the thread change independently speed up GN on Mac
Chrome by 150-300ms, so actual improvement should be 300-600ms. On beefy Windows
and Linux workstations, there seems to be no change in performance with these
changes, although the import change should help some worst-case situations.

BUG=599892
==========

[brettw, 2016-05-06 21:27:26]

brettw@chromium.org changed reviewers:
+ thakis@chromium.org

[brettw, 2016-05-06 21:28:02]

The CQ bit was checked by brettw@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-05-06 21:28:27]

Dry run: CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1957483004/20001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1957483004/20001

[Nico, 2016-05-06 21:33:51]

lgtm

https://codereview.chromium.org/1957483004/diff/20001/tools/gn/scheduler.cc
File tools/gn/scheduler.cc (right):

https://codereview.chromium.org/1957483004/diff/20001/tools/gn/scheduler.cc#n...
tools/gn/scheduler.cc:60: // The minimum thread count is based on the optiman
threads for the Chrome
typo (or super hero?) optiman

[commit-bot: I haz the power, 2016-05-06 22:23:12]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-05-06 22:23:13]

Dry run: This issue passed the CQ dry run.

[brettw, 2016-05-09 17:02:26]

typo

[brettw, 2016-05-09 17:02:27]

The CQ bit was checked by brettw@chromium.org

[brettw, 2016-05-09 17:02:27]

The patchset sent to the CQ was uploaded after l-g-t-m from thakis@chromium.org
Link to the patchset: https://codereview.chromium.org/1957483004/#ps40001
(title: "typo")

[commit-bot: I haz the power, 2016-05-09 17:03:00]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1957483004/40001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1957483004/40001

[commit-bot: I haz the power, 2016-05-09 17:41:09]

Description was changed from

==========
GN: Don't import a file more than once.

GN caches the results of imports so we don't have to load them more than once.
But if two BUILD files load the same import at the same time, there is a race

Previously GN would resolve the winner after the import ran on the assumption
that imports are fast and collisions are rare. But some imports run expensive
scripts meaning they take a long time and collisions are likely. The result is
that a file can get imported more than once.

This patch adds extra locking around the import to block other threads and
only do the import once.

Increase the minimum thread count for low-end systems to 8. This gives the
fastest running time on an older MacBook.

Both the import change and the thread change independently speed up GN on Mac
Chrome by 150-300ms, so actual improvement should be 300-600ms. On beefy Windows
and Linux workstations, there seems to be no change in performance with these
changes, although the import change should help some worst-case situations.

BUG=599892
==========

to

==========
GN: Don't import a file more than once.

GN caches the results of imports so we don't have to load them more than once.
But if two BUILD files load the same import at the same time, there is a race

Previously GN would resolve the winner after the import ran on the assumption
that imports are fast and collisions are rare. But some imports run expensive
scripts meaning they take a long time and collisions are likely. The result is
that a file can get imported more than once.

This patch adds extra locking around the import to block other threads and
only do the import once.

Increase the minimum thread count for low-end systems to 8. This gives the
fastest running time on an older MacBook.

Both the import change and the thread change independently speed up GN on Mac
Chrome by 150-300ms, so actual improvement should be 300-600ms. On beefy Windows
and Linux workstations, there seems to be no change in performance with these
changes, although the import change should help some worst-case situations.

BUG=599892
==========

[commit-bot: I haz the power, 2016-05-09 17:41:10]

Committed patchset #3 (id:40001)

[commit-bot: I haz the power, 2016-05-09 17:42:37]

Description was changed from

==========
GN: Don't import a file more than once.

GN caches the results of imports so we don't have to load them more than once.
But if two BUILD files load the same import at the same time, there is a race

Previously GN would resolve the winner after the import ran on the assumption
that imports are fast and collisions are rare. But some imports run expensive
scripts meaning they take a long time and collisions are likely. The result is
that a file can get imported more than once.

This patch adds extra locking around the import to block other threads and
only do the import once.

Increase the minimum thread count for low-end systems to 8. This gives the
fastest running time on an older MacBook.

Both the import change and the thread change independently speed up GN on Mac
Chrome by 150-300ms, so actual improvement should be 300-600ms. On beefy Windows
and Linux workstations, there seems to be no change in performance with these
changes, although the import change should help some worst-case situations.

BUG=599892
==========

to

==========
GN: Don't import a file more than once.

GN caches the results of imports so we don't have to load them more than once.
But if two BUILD files load the same import at the same time, there is a race

Previously GN would resolve the winner after the import ran on the assumption
that imports are fast and collisions are rare. But some imports run expensive
scripts meaning they take a long time and collisions are likely. The result is
that a file can get imported more than once.

This patch adds extra locking around the import to block other threads and
only do the import once.

Increase the minimum thread count for low-end systems to 8. This gives the
fastest running time on an older MacBook.

Both the import change and the thread change independently speed up GN on Mac
Chrome by 150-300ms, so actual improvement should be 300-600ms. On beefy Windows
and Linux workstations, there seems to be no change in performance with these
changes, although the import change should help some worst-case situations.

BUG=599892

Committed: https://crrev.com/a5b852cfbe6b6912ee6d1af145cbe9c090e40b42
Cr-Commit-Position: refs/heads/master@{#392353}
==========

[commit-bot: I haz the power, 2016-05-09 17:42:38]

Patchset 3 (id:??) landed as
https://crrev.com/a5b852cfbe6b6912ee6d1af145cbe9c090e40b42
Cr-Commit-Position: refs/heads/master@{#392353}