Chromium Code Reviews
|
|
Chromium Code Reviews
Issue
2559563002:
Limit the number of thinlto jobs to 8; also support limits in lld. (Closed)
DescriptionLimit the number of thinlto jobs to 8; also support limits in lld.
BUG=660216
Committed: https://crrev.com/a170f5836a8db250bdd60bb44c89d170f819090d
Cr-Commit-Position: refs/heads/master@{#436805}
Patch Set 1 #Patch Set 2 : improve a comment #
Total comments: 1
Messages
Total messages: 24 (10 generated)
krasin@chromium.org changed reviewers: + thakis@chromium.org
https://codereview.chromium.org/2559563002/diff/20001/build/config/compiler/B... File build/config/compiler/BUILD.gn (right): https://codereview.chromium.org/2559563002/diff/20001/build/config/compiler/B... build/config/compiler/BUILD.gn:448: # resource allocation scheme, but should be good enough. Shouldn't the OS scheduler be ok with linker threads being oversubscribed, as long as there's not a lot of memory use per thread? (Is there?)
On 2016/12/06 21:08:48, Nico wrote: > https://codereview.chromium.org/2559563002/diff/20001/build/config/compiler/B... > File build/config/compiler/BUILD.gn (right): > > https://codereview.chromium.org/2559563002/diff/20001/build/config/compiler/B... > build/config/compiler/BUILD.gn:448: # resource allocation scheme, but should be > good enough. > Shouldn't the OS scheduler be ok with linker threads being oversubscribed, as > long as there's not a lot of memory use per thread? (Is there?) ThinLTO jobs are about 8x less memory-hungry than FullLTO. So, it's mostly okay. At the same time, the linker is moving a lot of bytes around, so there is definitely some overhead there. I have not measured it scientifically, though. Another argument for going to -jobs=8 would be my benchmarking of ThinLTO scaling: https://docs.google.com/spreadsheets/d/18vi9p8ffIYNVPTyxtJwr-YrP4WJRbaQr_2nZ1... With -jobs=8 it takes 10% less user secs to link chrome than with -jobs=8. The wall time is still better for -jobs=16, but most of our bots have enough targets to utilize all cores anyway.
On 2016/12/06 21:15:55, krasin1 wrote: > On 2016/12/06 21:08:48, Nico wrote: > > > https://codereview.chromium.org/2559563002/diff/20001/build/config/compiler/B... > > File build/config/compiler/BUILD.gn (right): > > > > > https://codereview.chromium.org/2559563002/diff/20001/build/config/compiler/B... > > build/config/compiler/BUILD.gn:448: # resource allocation scheme, but should > be > > good enough. > > Shouldn't the OS scheduler be ok with linker threads being oversubscribed, as > > long as there's not a lot of memory use per thread? (Is there?) > ThinLTO jobs are about 8x less memory-hungry than FullLTO. So, it's mostly okay. > > At the same time, the linker is moving a lot of bytes around, so there is > definitely some overhead there. sorry, I missed a part of the sentence above. Please read: "At the same time, the linker is moving a lot of bytes around, so there is definitely some overhead there, when the context switch happens and all those caches are wiped" > I have not measured it scientifically, though. > > Another argument for going to -jobs=8 would be my benchmarking of ThinLTO > scaling: > https://docs.google.com/spreadsheets/d/18vi9p8ffIYNVPTyxtJwr-YrP4WJRbaQr_2nZ1... > > With -jobs=8 it takes 10% less user secs to link chrome than with -jobs=8. The > wall time is still better for -jobs=16, but most of our bots have enough targets > to utilize all cores anyway.
lgtm
The CQ bit was checked by krasin@chromium.org
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Try jobs failed on following builders: ios-device on master.tryserver.chromium.mac (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.mac/builders/ios-device/builds...)
The CQ bit was checked by krasin@chromium.org
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Try jobs failed on following builders: ios-device on master.tryserver.chromium.mac (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.mac/builders/ios-device/builds...)
The CQ bit was checked by krasin@chromium.org
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Try jobs failed on following builders: ios-device on master.tryserver.chromium.mac (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.mac/builders/ios-device/builds...)
The CQ bit was checked by krasin@chromium.org
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
CQ is committing da patch.
Bot data: {"patchset_id": 20001, "attempt_start_ts": 1481070123818370,
"parent_rev": "e785d0bac9cd21976fe80352847040e1bc8a3586", "commit_rev":
"4edbd22c890c4f4b14471de8fe4bda6a3b449afc"}
Message was sent while issue was closed.
Committed patchset #2 (id:20001)
Message was sent while issue was closed.
Description was changed from ========== Limit the number of thinlto jobs to 8; also support limits in lld. BUG=660216 ========== to ========== Limit the number of thinlto jobs to 8; also support limits in lld. BUG=660216 Committed: https://crrev.com/a170f5836a8db250bdd60bb44c89d170f819090d Cr-Commit-Position: refs/heads/master@{#436805} ==========
Message was sent while issue was closed.
Patchset 2 (id:??) landed as https://crrev.com/a170f5836a8db250bdd60bb44c89d170f819090d Cr-Commit-Position: refs/heads/master@{#436805} |
