ThinLTO: limit link / codegen parallelism to 16.

build/config/compiler/BUILD.gn

 # Copyright (c) 2013 The Chromium Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
 # found in the LICENSE file.
 
 import("//build/config/android/config.gni")
 import("//build/config/chrome_build.gni")
 import("//build/config/compiler/compiler.gni")
 import("//build/config/nacl/config.gni")
 import("//build/toolchain/cc_wrapper.gni")
 import("//build/toolchain/toolchain.gni")
@@ skipping 417 matching lines @@
   # time, so they need to be applied to all translation units, and we may end up
   # with miscompiles if only part of the program is compiled with LTO flags. For
   # that reason, we cannot allow targets to enable or disable these flags, for
   # example by disabling the optimize configuration.
   # TODO(pcc): Make this conditional on is_official_build rather than on gn
   # flags for specific features.
   if (!is_debug && (allow_posix_link_time_opt || is_cfi) && !is_nacl) {
     if (use_thin_lto) {
       cflags += [ "-flto=thin" ]
       ldflags += [ "-flto=thin" ]
+
+      if (is_linux && !use_lld) {
+        # Limit the parallelism to the sweet spot on most of the machines.
+        # As of now, ThinLTO does scale beyond 16 cores anyway.

[dpranke, 2016/09/15 03:35:20]

I think you mean "does not scale", right?

Does this flag impose a maximum number of jobs, or does it impose a fixed
number, i.e., would we still try to use 16 jobs on a 4-core machine?

Also, is the if (...) check even needed? At this time won't this always be true?

[krasin1, 2016/09/15 03:43:42]

right. Fixed.
Jobs is not really a good name here. It's actually a number of threads that the
linker will start to generate native code from the bitcode. It will still be a
single process (ld.gold with LLVMgold.so loaded) with 16 threads.

It's perfectly fine to run 16 threads on 4-core machine. While the performance
might be 10%-15% lower, I have verified locally that -jobs=48 and one core
(limited with schedtool -a 1) works reasonably fast.
with lld (which we aim to support), the flag will likely be -Wl,--job=16, but
it's not clear yet, as it's only to be implemented. On Mac, the flag will be
different as well.

So, this check is to avoid a time bomb: even though we only test ThinLTO with
Linux+Gold plugin now, I would like ThinLTO to still work for other options too,
may be not as fast as it could be with the proper limit.

+        ldflags += [ "-Wl,-plugin-opt,jobs=16" ]
+      }
     } else {
       # Note: ThinLTO does not currently have this feature implemented
       # For Full LTO, it provides a measurable runtime speedup of Chrome.
       cflags += [
         "-flto",
         "-fwhole-program-vtables",
       ]
       ldflags += [
         "-flto",
         "-fwhole-program-vtables",
@@ skipping 1140 matching lines @@
 
 if (is_ios || is_mac) {
   # On Mac and iOS, this enables support for ARC (automatic ref-counting).
   # See http://clang.llvm.org/docs/AutomaticReferenceCounting.html.
   config("enable_arc") {
     common_flags = [ "-fobjc-arc" ]
     cflags_objc = common_flags
     cflags_objcc = common_flags
   }
 }