Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(106)

Issue 2661023010: Allow using goma on Windows with symbol_level == 2 (Closed)

Created:
3 years, 10 months ago by brucedawson
Modified:
3 years, 10 months ago
Reviewers:
Dirk Pranke, scottmg
CC:
chromium-reviews, ukai, Sébastien Marchand
Target Ref:
refs/pending/heads/master
Project:
chromium
Visibility:
Public.

Description

Allow using goma on Windows with symbol_level == 2 Traditionally goma for Chrome has been less useful on Windows than on other platforms because it was incompatible with full debug information. Building with goma requires using /Z7 instead of /Zi, and this causes the linker's memory usage and runtime to blow up as all of the debug information is merged. However, /debug:fastlink makes this work. Because it doesn't merge all of the debug information it makes goma and /Z7 practical. Full release component builds can be done in less than fifteen minutes, with incremental builds taking just a few seconds. Without goma a full release component build of Chrome can easily take 40+ minutes, even on a Z840. Goma's speedup comes from massively parallelizing the compile phase, however even with /debug:fastlink the linking phases are longer with the /Z7 switch that is required by goma. A /debug:fastlink of chrome.dll in a component build goes from ~32 seconds to ~110 seconds on a Z620 when /Z7 (goma) is selected. This penalty will be reduced by VC++ 2017 which claims 30% speed improvements on /debug:fastlink. So, if you frequently need to do full relinks then goma may still be a bad choice. However for most scenarios this change should make goma a good choice for component builds of Chrome, even with full debug information enabled. To make use of this ability you need to explicitly specify the switches below - symbol_level will otherwise default to 1 when use_goma == true. use_goma = true is_win_fastlink = true symbol_level = 2 The fastlink PDBs work well for most scenarios but there are a few scenarios (rare for most people) where they do not work: - Copying PDBs to another machine (fails due to references to the .obj files) - Reporting on all symbols/globals with SymbolSort or similar fails - ETW tracing fails - VS heap profiling fails Ideally mspdbcmf.exe would let us create "normal" PDBs when needed but in practice this appears to be unusable with /Z7 created PDBs. R=scottmg@chromium.org Review-Url: https://codereview.chromium.org/2661023010 Cr-Commit-Position: refs/heads/master@{#447892} Committed: https://chromium.googlesource.com/chromium/src/+/4a3cadb2d998fb8f477d0ebc7ff13619b123a805

Patch Set 1 #

Patch Set 2 : Fix assert placement #

Total comments: 2
Unified diffs Side-by-side diffs Delta from patch set Stats (+9 lines, -1 line) Patch
M build/config/compiler/BUILD.gn View 1 1 chunk +6 lines, -1 line 2 comments Download
M build/config/compiler/compiler.gni View 1 1 chunk +3 lines, -0 lines 0 comments Download

Messages

Total messages: 30 (14 generated)
brucedawson
I noticed that the clang team was using /Z7 + goma + /debug:fastlink to get ...
3 years, 10 months ago (2017-02-01 21:40:05 UTC) #4
brucedawson
Huh. Several build failures. These may be uncovering existing problems or maybe I need to ...
3 years, 10 months ago (2017-02-01 21:44:44 UTC) #7
scottmg
https://codereview.chromium.org/2661023010/diff/20001/build/config/compiler/BUILD.gn File build/config/compiler/BUILD.gn (right): https://codereview.chromium.org/2661023010/diff/20001/build/config/compiler/BUILD.gn#newcode1585 build/config/compiler/BUILD.gn:1585: cflags = [ "/Z7" ] # Debug information in ...
3 years, 10 months ago (2017-02-01 21:56:43 UTC) #10
grt (UTC plus 2)
https://codereview.chromium.org/2661023010/diff/20001/build/config/compiler/BUILD.gn File build/config/compiler/BUILD.gn (right): https://codereview.chromium.org/2661023010/diff/20001/build/config/compiler/BUILD.gn#newcode1585 build/config/compiler/BUILD.gn:1585: cflags = [ "/Z7" ] # Debug information in ...
3 years, 10 months ago (2017-02-01 22:06:37 UTC) #11
brucedawson
On 2017/02/01 21:56:43, scottmg wrote: > https://codereview.chromium.org/2661023010/diff/20001/build/config/compiler/BUILD.gn > File build/config/compiler/BUILD.gn (right): > > https://codereview.chromium.org/2661023010/diff/20001/build/config/compiler/BUILD.gn#newcode1585 > ...
3 years, 10 months ago (2017-02-01 22:08:55 UTC) #12
brucedawson
> 1) You are awesome, Bruce. I was just searching for goma advice and came ...
3 years, 10 months ago (2017-02-01 22:13:15 UTC) #13
scottmg
On 2017/02/01 22:08:55, brucedawson (slow) wrote: > On 2017/02/01 21:56:43, scottmg wrote: > > > ...
3 years, 10 months ago (2017-02-01 22:24:34 UTC) #14
brucedawson
Good way of looking at it Scott. I'll check in with the other dev who ...
3 years, 10 months ago (2017-02-01 22:26:28 UTC) #15
brucedawson
dpranke@ - you landed crrev.com/2296953002 that tweaked my change to have goma imply symbol_level == ...
3 years, 10 months ago (2017-02-01 22:29:58 UTC) #17
brucedawson
On 2017/02/01 22:29:58, brucedawson (slow) wrote: > dpranke@ - you landed crrev.com/2296953002 that tweaked my ...
3 years, 10 months ago (2017-02-02 00:15:38 UTC) #20
Dirk Pranke
I do care about this! What's the perf difference if you set is_win_fastlink=true on a ...
3 years, 10 months ago (2017-02-02 01:49:18 UTC) #21
brucedawson
On 2017/02/02 01:49:18, Dirk Pranke wrote: > I do care about this! > > What's ...
3 years, 10 months ago (2017-02-02 01:54:28 UTC) #22
Dirk Pranke
lgtm as-is, then, and we can continue to iterate.
3 years, 10 months ago (2017-02-02 22:29:05 UTC) #23
commit-bot: I haz the power
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.org/2661023010/20001
3 years, 10 months ago (2017-02-02 22:33:17 UTC) #26
commit-bot: I haz the power
Committed patchset #2 (id:20001) as https://chromium.googlesource.com/chromium/src/+/4a3cadb2d998fb8f477d0ebc7ff13619b123a805
3 years, 10 months ago (2017-02-03 00:09:00 UTC) #29
xdai1
3 years, 10 months ago (2017-02-03 01:09:04 UTC) #30
Message was sent while issue was closed.
A revert of this CL (patchset #2 id:20001) has been created in
https://codereview.chromium.org/2669373002/ by xdai@chromium.org.

The reason for reverting is: It broke amd64-generic Trusty informational build
at generate_build_files:
ERROR at //build/config/compiler/compiler.gni:115:3: Assertion failed.
  assert(is_win_fastlink || !use_goma,
  ^-----
Goma builds that use symbol_level 2 must use is_win_fastlink.
See //BUILD.gn:11:1: whence it was imported.
import("//build/config/compiler/compiler.gni")
^--------------------------------------------
GN gen failed: 1
step returned non-zero exit code: 1
@@@STEP_FAILURE@@@

See crbug.com/688203 for more info..

Powered by Google App Engine
This is Rietveld 408576698