Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(89)

Issue 27418008: ninja: Write gypcmd files into the output directory instead of the tree. (Closed)

Created:
7 years, 2 months ago by Nico
Modified:
6 years, 9 months ago
Reviewers:
bradn, scottmg
CC:
gyp-developer_googlegroups.com, bradnelson
Visibility:
Public.

Description

ninja: Write gypcmd files into the output directory instead of the tree. gyp r789 added a filelist feature, where <|(file.foo arg1 arg2 arg3) would write "arg1 arg2 arg3" into file "file.foo" and evaluate to "file.foo". This is used to write very long commands into files to work around command line length limitations. The files written by this are just written into the source tree. (They are only written if they are different from the previous version of the file.) One problem with this is that the filenames are free form and so can't be reliably added to .gitignore -- and so our bots delete them all on sync, because they delete unknown files. So build edges using gyp filelists are always dirty on the bots. The filelist stuff is evaluated before generators run, and -- worse -- most generators rely on the build system creating the build directory at build time. Most build systems have a facility to change the build directory for every build (setting SYMROOT for xcodebuild, builddir for make, OutputDirectory for msvs). So it's not trivial to write filelist files into the build directory. For ninja, the output directory is known at gyp file however. So add an optional generator_file_path() function that generators can use to map filelist paths to paths in the build directory, and implement that for ninja. That way, gypcmd files won't clutter the src directory with the ninja generator at least. This should speed up incremental builds on most bots (== all bots using ninja) by a few minutes. It also makes filelists with work better with -Goutput_dir, as each output_dir now has an independent set of filelists. The make generator could use --generator-output-dir to do something similar, if someone feels motivated. Also add a better test for filelists. The existing test only used the gypd generator and compared to a golden file, which didn't catch several bugs that I introduced while writing this. The new test caught them all. BUG=chromium:297186, gyp:316 R=scottmg@chromium.org Committed: https://code.google.com/p/gyp/source/detail?r=1762

Patch Set 1 #

Patch Set 2 : #

Patch Set 3 : #

Patch Set 4 : #

Patch Set 5 : #

Patch Set 6 : #

Patch Set 7 : #

Total comments: 1

Patch Set 8 : #

Total comments: 5

Patch Set 9 : #

Patch Set 10 : #

Patch Set 11 : #

Unified diffs Side-by-side diffs Delta from patch set Stats (+150 lines, -44 lines) Patch
M pylib/gyp/__init__.py View 1 2 3 4 5 6 7 8 9 1 chunk +2 lines, -0 lines 0 comments Download
M pylib/gyp/generator/ninja.py View 1 2 3 4 5 6 7 8 9 3 chunks +34 lines, -10 lines 0 comments Download
M pylib/gyp/input.py View 1 2 3 4 5 6 7 8 9 3 chunks +11 lines, -4 lines 0 comments Download
M test/variables/filelist/gyptest-filelist.py View 1 2 3 4 5 6 7 8 9 10 1 chunk +7 lines, -30 lines 0 comments Download
A test/variables/filelist/gyptest-filelist-golden.py View 1 2 3 4 1 chunk +51 lines, -0 lines 0 comments Download
A test/variables/filelist/src/dummy.py View 1 2 3 4 1 chunk +5 lines, -0 lines 0 comments Download
A test/variables/filelist/src/filelist2.gyp View 1 2 3 4 5 6 7 8 1 chunk +40 lines, -0 lines 0 comments Download

Messages

Total messages: 7 (0 generated)
Nico
Fear not, the CL description is the longest part of this patch. I haven't tried ...
7 years, 2 months ago (2013-10-16 22:36:25 UTC) #1
Nico
https://codereview.chromium.org/27418008/diff/15001/test/variables/filelist/gyptest-filelist-golden.py File test/variables/filelist/gyptest-filelist-golden.py (right): https://codereview.chromium.org/27418008/diff/15001/test/variables/filelist/gyptest-filelist-golden.py#newcode8 test/variables/filelist/gyptest-filelist-golden.py:8: Test variable expansion of '<|(list.txt ...)' syntax commands. (This ...
7 years, 2 months ago (2013-10-16 22:40:40 UTC) #2
scottmg
https://codereview.chromium.org/27418008/diff/24001/pylib/gyp/generator/ninja.py File pylib/gyp/generator/ninja.py (right): https://codereview.chromium.org/27418008/diff/24001/pylib/gyp/generator/ninja.py#newcode1471 pylib/gyp/generator/ninja.py:1471: # relative path from source root to our output ...
7 years, 2 months ago (2013-10-16 23:29:48 UTC) #3
Nico
ptal! https://codereview.chromium.org/27418008/diff/24001/pylib/gyp/generator/ninja.py File pylib/gyp/generator/ninja.py (right): https://codereview.chromium.org/27418008/diff/24001/pylib/gyp/generator/ninja.py#newcode1471 pylib/gyp/generator/ninja.py:1471: # relative path from source root to our ...
7 years, 2 months ago (2013-10-17 00:00:26 UTC) #4
scottmg
lgtm
7 years, 2 months ago (2013-10-17 00:01:35 UTC) #5
Nico
Committed patchset #11 manually as r1762 (presubmit successful).
7 years, 2 months ago (2013-10-17 01:17:13 UTC) #6
bradn
7 years, 2 months ago (2013-10-17 03:41:59 UTC) #7
Message was sent while issue was closed.
LGTM
Was out today.
Thanks for fixing this!!

Powered by Google App Engine
This is Rietveld 408576698