Use GNU ar's thin archive if this feature is available....

Use GNU ar's thin archive if this feature is available.

The thin archives don't have actual copy of object files. Instead, they have the paths to object files like symbolic links. With this option, ar and ranlib in chromium build become much faster on linux.

Before this change:

Total size of .a files: 1726537946 (1726M)
./ar.sh  8.73s user 12.28s system 26% cpu 1:19.24 total
./link.sh  9.14s user 4.54s system 14% cpu 1:36.01 total

After this change:

Total size of .a files: 43157870 (43M)
./ar.sh  7.77s user 1.75s system 97% cpu 9.797 total
./link.sh  9.10s user 5.26s system 21% cpu 1:06.18 total

I'm not sure why link also becomes faster. Maybe other processes were running when I ran link.sh. The link.sh just links chrome and the ar.sh is generated by grepping our build process. The actual file is here: http://shinh.skr.jp/t/ar.sh.txt

Unfortunately, this feature is relatively new (2008-03-27, http://sourceware.org/ml/binutils/2008-03/msg00150.html), so I think we cannot expect everyone have this feature. My tiny sed script detects if ar --help has the line like

  [T]          - make a thin archive

So, we won't use this option if we cannot find this line.

[hamaji, 2010-06-16 06:24:40]

Hi Evan, I'm asking you to review this patch because IIRC you are the man behind
chromium's make build. Please let me know if you know other reviewer.

I couldn't run try jobs properly for this change, but I confirmed 1) I can build
out/Debug/chrome with this change and the size of the binary doesn't change and
2) my sed script correctly returns null string on my Mac OSX, where thin archive
isn't supported. Please let me know if there are something I should check.

Thanks!

[hamaji, 2010-06-22 08:03:20]

> Does the sed only work in English locale? Maybe better to run ar with flag and
check exit code to test

Very good point. Thanks for your review. I've noticed we should check the linker
as well as the ar command. I've updated the patch so now we check the exit code
of ar and ld. As this check would take one second or so, the updated Makefile
runs the shell commands only once per a target. Could you take another look?

Thanks!

[Evan Martin, 2010-06-22 15:55:43]

http://codereview.chromium.org/2841004/diff/3001/4001
File pylib/gyp/generator/make.py (right):

http://codereview.chromium.org/2841004/diff/3001/4001#newcode126
pylib/gyp/generator/make.py:126: # We don't want to run the detection multiple
times. So, we
I think as you've written it we'll do the above test every time you build --
like each time you run "make" while developing.  I've put a lot of effort into
making that quick, so I'd really like to avoid doing this too often.

Here's an idea:
- Make it so detect_arflags writes out a .mk file on the side that sets
ARFLAGS.target for you
- Then use 'include Makefile.arflags' or whatever here

http://codereview.chromium.org/2841004/diff/3001/4001#newcode128
pylib/gyp/generator/make.py:128: # - export the variable to our sub-makes, and
We only one run Make (it's all one file), so this comment isn't correct.

[hamaji, 2010-06-23 07:59:59]

> I think as you've written it we'll do the above test every time you build --
> like each time you run "make" while developing.  I've put a lot of effort into
> making that quick, so I'd really like to avoid doing this too often.

Nice suggestion. I've modified my patch so we don't run the detection twice if
you don't remove the "out" directory.

> http://codereview.chromium.org/2841004/diff/3001/4001#newcode128
> pylib/gyp/generator/make.py:128: # - export the variable to our sub-makes, and
> We only one run Make (it's all one file), so this comment isn't correct.

Ah, I see. I removed this comment and unnecessary export. I guessed we are
running sub-makes because I saw $(MAKE) in app/app.Makefile, but I've noticed
this file seems to be another root Makefile so this will never run another
*.Makefile.

[Evan Martin, 2010-06-23 14:44:10]

Awesome!  LGTM

[hamaji, 2010-06-23 15:08:07]

On 2010/06/23 14:44:10, Evan Martin wrote:
> Awesome!  LGTM

Thanks! Could you commit this on behalf of me? I don't think I'm a committer of
gyp.

[TVL, 2010-06-25 17:34:15]

What happens in the builder/tester setup?  The testers will download these
binaries and not have the .o files, so will we stop getting stacks on testers
when things crash/assert?  (i think this also would include the valgrind setup)

[Evan Martin, 2010-06-25 17:36:41]

Build happens in three steps:
.o => copied to .a => linked to executable
The executable is what contains all the symbols etc needed for debugging.  This
patch just simplifies the .a files, which aren't part of the resulting build.

[TVL, 2010-06-25 17:40:25]

On 2010/06/25 17:36:41, Evan Martin wrote:
> Build happens in three steps:
> .o => copied to .a => linked to executable
> The executable is what contains all the symbols etc needed for debugging. 
This
> patch just simplifies the .a files, which aren't part of the resulting build.

Ah ok.