Add gomadir for GYP_DEFINES

Add gomadir for GYP_DEFINES

If gomadir is specified in GYP_DEFINES (and use_goma=1, which is default),
it will start goma compiler_proxy and generates build rules file to use goma,
so no need to fix PATH when running ninja (or make).
Note: gomadir should not be in PATH with this use case.

Currently, it is for ninja (and for make in some cases).

If ANDROID_GOMA_WRAPPER is defined, its directory will be used as
gomadir's default (for backward compatibility).

Example 1

 $ GYP_DEFINES=gomadir=/path/to/goma build/gyp_chromium

build.ninja will have
  cc = /path/to/goma/gomacc gcc
  cxx = /path/to/goma/gomacc g++

user can run ninja without setting PATH or so.
 $ ninja -C out/Release -j100

It doesn't generate goma-ready Makefile yet in this case
(because no CC in make_global_settings)

Example 2
 $ GYP_DEFINES="clang=1 gomadir=/path/to/goma" build/gyp_chromium
(no need to set CC/CXX at this stage)

build.ninja will have
  cc = /path/to/goma/gomacc ../../third_party/llvm-build/Release+Asserts/bin/clang
  cxx = /path/to/goma/gomacc ../../third_party/llvm-build/Release+Asserts/bin/clang++

user can run ninja without setting PATH or so.
 $ ninja -C out/Release -j100

Makefile will be
 ifneq (,$(filter $(origin CC), undefined default))
  CC = $(abspath /usr/local/google/home/ukai/src/goma/client/out/Release/gomacc)
     $(abspath third_party/llvm-build/Release+Asserts/bin/clang)
 endif
 ifneq (,$(filter $(origin CXX), undefined default))
    CXX = $(abspath /usr/local/google/home/ukai/src/goma/client/out/Release/gomacc)
     $(abspath third_party/llvm-build/Release+Asserts/bin/clang++)
 endif
 ..
 CC.target = $(CC)
 CXX.target = $(CXX)

user can run make without setting PATH or so.
 $ make BUILDTYPE=Release -j100

R=iannucci@chromium.org,thakis@chromium.org,michaelbai@google.com,peter@chromium.org
BUG=173686

[ukai, 2013-03-07 05:38:41]

ping

[Nico, 2013-03-07 17:15:49]

When picking multiple reviewers, it's usually a good idea to tell everyone
what they should look at. Else, everyone might be waiting for someone else
to comment first, which is apparently what is happening here :-)


On Wed, Mar 6, 2013 at 9:38 PM, <ukai@chromium.org> wrote:

> ping
>
>
>
>
https://codereview.chromium.**org/12419003/<https://codereview.chromium.org/1...
>

[ukai, 2013-03-08 03:42:16]

On 2013/03/07 17:15:49, Nico wrote:
> When picking multiple reviewers, it's usually a good idea to tell everyone
> what they should look at. Else, everyone might be waiting for someone else
> to comment first, which is apparently what is happening here :-)

iannuncci: please review this would (be a step to) fix the issue 173686
thakis: please check it won't break use case with clang.
michaelbai,peter: please check it won't break use case on android.

thanks
> 
> 
> On Wed, Mar 6, 2013 at 9:38 PM, <mailto:ukai@chromium.org> wrote:
> 
> > ping
> >
> >
> >
> >
>
https://codereview.chromium.**org/12419003/%3Chttps://codereview.chromium.org...>
> >

[iannucci, 2013-03-08 04:04:09]

https://codereview.chromium.org/12419003/diff/1/build/common.gypi
File build/common.gypi (right):

https://codereview.chromium.org/12419003/diff/1/build/common.gypi#newcode662
build/common.gypi:662: '<!(python <(gomadir)/goma_ctl.py ensure_start >/dev/null
2>/dev/null && echo ok)'
It's really too bad we can't just use goma_ctl for both win and !win..

Also, I don't think >/dev/null is going to work how you expect on windows :).
Maybe add a --print-ok flag to goma_ctl?

I suppose it's OK to setup goma at gyptime... users will need to be educated
about this though. If I reboot my machine and try to build, I'll have a broken
config.

The BEST thing would be to generate an action which everything depends on
implicitly (I'm thinking: inject a dependency into everything with a global
target_defaults). If the goma setup fails at compile time, the user would need
to re-gyp with goma off (or try the build again after fixing whatever prevented
goma from starting).

https://codereview.chromium.org/12419003/diff/1/build/common.gypi#newcode1066
build/common.gypi:1066: 'fastbuild': 1,
At first I didn't like this, but now that I think about it, this seems pretty
good since as soon as goma supports PCH and/or PDB, we can just change it here
in the gypfile instead of on all the masters (and devs will get support for
these features too). Also, if we discover some optional feature doesn't work
with goma, we can disable it here too.

https://codereview.chromium.org/12419003/diff/1/build/common.gypi#newcode4114
build/common.gypi:4114: #  in make_global_settings yet.
AFAIK, nothing uses goma with Make currently, and we don't plan to in the
future.

[ukai, 2013-03-08 06:52:05]

https://codereview.chromium.org/12419003/diff/1/build/common.gypi
File build/common.gypi (right):

https://codereview.chromium.org/12419003/diff/1/build/common.gypi#newcode662
build/common.gypi:662: '<!(python <(gomadir)/goma_ctl.py ensure_start >/dev/null
2>/dev/null && echo ok)'
On 2013/03/08 04:04:09, iannucci wrote:
> It's really too bad we can't just use goma_ctl for both win and !win..
> 
> Also, I don't think >/dev/null is going to work how you expect on windows :).

Oh, fixed to use NUL.

> Maybe add a --print-ok flag to goma_ctl?

> 
> I suppose it's OK to setup goma at gyptime... users will need to be educated
> about this though. If I reboot my machine and try to build, I'll have a broken
> config.
> 
> The BEST thing would be to generate an action which everything depends on
> implicitly (I'm thinking: inject a dependency into everything with a global
> target_defaults). If the goma setup fails at compile time, the user would need
> to re-gyp with goma off (or try the build again after fixing whatever
prevented
> goma from starting).

Hmm, let me think...

https://codereview.chromium.org/12419003/diff/1/build/common.gypi#newcode4114
build/common.gypi:4114: #  in make_global_settings yet.
On 2013/03/08 04:04:09, iannucci wrote:
> AFAIK, nothing uses goma with Make currently, and we don't plan to in the
> future.

Oh, really?
Linux Builder x64, for instance, uses goma with make.
http://build.chromium.org/p/chromium.linux/builders/Linux%20Builder%20x64

/b/build/goma/goma_ctl.sh ensure_start
Using goma VERSION=37 (no_auto_update)...
Checking running compiler_proxy ...
GOMA version f0c845b57d1e5b51e44b672dec2be7e89f75b2e8@1360723428
waiting for compiler_proxy's unix domain socket...20
compiler proxy (pid=29804) status:  http://127.0.0.1:8088/ alive

Now goma is ready!

using goma
Environment variables set in compile.py:
  CC=gcc
  CXX=g++
  GOMA_API_KEY_FILE=/b/build/goma/goma.key
 
PATH=/b/build/goma:/home/chrome-bot/slavebin:/b/depot_tools:/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin


make -k CC.host=gcc CXX.host=g++ -j100 base_unittests browser_tests
cacheinvalidation_unittests cc_unittests chrome chromedriver2_unittests
components_unittests content_browsertests content_unittests crypto_unittests
dbus_unittests device_unittests googleurl_unittests gpu_unittests
interactive_ui_tests ipc_tests jingle_unittests media_unittests net_unittests
ppapi_unittests printing_unittests remoting_unittests sandbox_linux_unittests
sql_unittests sync_unit_tests ui_unittests unit_tests
webkit_compositor_bindings_unittests sync_integration_tests BUILDTYPE=Release

[iannucci, 2013-03-08 06:54:10]

https://codereview.chromium.org/12419003/diff/1/build/common.gypi
File build/common.gypi (right):

https://codereview.chromium.org/12419003/diff/1/build/common.gypi#newcode4114
build/common.gypi:4114: #  in make_global_settings yet.
On 2013/03/08 06:52:05, ukai wrote:
> On 2013/03/08 04:04:09, iannucci wrote:
> > AFAIK, nothing uses goma with Make currently, and we don't plan to in the
> > future.
> 
> Oh, really?
> Linux Builder x64, for instance, uses goma with make.
> http://build.chromium.org/p/chromium.linux/builders/Linux%2520Builder%2520x64
> 
> /b/build/goma/goma_ctl.sh ensure_start
> Using goma VERSION=37 (no_auto_update)...
> Checking running compiler_proxy ...
> GOMA version f0c845b57d1e5b51e44b672dec2be7e89f75b2e8@1360723428
> waiting for compiler_proxy's unix domain socket...20
> compiler proxy (pid=29804) status:  http://127.0.0.1:8088/ alive
> 
> Now goma is ready!
> 
> using goma
> Environment variables set in compile.py:
>   CC=gcc
>   CXX=g++
>   GOMA_API_KEY_FILE=/b/build/goma/goma.key
>  
>
PATH=/b/build/goma:/home/chrome-bot/slavebin:/b/depot_tools:/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin
> 
> 
> make -k CC.host=gcc CXX.host=g++ -j100 base_unittests browser_tests
> cacheinvalidation_unittests cc_unittests chrome chromedriver2_unittests
> components_unittests content_browsertests content_unittests crypto_unittests
> dbus_unittests device_unittests googleurl_unittests gpu_unittests
> interactive_ui_tests ipc_tests jingle_unittests media_unittests net_unittests
> ppapi_unittests printing_unittests remoting_unittests sandbox_linux_unittests
> sql_unittests sync_unit_tests ui_unittests unit_tests
> webkit_compositor_bindings_unittests sync_integration_tests BUILDTYPE=Release

You're completely right, I'm not sure what I was thinking >_<

[Nico, 2013-03-11 17:48:45]

I too am not sure if starting goma at gyp time will work well.

Most confusion seems to be caused by people not setting their PATH correctly,
which is solved by the _wrapper parts of this CL. Maybe we should require people
start up goma manually for now (i.e. omit that part from this CL) and see if
that's good enough? If people continue to get that wrong, we can add that part
later.