Update the gyp Linux build:

Update the gyp Linux build:
* Add Linux settings to target_defaults in common.gypi so
  gyp-generated SConscript files no longer depend on
  build/SConscript.main or the Hammer infrastructure.
* Copy the FilterOut() function from Hammer to the chromium_builders.py
  Tool module.
* Add a ChromiumLoadableModule() builder to chromium_builders.py.
* Add dependencies on the 'views' library to the chrome link (target 'app').
* Add missing views/*/*_unittest.cc modules to the 'unit_tests' target.
  Exclude all but the one that builds on Linux from the non-Windows builds.
* Crib a list of chrome/views files to exclude from the Linux build
  from the old SCons configuration.
* Add a new build/linux/system.gyp file with new 'settings' targets
  to encapsulate the pkg-config checks for gtk+-2.0, nss and pangoft2.
* Add depenedencies in the other targets on the new gtk, nss and
  pangoft2 'settings' targets from build/linux/system.gyp.
* Add a pkg_config_wrapper.py script that keeps gyp happy by
  simply exiting 0 if the package isn't found.
* DEPS roll for latest gyp changes to support the above.

Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=12228

[sgk, 2009-03-18 17:27:07]

This is preliminary.  Appreciate a thumbs up/down on whether this looks in the
ballpark before I go further with cleaning it up and finishing it off.

[agl, 2009-03-18 17:34:07]

This looks like what I was expecting. For certain build systems, we don't need
the pkg-config flags because they don't reference the system headers/libs, but I
can always filter out the extra flags.

http://codereview.chromium.org/42340/diff/1/3
File build/common.gypi (right):

http://codereview.chromium.org/42340/diff/1/3#newcode98
Line 98: '-march=i686',
march is pentium4 and we need -msse2 -mfpmath=sse (see comment in
build/SConscript)

http://codereview.chromium.org/42340/diff/1/2
File skia/skia.gyp (right):

http://codereview.chromium.org/42340/diff/1/2#newcode693
Line 693: '?@(python ../tools/pkg_config_wrapper/pkg_config_wrapper.py --cflags
gtk+-2.0)',
Skia needs GTK?!

http://codereview.chromium.org/42340/diff/1/5
File tools/pkg_config_wrapper/pkg_config_wrapper.py (right):

http://codereview.chromium.org/42340/diff/1/5#newcode1
Line 1: #!/bin/env python
The SVN properties are probably wrong for this file.

[sgk, 2009-03-18 21:18:43]

http://codereview.chromium.org/42340/diff/1/3
File build/common.gypi (right):

http://codereview.chromium.org/42340/diff/1/3#newcode98
Line 98: '-march=i686',
On 2009/03/18 17:34:07, agl wrote:
> march is pentium4 and we need -msse2 -mfpmath=sse (see comment in
> build/SConscript)

Done.

http://codereview.chromium.org/42340/diff/1/2
File skia/skia.gyp (right):

http://codereview.chromium.org/42340/diff/1/2#newcode693
Line 693: '?@(python ../tools/pkg_config_wrapper/pkg_config_wrapper.py --cflags
gtk+-2.0)',
On 2009/03/18 17:34:07, agl wrote:
> Skia needs GTK?!

Not GTK, but it looks like this was how we were getting the includes for
cairo-ft:

skia/ext/platform_canvas_linux.cc and
skia/ext/bitmap_platform_device_linux.cc
both #include <cairo/cairo.h>

and skia/ports/SkFontHost_FreeType.cc
#includes <freetype/config/ftheader.h>

I just rebuilt successfully using "pkg-config --cflags cairo-ft" and will make
that change when I submit, unless it seems more important to keep the argument
consistent with the uses in base.gyp.

http://codereview.chromium.org/42340/diff/1/5
File tools/pkg_config_wrapper/pkg_config_wrapper.py (right):

http://codereview.chromium.org/42340/diff/1/5#newcode1
Line 1: #!/bin/env python
On 2009/03/18 17:34:07, agl wrote:
> The SVN properties are probably wrong for this file.

Hmm, the other *.py files under tools all have svn:eol-style LF, too.

[sgk, 2009-03-20 00:47:41]

Mark, please take a look at the TODO(mmentovai) comments in chrome.gyp and let
me know if I'm overlooking a way to have those conditional targets use the
'<@(cflags_gtk)' variables from the definition of those variables in the earlier
'conditions' clause.

Note that this CL severs the gyp-based SCons build from the Hammer
infrastructure, but doesn't (yet) have the ability to let the user specify that
they want to use the system copies of libxml, libxslt or sqlite in preference to
the checked-in copies.  Next...

[Mark Mentovai, 2009-03-20 14:28:07]

http://codereview.chromium.org/42340/diff/31/2018
File base/base.gyp (right):

http://codereview.chromium.org/42340/diff/31/2018#newcode328
Line 328: 'libraries': [
If you are putting libraries into direct_dependent_settings, it's usually a bad
sign.  libraries belong in link_settings.  base's direct dependents are mostly
other static libraries, they can't do anything with a list of libraries either,
and ld will never see them.

http://codereview.chromium.org/42340/diff/31/2017
File build/common.gypi (right):

http://codereview.chromium.org/42340/diff/31/2017#newcode142
Line 142: # most significant example.) Since the GNU linker does only a
Revise this comment (if it even still applies).  These target names are no
longer applicable.  libport is gone, and V8Bindings and WebCore have been merged
into a single target, largely to resolve the silly circular dependency problem
we had prior to GYP.  We may no longer need this --start-group/--end-group
nonsense at all, but then again, we may still have a few circular dependencies
in the chrome static libraries.  Regardless, at the very least, this comment
shouldn't reference targets that don't exist in GYP.

http://codereview.chromium.org/42340/diff/31/2022
File chrome/chrome.gyp (right):

http://codereview.chromium.org/42340/diff/31/2022#newcode308
Line 308: 'cflags': [
Any place you say 'cflags': ['<@(cflags_gtk)'], you should probably also be
writing 'link_settings': {'libraries': ['<@(libs_gtk)']}.  Presumably things
need GTK CFLAGS because they use GTK, so they should declare that they need GTK
at link time as well.  Getting these right will make a shared library build so
much easier and cleaner.

http://codereview.chromium.org/42340/diff/31/2022#newcode1510
Line 1510: '<@(libs_nss)',
Again, with this too, we probably don't want libs_nss right here.  I doubt
anything the app target itself actually needs to link against nss.  Rather, you
should move libs_nss to a link_settings section of whatever is actually using
nss.

http://codereview.chromium.org/42340/diff/31/2022#newcode2333
Line 2333: # TODO(mmentovai):  this condition-within-condition
In order to make this work, don't set the "variables" you want within the
(outer) "conditions" section (line 2277), make the top-level "variables" section
(line 6) contain a "conditions" section and set cflags_gtk and friends in there.

http://codereview.chromium.org/42340/diff/31/2016
File skia/skia.gyp (right):

http://codereview.chromium.org/42340/diff/31/2016#newcode693
Line 693: '<!@(python ../tools/pkg_config_wrapper/pkg_config_wrapper.py --cflags
gtk+-2.0)',
I see you running this a bunch of times in lots of different GYP files.

It'd be an improvement if we collected all of these pkg-config runs into
common.gypi and assigned them into variables.  Still Linux-only, obviously...

http://codereview.chromium.org/42340/diff/31/2019
File tools/pkg_config_wrapper/pkg_config_wrapper.py (right):

http://codereview.chromium.org/42340/diff/31/2019#newcode2
Line 2: # Copyright (c) 2009. All rights reserved.
This line looks funny to me.  "The Chromium Authors," perhaps?

http://codereview.chromium.org/42340/diff/31/2019#newcode16
Line 16: stdout=subprocess.PIPE,
You're not using stdout for anything, so I don't think you need to pipe it
through this script.  Just leave this out so that pkg-config's stdout remains
hooked up.

http://codereview.chromium.org/42340/diff/31/2019#newcode26
Line 26: found_re = re.compile('No package.*found', re.S)
No need to compile/search for a one-shot RE, you can just use re.search.

re.S doesn't seem to be necessary for this expression, pkg-config always puts
this together on a single line.  (You can use ^ and $ if you want.)

[Mark Mentovai, 2009-03-20 15:37:34]

An additional idea I had that I actually like quite a lot is to provide "proxy"
targets for NSS, GTK, and anything else that we want to get from pkg-config.

build/linux/system.gyp:
{
  'targets': [
    {
      'target_name': 'gtk',
      'type': 'none',
      'direct_dependent_settings': {
        'cflags': [
          '<!@(python ../../tools/pkg_config_wrapper/pkg_config_wrapper.py
--cflags gtk+-2.0)',
        ],
      },
      'link_settings': {
        'libraries': [
          '<!@(python ../../tools/pkg_config_wrapper/pkg_config_wrapper.py
--libs gtk+-2.0)',
        ],
      },
    },
  ],
}

Then any target that needs to use GTK can just declare:

'dependencies': [
  '../build/linux/system.gyp:gtk',
],

I think this is probably the cleanest possible solution as it keeps everything
in "GYP-speak".  For libraries like NSS where we might want to be able to switch
between using a system version and an in-tree version, we could just build the
condition into nss.gyp making it a snap to switch between the two build types.

While writing the above sample, I also realized that pkg_config_wrapper probably
isn't "first-class" enough to warrant its own directory in tools.  I've been
sticking small scripts like this for the Mac in build/mac, something similar
might make sense for Linux.

Mark

[sgk, 2009-03-20 19:14:24]

Updated with new 'settings' targets.  Looking much cleaner.

[Mark Mentovai, 2009-03-20 19:24:22]

system.gyp is missing.

This change is a substantial improvement.  I still think we can probably make it
even simpler by dropping "nss" and "gtk" as dependencies from a bunch of targets
that themselves don't actually use NSS or GTK, but need to link against NSS or
GTK because they use other static libraries that depend on NSS or GTK. 
link_settings should cover us completely in that case.

http://codereview.chromium.org/42340/diff/2066/51
File DEPS (right):

http://codereview.chromium.org/42340/diff/2066/51#newcode29
Line 29: "http://gyp.googlecode.com/svn/trunk@382",
Note about what this did to the Windows tryserver - we can't update to any rev
from r378 on until http://codereview.chromium.org/49004 lands.

http://codereview.chromium.org/42340/diff/2066/45
File base/base.gyp (right):

http://codereview.chromium.org/42340/diff/2066/45#newcode322
Line 322: 'dependencies': [
This is definitely a substantial improvement!

http://codereview.chromium.org/42340/diff/2066/45#newcode563
Line 563: '../build/linux/system.gyp:gtk',
Fix indent.

http://codereview.chromium.org/42340/diff/2066/45#newcode616
Line 616: '../build/linux/system.gyp:gtk',
Here too.

(Does this actually need gtk?  Seems like it might not, but I'm not sure.)

http://codereview.chromium.org/42340/diff/2066/44
File build/common.gypi (right):

http://codereview.chromium.org/42340/diff/2066/44#newcode153
Line 153: # eliminated the circular dependencies.
Excellent.

http://codereview.chromium.org/42340/diff/2066/46
File build/linux/pkg_config_wrapper.py (right):

http://codereview.chromium.org/42340/diff/2066/46#newcode23
Line 23: found_re = re.compile('No package.*found', re.S)
Question again about whether re.compile is needed or if this can just be
condensed down to a single re.search on the next line.

http://codereview.chromium.org/42340/diff/2066/49
File chrome/chrome.gyp (right):

http://codereview.chromium.org/42340/diff/2066/49#newcode1505
Line 1505: '../build/linux/system.gyp:nss',
I'd be surprised if nss were needed at this level too (although it's certainly
possible).  Does any code directly in app actually use nss?  We should be
getting -lnss from the link_settings on other dependency targets that depend on
nss.

http://codereview.chromium.org/42340/diff/2066/49#newcode1792
Line 1792: '../build/linux/system.gyp:gtk',
Similar here - I'd be surprised if we actually needed gtk on this.  Same for
some of the other targets in this file.  Maybe we do, I'd just be surprised...

http://codereview.chromium.org/42340/diff/2066/49#newcode2164
Line 2164: }, { # else: OS != "win"
Ah, good cleanup, thanks.

[sgk, 2009-03-20 21:15:39]

Uploaded another version.  Added changes to media/media.gyp and
webkit/tools/test_shell/test_shell.gyp.

http://codereview.chromium.org/42340/diff/2066/45
File base/base.gyp (right):

http://codereview.chromium.org/42340/diff/2066/45#newcode563
Line 563: '../build/linux/system.gyp:gtk',
On 2009/03/20 19:24:22, Mark Mentovai wrote:
> Fix indent.

Done.

http://codereview.chromium.org/42340/diff/2066/45#newcode616
Line 616: '../build/linux/system.gyp:gtk',
On 2009/03/20 19:24:22, Mark Mentovai wrote:
> Here too.

Done.

> (Does this actually need gtk?  Seems like it might not, but I'm not sure.)

Yes, it does.  It ends up #include gtk/gtk.h.  I'll document the #include chain
in a comment so others don't have to track it down (agl was asking the same
questions, e.g.) and to help anyone who might want to refactor to get rid of the
dependency in the future.

http://codereview.chromium.org/42340/diff/2066/46
File build/linux/pkg_config_wrapper.py (right):

http://codereview.chromium.org/42340/diff/2066/46#newcode23
Line 23: found_re = re.compile('No package.*found', re.S)
On 2009/03/20 19:24:22, Mark Mentovai wrote:
> Question again about whether re.compile is needed or if this can just be
> condensed down to a single re.search on the next line.

Done.

http://codereview.chromium.org/42340/diff/2066/49
File chrome/chrome.gyp (right):

http://codereview.chromium.org/42340/diff/2066/49#newcode1505
Line 1505: '../build/linux/system.gyp:nss',
On 2009/03/20 19:24:22, Mark Mentovai wrote:
> I'd be surprised if nss were needed at this level too (although it's certainly
> possible).  Does any code directly in app actually use nss?  We should be
> getting -lnss from the link_settings on other dependency targets that depend
on
> nss.

Done.

http://codereview.chromium.org/42340/diff/2066/49#newcode1792
Line 1792: '../build/linux/system.gyp:gtk',
On 2009/03/20 19:24:22, Mark Mentovai wrote:
> Similar here - I'd be surprised if we actually needed gtk on this.  Same for
> some of the other targets in this file.  Maybe we do, I'd just be surprised...

run_all_unittests.cc ends up including gtk/gtk.h, so compile fails without it. 
Adding a comment.

[Mark Mentovai, 2009-03-20 21:20:28]

LGTM

I really like how this turned out!

Thanks for adding the comments on some of the weirder gtk dependencies.

http://codereview.chromium.org/42340/diff/56/2078
File DEPS (right):

http://codereview.chromium.org/42340/diff/56/2078#newcode29
Line 29: "http://gyp.googlecode.com/svn/trunk@382",
Brad should have this covered now.

http://codereview.chromium.org/42340/diff/56/2082
File build/linux/system.gyp (right):

http://codereview.chromium.org/42340/diff/56/2082#newcode1
Line 1: # Copyright (c) 2009 The Chromium Authors. All rights reserved.
svn:eol-style = LF

(sometimes Rietveld doesn't show these even when they're there, so I can't be
sure if you've set it or not.)

http://codereview.chromium.org/42340/diff/56/2082#newcode12
Line 12: '<!@(python ../build/linux/pkg_config_wrapper.py --cflags gtk+-2.0)',
Reminds me, I have to fix the cwd problem.  This will do for now.