xcode_emulation: work in the absence of xcodebuild

xcode_emulation: work in the absence of xcodebuild

OS X systems running only the Command Line Tools for Xcode package,
without the full Xcode, don't have a functioning xcodebuild, but this
isn't mandatory for building many gyp projects (e.g. node.js, v8).
This commit handles xcodebuild failures and avoids populating
Xcode-specific CFLAGS/LDFLAGS when xcodebuild can't be run.

This has been tested on both Xcode and CLT-only systems by successfully
building node.js. The behaviour can be simulated on systems with Xcode
by setting the xcode-select path to something nonsensical, e.g.
xcode-select -switch /usr/bin

BUG=https://code.google.com/p/gyp/issues/detail?id=292

[mistydemeo, 2013-12-15 09:43:09]

gyp's xcode_emulation component assumes that SDKs are available, and makes use
of the xcodebuild tool to locate the correct SDK. However, this assumption
doesn't hold when building with only the Command Line Tools for Xcode package,
and no Xcode. The CLT is a traditional Unix-style package that installs build
tools and headers in /usr; it has no concept of OS X SDKs, and does not install
the xcodebuild tool.

The way that xcodebuild is currently used to fetch the SDK causes gyp to error
out on CLT-only systems; however, not all packages that use gyp actually need
SDKs to build, such as node.js. They can easily build on CLT-only systems if the
xcodebuild check is removed. This handles the possibility of xcodebuild failing
in _GetSdkVersionInfoItem(), and updates a few places to avoid trying to use SDK
paths when they're not present.

The Homebrew package manager has been carrying a variation of this patch for
eight months in order to support node.js on CLT-only systems, without issue.

This obsoletes a previous submission I had made,
https://codereview.chromium.org/14887003/; apologies for not following up on
comments at that time, but I found myself in the middle of a major move and lost
track of this.

[Mark Mentovai, 2013-12-15 21:07:30]

Thanks for the contribution.

The unified diff works, but I’m getting “error: old chunk mismatch” on the
side-by-side diff. The base file upload must have failed. Can you try uploading
the patch again?

[mistydemeo, 2013-12-15 21:49:37]

On 2013/12/15 21:07:30, Mark Mentovai wrote:
> Thanks for the contribution.
> 
> The unified diff works, but I’m getting “error: old chunk mismatch” on the
> side-by-side diff. The base file upload must have failed. Can you try
uploading
> the patch again?

I've uploaded the patch again to this issue - looks like it worked this time.

[Nico, 2013-12-16 05:57:32]

lgtm

As said on the other review, since this has no tests, it'll possibly break in
the future.

mac_tool.py has a reference to xcrun now, so some stuff (such as
mac_bundle_resources) won't work for you. Apparently that stuff isn't important
for node :-)

Let me know if you want me to land this for you.

[mistydemeo, 2013-12-16 06:29:18]

On 2013/12/16 05:57:32, Nico wrote:
> lgtm
> 
> As said on the other review, since this has no tests, it'll possibly break in
> the future.

Yes, I'd taken a look at those, but the tests are a bit thorny. Many of them
call xcodebuild and friends directly instead of being proxied through the
methods in the xcode emulation parts of gyp, so there would be more cleanup than
I was really motivated to do at that time. ;)

> mac_tool.py has a reference to xcrun now, so some stuff (such as
> mac_bundle_resources) won't work for you. Apparently that stuff isn't
important
> for node :-)

That case calls ibtool, which is a part of the Xcode GUI stuff, and that *does*
require a full Xcode install. Ditto the ninja Xcode project generator.
(Something that checks for the dreaded xcrun infinite loop wouldn't be amiss in
both cases though.)

There were another few things I'm planning to clean up but I'd like to submit
those separately, since I'm not sure if I'll do that soon.

> Let me know if you want me to land this for you.

Please!

[Nico, 2013-12-16 19:00:42]

On 2013/12/16 06:29:18, mistydemeo wrote:
> On 2013/12/16 05:57:32, Nico wrote:
> > lgtm
> > 
> > As said on the other review, since this has no tests, it'll possibly break
in
> > the future.
> 
> Yes, I'd taken a look at those, but the tests are a bit thorny. Many of them
> call xcodebuild and friends directly instead of being proxied through the
> methods in the xcode emulation parts of gyp, so there would be more cleanup
than
> I was really motivated to do at that time. ;)
> 
> > mac_tool.py has a reference to xcrun now, so some stuff (such as
> > mac_bundle_resources) won't work for you. Apparently that stuff isn't
> important
> > for node :-)
> 
> That case calls ibtool, which is a part of the Xcode GUI stuff, and that
*does*
> require a full Xcode install. Ditto the ninja Xcode project generator.
> (Something that checks for the dreaded xcrun infinite loop wouldn't be amiss
in
> both cases though.)
> 
> There were another few things I'm planning to clean up but I'd like to submit
> those separately, since I'm not sure if I'll do that soon.
> 
> > Let me know if you want me to land this for you.
> 
> Please!

r1819

[scottmg, 2013-12-17 05:01:50]

On 2013/12/16 19:00:42, Nico wrote:
> On 2013/12/16 06:29:18, mistydemeo wrote:
> > On 2013/12/16 05:57:32, Nico wrote:
> > > lgtm
> > > 
> > > As said on the other review, since this has no tests, it'll possibly break
> in
> > > the future.
> > 
> > Yes, I'd taken a look at those, but the tests are a bit thorny. Many of them
> > call xcodebuild and friends directly instead of being proxied through the
> > methods in the xcode emulation parts of gyp, so there would be more cleanup
> than
> > I was really motivated to do at that time. ;)
> > 
> > > mac_tool.py has a reference to xcrun now, so some stuff (such as
> > > mac_bundle_resources) won't work for you. Apparently that stuff isn't
> > important
> > > for node :-)
> > 
> > That case calls ibtool, which is a part of the Xcode GUI stuff, and that
> *does*
> > require a full Xcode install. Ditto the ninja Xcode project generator.
> > (Something that checks for the dreaded xcrun infinite loop wouldn't be amiss
> in
> > both cases though.)
> > 
> > There were another few things I'm planning to clean up but I'd like to
submit
> > those separately, since I'm not sure if I'll do that soon.
> > 
> > > Let me know if you want me to land this for you.
> > 
> > Please!
> 
> r1819

It looks like this isn't working on the bot:
http://build.chromium.org/f/client/gyp/

[Nico, 2013-12-18 16:21:26]

mistydemeo, have you had a chance to look at the test failures this caused?

(Since my mail didn't make it through to rietveld, pasting it again:

http://build.chromium.org/p/tryserver.nacl/builders/gyp-mac/builds/1478/steps...

Failed the following 4 tests:
	test/ios/gyptest-app-ios.py
	test/ios/gyptest-archs.py
	test/ios/gyptest-per-config-settings.py
	test/mac/gyptest-framework-dirs.py

The Mac test:

/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/python
test/mac/gyptest-framework-dirs.py
FAILED test at line 423 of
/Volumes/data/b/build/slave/gyp/build/trunk/test/lib/TestCommon.py (_complete)
	from line 532 of
/Volumes/data/b/build/slave/gyp/build/trunk/test/lib/TestCommon.py (run)
	from line 277 of
/Volumes/data/b/build/slave/gyp/build/trunk/test/lib/TestGyp.py (run)
	from line 849 of
/Volumes/data/b/build/slave/gyp/build/trunk/test/lib/TestGyp.py (build)
	from line 21 of test/mac/gyptest-framework-dirs.py
None returned 1
STDOUT =========================================================================
ninja: Entering directory `out/Default'
[1/2] CC obj/calculate.calculate.o
[2/2] LINK calculate, POSTBUILDS
FAILED: cc -arch i386 -L. -o calculate obj/calculate.calculate.o  -framework
Calculate
ld: framework not found Calculate
collect2: ld returned 1 exit status
ninja: build stopped: subcommand failed.
)

[mistydemeo, 2013-12-18 16:43:25]

On 2013/12/18 16:21:26, Nico wrote:
> mistydemeo, have you had a chance to look at the test failures this caused?
> 
> (Since my mail didn't make it through to rietveld, pasting it again:
> 
>
http://build.chromium.org/p/tryserver.nacl/builders/gyp-mac/builds/1478/steps...
> 
> Failed the following 4 tests:
> 	test/ios/gyptest-app-ios.py
> 	test/ios/gyptest-archs.py
> 	test/ios/gyptest-per-config-settings.py
> 	test/mac/gyptest-framework-dirs.py
> 
> The Mac test:
> 
>
/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/python
> test/mac/gyptest-framework-dirs.py
> FAILED test at line 423 of
> /Volumes/data/b/build/slave/gyp/build/trunk/test/lib/TestCommon.py (_complete)
> 	from line 532 of
> /Volumes/data/b/build/slave/gyp/build/trunk/test/lib/TestCommon.py (run)
> 	from line 277 of
> /Volumes/data/b/build/slave/gyp/build/trunk/test/lib/TestGyp.py (run)
> 	from line 849 of
> /Volumes/data/b/build/slave/gyp/build/trunk/test/lib/TestGyp.py (build)
> 	from line 21 of test/mac/gyptest-framework-dirs.py
> None returned 1
> STDOUT
=========================================================================
> ninja: Entering directory `out/Default'
> [1/2] CC obj/calculate.calculate.o
> [2/2] LINK calculate, POSTBUILDS
> FAILED: cc -arch i386 -L. -o calculate obj/calculate.calculate.o  -framework
> Calculate
> ld: framework not found Calculate
> collect2: ld returned 1 exit status
> ninja: build stopped: subcommand failed.
> )

Yes, sorry I didn't reply sooner.

My computer doesn't have Xcode installed, so I can't run those tests, but I have
access to a few VMs with Xcode so I'll dig into this.