Fix XCObject._EncodeString() for unicode characters.

The old _escaped regex matched any characters not in range from space to tilde, i.e. from \x20 to \x7e. Therefore it matched all non-ASCII characters, despite this comment in _EncodeString: "Characters above the ASCII range are passed through to the output encoded as UTF-8 without any escaping."

This changes the regex to match the same set of ASCII characters as before, without matching characters outside the ASCII range.

Committed GYP r1728

[Nico, 2013-09-12 20:19:02]

Mark knows the xcode generator better.

Please add a test (see test/mac, use gyptest.py -a to run all tests, gyptest.py
test/mac/gyptest-foo.py to run a single test).

[kbongort, 2013-09-12 20:24:25]

On 2013/09/12 20:19:02, Nico wrote:
> Mark knows the xcode generator better.
> 
> Please add a test (see test/mac, use gyptest.py -a to run all tests,
gyptest.py
> test/mac/gyptest-foo.py to run a single test).

Does a test suite exist for xcodeproj_file.py? I looked around, but couldn't
figure out where to add a test case.

[Nico, 2013-09-12 20:26:04]

On 2013/09/12 20:24:25, kbongort wrote:
> On 2013/09/12 20:19:02, Nico wrote:
> > Mark knows the xcode generator better.
> > 
> > Please add a test (see test/mac, use gyptest.py -a to run all tests,
> gyptest.py
> > test/mac/gyptest-foo.py to run a single test).
> 
> Does a test suite exist for xcodeproj_file.py? I looked around, but couldn't
> figure out where to add a test case.

gyp is mostly system tests (to make sure things build the same under different
generators). Can you create a gyp file with characters in that range? How are
you running into this problem in practice; can you replicate this in a small
test.gyp file?

[kbongort, 2013-09-12 21:14:24]

On 2013/09/12 20:26:04, Nico wrote:
> On 2013/09/12 20:24:25, kbongort wrote:
> > On 2013/09/12 20:19:02, Nico wrote:
> > > Mark knows the xcode generator better.
> > > 
> > > Please add a test (see test/mac, use gyptest.py -a to run all tests,
> > gyptest.py
> > > test/mac/gyptest-foo.py to run a single test).
> > 
> > Does a test suite exist for xcodeproj_file.py? I looked around, but couldn't
> > figure out where to add a test case.
> 
> gyp is mostly system tests (to make sure things build the same under different
> generators). Can you create a gyp file with characters in that range? How are
> you running into this problem in practice; can you replicate this in a small
> test.gyp file?

Ok, I added a test case. The issue is making setting 'xcode_settings' variables
that contain unicode characters.

[Mark Mentovai, 2013-09-12 21:32:18]

https://codereview.chromium.org/23892016/diff/8001/pylib/gyp/xcodeproj_file.py
File pylib/gyp/xcodeproj_file.py (right):

https://codereview.chromium.org/23892016/diff/8001/pylib/gyp/xcodeproj_file.p...
pylib/gyp/xcodeproj_file.py:172: _escaped =
re.compile('[\\\\"]|[\x00-\x1f\x7f]')
Why is 0x7f in here? If the input contains a DEL character…

https://codereview.chromium.org/23892016/diff/8001/pylib/gyp/xcodeproj_file.p...
pylib/gyp/xcodeproj_file.py:275: while i < ord(' '):
…it won’t be matched by a transform in this list…

https://codereview.chromium.org/23892016/diff/8001/pylib/gyp/xcodeproj_file.p...
pylib/gyp/xcodeproj_file.py:531: return self._encode_transforms[ord(char)]
…and then this will raise an exception.

https://codereview.chromium.org/23892016/diff/8001/pylib/gyp/xcodeproj_file.p...
pylib/gyp/xcodeproj_file.py:560: # All other nonprintable characters within the
ASCII range (0 through 127
I just consulted with both Xcode 4.6.3 and 3.2.6 and found that it doesn’t do
anything special to DEL characters, it just passes ’em right through, so this
comment about nonprintable characters is either newly wrong or was always wrong.
It should say “All other characters within the ASCII control character range (0
through 31 inclusive).”

NUL characters in the path of a PBXFileReference crash Xcode 4. Xcode 3 deals
with ’em. Progress.

https://codereview.chromium.org/23892016/diff/8001/test/mac/unicode-settings/...
File test/mac/unicode-settings/test.gyp (right):

https://codereview.chromium.org/23892016/diff/8001/test/mac/unicode-settings/...
test/mac/unicode-settings/test.gyp:13: },
Maybe add a couple more funny characters here, like the DEL character and some
control characters, just to make sure that they’re accepted and don’t cause the
generator to croak.

[kbongort, 2013-09-12 22:41:40]

https://codereview.chromium.org/23892016/diff/8001/pylib/gyp/xcodeproj_file.py
File pylib/gyp/xcodeproj_file.py (right):

https://codereview.chromium.org/23892016/diff/8001/pylib/gyp/xcodeproj_file.p...
pylib/gyp/xcodeproj_file.py:172: _escaped =
re.compile('[\\\\"]|[\x00-\x1f\x7f]')
On 2013/09/12 21:32:18, Mark Mentovai wrote:
> Why is 0x7f in here? If the input contains a DEL character…

This was to keep the former behavior, but the comment in _EncodeString says "DEL
is passed through as-is without escaping" so it was wrong. And as you note,
would cause a crash. Fixed.

https://codereview.chromium.org/23892016/diff/8001/pylib/gyp/xcodeproj_file.p...
pylib/gyp/xcodeproj_file.py:275: while i < ord(' '):
On 2013/09/12 21:32:18, Mark Mentovai wrote:
> …it won’t be matched by a transform in this list…

Done.

https://codereview.chromium.org/23892016/diff/8001/pylib/gyp/xcodeproj_file.p...
pylib/gyp/xcodeproj_file.py:531: return self._encode_transforms[ord(char)]
On 2013/09/12 21:32:18, Mark Mentovai wrote:
> …and then this will raise an exception.

Done.

https://codereview.chromium.org/23892016/diff/8001/pylib/gyp/xcodeproj_file.p...
pylib/gyp/xcodeproj_file.py:560: # All other nonprintable characters within the
ASCII range (0 through 127
I fixed the comment.

https://codereview.chromium.org/23892016/diff/8001/test/mac/unicode-settings/...
File test/mac/unicode-settings/test.gyp (right):

https://codereview.chromium.org/23892016/diff/8001/test/mac/unicode-settings/...
test/mac/unicode-settings/test.gyp:13: },
On 2013/09/12 21:32:18, Mark Mentovai wrote:
> Maybe add a couple more funny characters here, like the DEL character and some
> control characters, just to make sure that they’re accepted and don’t cause
the
> generator to croak.

I added a check for the tab character.

[Mark Mentovai, 2013-09-13 15:28:56]

LGTM and committed GYP r1728.