Fix XCObject._EncodeString() for unicode characters.

pylib/gyp/xcodeproj_file.py

 # Copyright (c) 2012 Google Inc. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
 # found in the LICENSE file.
 
 """Xcode project file generator.
 
 This module is both an Xcode project file generator and a documentation of the
 Xcode project file format.  Knowledge of the project file format was gained
 based on extensive experience with Xcode, and by making changes to projects in
 Xcode.app and observing the resultant changes in the associated project files.
@@ skipping 151 matching lines @@
 # characters listed with "+", for 1 or more occurrences: if a string is empty,
 # it must not match this pattern, because it needs to be encoded as "".
 _unquoted = re.compile('^[A-Za-z0-9$./_]+$')
 
 # Strings that match this pattern are quoted regardless of what _unquoted says.
 # Oddly, Xcode will quote any string with a run of three or more underscores.
 _quoted = re.compile('___')
 
 # This pattern should match any character that needs to be escaped by
 # XCObject._EncodeString.  See that function.
-_escaped = re.compile('[\\\\"]|[^ -~]')
+_escaped = re.compile('[\\\\"]|[\x00-\x1f\x7f]')

[Mark Mentovai, 2013/09/12 21:32:18]

Why is 0x7f in here? If the input contains a DEL character…

[kbongort, 2013/09/12 22:41:40]

This was to keep the former behavior, but the comment in _EncodeString says "DEL
is passed through as-is without escaping" so it was wrong. And as you note,
would cause a crash. Fixed.

 
 
 # Used by SourceTreeAndPathFromPath
 _path_leading_variable = re.compile('^\$\((.*?)\)(/(.*))?$')
 
 def SourceTreeAndPathFromPath(input_path):
   """Given input_path, returns a tuple with sourceTree and path values.
 
   Examples:
     input_path     (source_tree, output_path)
@@ skipping 82 matching lines @@
     _properties: The object's property dictionary.  An object's properties are
                  described by its class' _schema variable.
   """
 
   _schema = {}
   _should_print_single_line = False
 
   # See _EncodeString.
   _encode_transforms = []
   i = 0
   while i < ord(' '):

[Mark Mentovai, 2013/09/12 21:32:18]

…it won’t be matched by a transform in this list…

[kbongort, 2013/09/12 22:41:40]

Done.

     _encode_transforms.append('\\U%04x' % i)
     i = i + 1
   _encode_transforms[7] = '\\a'
   _encode_transforms[8] = '\\b'
   _encode_transforms[9] = '\\t'
   _encode_transforms[10] = '\\n'
   _encode_transforms[11] = '\\v'
   _encode_transforms[12] = '\\f'
   _encode_transforms[13] = '\\n'
 
@@ skipping 235 matching lines @@
     # the _escaped expression.
     char = match.group(0)
 
     # Backslashes (\) and quotation marks (") are always replaced with a
     # backslash-escaped version of the same.  Everything else gets its
     # replacement from the class' _encode_transforms array.
     if char == '\\':
       return '\\\\'
     if char == '"':
       return '\\"'
     return self._encode_transforms[ord(char)]

[Mark Mentovai, 2013/09/12 21:32:18]

…and then this will raise an exception.

[kbongort, 2013/09/12 22:41:40]

Done.

 
   def _EncodeString(self, value):
     """Encodes a string to be placed in the project file output, mimicing
     Xcode behavior.
     """
 
     # Use quotation marks when any character outside of the range A-Z, a-z, 0-9,
     # $ (dollar sign), . (period), and _ (underscore) is present.  Also use
     # quotation marks to represent empty strings.
     #
     # Escape " (double-quote) and \ (backslash) by preceding them with a
     # backslash.
     #
     # Some characters below the printable ASCII range are encoded specially:
     #     7 ^G BEL is encoded as "\a"
     #     8 ^H BS  is encoded as "\b"
     #    11 ^K VT  is encoded as "\v"
     #    12 ^L NP  is encoded as "\f"
     #   127 ^? DEL is passed through as-is without escaping
     #  - In PBXFileReference and PBXBuildFile objects:
     #     9 ^I HT  is passed through as-is without escaping
     #    10 ^J NL  is passed through as-is without escaping
     #    13 ^M CR  is passed through as-is without escaping
     #  - In other objects:
     #     9 ^I HT  is encoded as "\t"
     #    10 ^J NL  is encoded as "\n"
     #    13 ^M CR  is encoded as "\n" rendering it indistinguishable from
     #              10 ^J NL
     # All other nonprintable characters within the ASCII range (0 through 127

[Mark Mentovai, 2013/09/12 21:32:18]

I just consulted with both Xcode 4.6.3 and 3.2.6 and found that it doesn’t do
anything special to DEL characters, it just passes ’em right through, so this
comment about nonprintable characters is either newly wrong or was always wrong.
It should say “All other characters within the ASCII control character range (0
through 31 inclusive).”

NUL characters in the path of a PBXFileReference crash Xcode 4. Xcode 3 deals
with ’em. Progress.

[kbongort, 2013/09/12 22:41:40]

I fixed the comment.

     # inclusive) are encoded as "\U001f" referring to the Unicode code point in
     # hexadecimal.  For example, character 14 (^N SO) is encoded as "\U000e".
     # Characters above the ASCII range are passed through to the output encoded
     # as UTF-8 without any escaping.  These mappings are contained in the
     # class' _encode_transforms list.
 
     if _unquoted.search(value) and not _quoted.search(value):
       return value
 
     return '"' + _escaped.sub(self._EncodeTransform, value) + '"'
@@ skipping 2290 matching lines @@
       self._XCPrint(file, 0, '/* Begin ' + class_name + ' section */\n')
       for object in sorted(objects_by_class[class_name],
                            cmp=lambda x, y: cmp(x.id, y.id)):
         object.Print(file)
       self._XCPrint(file, 0, '/* End ' + class_name + ' section */\n')
 
     if self._should_print_single_line:
       self._XCPrint(file, 0, '}; ')
     else:
       self._XCPrint(file, 1, '};\n')