fix stringToCodePoints on VM

fix stringToCodePoints on VM
http://dartbug.com/6476

Committed: https://code.google.com/p/dart/source/detail?r=14585

[Jennifer Messerly, 2012-11-05 18:45:35]

The context behind this: when the Dart VM removed Unicode string support, it
broke our html5 parser. I have a workaround
(https://github.com/dart-lang/html5lib/blob/f799cdc78499d407402e76cb97f188969f...),
but it would be nice to get things fixed up in the core libraries and dart:utf.
String and dart:utf were designed under the assumption that charCodes == code
points, and since that has changed, they will need some changes.

[siva, 2012-11-05 22:03:57]

I am not very familiar with this libary.
The change looks consistent with how things are implemented
in the dart VM so LGTM from that angle.

[Lasse Reichstein Nielsen, 2012-11-06 09:57:38]

LGTM

https://codereview.chromium.org/11293080/diff/2001/sdk/lib/utf/utf_core.dart
File sdk/lib/utf/utf_core.dart (right):

https://codereview.chromium.org/11293080/diff/2001/sdk/lib/utf/utf_core.dart#...
sdk/lib/utf/utf_core.dart:13: return _utf16CodeUnitsToCodepoints(str.charCodes);
This seems wasteful since charCodes currently creates a new list, and not just a
thin wrapper around a String.
We should probably both change that, and make this function return an
unmodifiable list (so that it can return the same list if the code units contain
no surrogates, which will *often* be the case).
I'll file bugs for that, this is OK for now.

https://codereview.chromium.org/11293080/diff/2001/tests/utils/utf_test.dart
File tests/utils/utf_test.dart (right):

https://codereview.chromium.org/11293080/diff/2001/tests/utils/utf_test.dart#...
tests/utils/utf_test.dart:12: Expect.listEquals(str.charCodes, [0xd835,
0xdd37]);
Expected values should be first argument.

[Jennifer Messerly, 2012-11-06 18:41:20]

Thanks!

https://codereview.chromium.org/11293080/diff/2001/sdk/lib/utf/utf_core.dart
File sdk/lib/utf/utf_core.dart (right):

https://codereview.chromium.org/11293080/diff/2001/sdk/lib/utf/utf_core.dart#...
sdk/lib/utf/utf_core.dart:13: return _utf16CodeUnitsToCodepoints(str.charCodes);
On 2012/11/06 09:57:38, Lasse Reichstein Nielsen wrote:
> This seems wasteful since charCodes currently creates a new list, and not just
a
> thin wrapper around a String.
> We should probably both change that, and make this function return an
> unmodifiable list (so that it can return the same list if the code units
contain
> no surrogates, which will *often* be the case).
> I'll file bugs for that, this is OK for now.

Yeah, that sounds good. Thanks!

https://codereview.chromium.org/11293080/diff/2001/tests/utils/utf_test.dart
File tests/utils/utf_test.dart (right):

https://codereview.chromium.org/11293080/diff/2001/tests/utils/utf_test.dart#...
tests/utils/utf_test.dart:12: Expect.listEquals(str.charCodes, [0xd835,
0xdd37]);
On 2012/11/06 09:57:38, Lasse Reichstein Nielsen wrote:
> Expected values should be first argument.

Good catch. Done. I've been using the unittest "expect" for a while, which for
some reason has the opposite order.