Move unicode tests to utf package.

Move unicode tests to utf package.

BUG= http://dartbug.com/2252
R=lrn@google.com

Committed: https://code.google.com/p/dart/source/detail?r=30409

[floitsch, 2013-11-18 17:08:16]

https://codereview.chromium.org/68563004/diff/50001/pkg/utf/lib/utf16.dart
File pkg/utf/lib/utf16.dart (right):

https://codereview.chromium.org/68563004/diff/50001/pkg/utf/lib/utf16.dart#ne...
pkg/utf/lib/utf16.dart:7: // TODO(jmesserly): would be nice to have this on
String (dartbug.com/6501).
Whole section copied verbatim.

https://codereview.chromium.org/68563004/diff/50001/pkg/utf/test/utf16_test.dart
File pkg/utf/test/utf16_test.dart (right):

https://codereview.chromium.org/68563004/diff/50001/pkg/utf/test/utf16_test.d...
pkg/utf/test/utf16_test.dart:126: 0, null, stripBom)))[0]);
needed to change from named to positional arguments.

https://codereview.chromium.org/68563004/diff/50001/pkg/utf/test/utf32_test.dart
File pkg/utf/test/utf32_test.dart (right):

https://codereview.chromium.org/68563004/diff/50001/pkg/utf/test/utf32_test.d...
pkg/utf/test/utf32_test.dart:176: 0, null, stripBom)))[0]);
needed to change from named to positional arguments.

[Lasse Reichstein Nielsen, 2013-11-19 07:42:37]

Do any of these tests make sense for dart:convert too? Or instead?

https://codereview.chromium.org/68563004/diff/50001/pkg/utf/lib/utf16.dart
File pkg/utf/lib/utf16.dart (right):

https://codereview.chromium.org/68563004/diff/50001/pkg/utf/lib/utf16.dart#ne...
pkg/utf/lib/utf16.dart:7: // TODO(jmesserly): would be nice to have this on
String (dartbug.com/6501).
I don't think I want to add anything new to the utf package now.
It was kept for backwards compatibility, but new features should go into
dart:convert instead. The utf package isn't something we want to continue
developing.

https://codereview.chromium.org/68563004/diff/50001/pkg/utf/lib/utf16.dart#ne...
pkg/utf/lib/utf16.dart:14: return _utf16CodeUnitsToCodepoints(str.codeUnits);
If we keep it (and I don't think we should - if it's used by the tests, put it
there instead), then update it:
- Change this to "return str.runes.toList();" and remove the comment above.
- If _utf16CodeUnitsToCodepoints isn't used in other places, it can be removed
then.

https://codereview.chromium.org/68563004/diff/50001/pkg/utf/lib/utf16.dart#ne...
pkg/utf/lib/utf16.dart:20: * *Deprecated* Use [String.fromCharCodes] instead.
Ditto - remove this. Definitely remove the "Deprecated".

https://codereview.chromium.org/68563004/diff/50001/pkg/utf/lib/utf16.dart#ne...
pkg/utf/lib/utf16.dart:32: class Utf16CodeUnitDecoder implements Iterator<int> {
Do we have a way to use a Converter to go from input iterator to output
iterator? If not, we probably should.

https://codereview.chromium.org/68563004/diff/50001/pkg/utf/lib/utf16.dart#ne...
pkg/utf/lib/utf16.dart:102: List<int> _codepointsToUtf16CodeUnits(
If this isn't used, remove it. If it is, consider whether it can be replaced by:
  new String.fromCharCodes(codePoints.getRange(offset, length)).codeUnits;

https://codereview.chromium.org/68563004/diff/50001/pkg/utf/lib/utf16.dart#ne...
pkg/utf/lib/utf16.dart:147: List<int> _utf16CodeUnitsToCodepoints(
If not used, remove.
If used, consider rewriting as 
  new String.fromCharCodes(utf16CodeUnits.getRange(offset,
length)).runes.toList();

[floitsch, 2013-11-19 10:40:32]

I think we have pretty good UTF-8 coverage already. But I will look if they make
sense.

https://codereview.chromium.org/68563004/diff/50001/pkg/utf/lib/utf16.dart
File pkg/utf/lib/utf16.dart (right):

https://codereview.chromium.org/68563004/diff/50001/pkg/utf/lib/utf16.dart#ne...
pkg/utf/lib/utf16.dart:7: // TODO(jmesserly): would be nice to have this on
String (dartbug.com/6501).
On 2013/11/19 07:42:38, Lasse Reichstein Nielsen wrote:
> I don't think I want to add anything new to the utf package now.
> It was kept for backwards compatibility, but new features should go into
> dart:convert instead. The utf package isn't something we want to continue
> developing.

I don't agree.
The utf-package contains much more features than convert. I would like it to see
getting improved as a package.

When I say "copied verbatim" I meant from within the same package.

https://codereview.chromium.org/68563004/diff/50001/pkg/utf/lib/utf16.dart#ne...
pkg/utf/lib/utf16.dart:14: return _utf16CodeUnitsToCodepoints(str.codeUnits);
On 2013/11/19 07:42:38, Lasse Reichstein Nielsen wrote:
> If we keep it (and I don't think we should - if it's used by the tests, put it
> there instead), then update it:
> - Change this to "return str.runes.toList();" and remove the comment above.
> - If _utf16CodeUnitsToCodepoints isn't used in other places, it can be removed
> then.

This was code that already existed in the package. Not changing in this CL.

https://codereview.chromium.org/68563004/diff/50001/pkg/utf/lib/utf16.dart#ne...
pkg/utf/lib/utf16.dart:20: * *Deprecated* Use [String.fromCharCodes] instead.
On 2013/11/19 07:42:38, Lasse Reichstein Nielsen wrote:
> Ditto - remove this. Definitely remove the "Deprecated".

Not in this CL.

https://codereview.chromium.org/68563004/diff/50001/pkg/utf/lib/utf16.dart#ne...
pkg/utf/lib/utf16.dart:32: class Utf16CodeUnitDecoder implements Iterator<int> {
On 2013/11/19 07:42:38, Lasse Reichstein Nielsen wrote:
> Do we have a way to use a Converter to go from input iterator to output
> iterator? If not, we probably should.

Again. this is code that already existed. Not changing in this CL.

https://codereview.chromium.org/68563004/diff/50001/pkg/utf/lib/utf16.dart#ne...
pkg/utf/lib/utf16.dart:102: List<int> _codepointsToUtf16CodeUnits(
On 2013/11/19 07:42:38, Lasse Reichstein Nielsen wrote:
> If this isn't used, remove it. If it is, consider whether it can be replaced
by:
>   new String.fromCharCodes(codePoints.getRange(offset, length)).codeUnits;
> 

ditto.

https://codereview.chromium.org/68563004/diff/50001/pkg/utf/lib/utf16.dart#ne...
pkg/utf/lib/utf16.dart:147: List<int> _utf16CodeUnitsToCodepoints(
On 2013/11/19 07:42:38, Lasse Reichstein Nielsen wrote:
> If not used, remove.
> If used, consider rewriting as 
>   new String.fromCharCodes(utf16CodeUnits.getRange(offset,
> length)).runes.toList();

ditto.

[Lasse Reichstein Nielsen, 2013-11-19 12:25:42]

LGTM, and we should consider cleaning up the package at some point.

https://codereview.chromium.org/68563004/diff/50001/pkg/utf/lib/utf16.dart
File pkg/utf/lib/utf16.dart (right):

https://codereview.chromium.org/68563004/diff/50001/pkg/utf/lib/utf16.dart#ne...
pkg/utf/lib/utf16.dart:7: // TODO(jmesserly): would be nice to have this on
String (dartbug.com/6501).
From the same package - in that case, LGTM.

[floitsch, 2013-11-19 12:58:27]

Committed patchset #3 manually as r30409 (presubmit successful).