Update dart:io to convert strings between UTF8 and current code page

Update dart:io to convert strings between UTF8 and current code page
when interacting with the system.

What we get from and need to hand to the VM is utf8. What we
get from and need to hand to the system is in the current
code page.

R=sgjesse@google.com
BUG=

Committed: https://code.google.com/p/dart/source/detail?r=14851

[Mads Ager (google), 2012-11-13 12:03:20]

The combination of "const char*" and "malloc/free" is annoying. This is the
current local optimum. Added tests that should pass once strings posted through
native ports are utf8 encoded.

[Mads Ager (google), 2012-11-13 12:04:15]

https://codereview.chromium.org/11275281/diff/1/tests/standalone/float_array_...
File tests/standalone/float_array_test.dart (left):

https://codereview.chromium.org/11275281/diff/1/tests/standalone/float_array_...
tests/standalone/float_array_test.dart:1: // Copyright (c) 2012, the Dart
project authors.  Please see the AUTHORS file
This file has dos line-endings for no reason. Removing those is the only change.

[Søren Gjesse, 2012-11-13 12:48:45]

lgtm

https://codereview.chromium.org/11275281/diff/1/runtime/bin/directory_win.cc
File runtime/bin/directory_win.cc (right):

https://codereview.chromium.org/11275281/diff/1/runtime/bin/directory_win.cc#...
runtime/bin/directory_win.cc:56: char* utf8_encoded_path =
StringUtils::SystemStringToUtf8(path);
Maybe remove "encoded" from utf8_encoded_path to shorten the name.

https://codereview.chromium.org/11275281/diff/1/runtime/bin/directory_win.cc#...
runtime/bin/directory_win.cc:57: bool ok =
listing->HandleDirectory(utf8_encoded_path);
Maybe add ASSERT(utf8_encoded_path != path).

https://codereview.chromium.org/11275281/diff/1/runtime/bin/main.cc
File runtime/bin/main.cc (right):

https://codereview.chromium.org/11275281/diff/1/runtime/bin/main.cc#newcode236
runtime/bin/main.cc:236: // encoded in the current code page and not UTF8.
Document return value.

https://codereview.chromium.org/11275281/diff/1/runtime/bin/main.cc#newcode241
runtime/bin/main.cc:241: if (argv[i] != arg) result = true;
Maybe change this to

if (argv == 0) {
  result = argv[i] != arg;
} else {
  ASSERT(result == (argv[i] != arg));
}

https://codereview.chromium.org/11275281/diff/1/runtime/bin/main.cc#newcode652
runtime/bin/main.cc:652: // On Windows, the argv strings are code page incoded
and not
incoded -> encoded

https://codereview.chromium.org/11275281/diff/1/runtime/bin/utils_win.cc
File runtime/bin/utils_win.cc (right):

https://codereview.chromium.org/11275281/diff/1/runtime/bin/utils_win.cc#newc...
runtime/bin/utils_win.cc:54: len = WideCharToMultiByte(CP_UTF8, 0, unicode, -1,
NULL, 0, NULL, NULL);
If the len here and the len above are then same then it must be an ASCII string
and you should be able to just return str. That of cause means that you will
have to check whether to free or not every time this method is used.

Or maybe check for ASCII string up-front.

[Mads Ager (google), 2012-11-13 14:50:17]

https://codereview.chromium.org/11275281/diff/1/runtime/bin/directory_win.cc
File runtime/bin/directory_win.cc (right):

https://codereview.chromium.org/11275281/diff/1/runtime/bin/directory_win.cc#...
runtime/bin/directory_win.cc:56: char* utf8_encoded_path =
StringUtils::SystemStringToUtf8(path);
On 2012/11/13 12:48:45, Søren Gjesse wrote:
> Maybe remove "encoded" from utf8_encoded_path to shorten the name.

Done for all of them.

https://codereview.chromium.org/11275281/diff/1/runtime/bin/directory_win.cc#...
runtime/bin/directory_win.cc:57: bool ok =
listing->HandleDirectory(utf8_encoded_path);
On 2012/11/13 12:48:45, Søren Gjesse wrote:
> Maybe add ASSERT(utf8_encoded_path != path).

I think I would rather go with your suggestion of not converting when getting
ascii. In that case I need to add a guarding test here.

https://codereview.chromium.org/11275281/diff/1/runtime/bin/main.cc
File runtime/bin/main.cc (right):

https://codereview.chromium.org/11275281/diff/1/runtime/bin/main.cc#newcode236
runtime/bin/main.cc:236: // encoded in the current code page and not UTF8.
On 2012/11/13 12:48:45, Søren Gjesse wrote:
> Document return value.

Done.

https://codereview.chromium.org/11275281/diff/1/runtime/bin/main.cc#newcode241
runtime/bin/main.cc:241: if (argv[i] != arg) result = true;
On 2012/11/13 12:48:45, Søren Gjesse wrote:
> Maybe change this to
> 
> if (argv == 0) {
>   result = argv[i] != arg;
> } else {
>   ASSERT(result == (argv[i] != arg));
> }

Done.

https://codereview.chromium.org/11275281/diff/1/runtime/bin/main.cc#newcode652
runtime/bin/main.cc:652: // On Windows, the argv strings are code page incoded
and not
On 2012/11/13 12:48:45, Søren Gjesse wrote:
> incoded -> encoded

Done.

https://codereview.chromium.org/11275281/diff/1/runtime/bin/utils_win.cc
File runtime/bin/utils_win.cc (right):

https://codereview.chromium.org/11275281/diff/1/runtime/bin/utils_win.cc#newc...
runtime/bin/utils_win.cc:54: len = WideCharToMultiByte(CP_UTF8, 0, unicode, -1,
NULL, 0, NULL, NULL);
On 2012/11/13 12:48:45, Søren Gjesse wrote:
> If the len here and the len above are then same then it must be an ASCII
string
> and you should be able to just return str. That of cause means that you will
> have to check whether to free or not every time this method is used.
> 
> Or maybe check for ASCII string up-front.

Yeah, I think I'll save that for a separate change. Using the standard trick of
taking four bytes at a time and masking off an iteration up front for the common
ascii case is probably the cheapest.