Add support for non ASCII strings when communicating with native ports

Add support for non ASCII strings when communicating with native ports

The string representation in a Dart_CObject strructure is now
UTF8. All strings read are now converted to UTF8 from either ASCII or
UTF16 serialization. All strings posted should be valid UTF8 and are
serialized as either ASCII or UTF16 depending on the content.

Proper andling of surrogate pairs is missing, but will be added when
https://codereview.chromium.org/11368138/ lands.

R=ager@google.com, erikcorry@google.com, asiva@google.com

BUG=

Committed: https://code.google.com/p/dart/source/detail?r=14887

[erikcorry, 2012-11-12 11:44:06]

lgtm

https://codereview.chromium.org/11410032/diff/1/runtime/vm/snapshot_test.cc
File runtime/vm/snapshot_test.cc (right):

https://codereview.chromium.org/11410032/diff/1/runtime/vm/snapshot_test.cc#n...
runtime/vm/snapshot_test.cc:538: // TODO(sgjesse): Add tests with non-BMP
characters.
You could add tests for isolated surrogates like 0xd800, 0xdbff right now.

[Mads Ager (google), 2012-11-12 12:11:26]

lgtm

[Søren Gjesse, 2012-11-12 16:18:10]

Mads, please take a look at patch set #2 as well.

https://codereview.chromium.org/11410032/diff/1/runtime/vm/snapshot_test.cc
File runtime/vm/snapshot_test.cc (right):

https://codereview.chromium.org/11410032/diff/1/runtime/vm/snapshot_test.cc#n...
runtime/vm/snapshot_test.cc:538: // TODO(sgjesse): Add tests with non-BMP
characters.
On 2012/11/12 11:44:07, erikcorry wrote:
> You could add tests for isolated surrogates like 0xd800, 0xdbff right now.

I can't do that. If I create a message with an valid UTF-8 encoded string, and
the UTF-8 encoding of r.g. 0xd800 is not considered valid by the Dart VM unicode
library.

However as messages posted from native threads can contain strings with invalid
UTF-8 sequences we need to check that when posting a Dart_CObject structure.
Added checking for thet when writing.

[Mads Ager (google), 2012-11-12 18:12:15]

LGTM. Thanks for adding the extra checking. That should save me some time when I
test this with dart:io! :)

[Ivan Posva, 2012-11-14 19:07:15]

DBC -ip

https://codereview.chromium.org/11410032/diff/9003/runtime/vm/dart_api_impl.cc
File runtime/vm/dart_api_impl.cc (right):

https://codereview.chromium.org/11410032/diff/9003/runtime/vm/dart_api_impl.c...
runtime/vm/dart_api_impl.cc:1023: Dart_CObject* message) {
No need to wrap the line.

https://codereview.chromium.org/11410032/diff/9003/tests/standalone/standalon...
File tests/standalone/standalone.status (right):

https://codereview.chromium.org/11410032/diff/9003/tests/standalone/standalon...
tests/standalone/standalone.status:9: [ $system == macos ]
This empty qualifier makes no sense.

[Søren Gjesse, 2012-11-15 08:16:53]

https://codereview.chromium.org/11410032/diff/9003/runtime/vm/dart_api_impl.cc
File runtime/vm/dart_api_impl.cc (right):

https://codereview.chromium.org/11410032/diff/9003/runtime/vm/dart_api_impl.c...
runtime/vm/dart_api_impl.cc:1023: Dart_CObject* message) {
On 2012/11/14 19:07:15, Ivan Posva wrote:
> No need to wrap the line.

https://codereview.chromium.org//11308031.

https://codereview.chromium.org/11410032/diff/9003/tests/standalone/standalon...
File tests/standalone/standalone.status (right):

https://codereview.chromium.org/11410032/diff/9003/tests/standalone/standalon...
tests/standalone/standalone.status:9: [ $system == macos ]
On 2012/11/14 19:07:15, Ivan Posva wrote:
> This empty qualifier makes no sense.

Mads removed this in
https://dart.googlecode.com/svn/branches/bleeding_edge/dart@14891.