Added back some utf8 optimizations

Added back some utf8 optimizations

R=yangguo@chromium.org
BUG=https://code.google.com/p/v8/issues/detail?id=2551

Committed: https://code.google.com/p/v8/source/detail?r=13842

[Yang, 2013-03-05 15:32:51]

LGTM with comments.

https://codereview.chromium.org/12390057/diff/1/src/api.cc
File src/api.cc (right):

https://codereview.chromium.org/12390057/diff/1/src/api.cc#newcode3919
src/api.cc:3919: if (EndsWithSurrogate(leaf_state)) {
something to consider here:
http://graphics.stanford.edu/~seander/bithacks.html#ConditionalSetOrClearBits...

https://codereview.chromium.org/12390057/diff/1/src/api.cc#newcode4112
src/api.cc:4112: if (sizeof(Char) == 1) {
It looks like this if should be hoisted to outside the for-loop.

[drcarney, 2013-03-06 15:39:21]

https://codereview.chromium.org/12390057/diff/1/src/api.cc
File src/api.cc (right):

https://codereview.chromium.org/12390057/diff/1/src/api.cc#newcode3919
src/api.cc:3919: if (EndsWithSurrogate(leaf_state)) {
On 2013/03/05 15:32:51, Yang wrote:
> something to consider here:
>
http://graphics.stanford.edu/%7Eseander/bithacks.html#ConditionalSetOrClearBi...

will handle in the next patch which is a performance tweak

https://codereview.chromium.org/12390057/diff/1/src/api.cc#newcode4112
src/api.cc:4112: if (sizeof(Char) == 1) {
On 2013/03/05 15:32:51, Yang wrote:
> It looks like this if should be hoisted to outside the for-loop.

done

[dcarney, 2013-03-06 15:40:03]

Committed manually as r13842 (presubmit successful).

[Erik Corry, 2013-03-07 09:38:37]

https://codereview.chromium.org/12390057/diff/4001/src/api.cc
File src/api.cc (right):

https://codereview.chromium.org/12390057/diff/4001/src/api.cc#newcode3864
src/api.cc:3864: int last_character = unibrow::Utf16::kNoPreviousCharacter;
I wonder if it's worth modifying this to take advantage of the fact that Latin1
never has surrogate pairs and the high bit of the character distinguishes 1-byte
from 2-byte UTF-8 encodings.

if (sizeof(Char) == 2) {
  // Current implementation
} else {
  for (int i = 0; i < length; i++) {
    utf8_length += (chars[i] >> 7);  // Assume unsigned.
  }
  utf8_length += length;
}

This has the advantage that there is no branch in the inner loop, which is
normally great for deeply pipelined CPUs.

In general it seems that this could be made vastly simpler for Latin1 strings,
perhaps by having a completely different visitor.  It's really a rather trivial
operation for Latin1 (not as simple as ASCII, but still...)