Make scanner handle invalid unicode escapes in identifiers correctly.

Make scanner handle invalid unicode escapes in identifiers correctly.

I.e., don't just convert \u to u in identifiers (like in strings and regexps).

Also make the scanning of RegExp flags not interpret the escapes.

(Fix and reapply of r8942)


BUG=v8:1620
TEST=mjsunit/regress/regress-1620

Committed: http://code.google.com/p/v8/source/detail?r=8969

[Rico, 2011-08-18 11:43:13]

LGTM

http://codereview.chromium.org/7677012/diff/1/src/scanner-base.cc
File src/scanner-base.cc (right):

http://codereview.chromium.org/7677012/diff/1/src/scanner-base.cc#newcode1068
src/scanner-base.cc:1068: return true;
Indention seems wrong

http://codereview.chromium.org/7677012/diff/1/test/mjsunit/regress/regress-16...
File test/mjsunit/regress/regress-1620.js (right):

http://codereview.chromium.org/7677012/diff/1/test/mjsunit/regress/regress-16...
test/mjsunit/regress/regress-1620.js:28: // See:
http://code.google.com/p/v8/issues/detail?id=1620
Please also state the reason here, this code could easily outlive the issue
tracker

[Lasse Reichstein, 2011-08-24 13:36:28]

http://codereview.chromium.org/7677012/diff/1/src/scanner-base.cc
File src/scanner-base.cc (right):

http://codereview.chromium.org/7677012/diff/1/src/scanner-base.cc#newcode1068
src/scanner-base.cc:1068: return true;
Argh, more than wrong. The return has moved itself above the end '}' of the
loop. Will add more tests since we didn't catch such a blatant bug.