Reject high- and low-surrogates in escaped code points.

Reject high- and low-surrogates in escaped code points.

BUG=3097
Committed: https://code.google.com/p/dart/source/detail?r=7725

[cshapiro, 2012-05-17 04:01:31]

I would like to validate more than just the first character of the token stream.
 However, I get junk because the literal parsing aborts midway.

Scanning: <"\uD800">
pos  1:7-7 token kERROR  invalid code point
pos  1:7-8 token kINTEGER  0
pos  1:9-9 token kERROR  unterminated string literal
pos  1:9-9 token kEOS  
4 tokens in stream.

Scanning: <"\u{D800}">
pos  1:9-9 token kERROR  invalid code point
pos  1:9-10 token kRBRACE  
pos  1:11-11 token kERROR  unterminated string literal
pos  1:11-11 token kEOS  
4 tokens in stream.

Is it safe to assume that, at least in the near future, the stuff after the
kERROR will remain the same with time?

[hausner, 2012-05-17 15:37:53]

I'm not quite sure what you are asking about the junk after kERROR. I guess the
scanner could try to sync to a safe point after a malformed code point and
continue to scan. In your examples, it seems that the problem is the
EndStringLiteral() call in line 541 (in your change). When the code point is not
malformed syntactically but is simply an illegal value, maybe we should not
abort parsing the string.

Otherwise LGTM.

[hausner, 2012-05-17 22:52:10]

lgtm