Scanner: remove PushBack calls when we're going to return ILLEGAL.

Scanner: remove PushBack calls when we're going to return ILLEGAL.

This simplifies escape handling and makes it easier to extend escapes for ES6.

PushBack just before detecting ILLEGAL is unnecessary, since we will abort the
scanning / parsing anyway at that point, and it doesn't matter where the cursor
exactly is. The error messages w/ PushBack are not any better or more correct
than without.

In addition: remove a comment about handling invalid escapes gracefully when we
no longer do. (*)

This CL includes a behavioral change: For input "var r = /foobar/g\urrrr;" we
used to report "unexpected_token: ILLEGAL" for "\u", but now we report
malformed_regexp_flags which is a more correct error message. (Note that the
code for reporting invalid_regexp_flags was dead, and invalid_regexp_flags is
not the right error message.)

Note that the V8 is more relaxed about unicode escapes in regexp flags than ES6
(see
http://people.mozilla.org/~jorendorff/es6-draft.html#sec-regular-expressions )
and this CL doesn't change it. (V8 accepts any \uxxxx, ES6 spec says only a
certain value range is acceptable.)

(*) Code archaeology:

Originally, doing PushBack in ScanHexEscape made sense (see e.g., here
https://codereview.chromium.org/5063003/diff/6001/src/prescanner.h ), since we
wouldn't return ILLEGAL but treat an invalid escape sequence "\uxxxx" as
"uxxxx".

(The repo at that point contains another instance of the same function, from the
initial commit. The logic is the same.)

This behavior was changed in a "renaming" commit
https://codereview.chromium.org/7739020.

BUG=
R=rossberg@chromium.org

Committed: https://code.google.com/p/v8/source/detail?r=25031

[marja, 2014-10-31 09:02:59]

marja@chromium.org changed reviewers:
+ rossberg@chromium.org

[marja, 2014-10-31 09:03:00]

rossberg, ptal

[rossberg, 2014-10-31 10:02:59]

lgtm

[marja, 2014-10-31 13:03:27]

Committed patchset #3 (id:40001) manually as 25031 (presubmit successful).