Sloppy-mode let parsing

Sloppy-mode let parsing

This patch makes 'let' a contextual keyword in both strict and sloppy mode.
It behaves as a keyword when used at the beginning of a StatementListItem
or lexical declaration at the beginning of a for statement, if it is followed
by an identifier, [ or {. Implementing this change requires an extra token
look-ahead by the parser which is only invoked in certain cases (so as to
avoid parsing RegExps as ECMAScript tokens). This might result in a slowdown
of the scanner, but performance testing of this patch hasn't yet found much
of a regression.

BUG=v8:3305
LOG=Y
R=adamk,vogelheim

Committed: https://crrev.com/decc7b092ae90a378423df760690d58585e9ac71
Cr-Commit-Position: refs/heads/master@{#30451}

[Dan Ehrenberg, 2015-08-27 22:24:22]

This was previously reviewed at https://codereview.chromium.org/1295883002 ;
uploaded here because I couldn't get Rietveldt to resume a patch from a
different checkout when there were conflicts in the cherry-pick.

Compared to the last version, this version tries to fix a bookmark-related issue
where next_next_ is not stored in the bookmark by refusing to save a bookmark
when it is initialized.

[Dan Ehrenberg, 2015-08-27 22:24:26]

The CQ bit was checked by littledan@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2015-08-27 22:24:34]

Dry run: CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1315673009/1
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1315673009/1

[adamk, 2015-08-28 00:13:22]

lgtm, but would like to see vogelheim's OK on the bookmark stuff (wonder if
there's a way to test that).

Also, could you update the CL description with the 2-3% number? Or do you want
to wait to see if the UNLIKELY macro smooths that out?

[vogelheim, 2015-08-28 11:01:04]

lgtm - bookmarks look fine.

https://codereview.chromium.org/1315673009/diff/20001/src/scanner.h
File src/scanner.h (right):

https://codereview.chromium.org/1315673009/diff/20001/src/scanner.h#newcode714
src/scanner.h:714: TokenDesc next_next_;  // desc for the token after next
(push-back)
What do you mean with "(push-back)"?

[vogelheim, 2015-08-28 11:03:45]

On 2015/08/28 00:13:22, adamk wrote:
> lgtm, but would like to see vogelheim's OK on the bookmark stuff (wonder if
> there's a way to test that).

Unfortunately I didn't put in proper tests for the bookmarks. Sorry.

Bookmarking may fail (for any reason), so just not bookmarking is fine. The only
use-case for the bookmarking is re-deciding whether to eager/lazy-parse a
function and as I read the code I wouldn't expect next_next_ to ever have a
value in that case.

[Dan Ehrenberg, 2015-08-28 17:39:55]

The CQ bit was checked by littledan@chromium.org

[Dan Ehrenberg, 2015-08-28 17:39:55]

The patchset sent to the CQ was uploaded after l-g-t-m from adamk@chromium.org,
vogelheim@chromium.org
Link to the patchset: https://codereview.chromium.org/1315673009/#ps40001
(title: "clarify comment")

[commit-bot: I haz the power, 2015-08-28 17:40:03]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1315673009/40001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1315673009/40001

[commit-bot: I haz the power, 2015-08-28 18:47:37]

Committed patchset #3 (id:40001)

[commit-bot: I haz the power, 2015-08-28 18:47:51]

Patchset 3 (id:??) landed as
https://crrev.com/decc7b092ae90a378423df760690d58585e9ac71
Cr-Commit-Position: refs/heads/master@{#30451}