Fixes in dart2js parser to be able to parse files without the diet-parser

Fixes in dart2js parser to be able to parse files without the diet-parser

R=ahe@google.com

Committed: https://github.com/dart-lang/sdk/commit/b39b0d181178e0b12fe9e6d376b7151479db6cb4

[Siggi Cherem (dart-lang), 2016-09-30 22:21:22]

Patchset #1 (id:1) has been deleted

[Siggi Cherem (dart-lang), 2016-09-30 22:28:07]

sigmund@google.com changed reviewers:
+ ahe@google.com, johnniwinther@google.com

[Siggi Cherem (dart-lang), 2016-09-30 22:28:07]

dart2js always uses the "full-parser" for members, but not to do the full parse
of a file because we use the diet-parser for that. I discovered that the parser
breaks if you try to do a full parse, this CL fixes those issues.

Peter - I'll wait for your review, I'm curious to hear if you prefer a different
approach than what I'm doing here.

Background: I'm creating benchmarks for the frontend. See
https://codereview.chromium.org/2385893002/ for an example script that loads a
program and measures the time it takes to scan and parse it end-to-end.

https://codereview.chromium.org/2385643003/diff/20001/pkg/compiler/lib/src/pa...
File pkg/compiler/lib/src/parser/member_listener.dart (right):

https://codereview.chromium.org/2385643003/diff/20001/pkg/compiler/lib/src/pa...
pkg/compiler/lib/src/parser/member_listener.dart:150:
super.endMetadata(beginToken, periodBeforeName, endToken);
note: this will keep the metadata node in the stack

https://codereview.chromium.org/2385643003/diff/20001/pkg/compiler/lib/src/pa...
File pkg/compiler/lib/src/parser/node_listener.dart (right):

https://codereview.chromium.org/2385643003/diff/20001/pkg/compiler/lib/src/pa...
pkg/compiler/lib/src/parser/node_listener.dart:148: pushNode(new
FunctionExpression(name, typeVariables, formals, body, type,
note: this was inconsistent (likely copy/paste error?) - it was never reached in
dart2js, but it will be reached when using the parser alone without the
diet-parser.

https://codereview.chromium.org/2385643003/diff/20001/pkg/compiler/lib/src/pa...
File pkg/compiler/lib/src/parser/parser.dart (right):

https://codereview.chromium.org/2385643003/diff/20001/pkg/compiler/lib/src/pa...
pkg/compiler/lib/src/parser/parser.dart:123:
listener.endTopLevelDeclaration(token);
note: this moved down here because when we parse a single element we ended up
with a stack containing an extra node (the metadata). Moving this here makes the
parsing and listening always happen together, so calling
parseTopLevelDeclaration after doing a diet-parse continues to work as expected.

https://codereview.chromium.org/2385643003/diff/20001/pkg/compiler/lib/src/pa...
File pkg/compiler/lib/src/parser/partial_parser.dart (right):

https://codereview.chromium.org/2385643003/diff/20001/pkg/compiler/lib/src/pa...
pkg/compiler/lib/src/parser/partial_parser.dart:29:
listener.handleNoArguments(token);
note: listeners now assume that metadata is always parsed, without this we had
an inconsistent state where the arguments of an annotation where missing.

question: in reason #1 above: was "generating events" meant calling listeners
like I do here? Should we remove that reason now?

[ahe, 2016-10-03 10:14:19]

Siggi, if it's OK with you, I'd like to review this after our 1:1 on Thursday.
This way I'll be able to understand your approach and constraints before I start
looking at the code.

[Siggi Cherem (dart-lang), 2016-10-07 20:55:41]

friendly ping :)

[ahe, 2016-10-10 10:55:48]

lgtm

[Siggi Cherem (dart-lang), 2016-10-10 19:22:01]

Description was changed from

==========
Fixes in dart2js parser to be able to parse files without the diet-parser
==========

to

==========
Fixes in dart2js parser to be able to parse files without the diet-parser

R=ahe@google.com

Committed:
https://github.com/dart-lang/sdk/commit/b39b0d181178e0b12fe9e6d376b7151479db6cb4
==========

[Siggi Cherem (dart-lang), 2016-10-10 19:22:03]

Committed patchset #1 (id:20001) manually as
b39b0d181178e0b12fe9e6d376b7151479db6cb4 (presubmit successful).