Change toAnalyzerTokenStream into a class.

Change toAnalyzerTokenStream into a class.

This will allow it to be subclassed in parser tests, which will need
to record the relationship between analyzer and fasta tokens.

R=ahe@google.com

Committed: https://github.com/dart-lang/sdk/commit/af9e2fb7918bc8e64d7d6ff8a07eabc3b8503b94

[Paul Berry, 2017-02-15 21:55:42]

paulberry@google.com changed reviewers:
+ ahe@google.com

[ahe, 2017-02-15 22:09:20]

lgtm

https://codereview.chromium.org/2693403002/diff/20001/pkg/front_end/lib/src/f...
File pkg/front_end/lib/src/fasta/analyzer/token_utils.dart (right):

https://codereview.chromium.org/2693403002/diff/20001/pkg/front_end/lib/src/f...
pkg/front_end/lib/src/fasta/analyzer/token_utils.dart:103: while (true) {
Very very optional: I'm always looking for infinite loops in parsers and
scanners, so while (true) always raises a flag for me. For this reason, I like
to make the loop condition be "!token.isEof". In this case, I might even go as
far as to use a for-loop:

for (; !token.isEof; token = token.next) {
...

However, I have no problem in seeing that this loops terminates as it is.

https://codereview.chromium.org/2693403002/diff/20001/pkg/front_end/lib/src/f...
pkg/front_end/lib/src/fasta/analyzer/token_utils.dart:164: /// in the way
analyzer expects.
FYI: If you haven't noticed it already: there's documentation about these
synthetic tokens in AbstractScanner.unmatchedBeginGroup.

[Paul Berry, 2017-02-16 02:01:23]

https://codereview.chromium.org/2693403002/diff/20001/pkg/front_end/lib/src/f...
File pkg/front_end/lib/src/fasta/analyzer/token_utils.dart (right):

https://codereview.chromium.org/2693403002/diff/20001/pkg/front_end/lib/src/f...
pkg/front_end/lib/src/fasta/analyzer/token_utils.dart:103: while (true) {
On 2017/02/15 22:09:20, ahe wrote:
> Very very optional: I'm always looking for infinite loops in parsers and
> scanners, so while (true) always raises a flag for me. For this reason, I like
> to make the loop condition be "!token.isEof". In this case, I might even go as
> far as to use a for-loop:
> 
> for (; !token.isEof; token = token.next) {
> ...
> 
> However, I have no problem in seeing that this loops terminates as it is.

Acknowledged.

https://codereview.chromium.org/2693403002/diff/20001/pkg/front_end/lib/src/f...
pkg/front_end/lib/src/fasta/analyzer/token_utils.dart:164: /// in the way
analyzer expects.
On 2017/02/15 22:09:20, ahe wrote:
> FYI: If you haven't noticed it already: there's documentation about these
> synthetic tokens in AbstractScanner.unmatchedBeginGroup.

Thanks for the pointer!

[Paul Berry, 2017-02-16 02:09:40]

Description was changed from

==========
Change toAnalyzerTokenStream into a class.

This will allow it to be subclassed in parser tests, which will need
to record the relationship between analyzer and fasta tokens.
==========

to

==========
Change toAnalyzerTokenStream into a class.

This will allow it to be subclassed in parser tests, which will need
to record the relationship between analyzer and fasta tokens.

R=ahe@google.com

Committed:
https://github.com/dart-lang/sdk/commit/af9e2fb7918bc8e64d7d6ff8a07eabc3b8503b94
==========

[Paul Berry, 2017-02-16 02:09:44]

Committed patchset #2 (id:20001) manually as
af9e2fb7918bc8e64d7d6ff8a07eabc3b8503b94 (presubmit successful).