more work aligning fasta.Token with analyzer.Token

more work aligning fasta.Token with analyzer.Token

* move isPseudo getter out of Token into TokenType
* move special case charCount logic out of fasta.Token into fasta.ErrorToken
* move more accessors from fasta.Token into analyzer.Token
* add @overrides to denote which methods have been declared in analyzer.Token
* make fasta LazySubstring and its subclasses private
* rename fasta.previousToken to previous to match analyzer.Token
* flatten fasta.Token.preceedingCommentTokens into preceedingComments
* normalize fasta.Token.next and analyzer.Token.next

R=scheglov@google.com

Committed: https://github.com/dart-lang/sdk/commit/81d25f4c71e44e08fa94ea1616e7b5d97af20987

[danrubel, 2017-05-09 01:21:13]

danrubel@google.com changed reviewers:
+ paulberry@google.com, scheglov@google.com

[scheglov, 2017-05-09 02:14:40]

LGTM

[danrubel, 2017-05-09 04:10:03]

Description was changed from

==========
more work aligning fasta.Token with analyzer.Token

* move isPseudo getter out of Token into TokenType
* move special case charCount logic out of fasta.Token into fasta.ErrorToken
* move more accessors from fasta.Token into analyzer.Token
* add @overrides to denote which methods have been declared in analyzer.Token
* make fasta LazySubstring and its subclasses private
* rename fasta.previousToken to previous to match analyzer.Token
* flatten fasta.Token.preceedingCommentTokens into preceedingComments
* normalize fasta.Token.next and analyzer.Token.next
==========

to

==========
more work aligning fasta.Token with analyzer.Token

* move isPseudo getter out of Token into TokenType
* move special case charCount logic out of fasta.Token into fasta.ErrorToken
* move more accessors from fasta.Token into analyzer.Token
* add @overrides to denote which methods have been declared in analyzer.Token
* make fasta LazySubstring and its subclasses private
* rename fasta.previousToken to previous to match analyzer.Token
* flatten fasta.Token.preceedingCommentTokens into preceedingComments
* normalize fasta.Token.next and analyzer.Token.next

R=scheglov@google.com

Committed:
https://github.com/dart-lang/sdk/commit/81d25f4c71e44e08fa94ea1616e7b5d97af20987
==========

[danrubel, 2017-05-09 04:10:04]

Committed patchset #1 (id:1) manually as
81d25f4c71e44e08fa94ea1616e7b5d97af20987 (presubmit successful).

[ahe, 2017-05-15 09:23:12]

https://codereview.chromium.org/2872433005/diff/1/pkg/front_end/lib/src/fasta...
File pkg/front_end/lib/src/fasta/parser/parser.dart (right):

https://codereview.chromium.org/2872433005/diff/1/pkg/front_end/lib/src/fasta...
pkg/front_end/lib/src/fasta/parser/parser.dart:1054: } else if (!inPlainSync &&
token.type.isPseudo) {
Why this change?

[danrubel, 2017-05-15 13:03:18]

https://codereview.chromium.org/2872433005/diff/1/pkg/front_end/lib/src/fasta...
File pkg/front_end/lib/src/fasta/parser/parser.dart (right):

https://codereview.chromium.org/2872433005/diff/1/pkg/front_end/lib/src/fasta...
pkg/front_end/lib/src/fasta/parser/parser.dart:1054: } else if (!inPlainSync &&
token.type.isPseudo) {
On 2017/05/15 09:23:12, ahe wrote:
> Why this change?

There was no corresponding `isPseudo` accessor in analyzer.Token and that
accessor always forwards to the type. Rather than add yet another accessor to
Token, I simply call `token.type.isPseudo`.  There are two other accessors in
analyzer.Token that do the same that I'd also like to remove, but I'll leave
that for another day.

All that said, if its important to have an `isPseudo` accessor on analyzer.Token
for performance or other reasons, then please let me know and I can add it and
refactor this call site to use it.