add previous token field

add previous token field

This adds fasta Token.previous and sets that field to point
to the previous token in the fasta token stream.

R=paulberry@google.com

Committed: https://github.com/dart-lang/sdk/commit/dd3ab83eee64be2646228efbd12d4189f9806297

[danrubel, 2017-03-07 03:12:55]

danrubel@google.com changed reviewers:
+ ahe@google.com, paulberry@google.com

[Paul Berry, 2017-03-07 13:47:28]

lgtm

https://codereview.chromium.org/2739583002/diff/1/pkg/front_end/lib/src/fasta...
File pkg/front_end/lib/src/fasta/scanner/token.dart (right):

https://codereview.chromium.org/2739583002/diff/1/pkg/front_end/lib/src/fasta...
pkg/front_end/lib/src/fasta/scanner/token.dart:33: * @deprecated This exists for
compatibility with the Analyzer token stream
Saying "@deprecated" here doesn't have any effect on analysis.  If we want to
deprecate it, we should do:

@deprecated
Token previous;

so that references to it will be flagged by analyzer.

(Note that we can always suppress such warnings using the "// ignore:" syntax)

[ahe, 2017-03-07 14:23:51]

I have a strong preference for not adding stuff that is already deprecated. If
you want to remove something, don't add it.

I'd like to make sure that we understand the performance of this before we land
it.

[danrubel, 2017-03-07 14:33:45]

On 2017/03/07 14:23:51, ahe wrote:
> I have a strong preference for not adding stuff that is already deprecated. If
> you want to remove something, don't add it.
> 
> I'd like to make sure that we understand the performance of this before we
land
> it.

My long term goal is to have fasta.Token implement analyzer.Token and work to
minimize any performance impact. And in the long run I want to remove the
previous field from analyzer tokens and from fasta tokens. I'll have a
performance impact analysis for your later today.

In the meantime, I've updated this CL to rename "previous" --> "previousToken"
so that the name does not collide with analyzer.Token.previous.

[danrubel, 2017-03-07 14:40:54]

https://codereview.chromium.org/2739583002/diff/1/pkg/front_end/lib/src/fasta...
File pkg/front_end/lib/src/fasta/scanner/token.dart (right):

https://codereview.chromium.org/2739583002/diff/1/pkg/front_end/lib/src/fasta...
pkg/front_end/lib/src/fasta/scanner/token.dart:33: * @deprecated This exists for
compatibility with the Analyzer token stream
On 2017/03/07 13:47:28, Paul Berry wrote:
> Saying "@deprecated" here doesn't have any effect on analysis.  If we want to
> deprecate it, we should do:
> 
> @deprecated
> Token previous;
> 
> so that references to it will be flagged by analyzer.
> 
> (Note that we can always suppress such warnings using the "// ignore:" syntax)

Good catch. That's what I meant to do. Fixed.

[danrubel, 2017-03-07 18:03:59]

Description was changed from

==========
add previous token field

This adds fasta Token.previous and sets that field to point
to the previous token in the fasta token stream.
==========

to

==========
add previous token field

This adds fasta Token.previous and sets that field to point
to the previous token in the fasta token stream.

R=paulberry@google.com

Committed:
https://github.com/dart-lang/sdk/commit/dd3ab83eee64be2646228efbd12d4189f9806297
==========

[danrubel, 2017-03-07 18:04:01]

Committed patchset #5 (id:80001) manually as
dd3ab83eee64be2646228efbd12d4189f9806297 (presubmit successful).