Extract DocumentationCommentToken with field.

Extract DocumentationCommentToken with  field.

R=brianwilkerson@google.com
BUG=

Committed: https://code.google.com/p/dart/source/detail?r=42056

[Brian Wilkerson, 2014-12-01 23:08:43]

LGTM once the parsing problem is fixed.

https://codereview.chromium.org/768233002/diff/1/pkg/analyzer/lib/src/generat...
File pkg/analyzer/lib/src/generated/parser.dart (right):

https://codereview.chromium.org/768233002/diff/1/pkg/analyzer/lib/src/generat...
pkg/analyzer/lib/src/generated/parser.dart:6027:
documentationTokens.add(commentToken);
This doesn't give us the semantics we want (obviously we're missing test cases)
because it allows a combination of block and end-of-line comments to all provide
a single documentation comment, such as

  /**
   * This is a single
   */
  /// documentation comment.

Previously we would keep either the last block comment or the end-of-line
comments that were not followed by a block comment.

[scheglov, 2014-12-01 23:27:53]

PTAL

https://codereview.chromium.org/768233002/diff/1/pkg/analyzer/lib/src/generat...
File pkg/analyzer/lib/src/generated/parser.dart (right):

https://codereview.chromium.org/768233002/diff/1/pkg/analyzer/lib/src/generat...
pkg/analyzer/lib/src/generated/parser.dart:6027:
documentationTokens.add(commentToken);
On 2014/12/01 23:08:43, Brian Wilkerson wrote:
> This doesn't give us the semantics we want (obviously we're missing test
cases)
> because it allows a combination of block and end-of-line comments to all
provide
> a single documentation comment, such as
> 
>   /**
>    * This is a single
>    */
>   /// documentation comment.
> 
> Previously we would keep either the last block comment or the end-of-line
> comments that were not followed by a block comment.

Done.

[Brian Wilkerson, 2014-12-02 00:52:57]

LGTM

[scheglov, 2014-12-02 01:39:30]

Committed patchset #2 (id:20001) manually as 42056 (presubmit successful).