- Ensure that the token stream from generated source matches the

- Ensure that the token stream from generated source matches the
  original token stream in the presence of combinations of
  multiline string and string interpolation. If not this will
  cause mismatches when accessing code snippets, e.g when
  reporting parser errors or assertions.
- Ensure that ScanAll and ScanTo are in sync when adding or
  counting tokens.
- Added unit test for multiline strings and interpolation.
- Expanded GenerateSource unit test to iterate through all
  loaded libraries.
- Do not report columns for compilation errors in snapshotted
  code.
- Do not report columns from exceptions and errors when dealing
  with snapshotted code.
- Remove length field from TokenDescriptor. It was unused.
- Remove TokenStream::ComputeTokenPosition. It was unused.

R=fschneider@google.com, hausner@google.com

Committed: https://code.google.com/p/dart/source/detail?r=29110

[hausner, 2013-10-23 16:44:50]

LGTM

https://codereview.chromium.org/36323003/diff/1/runtime/vm/scanner.cc
File runtime/vm/scanner.cc (right):

https://codereview.chromium.org/36323003/diff/1/runtime/vm/scanner.cc#newcode923
runtime/vm/scanner.cc:923: index++;  // We need to advance the index everytime a
token is added.
The comment is a bit misleading, since no tokens are added. Maybe: Advance index
to account for newline tokens.

https://codereview.chromium.org/36323003/diff/1/runtime/vm/scanner.cc#newcode930
runtime/vm/scanner.cc:930: index++;  // We need to advance the index everytime a
token is added.
ditto

[Florian Schneider, 2013-10-23 16:54:39]

lgtm

https://codereview.chromium.org/36323003/diff/1/runtime/vm/scanner.cc
File runtime/vm/scanner.cc (right):

https://codereview.chromium.org/36323003/diff/1/runtime/vm/scanner.cc#newcode931
runtime/vm/scanner.cc:931: }
Strange indentation.

[Ivan Posva, 2013-10-23 18:20:00]

https://codereview.chromium.org/36323003/diff/1/runtime/vm/scanner.cc
File runtime/vm/scanner.cc (right):

https://codereview.chromium.org/36323003/diff/1/runtime/vm/scanner.cc#newcode923
runtime/vm/scanner.cc:923: index++;  // We need to advance the index everytime a
token is added.
On 2013/10/23 16:44:51, hausner wrote:
> The comment is a bit misleading, since no tokens are added. Maybe: Advance
index
> to account for newline tokens.

  // Advance the index to account for tokens added in ScanAll.

https://codereview.chromium.org/36323003/diff/1/runtime/vm/scanner.cc#newcode930
runtime/vm/scanner.cc:930: index++;  // We need to advance the index everytime a
token is added.
On 2013/10/23 16:44:51, hausner wrote:
> ditto

ditto

https://codereview.chromium.org/36323003/diff/1/runtime/vm/scanner.cc#newcode931
runtime/vm/scanner.cc:931: }
On 2013/10/23 16:54:40, Florian Schneider wrote:
> Strange indentation.

Done.

[Ivan Posva, 2013-10-23 19:08:46]

Committed patchset #1 manually as r29110 (presubmit successful).