sdk/lib/_internal/compiler/implementation/scanner/token.dart - Issue 11368138: Add some support for the code-point code-unit distinction.

Keyboard Shortcuts

	File
u :	up to issue
j / k :	jump to file after / before current file
J / K :	jump to next file with a comment after / before current file
	Side-by-side diff
i :	toggle intra-line diffs
e :	expand all comments
c :	collapse all comments
s :	toggle showing all comments
n / p :	next / previous diff chunk or comment
N / P :	next / previous comment
<Up> / <Down> :	next / previous line

	Issue
u :	up to list of issues
j / k :	jump to patch after / before current patch
o / <Enter> :	open current patch in side-by-side view
i :	open current patch in unified diff view

	Issue List
j / k :	jump to issue after / before current issue
o / <Enter> :	open current issue

Unified Diff: sdk/lib/_internal/compiler/implementation/scanner/token.dart

Issue 11368138: Add some support for the code-point code-unit distinction. (Closed) Base URL: https://dart.googlecode.com/svn/branches/bleeding_edge/dart

Patch Set: Created 8 years, 1 month ago

Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.

Jump to:

View side-by-side diff with in-line comments

Download patch

« sdk/lib/_internal/compiler/implementation/lib/js_helper.dart ('K') | « sdk/lib/_internal/compiler/implementation/lib/js_helper.dart ('k') | sdk/lib/_internal/compiler/implementation/string_validator.dart » ('j') | sdk/lib/_internal/compiler/implementation/tree/dartstring.dart » ('J')
Expand Comments ('e') | Collapse Comments ('c') | Hide Comments ('s')

Index: sdk/lib/_internal/compiler/implementation/scanner/token.dart

diff --git a/sdk/lib/_internal/compiler/implementation/scanner/token.dart b/sdk/lib/_internal/compiler/implementation/scanner/token.dart

index 901202139bcdf4c3205b7c3b2e56229eef259021..28ba95f6180f7fabbcddeadb144e74e1a974ea6b 100644

--- a/sdk/lib/_internal/compiler/implementation/scanner/token.dart

+++ b/sdk/lib/_internal/compiler/implementation/scanner/token.dart

@@ -246,7 +246,11 @@ class StringCodeIterator implements Iterator<int> {

}

bool get hasNext => index < end;

- int next() => string.charCodeAt(index++);

+ int next() {

floitsch 2012/11/08 15:28:21 maybe add TODO to switch to charcode-iterator once

erikcorry 2012/11/15 13:28:25 Done.

+ int charCode = string.charCodeAt(index++);

+ if (charCode >= 0x10000) index++; // Skip trail surrogate.

+ return charCode;

+ }

}

class BeginGroupToken extends StringToken {