tools/lexer_generator/code_generator.jinja - Issue 78933002: Experimental lexer generator / utf16 support: Fix + add test files.

Keyboard Shortcuts

	File
u :	up to issue
j / k :	jump to file after / before current file
J / K :	jump to next file with a comment after / before current file
	Side-by-side diff
i :	toggle intra-line diffs
e :	expand all comments
c :	collapse all comments
s :	toggle showing all comments
n / p :	next / previous diff chunk or comment
N / P :	next / previous comment
<Up> / <Down> :	next / previous line

	Issue
u :	up to list of issues
j / k :	jump to patch after / before current patch
o / <Enter> :	open current patch in side-by-side view
i :	open current patch in unified diff view

	Issue List
j / k :	jump to issue after / before current issue
o / <Enter> :	open current issue

Unified Diff: tools/lexer_generator/code_generator.jinja

Issue 78933002: Experimental lexer generator / utf16 support: Fix + add test files. (Closed) Base URL: https://v8.googlecode.com/svn/branches/experimental/parser

Patch Set: Created 7 years, 1 month ago

Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.

Jump to:

View side-by-side diff with in-line comments

Download patch

Index: tools/lexer_generator/code_generator.jinja

diff --git a/tools/lexer_generator/code_generator.jinja b/tools/lexer_generator/code_generator.jinja

index 4ed43af92bed51c5143b80432a0271a028032c87..a6a381d83ca5c96abe4affb17dc9ea8ea262dea5 100644

--- a/tools/lexer_generator/code_generator.jinja

+++ b/tools/lexer_generator/code_generator.jinja

@@ -19,7 +19,8 @@

{%- elif r[1] == 'zero' -%}

(yych == 0 && cursor_ < buffer_end_)

{%- elif r[1] == 'whitespace' and encoding == 'utf16'-%}

- unicode_cache_->IsWhiteSpace(yych)

+ {# FIXME: Add and use unicode_cache_->InNonAsciiWhitespace #}

+ (yych != ' ' && yych != '\t' && yych != '\v' && yych != '\f' && yych != '\r' && yych != '\n' && unicode_cache_->IsWhiteSpace(yych))

{%- elif r[1] == 'letter' and encoding == 'utf16'-%}

{# FIXME: Add and use unicode_cache_->InNonAsciiLetter #}

(!(yych >= 'a' && yych <= 'z') && !(yych >= 'A' && yych <= 'Z') && unicode_cache_->IsLetter(yych))

« no previous file with comments | « test/lexer/utf-16-2.js ('k') | no next file » | no next file with comments »