tools/lexer_generator/transition_keys.py - Issue 148283007: Experimental parser: remove bom handling

Keyboard Shortcuts

	File
u :	up to issue
j / k :	jump to file after / before current file
J / K :	jump to next file with a comment after / before current file
	Side-by-side diff
i :	toggle intra-line diffs
e :	expand all comments
c :	collapse all comments
s :	toggle showing all comments
n / p :	next / previous diff chunk or comment
N / P :	next / previous comment
<Up> / <Down> :	next / previous line

	Issue
u :	up to list of issues
j / k :	jump to patch after / before current patch
o / <Enter> :	open current patch in side-by-side view
i :	open current patch in unified diff view

	Issue List
j / k :	jump to issue after / before current issue
o / <Enter> :	open current issue

Unified Diff: tools/lexer_generator/transition_keys.py

Issue 148283007: Experimental parser: remove bom handling (Closed) Base URL: https://v8.googlecode.com/svn/branches/experimental/parser

Patch Set: Created 6 years, 11 months ago

Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.

Jump to:

View side-by-side diff with in-line comments

Download patch

Index: tools/lexer_generator/transition_keys.py

diff --git a/tools/lexer_generator/transition_keys.py b/tools/lexer_generator/transition_keys.py

index e3664d4ea7e1415555c73ed783cd8dd9c1506c5e..33019b99a09ba92c78244a84553debdc265ec1d4 100644

--- a/tools/lexer_generator/transition_keys.py

+++ b/tools/lexer_generator/transition_keys.py

@@ -493,8 +493,7 @@ class Utf16Encoding(KeyEncoding):

super(Utf16Encoding, self).__init__(

'utf16',

(0, 255),

- ['byte_order_mark',

- 'non_primary_whitespace',

+ ['non_primary_whitespace',

'non_primary_letter',

'non_primary_identifier_part_not_letter',

'non_primary_line_terminator',

@@ -502,7 +501,6 @@ class Utf16Encoding(KeyEncoding):

self.add_predefined_range(

'whitespace',

[(9, 9), (11, 12), (32, 32), (133, 133), (160, 160),

- self.class_range('byte_order_mark'),

self.class_range('non_primary_whitespace')])

self.add_predefined_range(

'letter', [

@@ -523,8 +521,7 @@ class Utf8Encoding(KeyEncoding):

super(Utf8Encoding, self).__init__(

'utf8',

(0, 127),

- ['byte_order_mark',

- 'non_primary_whitespace',

+ ['non_primary_whitespace',

'non_primary_letter',

'non_primary_identifier_part_not_letter',

'non_primary_line_terminator',

@@ -532,7 +529,6 @@ class Utf8Encoding(KeyEncoding):

self.add_predefined_range(

'whitespace',

[(9, 9), (11, 12), (32, 32),

- self.class_range('byte_order_mark'),

self.class_range('non_primary_whitespace')])

self.add_predefined_range(

'letter', [(65, 90), (97, 122), self.class_range('non_primary_letter')])

« no previous file with comments | « tools/lexer_generator/code_generator.py ('k') | no next file » | no next file with comments »