tools/lexer_generator/code_generator.py - Issue 82803003: Experimental parser: add utf8 encoding

Keyboard Shortcuts

	File
u :	up to issue
j / k :	jump to file after / before current file
J / K :	jump to next file with a comment after / before current file
	Side-by-side diff
i :	toggle intra-line diffs
e :	expand all comments
c :	collapse all comments
s :	toggle showing all comments
n / p :	next / previous diff chunk or comment
N / P :	next / previous comment
<Up> / <Down> :	next / previous line

	Issue
u :	up to list of issues
j / k :	jump to patch after / before current patch
o / <Enter> :	open current patch in side-by-side view
i :	open current patch in unified diff view

	Issue List
j / k :	jump to issue after / before current issue
o / <Enter> :	open current issue

Unified Diff: tools/lexer_generator/code_generator.py

Issue 82803003: Experimental parser: add utf8 encoding (Closed) Base URL: https://v8.googlecode.com/svn/branches/experimental/parser

Patch Set: Created 7 years, 1 month ago

Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.

Jump to:

Index: tools/lexer_generator/code_generator.py

diff --git a/tools/lexer_generator/code_generator.py b/tools/lexer_generator/code_generator.py

index a703890e076c193046d23f98ab84333bc8f23835..e6d44662908f0e8bc32decc2d3a006cddebaae0f 100644

--- a/tools/lexer_generator/code_generator.py

+++ b/tools/lexer_generator/code_generator.py

@@ -247,12 +247,9 @@ class CodeGenerator:

template = template_env.get_template('code_generator.jinja')

encoding = self.__dfa.encoding().name()

- if encoding == 'latin1':

- char_type = 'uint8_t'

- elif encoding == 'utf16':

- char_type = 'uint16_t'

- else:

- raise Exception('Unsupported encoding %s' % encoding)

+ char_types = {'latin1': 'uint8_t', 'utf16': 'uint16_t', 'utf8': 'int8_t'}

+ char_type = char_types[encoding]

return template.render(

start_node_number = 0,

debug_print = self.__debug_print,

« no previous file with comments | « tools/lexer_generator/automaton.py ('k') | tools/lexer_generator/dfa.py » ('j') | no next file with comments »