tools/lexer_generator/rule_lexer.py - Issue 48783004: Experimental lexer generator: fixing the rule parser + adding tests.

Keyboard Shortcuts

	File
u :	up to issue
j / k :	jump to file after / before current file
J / K :	jump to next file with a comment after / before current file
	Side-by-side diff
i :	toggle intra-line diffs
e :	expand all comments
c :	collapse all comments
s :	toggle showing all comments
n / p :	next / previous diff chunk or comment
N / P :	next / previous comment
<Up> / <Down> :	next / previous line

	Issue
u :	up to list of issues
j / k :	jump to patch after / before current patch
o / <Enter> :	open current patch in side-by-side view
i :	open current patch in unified diff view

	Issue List
j / k :	jump to issue after / before current issue
o / <Enter> :	open current issue

Unified Diff: tools/lexer_generator/rule_lexer.py

Issue 48783004: Experimental lexer generator: fixing the rule parser + adding tests. (Closed) Base URL: https://v8.googlecode.com/svn/branches/experimental/parser

Patch Set: Created 7 years, 2 months ago

Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.

Jump to:

View side-by-side diff with in-line comments

Download patch

Index: tools/lexer_generator/rule_lexer.py

diff --git a/tools/lexer_generator/rule_lexer.py b/tools/lexer_generator/rule_lexer.py

index 5dc4936a3802eaafc7e6fe672c099380eefe10f6..b5bb8c403530818c32b5c601197a47736fd07465 100644

--- a/tools/lexer_generator/rule_lexer.py

+++ b/tools/lexer_generator/rule_lexer.py

@@ -36,8 +36,9 @@ class RuleLexer:

'CONDITION',

'CONDITION_BEGIN',

'CONDITION_END',

- 'REGEX_AND_TRANSITION',

- 'REGEX_AND_BODY',

+ 'REGEX_TRANSITION_BODY',

+ 'REGEX_TRANSITION',

+ 'REGEX_BODY',

)

t_ANY_ignore = " \t\n"

@@ -75,21 +76,26 @@ class RuleLexer:

return t

def t_seenCondition_CONDITION_END(self, t):

- r'>'

+ r'>\s*'

self.lexer.begin('afterCondition')

return t

- def t_afterCondition_REGEX_AND_TRANSITION(self, t):

+ def t_afterCondition_REGEX_TRANSITION_BODY(self, t):

+ r'(?P<regex>.+)\s*=>\s*(?P<new>.+)\s*{\s*(?P<body>.+)\s*}\s*'

+ self.lexer.begin('INITIAL')

+ return t

+ def t_afterCondition_REGEX_TRANSITION(self, t):

r'(?P<regex>.+)\s*:=>\s*(?P<new>.+)\s*'

self.lexer.begin('INITIAL')

return t

- def t_afterCondition_REGEX_AND_BODY(self, t):

- r'(?P<regex>.+)\s*{\s*(?P<body>.+)\s*}\s*'

+ def t_afterCondition_REGEX_BODY(self, t):

+ r'(?P<regex>.+?)\s+{\s*(?P<body>.+)\s*}\s*'

self.lexer.begin('INITIAL')

return t

- def t_error(self, t):

+ def t_ANY_error(self, t):

raise Exception("Illegal character '%s'" % t.value[0])

def build(self, **kwargs):

« no previous file with comments | « no previous file | tools/lexer_generator/rule_parser.py » ('j') | no next file with comments »