Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(116)

Unified Diff: src/lexer/lexer_py.re

Issue 71303005: Experimental parser: add back last re2c rules (Closed) Base URL: https://v8.googlecode.com/svn/branches/experimental/parser
Patch Set: Created 7 years, 1 month ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View side-by-side diff with in-line comments
Download patch
« no previous file with comments | « src/lexer/even-more-experimental-scanner.cc ('k') | no next file » | no next file with comments »
Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
Index: src/lexer/lexer_py.re
diff --git a/src/lexer/lexer_py.re b/src/lexer/lexer_py.re
index b18399a73e7f3ad0f383a8fc2c2bc50eda785a01..481d8e488d18081d7979384de08db6ba874640a9 100644
--- a/src/lexer/lexer_py.re
+++ b/src/lexer/lexer_py.re
@@ -65,7 +65,13 @@ number =
"/*" <||MultiLineComment>
"<!--" <||HtmlComment>
-#whitespace* "-->" { if (just_seen_line_terminator_) { YYSETCONDITION(kConditionSingleLineComment); goto yyc_SingleLineComment; } else { --cursor_; send(Token::DEC); start_ = cursor_; goto yyc_Normal; } }
+whitespace? "-->" <{
+ if (!just_seen_line_terminator_) {
+ PUSH_TOKEN(Token::DEC);
+ start_ = cursor_ - 1;
+ goto code_start;
+ }
+}||SingleLineComment>
">>>=" <|push_token(ASSIGN_SHR)|>
">>>" <|push_token(SHR)|>
@@ -165,8 +171,7 @@ whitespace <|skip|>
identifier_start <|push_token(IDENTIFIER)|Identifier>
/\\u[0-9a-fA-F]{4}/ <{
if (V8_UNLIKELY(!ValidIdentifierStart())) {
- PUSH_TOKEN(Token::ILLEGAL);
- // need to goto something here
+ goto default_action;
}
}|push_token(IDENTIFIER)|Identifier>
@@ -194,9 +199,8 @@ catch_all <||continue>
<<Identifier>>
identifier_char <|push_token(IDENTIFIER)|continue>
/\\u[0-9a-fA-F]{4}/ <{
- if (V8_UNLIKELY(!ValidIdentifierStart())) {
- PUSH_TOKEN(Token::ILLEGAL);
- // need to goto something here
+ if (V8_UNLIKELY(!ValidIdentifierPart())) {
+ goto default_action;
}
}|push_token(IDENTIFIER)|continue>
« no previous file with comments | « src/lexer/even-more-experimental-scanner.cc ('k') | no next file » | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698