Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(382)

Unified Diff: src/lexer/lexer_py.re

Issue 83243004: Experimental parser: cleanup code requiring calls to the unicode cache (Closed) Base URL: https://v8.googlecode.com/svn/branches/experimental/parser
Patch Set: Created 7 years, 1 month ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View side-by-side diff with in-line comments
Download patch
« no previous file with comments | « no previous file | tools/lexer_generator/code_generator.jinja » ('j') | no next file with comments »
Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
Index: src/lexer/lexer_py.re
diff --git a/src/lexer/lexer_py.re b/src/lexer/lexer_py.re
index 794350accd962d97d62c4f3a9388ccacb3f3e754..1414ff5f0ae3b830ef49976165d3a4b2b9f8f51f 100644
--- a/src/lexer/lexer_py.re
+++ b/src/lexer/lexer_py.re
@@ -68,21 +68,19 @@ eos = [:eos:];
"<!--" <||SingleLineComment>
"<!-" <|{
- cursor_ -= 2;
- yych = *(cursor_);
+ BACKWARD(2);
PUSH_TOKEN(Token::LT);
}|>
"<!" <|{
- cursor_ -= 1;
- yych = *(cursor_);
+ BACKWARD(1);
PUSH_TOKEN(Token::LT);
}|>
"-->" <{
if (!just_seen_line_terminator_) {
- yych = *(--cursor_);
+ BACKWARD(1);
PUSH_TOKEN(Token::DEC);
}
}||SingleLineComment>
@@ -134,7 +132,7 @@ harmony_number "\\" <|push_token(ILLEGAL)|>
"," <|push_token(COMMA)|>
line_terminator+ <|push_line_terminator|>
-/[:whitespace::byte_order_mark:]+/ <|skip|>
+/[:whitespace:]+/ <|skip|>
"\"" <set_marker(1)||DoubleQuoteString>
"'" <set_marker(1)||SingleQuoteString>
« no previous file with comments | « no previous file | tools/lexer_generator/code_generator.jinja » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698