Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(36)

Side by Side Diff: tools/lexer_generator/code_generator.jinja

Issue 83073004: Experimental parser: fix off by one error in utf8 decoding (Closed) Base URL: https://v8.googlecode.com/svn/branches/experimental/parser
Patch Set: Created 7 years ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch | Annotate | Revision Log
« no previous file with comments | « no previous file | no next file » | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 #include "lexer/experimental-scanner.h" 1 #include "lexer/experimental-scanner.h"
2 2
3 {%- macro do_key(key) -%} 3 {%- macro do_key(key) -%}
4 {%- for r in key -%} 4 {%- for r in key -%}
5 {%- if not loop.first %} || {% endif -%} 5 {%- if not loop.first %} || {% endif -%}
6 {%- if r[0] == 'PRIMARY_RANGE' -%} 6 {%- if r[0] == 'PRIMARY_RANGE' -%}
7 {%- if r[1][0] == r[1][1] -%} 7 {%- if r[1][0] == r[1][1] -%}
8 primary_char == {{r[1][0]}} 8 primary_char == {{r[1][0]}}
9 {%- elif r[1][0] == 0 -%} 9 {%- elif r[1][0] == 0 -%}
10 primary_char <= {{r[1][1]}} 10 primary_char <= {{r[1][1]}}
(...skipping 76 matching lines...) Expand 10 before | Expand all | Expand 10 after
87 87
88 {%- macro long_char_create() -%} 88 {%- macro long_char_create() -%}
89 {%- if encoding == 'utf16'-%} 89 {%- if encoding == 'utf16'-%}
90 const uint32_t long_char = primary_char; 90 const uint32_t long_char = primary_char;
91 {%- elif encoding == 'utf8'-%} 91 {%- elif encoding == 'utf8'-%}
92 unsigned bytes_read = 0; 92 unsigned bytes_read = 0;
93 const uint32_t long_char = unibrow::Utf8::CalculateValue( 93 const uint32_t long_char = unibrow::Utf8::CalculateValue(
94 reinterpret_cast<uint8_t*>(cursor_), 94 reinterpret_cast<uint8_t*>(cursor_),
95 buffer_end_ - cursor_, 95 buffer_end_ - cursor_,
96 &bytes_read); 96 &bytes_read);
97 cursor_ += bytes_read; 97 cursor_ += bytes_read - 1;
98 if (long_char == unibrow::Utf8::kBadChar) goto default_action; 98 if (long_char == unibrow::Utf8::kBadChar) goto default_action;
99 {%- else -%} 99 {%- else -%}
100 uncompilable code for {{encoding}} 100 uncompilable code for {{encoding}}
101 {%- endif -%} 101 {%- endif -%}
102 {%- endmacro -%} 102 {%- endmacro -%}
103 103
104 104
105 {%- macro do_dfa_state(node_number, inline) -%} 105 {%- macro do_dfa_state(node_number, inline) -%}
106 106
107 {%- set state = dfa_states[node_number] -%} 107 {%- set state = dfa_states[node_number] -%}
(...skipping 151 matching lines...) Expand 10 before | Expand all | Expand 10 after
259 {% endif -%} 259 {% endif -%}
260 {{dispatch_action(default_action[0], default_action[1])}} 260 {{dispatch_action(default_action[0], default_action[1])}}
261 FORWARD(); 261 FORWARD();
262 goto code_start; 262 goto code_start;
263 263
264 fell_through: 264 fell_through:
265 CHECK(false); 265 CHECK(false);
266 } 266 }
267 } } 267 } }
268 268
OLDNEW
« no previous file with comments | « no previous file | no next file » | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698