Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(175)

Side by Side Diff: tools/lexer_generator/transition_keys.py

Issue 85413006: Experimental parser: dfa optimization to remove transitions from keyword lexing (Closed) Base URL: https://v8.googlecode.com/svn/branches/experimental/parser
Patch Set: Created 7 years ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch | Annotate | Revision Log
« no previous file with comments | « tools/lexer_generator/rule_parser.py ('k') | no next file » | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 # Copyright 2013 the V8 project authors. All rights reserved. 1 # Copyright 2013 the V8 project authors. All rights reserved.
2 # Redistribution and use in source and binary forms, with or without 2 # Redistribution and use in source and binary forms, with or without
3 # modification, are permitted provided that the following conditions are 3 # modification, are permitted provided that the following conditions are
4 # met: 4 # met:
5 # 5 #
6 # * Redistributions of source code must retain the above copyright 6 # * Redistributions of source code must retain the above copyright
7 # notice, this list of conditions and the following disclaimer. 7 # notice, this list of conditions and the following disclaimer.
8 # * Redistributions in binary form must reproduce the above 8 # * Redistributions in binary form must reproduce the above
9 # copyright notice, this list of conditions and the following 9 # copyright notice, this list of conditions and the following
10 # disclaimer in the documentation and/or other materials provided 10 # disclaimer in the documentation and/or other materials provided
(...skipping 311 matching lines...) Expand 10 before | Expand all | Expand 10 after
322 self.__ranges = tuple(ranges) # immutable 322 self.__ranges = tuple(ranges) # immutable
323 self.__cached_hash = None 323 self.__cached_hash = None
324 324
325 def to_string(self, encoding): 325 def to_string(self, encoding):
326 if self.__name: 326 if self.__name:
327 return self.__name 327 return self.__name
328 strings = [TransitionKey.__range_str(encoding, x) for x in self.__ranges] 328 strings = [TransitionKey.__range_str(encoding, x) for x in self.__ranges]
329 return ', '.join(strings) 329 return ', '.join(strings)
330 330
331 def __str__(self): 331 def __str__(self):
332 self.to_string(None) 332 return self.to_string(None)
333 333
334 @staticmethod 334 @staticmethod
335 def __disjoint_keys(encoding, range_map): 335 def __disjoint_keys(encoding, range_map):
336 '''Takes a set of possibly overlapping ranges, returns a list of ranges 336 '''Takes a set of possibly overlapping ranges, returns a list of ranges
337 which don't overlap and which cover the same points as the original 337 which don't overlap and which cover the same points as the original
338 set. range_map is a map from lower bounds to a list of upper bounds.''' 338 set. range_map is a map from lower bounds to a list of upper bounds.'''
339 sort = lambda x : sorted(set(x)) 339 sort = lambda x : sorted(set(x))
340 range_map = sorted(map(lambda (k, v): (k, sort(v)), range_map.items())) 340 range_map = sorted(map(lambda (k, v): (k, sort(v)), range_map.items()))
341 ranges = [] 341 ranges = []
342 upper_bound = encoding.upper_bound() + 1 342 upper_bound = encoding.upper_bound() + 1
(...skipping 182 matching lines...) Expand 10 before | Expand all | Expand 10 after
525 self.class_range('non_primary_whitespace')]) 525 self.class_range('non_primary_whitespace')])
526 self.add_predefined_range( 526 self.add_predefined_range(
527 'letter', [(65, 90), (97, 122), self.class_range('non_primary_letter')]) 527 'letter', [(65, 90), (97, 122), self.class_range('non_primary_letter')])
528 self.add_predefined_range( 528 self.add_predefined_range(
529 'line_terminator', 529 'line_terminator',
530 [(10, 10), (13, 13), self.class_range('non_primary_line_terminator')]) 530 [(10, 10), (13, 13), self.class_range('non_primary_line_terminator')])
531 self.add_predefined_range( 531 self.add_predefined_range(
532 'identifier_part_not_letter', 532 'identifier_part_not_letter',
533 [(48, 57), (95, 95), 533 [(48, 57), (95, 95),
534 self.class_range('non_primary_identifier_part_not_letter')]) 534 self.class_range('non_primary_identifier_part_not_letter')])
OLDNEW
« no previous file with comments | « tools/lexer_generator/rule_parser.py ('k') | no next file » | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698