Update Lexer/Parser to support '#inline' and 'label'

Update Lexer/Parser to support '#inline' and 'label'

Added the keyword 'label' to support versioning.
Added the token INLINE to capture blocks of text within
  #inline XXX #endinl
Updated tests

TEST= python idl_lexer.py --test && python idl_parser.py --test
BUG= http://code.google.com/p/chromium/issues/detail?id=87684
Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=91426

[sehr (please use chromium), 2011-06-29 16:43:06]

A few small issues.  One request for a clarification.

http://codereview.chromium.org/7272043/diff/1/ppapi/generators/idl_lexer.py
File ppapi/generators/idl_lexer.py (right):

http://codereview.chromium.org/7272043/diff/1/ppapi/generators/idl_lexer.py#n...
ppapi/generators/idl_lexer.py:49: 'COMMENT',
should 'ATTRIBUTE' be here also?

http://codereview.chromium.org/7272043/diff/1/ppapi/generators/idl_lexer.py#n...
ppapi/generators/idl_lexer.py:124: # For C++ style, remove the preceeding '//'
s/preceeding/preceding/

http://codereview.chromium.org/7272043/diff/1/ppapi/generators/idl_lexer.py#n...
ppapi/generators/idl_lexer.py:156: msg = "Unrecognized intput"
s/intput/input/

http://codereview.chromium.org/7272043/diff/1/ppapi/generators/idl_lexer.py#n...
ppapi/generators/idl_lexer.py:160: # EoF, so compute the beginning of the that
caused the problem.
the what that?

http://codereview.chromium.org/7272043/diff/1/ppapi/generators/idl_lexer.py#n...
ppapi/generators/idl_lexer.py:165: # Add the computed line's starting possition
s/possition/position/

http://codereview.chromium.org/7272043/diff/1/ppapi/generators/idl_parser.py
File ppapi/generators/idl_parser.py (right):

http://codereview.chromium.org/7272043/diff/1/ppapi/generators/idl_parser.py#...
ppapi/generators/idl_parser.py:450: array = self.BuildProduction('Array', p, 1)
I don't understand how this branch of the if works.  How does this handle the
recursive array declaration case?

http://codereview.chromium.org/7272043/diff/1/ppapi/generators/test_lex.in
File ppapi/generators/test_lex.in (right):

http://codereview.chromium.org/7272043/diff/1/ppapi/generators/test_lex.in#ne...
ppapi/generators/test_lex.in:49: Contains /* Comments */
Your matcher includes C-style comments that break across lines.  A test might be
nice for that.

[noelallen1, 2011-06-29 18:59:01]

PTAL

Multi-line/word testing for comments was broken in the lexer because of the way
I pre-processed comment blocks.  I moved that code out to the parser, so that
the data processed by the lexer was more true to the original, and thus easier
to reproduce and compare.

http://codereview.chromium.org/7272043/diff/1/ppapi/generators/idl_lexer.py
File ppapi/generators/idl_lexer.py (right):

http://codereview.chromium.org/7272043/diff/1/ppapi/generators/idl_lexer.py#n...
ppapi/generators/idl_lexer.py:49: 'COMMENT',
On 2011/06/29 16:43:06, sehr wrote:
> should 'ATTRIBUTE' be here also?

An extended attribute is a grammar component consisting of three or more lexer
tokens in a form like:
  '[' SYMBOL ']'

'[' and ']' are literal tokens listed a few lines bellow in the member variable
"literals", so they do not have a regex t_XXX function like other tokens.

http://codereview.chromium.org/7272043/diff/1/ppapi/generators/idl_lexer.py#n...
ppapi/generators/idl_lexer.py:124: # For C++ style, remove the preceeding '//'
On 2011/06/29 16:43:06, sehr wrote:
> s/preceeding/preceding/

Done.

http://codereview.chromium.org/7272043/diff/1/ppapi/generators/idl_lexer.py#n...
ppapi/generators/idl_lexer.py:156: msg = "Unrecognized intput"
On 2011/06/29 16:43:06, sehr wrote:
> s/intput/input/

Done.

http://codereview.chromium.org/7272043/diff/1/ppapi/generators/idl_lexer.py#n...
ppapi/generators/idl_lexer.py:160: # EoF, so compute the beginning of the that
caused the problem.
On 2011/06/29 16:43:06, sehr wrote:
> the what that?

Done.

http://codereview.chromium.org/7272043/diff/1/ppapi/generators/idl_lexer.py#n...
ppapi/generators/idl_lexer.py:165: # Add the computed line's starting possition
On 2011/06/29 16:43:06, sehr wrote:
> s/possition/position/

Done.

http://codereview.chromium.org/7272043/diff/1/ppapi/generators/idl_parser.py
File ppapi/generators/idl_parser.py (right):

http://codereview.chromium.org/7272043/diff/1/ppapi/generators/idl_parser.py#...
ppapi/generators/idl_parser.py:450: array = self.BuildProduction('Array', p, 1)
I have re-ordered the size checks and commented them to make the definition more
obvious.

On 2011/06/29 16:43:06, sehr wrote:
> I don't understand how this branch of the if works.  How does this handle the
> recursive array declaration case?

Done.

http://codereview.chromium.org/7272043/diff/1/ppapi/generators/test_lex.in
File ppapi/generators/test_lex.in (right):

http://codereview.chromium.org/7272043/diff/1/ppapi/generators/test_lex.in#ne...
ppapi/generators/test_lex.in:49: Contains /* Comments */
On 2011/06/29 16:43:06, sehr wrote:
> Your matcher includes C-style comments that break across lines.  A test might
be
> nice for that.

Done.

[sehr (please use chromium), 2011-07-01 22:59:22]

On 2011/07/01 08:33:11, noelallen1 wrote:

LGTM.