Rewrite generate-bindings.pl in Python

Closed; superseded by separate flows CL
Issue 18190004: Add Python flow to bindings generation, move dummy-generating IDL files over
https://codereview.chromium.org/18190004/

Rewrite generate-bindings.pl in Python

This rewrites (and refactors) the bindings generation "driver script"
generate-bindings.pl (i.e., the script that the build actually calls,
which calls the parser and code generator) in Python.

Once this lands, we can start work on the backend!


This also includes:
* a Perl Parser to Python wrapper (via JSON),
  so can use the Perl parser in Python
* a stub rewrite of CodeGeneratorV8.pl into Python

code_generator_v8.py is where the bulk of the overall rewrite work will be.
I’ve written just enough so the interface for output functions works, but there’s no content.

The 7 files are:
generate_bindings.py  –  rewrite of generate-bindings.pl
    interface_merger.py  –  refactored
    semantic_analyzer.py  –  refactored

idl-to-json.pl  –  Perl Parser to Python wrapper, Perl side
blink_idl_parser_perl.py  –  Perl Parser to Python wrapper, Python side

code_generator_v8.py  –  stub
compare-python-to-perl  –  utility script to compare Perl and Python output

This is a bit long (800+ lines), since it's rewriting the overall script *and*
enough of the dependencies (parser wrapper + stub code generator) that we can
actually run the script.

Some command lines you can use to test include:
Easy wrapper:
./compare-python-to-perl --generate-only BarProp.idl

Directly call key script:
./generate_bindings.py --perl-parser --idl-attributes-file=IDLAttributes.txt --output-directory=test_output ../../../Source/core/page/BarProp.idl

(Does not generate any C++ code – just empty files – but exercises the scripts.)

The wrapper script updates:
http://crrev.com/16185003


BTW, weird diff for idl-to-json.pl is b/c git confused it with:
core/scripts/make_internal_runtime_flags.py
...due to shortness (large copyright relative to size of script means it's viewed as a diff off that one)

BUG=239771

[Nils Barth (inactive), 2013-05-31 11:45:05]

Hi haraken,
Here's the generate-bindings.pl rewrite!

From here out it should be easy to work incrementally – implementing the backend
gradually – but the first bit is a bit big.
Hopefully the individual files are legible, even breezy.
If you'd rather not do it all in one go, I could break this up into pieces like
"comparison script", "Perl Parser to Python wrapper", and "generate bindings".

I've intentionally done very little of the backend, as that's very long and
complex, and we'll want to look at it carefully, but I can fill out a lot more
of the framework (if not the gory details) in a follow-up patch.

[haraken, 2013-05-31 12:06:11]

At a glance, not only is the CL too big but also I'm not sure you're going a
right way. Repeatedly, we don't want to introduce unnecessary complexity to the
code base.

Why is it hard to proceed with the work incrementally as discussed before
(https://code.google.com/p/chromium/issues/detail?id=239771#c8)?

(1) Remove 'signature->' indirection.
(2) Connect IDLParser.pm and CodeGeneratorV8.pm via a JSON file. This will
unblock koji-san's work, because koji-san can just depend on the JSON file,
whether the JSON file is generated by Perl or Python.
(3) You can try to generate the JSON file from Python.

It's important to break down changes so that we can verify that each change is
correct.

[Nils Barth (inactive), 2013-05-31 12:13:07]

On 2013/05/31 12:06:11, haraken wrote:
> At a glance, not only is the CL too big but also I'm not sure you're going a
> right way. Repeatedly, we don't want to introduce unnecessary complexity to
the
> code base.
> 
> Why is it hard to proceed with the work incrementally as discussed before
> (https://code.google.com/p/chromium/issues/detail?id=239771#c8%29?
> 
> (1) Remove 'signature->' indirection.
> (2) Connect IDLParser.pm and CodeGeneratorV8.pm via a JSON file. This will
> unblock koji-san's work, because koji-san can just depend on the JSON file,
> whether the JSON file is generated by Perl or Python.
> (3) You can try to generate the JSON file from Python.
> 
> It's important to break down changes so that we can verify that each change is
> correct.

This CL does exactly that (uses JSON to connect frontend and backend):
(2) IDLParser.pm generates a JSON file (via idl-to-json.pl)
[We don't need to do (3) yet.]

blink_idl_parser_perl.py reads in the JSON
generate_bindings.py generates the bindings
...by calling code_generator_v8.py

We can break this up by doing the JSON step separately:
idl-to-json.pl – Perl Parser to Python wrapper, Perl side 
blink_idl_parser_perl.py – Perl Parser to Python wrapper, Python side