Add Python flow to bindings generation, move dummy-generating IDL files over

Source/bindings/scripts/idl_compiler.py

+#!/usr/bin/python
+# Copyright (C) 2013 Google Inc. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met:
+#
+#     * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above
+# copyright notice, this list of conditions and the following disclaimer
+# in the documentation and/or other materials provided with the
+# distribution.
+#     * Neither the name of Google Inc. nor the names of its
+# contributors may be used to endorse or promote products derived from
+# this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+import optparse
+import os
+import shlex
+import sys
+
+import build_ir
+import code_generator_v8
+
+
+def parse_options():
+    parser = optparse.OptionParser()
+    parser.add_option('--additional-idl-files')
+    parser.add_option('--idl-attributes-file')
+    parser.add_option('--include', dest='idl_directories', action='append')
+    parser.add_option('--output-directory')
+    parser.add_option('--supplemental-dependency-file')

[haraken, 2013/07/09 11:16:48]

--interface-dependency-file ?

[Nils Barth (inactive), 2013/07/10 02:38:32]

Oops (>,<). Fixed (throughout, plus rebase).

+    parser.add_option('--verbose', action='store_true', default=False)
+    parser.add_option('--write-file-only-if-changed', type='int')
+    # ensure output comes last, so command line easy to parse via regexes
+    parser.disable_interspersed_args()
+
+    options, args = parser.parse_args()
+    if options.output_directory is None:
+        parser.error('Must specify output directory using --output-directory.')
+    if options.additional_idl_files is not None:
+        # additional_idl_files is passed as a string with varied (shell-style)
+        # quoting, hence needs parsing.
+        options.additional_idl_files = shlex.split(options.additional_idl_files)
+    if len(args) != 1:
+        parser.error('Must specify exactly 1 input file as argument, but %d given.' % len(args))
+    options.target_idl_filename = os.path.realpath(args[0])
+    return options
+
+
+def main():
+    options = parse_options()
+    target_idl_filename = options.target_idl_filename
+    target_basename = os.path.basename(target_idl_filename)
+    target_interface_name, _ = os.path.splitext(target_basename)
+    if options.verbose:
+        print target_idl_filename
+
+    try:
+        # FIXME: Currently read_merged_idl does not return, since just a stub
+        # Ignore E1111: Assigning to function call which doesn't return
+        # pylint: disable=E1111

[haraken, 2013/07/09 11:16:48]

Remove the part after "Ignore E111...". This part looks uninformative for
others.

[Nils Barth (inactive), 2013/07/10 02:38:32]

We don't need this anymore now that we return a dummy IdlDocument (from
read_idl_file)
and bool (from merge_interface_dependencies), which fixes this.

+        blink_ir = build_ir.read_merged_idl(target_idl_filename, supplemental_dependency_file=options.supplemental_dependency_file, additional_idl_files=options.additional_idl_files)

[haraken, 2013/07/09 11:16:48]

I feel that read_merged_idl() and methods in read_merged_idl() are not well
organized. How about organizing them like this?

def main():
  options = parse_options()
  parse_tree_of_idl = parse_idl(options, ...)
  check_attributes(parse_tree_of_idl)
  parse_tree_of_interface = resolve_idl_dependency(parse_tree_of_idl, ...)
  generate_code(parse_tree_of_interface, ...)

def parse_idl():  # This can be used in other bindings like Dart.
  tokens = lex_idl(...)
  abstract_tree = build_abstract_tree(tokens)
  parse_tree = build_parse_tree(abstract_tree)
  return parse_tree

def generate_code():
  ...

 (It's up to you how the methods are split into files.)

[Nils Barth (inactive), 2013/07/10 02:38:32]

Got it, changed; how does it look?

I wanted to have a single function that wrapped all the steps (read file,
resolve dependencies, check attributes), but admittedly it results in
a function with a huge signature.
Instead, we can have a few more lines in main(), and simpler calls.

This required a few other changes to the code:
* Dummy IdlDocument (will be full object for interface once parser lands),
  since parsing base file before resolving dependencies.
* Returning bool indicating whether bindings should be generated or not,
  since not raising exception.

(I think I can simplify this even more in future, but that requires
some GYP changes too, so I'll do it later.)
Pedantry:
"parse tree" is the CST, and comes before the AST:
the flow is CST > AST > IR:
* CST is aka "parse tree" or "parsing tree" = exactly the grammar,
* AST is aka "syntax tree".
These names are very confusing, hence why I always use "CST" and "AST".

Anyway, I've replaced all references in the code to "AST" or "IR"
with more explicit names of "node" (in AST) or "idl_document",
since the IR is an object of type IdlDocument.
I've left the generic terms "AST" and "IR" in the intro comment to read_idl.py
(which seems a better name than the cryptic "build_ir.py"),
since they're useful for an overview.
However, in the actual code more explicit names are clearer.

+    except build_ir.IdlNotFoundError:
+        # We generate dummy .h and .cpp files just to tell build scripts
+        # that outputs have been created.
+        code_generator_v8.generate_dummy_header_and_cpp(target_interface_name, options.output_directory)
+        return
+    # FIXME: currently IdlNotFoundError *must* be raised, and dummy .cpp and
+    # .h files generated, as actual code generator not present yet.
+    raise RuntimeError('Stub: real code generator not implemented yet')
+
+
+if __name__ == '__main__':
+    sys.exit(main())