Add IDL capability to json_schema_compiler.

- Extend compiler.py to recognize .idl files.
  - Extend .gyp to pass certain .idl files to compiler.py.
  - Implement a translator taking IDL parser output and creating
    a schema that the JSON compiler recognizes.
  - Convert one API to IDL using asargent's JSON cat JSON-to-IDL
    converter, and adapt API's implementation to use the newly generated
    output.
    
TEST=no changes
BUG=none

Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=125300

[asargent_no_longer_on_chrome, 2012-03-06 22:27:57]

mostly nits

http://codereview.chromium.org/9600050/diff/1008/chrome/browser/extensions/ap...
File chrome/browser/extensions/api/dns/dns_api.cc (right):

http://codereview.chromium.org/9600050/diff/1008/chrome/browser/extensions/ap...
chrome/browser/extensions/api/dns/dns_api.cc:11: #include
"chrome/common/extensions/api/experimental.dns.h"
This experimental.dns.h file gets auto-generated during build time right? Does
it literally end up at src/chrome/common/extensions/api/experimental.dns.h, or
is it somewhere in $BUILDDIR?

http://codereview.chromium.org/9600050/diff/1008/tools/json_schema_compiler/i...
File tools/json_schema_compiler/idl_schema.py (right):

http://codereview.chromium.org/9600050/diff/1008/tools/json_schema_compiler/i...
tools/json_schema_compiler/idl_schema.py:4: 
nit: might be worth having an overview comment here explaining what the code in
this file is for.

http://codereview.chromium.org/9600050/diff/1008/tools/json_schema_compiler/i...
tools/json_schema_compiler/idl_schema.py:16: Given a Callspec node representing
an IDL namespace, converts into a Python
"an IDL namespace" -> "an IDL function declaration"

"a Python dictionary" -> "a name, value pair where the value is a list of
parameters for the function"

http://codereview.chromium.org/9600050/diff/1008/tools/json_schema_compiler/i...
tools/json_schema_compiler/idl_schema.py:30: Given a Param node representing an
IDL namespace, converts into a Python
"an IDL namespace" -> "a function parameter"

http://codereview.chromium.org/9600050/diff/1008/tools/json_schema_compiler/i...
tools/json_schema_compiler/idl_schema.py:44: Given a Dictionary node
representing an IDL namespace, converts into a Python
"an IDL namespace" -> "an IDL dictionary"

http://codereview.chromium.org/9600050/diff/1008/tools/json_schema_compiler/i...
tools/json_schema_compiler/idl_schema.py:62: Given a Member node representing an
IDL namespace, converts into a Python
"an IDL namespace" -> "an IDL dictionary or interface member"

"converts into a Python dictionary..." -> "converts into a name, value pair
where the value is a dictionary the JSON schema compiler expects to see"

http://codereview.chromium.org/9600050/diff/1008/tools/json_schema_compiler/i...
tools/json_schema_compiler/idl_schema.py:77: name, parameters =
Callspec(node).process(refs)
It would be nice to enforce somewhere that this callspec member has the "STATIC"
attribute, since we don't support object-oriented interfaces. We could either do
that here or in a separate idl linter that is run by a presubmit check or
something.

(We may someday want to support object-oriented interfaces, in which case
non-static methods would be required to be called on an object that implements
an interface).

http://codereview.chromium.org/9600050/diff/1008/tools/json_schema_compiler/i...
tools/json_schema_compiler/idl_schema.py:81: if typeref == 'void':
Instead of looking for typeref==void to tell if this is a function, we should
instead look to see if this node has a Callspec node attached. (We actually have
a few extension functions that return a non-void value synchronously, e.g.
chrome.contextMenus.create, chrome.i18n.getMessage).

Or if for some reason that isn't sufficient, we could attach an additional
property or something in the idl parser - we know from the grammar whether this
is a member variable or a member function.

http://codereview.chromium.org/9600050/diff/1008/tools/json_schema_compiler/i...
tools/json_schema_compiler/idl_schema.py:85: Typeref(typeref,
self.node).process(refs).items())
would it be clearer if instead of merging dict's here, you just passed in
properties to the Typeref process method, and had it add the appropriate values
to that?

http://codereview.chromium.org/9600050/diff/1008/tools/json_schema_compiler/i...
tools/json_schema_compiler/idl_schema.py:90: Given a TYPEREF property
representing an IDL namespace, converts into a
"an IDL namespace" -> "the type of a dictionary member or function parameter"

http://codereview.chromium.org/9600050/diff/1008/tools/json_schema_compiler/i...
tools/json_schema_compiler/idl_schema.py:111: properties['$ref'] = self.typeref
Someday it would be cool to take note of not-yet-defined types that we see
during compile, and compare that to the full list of defined types when we're
finished, and emit an error or warning for any that were never defined. That
could help catch cases of typos, etc.

http://codereview.chromium.org/9600050/diff/1008/tools/json_schema_compiler/i...
tools/json_schema_compiler/idl_schema.py:143: 'namespace':
self.namespace.GetName() }
looks like you're missing the 'nodoc' attribute if it's present?

[miket_OOO, 2012-03-07 00:23:01]

Looking better, representing an IDL namespace. Representing an IDL namespace.
(Also: representing an IDL namespace.)

http://codereview.chromium.org/9600050/diff/1008/chrome/browser/extensions/ap...
File chrome/browser/extensions/api/dns/dns_api.cc (right):

http://codereview.chromium.org/9600050/diff/1008/chrome/browser/extensions/ap...
chrome/browser/extensions/api/dns/dns_api.cc:11: #include
"chrome/common/extensions/api/experimental.dns.h"
On 2012/03/06 22:27:57, Antony Sargent wrote:
> This experimental.dns.h file gets auto-generated during build time right? Does
> it literally end up at src/chrome/common/extensions/api/experimental.dns.h, or
> is it somewhere in $BUILDDIR?

Don't worry! If you take a look at http://codereview.chromium.org/9359040/,
particularly build/apiidlc.gypi, then you will see that generated files are
emitted into a subdirectory of $SHARED_INTERMEDIATE_DIR (e.g.,
out/Debug/obj/gen/foo/bar/baz). Though that CL wasn't checked in, Chris did
roughly the same thing in the code that this CL is based on.

The tricky part was getting the compiler to see those generated files; the
"direct_dependent_settings" item in the .gypi is what makes what work (again,
that's in the other CL).

http://codereview.chromium.org/9600050/diff/1008/tools/json_schema_compiler/i...
File tools/json_schema_compiler/idl_schema.py (right):

http://codereview.chromium.org/9600050/diff/1008/tools/json_schema_compiler/i...
tools/json_schema_compiler/idl_schema.py:81: if typeref == 'void':
On 2012/03/06 22:27:57, Antony Sargent wrote:
> Instead of looking for typeref==void to tell if this is a function, we should
> instead look to see if this node has a Callspec node attached. (We actually
have
> a few extension functions that return a non-void value synchronously, e.g.
> chrome.contextMenus.create, chrome.i18n.getMessage).
> 
> Or if for some reason that isn't sufficient, we could attach an additional
> property or something in the idl parser - we know from the grammar whether
this
> is a member variable or a member function. 

Done.

http://codereview.chromium.org/9600050/diff/1008/tools/json_schema_compiler/i...
tools/json_schema_compiler/idl_schema.py:85: Typeref(typeref,
self.node).process(refs).items())
On 2012/03/06 22:27:57, Antony Sargent wrote:
> would it be clearer if instead of merging dict's here, you just passed in
> properties to the Typeref process method, and had it add the appropriate
values
> to that?

Yes, great idea! Done.

http://codereview.chromium.org/9600050/diff/1008/tools/json_schema_compiler/i...
tools/json_schema_compiler/idl_schema.py:111: properties['$ref'] = self.typeref
On 2012/03/06 22:27:57, Antony Sargent wrote:
> Someday it would be cool to take note of not-yet-defined types that we see
> during compile, and compare that to the full list of defined types when we're
> finished, and emit an error or warning for any that were never defined. That
> could help catch cases of typos, etc. 

Agreed -- though at present, it's caught loudly and fatally during C++
compilation.

http://codereview.chromium.org/9600050/diff/1008/tools/json_schema_compiler/i...
tools/json_schema_compiler/idl_schema.py:143: 'namespace':
self.namespace.GetName() }
On 2012/03/06 22:27:57, Antony Sargent wrote:
> looks like you're missing the 'nodoc' attribute if it's present?

Fixed. I know I'm being inconsistent here because I handled it elsewhere, but
what difference does nodoc make at this point (about to generate C code)?

[asargent_no_longer_on_chrome, 2012-03-07 00:42:09]

LGTM!

http://codereview.chromium.org/9600050/diff/1008/tools/json_schema_compiler/i...
File tools/json_schema_compiler/idl_schema.py (right):

http://codereview.chromium.org/9600050/diff/1008/tools/json_schema_compiler/i...
tools/json_schema_compiler/idl_schema.py:111: properties['$ref'] = self.typeref
On 2012/03/07 00:23:01, miket wrote:
> On 2012/03/06 22:27:57, Antony Sargent wrote:
> > Someday it would be cool to take note of not-yet-defined types that we see
> > during compile, and compare that to the full list of defined types when
we're
> > finished, and emit an error or warning for any that were never defined. That
> > could help catch cases of typos, etc. 
> 
> Agreed -- though at present, it's caught loudly and fatally during C++
> compilation.

Good point!

http://codereview.chromium.org/9600050/diff/1008/tools/json_schema_compiler/i...
tools/json_schema_compiler/idl_schema.py:143: 'namespace':
self.namespace.GetName() }
On 2012/03/07 00:23:01, miket wrote:
> On 2012/03/06 22:27:57, Antony Sargent wrote:
> > looks like you're missing the 'nodoc' attribute if it's present?
> 
> Fixed. I know I'm being inconsistent here because I handled it elsewhere, but
> what difference does nodoc make at this point (about to generate C code)?

Good point. (It was actually seeing the code elsewhere to handle nodoc that made
me think about the fact that it's missing here, but I didn't stop to think about
what it's used for :) )

[commit-bot: I haz the power, 2012-03-07 00:43:28]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/miket@chromium.org/9600050/7001

[commit-bot: I haz the power, 2012-03-07 02:33:37]

Change committed as 125300