[mojo] Add mojom parser support for native-only structs

[mojo] Add mojom parser support for native-only structs

This allows struct declarations to be parsed in mojom with no
struct body. Such structs are tagged as native-only and this
information is propagated to different language generators.

Part of a series of changes to support custom mojom serialization:

  1. This CL
  2. https://codereview.chromium.org/1517043004
  3. https://codereview.chromium.org/1524693002
  4. https://codereview.chromium.org/1520153002
  5. https://codereview.chromium.org/1524613002
  6. https://codereview.chromium.org/1526533002
  7. https://codereview.chromium.org/1524703002

BUG=569669
Committed: https://crrev.com/e1543ddc1e3cfeb3019cdda3357240727975d9d5
Cr-Commit-Position: refs/heads/master@{#365467}

[Ken Rockot(use gerrit already), 2015-12-12 04:50:20]

Description was changed from

==========
[mojo] Add basic bindings support for native-only structs

This adds support to the parser for two precursors for
ParamTraits+Mojo serialization:

  1. Compound attribute literals, for lists of metadata. This
     will be used to associate a list of C++ headers with an
     empty struct declaration.

  2. Empty mojom struct declarations, used to declare aliases to
     native types.

This stuff is also plumbed through to the generators, though no
real implementation support is provided in this CL.

BUG=
==========

to

==========
[mojo] Add basic bindings support for native-only structs

This adds support to the parser for two precursors for
ParamTraits+Mojo serialization:

  1. Compound attribute literals, for lists of metadata. This
     will be used to associate a list of C++ headers with an
     empty struct declaration.

  2. Empty mojom struct declarations, used to declare aliases to
     native types.

This stuff is also plumbed through to the generators, though no
real implementation support is provided in this CL.

Part of a series of changes to reuse existing serialization
traits with new Mojo bindings:

  1. https://codereview.chromium.org/1524613002
  2. This CL
  3. https://codereview.chromium.org/1517043004
  4. https://codereview.chromium.org/1518293002

BUG=
==========

[Ken Rockot(use gerrit already), 2015-12-14 07:05:08]

Description was changed from

==========
[mojo] Add basic bindings support for native-only structs

This adds support to the parser for two precursors for
ParamTraits+Mojo serialization:

  1. Compound attribute literals, for lists of metadata. This
     will be used to associate a list of C++ headers with an
     empty struct declaration.

  2. Empty mojom struct declarations, used to declare aliases to
     native types.

This stuff is also plumbed through to the generators, though no
real implementation support is provided in this CL.

Part of a series of changes to reuse existing serialization
traits with new Mojo bindings:

  1. https://codereview.chromium.org/1524613002
  2. This CL
  3. https://codereview.chromium.org/1517043004
  4. https://codereview.chromium.org/1518293002

BUG=
==========

to

==========
[mojo] Add mojom parser support for native-only structs

This allows struct declarations to be parsed in mojom with no
struct body. Such structs are tagged as native-only and this
information is propagated to different language generators.

Part of a series of changes to support custom mojom serialization:

  1. https://codereview.chromium.org/1524613002
  2. This CL
  3. https://codereview.chromium.org/1517043004
  4. https://codereview.chromium.org/1524693002
  5. https://codereview.chromium.org/1520153002
  6. https://codereview.chromium.org/1526533002
  7. https://codereview.chromium.org/1524703002

BUG=
==========

[Ken Rockot(use gerrit already), 2015-12-14 18:40:37]

rockot@chromium.org changed reviewers:
+ yzshen@chromium.org

[Ken Rockot(use gerrit already), 2015-12-14 18:40:38]

Please take a look

[Ken Rockot(use gerrit already), 2015-12-14 18:45:39]

Description was changed from

==========
[mojo] Add mojom parser support for native-only structs

This allows struct declarations to be parsed in mojom with no
struct body. Such structs are tagged as native-only and this
information is propagated to different language generators.

Part of a series of changes to support custom mojom serialization:

  1. https://codereview.chromium.org/1524613002
  2. This CL
  3. https://codereview.chromium.org/1517043004
  4. https://codereview.chromium.org/1524693002
  5. https://codereview.chromium.org/1520153002
  6. https://codereview.chromium.org/1526533002
  7. https://codereview.chromium.org/1524703002

BUG=
==========

to

==========
[mojo] Add mojom parser support for native-only structs

This allows struct declarations to be parsed in mojom with no
struct body. Such structs are tagged as native-only and this
information is propagated to different language generators.

Part of a series of changes to support custom mojom serialization:

  1. This CL
  2. https://codereview.chromium.org/1517043004
  3. https://codereview.chromium.org/1524693002
  4. https://codereview.chromium.org/1520153002
  5. https://codereview.chromium.org/1526533002
  6. https://codereview.chromium.org/1524703002

BUG=
==========

[Ken Rockot(use gerrit already), 2015-12-14 20:18:38]

Description was changed from

==========
[mojo] Add mojom parser support for native-only structs

This allows struct declarations to be parsed in mojom with no
struct body. Such structs are tagged as native-only and this
information is propagated to different language generators.

Part of a series of changes to support custom mojom serialization:

  1. This CL
  2. https://codereview.chromium.org/1517043004
  3. https://codereview.chromium.org/1524693002
  4. https://codereview.chromium.org/1520153002
  5. https://codereview.chromium.org/1526533002
  6. https://codereview.chromium.org/1524703002

BUG=
==========

to

==========
[mojo] Add mojom parser support for native-only structs

This allows struct declarations to be parsed in mojom with no
struct body. Such structs are tagged as native-only and this
information is propagated to different language generators.

Part of a series of changes to support custom mojom serialization:

  1. This CL
  2. https://codereview.chromium.org/1517043004
  3. https://codereview.chromium.org/1524693002
  4. https://codereview.chromium.org/1520153002
  5. https://codereview.chromium.org/1524613002
  6. https://codereview.chromium.org/1526533002
  7. https://codereview.chromium.org/1524703002

BUG=
==========

[Ken Rockot(use gerrit already), 2015-12-14 21:08:26]

Description was changed from

==========
[mojo] Add mojom parser support for native-only structs

This allows struct declarations to be parsed in mojom with no
struct body. Such structs are tagged as native-only and this
information is propagated to different language generators.

Part of a series of changes to support custom mojom serialization:

  1. This CL
  2. https://codereview.chromium.org/1517043004
  3. https://codereview.chromium.org/1524693002
  4. https://codereview.chromium.org/1520153002
  5. https://codereview.chromium.org/1524613002
  6. https://codereview.chromium.org/1526533002
  7. https://codereview.chromium.org/1524703002

BUG=
==========

to

==========
[mojo] Add mojom parser support for native-only structs

This allows struct declarations to be parsed in mojom with no
struct body. Such structs are tagged as native-only and this
information is propagated to different language generators.

Part of a series of changes to support custom mojom serialization:

  1. This CL
  2. https://codereview.chromium.org/1517043004
  3. https://codereview.chromium.org/1524693002
  4. https://codereview.chromium.org/1520153002
  5. https://codereview.chromium.org/1524613002
  6. https://codereview.chromium.org/1526533002
  7. https://codereview.chromium.org/1524703002

BUG=569669
==========

[yzshen1, 2015-12-14 21:33:10]

https://codereview.chromium.org/1515423002/diff/100001/mojo/public/interfaces...
File mojo/public/interfaces/bindings/tests/BUILD.gn (right):

https://codereview.chromium.org/1515423002/diff/100001/mojo/public/interfaces...
mojo/public/interfaces/bindings/tests/BUILD.gn:23: "test_native_types.mojom",
Do you think it will break JS/Java bindings?

FYI:
I just got mojo_js_unittests compile and pass with GN+new EDK. It is now running
on FYI bots:
http://build.chromium.org/p/chromium.mojo/console

I will need to enable it on chromium bots and try bots.

Unfortunately I haven't got a chance to enable Java unittests. At least if the
resulting Java code cannot compile we will receive some chromium bot failures, I
think.

My plan is to look into the enable Java unittests when I get back.

https://codereview.chromium.org/1515423002/diff/100001/mojo/public/interfaces...
File mojo/public/interfaces/bindings/tests/test_native_types.mojom (right):

https://codereview.chromium.org/1515423002/diff/100001/mojo/public/interfaces...
mojo/public/interfaces/bindings/tests/test_native_types.mojom:9: struct
PickledStruct;
This looks like a "forward declaration".

What do you think if we use something more explicit, say:
[native]
struct PickledStruct;

And only allow struct-without-body if attribute [native] is present.

Another thing is whether it is necessary to be a "struct"? From users'
perspective, they can put anything in this blob of memory, right? E.g., simply
an int32 or a vector. From bindings' perspective, maybe the bindings doesn't
have to encode the memory as a mojo struct either.
(But maybe it is okay comparing with introducing a new keyword. :))

[Ken Rockot(use gerrit already), 2015-12-14 23:50:57]

On 2015/12/14 at 21:33:10, yzshen wrote:
>
https://codereview.chromium.org/1515423002/diff/100001/mojo/public/interfaces...
> File mojo/public/interfaces/bindings/tests/BUILD.gn (right):
> 
>
https://codereview.chromium.org/1515423002/diff/100001/mojo/public/interfaces...
> mojo/public/interfaces/bindings/tests/BUILD.gn:23: "test_native_types.mojom",
> Do you think it will break JS/Java bindings?
> 
> FYI:
> I just got mojo_js_unittests compile and pass with GN+new EDK. It is now
running on FYI bots:
> http://build.chromium.org/p/chromium.mojo/console
> 
> I will need to enable it on chromium bots and try bots.
> 
> Unfortunately I haven't got a chance to enable Java unittests. At least if the
resulting Java code cannot compile we will receive some chromium bot failures, I
think.
> 
> My plan is to look into the enable Java unittests when I get back.

I changed the code a bit so that the JS and Java generators will still emit
bindings for the equivalent of an empty struct in place of native-only types.
This means the code will be valid, but it will not be sufficient to use the
generated bindings. Existing code is unaffected and I think later we can add
support for making native types look like variable-length arrays in other
languages.

> 
>
https://codereview.chromium.org/1515423002/diff/100001/mojo/public/interfaces...
> File mojo/public/interfaces/bindings/tests/test_native_types.mojom (right):
> 
>
https://codereview.chromium.org/1515423002/diff/100001/mojo/public/interfaces...
> mojo/public/interfaces/bindings/tests/test_native_types.mojom:9: struct
PickledStruct;
> This looks like a "forward declaration".
> 
> What do you think if we use something more explicit, say:
> [native]
> struct PickledStruct;
> 
> And only allow struct-without-body if attribute [native] is present.

SGTM, done.

> 
> Another thing is whether it is necessary to be a "struct"? From users'
perspective, they can put anything in this blob of memory, right? E.g., simply
an int32 or a vector. From bindings' perspective, maybe the bindings doesn't
have to encode the memory as a mojo struct either.
> (But maybe it is okay comparing with introducing a new keyword. :))

I think it's OK to leave it as struct IMHO. In a perfect world (IMHO) we
wouldn't need to support native-only types, and all wire formats would be
formally specified by mojom. Since this is meant as a transitional feature for
porting chromium IPC, and existing chromium IPC types are also "structs", this
should be OK.

[yzshen1, 2015-12-15 23:16:04]

LGTM

Please also consider adding tests for this change. (Fine if you would like to do
it in a separate CL later.)

[Ken Rockot(use gerrit already), 2015-12-16 00:41:18]

On 2015/12/15 at 23:16:04, yzshen wrote:
> LGTM
> 
> Please also consider adding tests for this change. (Fine if you would like to
do it in a separate CL later.)

OK - we get a little coverage just by virtue of the fact that the test bindings
process test_native_types.mojom. I'm planning to hack on a bindings generator
test suite this week.

[Ken Rockot(use gerrit already), 2015-12-16 02:03:01]

The CQ bit was checked by rockot@chromium.org

[Ken Rockot(use gerrit already), 2015-12-16 02:03:01]

The patchset sent to the CQ was uploaded after l-g-t-m from yzshen@chromium.org
Link to the patchset: https://codereview.chromium.org/1515423002/#ps160001
(title: "oops")

[commit-bot: I haz the power, 2015-12-16 02:11:29]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1515423002/160001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1515423002/160001

[commit-bot: I haz the power, 2015-12-16 02:52:59]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2015-12-16 02:53:00]

Try jobs failed on following builders:
  android_arm64_dbg_recipe on tryserver.chromium.linux (JOB_TIMED_OUT, no build
URL)

[Ken Rockot(use gerrit already), 2015-12-16 04:11:12]

The CQ bit was checked by rockot@chromium.org

[commit-bot: I haz the power, 2015-12-16 04:13:21]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1515423002/160001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1515423002/160001

[commit-bot: I haz the power, 2015-12-16 05:36:18]

Committed patchset #9 (id:160001)

[commit-bot: I haz the power, 2015-12-16 05:37:17]

Description was changed from

==========
[mojo] Add mojom parser support for native-only structs

This allows struct declarations to be parsed in mojom with no
struct body. Such structs are tagged as native-only and this
information is propagated to different language generators.

Part of a series of changes to support custom mojom serialization:

  1. This CL
  2. https://codereview.chromium.org/1517043004
  3. https://codereview.chromium.org/1524693002
  4. https://codereview.chromium.org/1520153002
  5. https://codereview.chromium.org/1524613002
  6. https://codereview.chromium.org/1526533002
  7. https://codereview.chromium.org/1524703002

BUG=569669
==========

to

==========
[mojo] Add mojom parser support for native-only structs

This allows struct declarations to be parsed in mojom with no
struct body. Such structs are tagged as native-only and this
information is propagated to different language generators.

Part of a series of changes to support custom mojom serialization:

  1. This CL
  2. https://codereview.chromium.org/1517043004
  3. https://codereview.chromium.org/1524693002
  4. https://codereview.chromium.org/1520153002
  5. https://codereview.chromium.org/1524613002
  6. https://codereview.chromium.org/1526533002
  7. https://codereview.chromium.org/1524703002

BUG=569669

Committed: https://crrev.com/e1543ddc1e3cfeb3019cdda3357240727975d9d5
Cr-Commit-Position: refs/heads/master@{#365467}
==========

[commit-bot: I haz the power, 2015-12-16 05:37:18]

Patchset 9 (id:??) landed as
https://crrev.com/e1543ddc1e3cfeb3019cdda3357240727975d9d5
Cr-Commit-Position: refs/heads/master@{#365467}