[Do not commit] Sample USB layout tests with Mojo service mocks

[Do not commit] Sample USB layout tests with Mojo service mocks

Just showing off basic browser service mocking for layout tests.
This demonstrates mocking out the browser's USB DeviceManager service
to test navigator.usb.

BUG=

[Ken Rockot(use gerrit already), 2015-11-24 01:19:56]

rockot@chromium.org changed reviewers:
+ reillyg@chromium.org

[Ken Rockot(use gerrit already), 2015-11-24 01:19:57]

FYI - what testing may sort of look like when all the dependent patches land

[Ken Rockot(use gerrit already), 2015-11-24 01:27:05]

On 2015/11/24 at 01:19:57, Ken Rockot wrote:
> FYI - what testing may sort of look like when all the dependent patches land

Also, dependent patches start here: https://codereview.chromium.org/1468903002

[Ken Rockot(use gerrit already), 2016-01-25 18:01:04]

Description was changed from

==========
[Do not commit] Sample USB layout tests

Just showing off basic browser service mocking for layout tests.
This demonstrates mocking out the browser's USB DeviceManager service
to test navigator.usb.

Note that at least for now, generated mojom JS bindings are simply
lifted into the layout tests dir and manually loaded from there.
This seems better than hacking dynamic generated resource loading
into content_shell or requiring people to maintain a list of
usable bindings resources in shell_resources.grd.

Maybe a good idea to at least automate the copying with build rules
where appropriate and/or update the bindings generator so that
the generated code meets basic style guidelines.

BUG=
==========

to

==========
[Do not commit] Sample USB layout tests with Mojo service mocks

Just showing off basic browser service mocking for layout tests.
This demonstrates mocking out the browser's USB DeviceManager service
to test navigator.usb.

Note that at least for now, generated mojom JS bindings are simply
lifted into the layout tests dir and manually loaded from there.
This seems better than hacking dynamic generated resource loading
into content_shell or requiring people to maintain a list of
usable bindings resources in shell_resources.grd.

Maybe a good idea to at least automate the copying with build rules
where appropriate and/or update the bindings generator so that
the generated code meets basic style guidelines.

BUG=
==========

[Ken Rockot(use gerrit already), 2016-01-28 04:39:53]

rockot@chromium.org changed reviewers:
+ haraken@chromium.org, jam@chromium.org, sammc@chromium.org,
yukishiino@chromium.org

[Ken Rockot(use gerrit already), 2016-01-28 04:39:53]

Hey folks, here's a partial CL which uses the new layout test mojo bindings to
mock a service consumed by web API. I think this will be a great way in general
to mock out new service implementations going forward.

One major concern I have is how to deal with the generated JS bindings. I see
three OK options:

  1. Build them into content resources so they're always available by name to
the module system

  2. Have a standard generated output location to load them from the test
runner; add build rules to copy them there.

  3. Do what this CL does and keep them in version control next to the tests,
optionally with build rules to copy them in place on bindings update.


I don't like #3 because it means committing generated resources (ick); but it's
probably the fastest path forward.

#2 may be doable but I'm not sure what precedent there is for layout tests
depending on generated output files in the build. seems like there's potential
for some ugliness here.

#1 seems OK if we restrict the bindings content_shell only, and simplify the
process of adding new JS bindings -- e.g. we could add a build rule that is just
a list of mojom targets, and a tool that slurps them all into a generated
resource file.

WDYT?

[Ken Rockot(use gerrit already), 2016-01-28 05:29:58]

Description was changed from

==========
[Do not commit] Sample USB layout tests with Mojo service mocks

Just showing off basic browser service mocking for layout tests.
This demonstrates mocking out the browser's USB DeviceManager service
to test navigator.usb.

Note that at least for now, generated mojom JS bindings are simply
lifted into the layout tests dir and manually loaded from there.
This seems better than hacking dynamic generated resource loading
into content_shell or requiring people to maintain a list of
usable bindings resources in shell_resources.grd.

Maybe a good idea to at least automate the copying with build rules
where appropriate and/or update the bindings generator so that
the generated code meets basic style guidelines.

BUG=
==========

to

==========
[Do not commit] Sample USB layout tests with Mojo service mocks

Just showing off basic browser service mocking for layout tests.
This demonstrates mocking out the browser's USB DeviceManager service
to test navigator.usb.

BUG=
==========

[haraken, 2016-01-28 06:33:15]

On 2016/01/28 04:39:53, Ken Rockot wrote:
> Hey folks, here's a partial CL which uses the new layout test mojo bindings to
> mock a service consumed by web API. I think this will be a great way in
general
> to mock out new service implementations going forward.
> 
> One major concern I have is how to deal with the generated JS bindings. I see
> three OK options:
> 
>   1. Build them into content resources so they're always available by name to
> the module system
> 
>   2. Have a standard generated output location to load them from the test
> runner; add build rules to copy them there.
> 
>   3. Do what this CL does and keep them in version control next to the tests,
> optionally with build rules to copy them in place on bindings update.
> 
> 
> I don't like #3 because it means committing generated resources (ick); but
it's
> probably the fastest path forward.
> 
> #2 may be doable but I'm not sure what precedent there is for layout tests
> depending on generated output files in the build. seems like there's potential
> for some ugliness here.
> 
> #1 seems OK if we restrict the bindings content_shell only, and simplify the
> process of adding new JS bindings -- e.g. we could add a build rule that is
just
> a list of mojom targets, and a tool that slurps them all into a generated
> resource file.
> 
> WDYT?

I think #1 sounds reasonable. Blink generates various bindings into
out/{Release,Debug}/gen/blink/*.

Just to confirm: We're aiming at the following design, right?

  Mojo services (in the browser process) <== Mojo ==> Mojo clients (in the
renderer process; written in C++ in Blink's platform/)
                                  ^========= Mojo ==> Tests of the Mojo services
(in the renderer process; written in JS in LayoutTests)

Would you help me understand why we're not aiming at the following design? (I'm
not objecting to the above design; just want to understand the background.)

  Mojo services (in the browser process) <== Mojo ==> Mojo clients (in the
renderer process; written in C++ in Blink's platform/) <== Mojo ==> Tests of the
Mojo clients (in the renderer process; written in C++ in platform/'s unittests)

[Ken Rockot(use gerrit already), 2016-01-28 06:39:02]

On 2016/01/28 at 06:33:15, haraken wrote:
> On 2016/01/28 04:39:53, Ken Rockot wrote:
> > Hey folks, here's a partial CL which uses the new layout test mojo bindings
to
> > mock a service consumed by web API. I think this will be a great way in
general
> > to mock out new service implementations going forward.
> > 
> > One major concern I have is how to deal with the generated JS bindings. I
see
> > three OK options:
> > 
> >   1. Build them into content resources so they're always available by name
to
> > the module system
> > 
> >   2. Have a standard generated output location to load them from the test
> > runner; add build rules to copy them there.
> > 
> >   3. Do what this CL does and keep them in version control next to the
tests,
> > optionally with build rules to copy them in place on bindings update.
> > 
> > 
> > I don't like #3 because it means committing generated resources (ick); but
it's
> > probably the fastest path forward.
> > 
> > #2 may be doable but I'm not sure what precedent there is for layout tests
> > depending on generated output files in the build. seems like there's
potential
> > for some ugliness here.
> > 
> > #1 seems OK if we restrict the bindings content_shell only, and simplify the
> > process of adding new JS bindings -- e.g. we could add a build rule that is
just
> > a list of mojom targets, and a tool that slurps them all into a generated
> > resource file.
> > 
> > WDYT?
> 
> I think #1 sounds reasonable. Blink generates various bindings into
out/{Release,Debug}/gen/blink/*.
> 
> Just to confirm: We're aiming at the following design, right?
> 
>   Mojo services (in the browser process) <== Mojo ==> Mojo clients (in the
renderer process; written in C++ in Blink's platform/)
>                                   ^========= Mojo ==> Tests of the Mojo
services (in the renderer process; written in JS in LayoutTests)
> 
> Would you help me understand why we're not aiming at the following design?
(I'm not objecting to the above design; just want to understand the background.)
> 
>   Mojo services (in the browser process) <== Mojo ==> Mojo clients (in the
renderer process; written in C++ in Blink's platform/) <== Mojo ==> Tests of the
Mojo clients (in the renderer process; written in C++ in platform/'s unittests)

There isn't much background, and we could really go either way here. :)

In my opinion since all layout tests will be written in JS already, writing
their corresponding mock services in JS will allow developers to move more
quickly. All the test code is in one place and there's no need to switch between
the different bindings styles of C++ and JS.

Do you see any particular advantages to writing the mocks alongside platform
unittests? If it's likely that these unittests will also consume the mock
services in many cases, then that makes sense to me. But perhaps that won't be
the case.

Of course a third option is that we can make it really easy for developers to do
either one and choose the approach that works best for their specific needs.

[haraken, 2016-01-28 06:46:36]

haraken@chromium.org changed reviewers:
+ esprehn@chromium.org

[haraken, 2016-01-28 06:46:37]

On 2016/01/28 06:39:02, Ken Rockot wrote:
> On 2016/01/28 at 06:33:15, haraken wrote:
> > On 2016/01/28 04:39:53, Ken Rockot wrote:
> > > Hey folks, here's a partial CL which uses the new layout test mojo
bindings
> to
> > > mock a service consumed by web API. I think this will be a great way in
> general
> > > to mock out new service implementations going forward.
> > > 
> > > One major concern I have is how to deal with the generated JS bindings. I
> see
> > > three OK options:
> > > 
> > >   1. Build them into content resources so they're always available by name
> to
> > > the module system
> > > 
> > >   2. Have a standard generated output location to load them from the test
> > > runner; add build rules to copy them there.
> > > 
> > >   3. Do what this CL does and keep them in version control next to the
> tests,
> > > optionally with build rules to copy them in place on bindings update.
> > > 
> > > 
> > > I don't like #3 because it means committing generated resources (ick); but
> it's
> > > probably the fastest path forward.
> > > 
> > > #2 may be doable but I'm not sure what precedent there is for layout tests
> > > depending on generated output files in the build. seems like there's
> potential
> > > for some ugliness here.
> > > 
> > > #1 seems OK if we restrict the bindings content_shell only, and simplify
the
> > > process of adding new JS bindings -- e.g. we could add a build rule that
is
> just
> > > a list of mojom targets, and a tool that slurps them all into a generated
> > > resource file.
> > > 
> > > WDYT?
> > 
> > I think #1 sounds reasonable. Blink generates various bindings into
> out/{Release,Debug}/gen/blink/*.
> > 
> > Just to confirm: We're aiming at the following design, right?
> > 
> >   Mojo services (in the browser process) <== Mojo ==> Mojo clients (in the
> renderer process; written in C++ in Blink's platform/)
> >                                   ^========= Mojo ==> Tests of the Mojo
> services (in the renderer process; written in JS in LayoutTests)
> > 
> > Would you help me understand why we're not aiming at the following design?
> (I'm not objecting to the above design; just want to understand the
background.)
> > 
> >   Mojo services (in the browser process) <== Mojo ==> Mojo clients (in the
> renderer process; written in C++ in Blink's platform/) <== Mojo ==> Tests of
the
> Mojo clients (in the renderer process; written in C++ in platform/'s
unittests)
> 
> There isn't much background, and we could really go either way here. :)
> 
> In my opinion since all layout tests will be written in JS already, writing
> their corresponding mock services in JS will allow developers to move more
> quickly. All the test code is in one place and there's no need to switch
between
> the different bindings styles of C++ and JS.
> 
> Do you see any particular advantages to writing the mocks alongside platform
> unittests? If it's likely that these unittests will also consume the mock
> services in many cases, then that makes sense to me. But perhaps that won't be
> the case.
> 
> Of course a third option is that we can make it really easy for developers to
do
> either one and choose the approach that works best for their specific needs.

Given that layout tests are heavy, Blink has preferred unittests (written in
C++) than layout tests wherever possible. In other words, if you just want to
test C++ functionalities, unittests are preferred. (That said, most tests in
Blink are written in layout tests because most Blink features cannot be tested
without having a full context of Page.)

So, my opinion is that if layout tests are much easier than unittests, we should
go with layout tests. Otherwise we should go with unittests.

Let's see how others think about it.

[Yuki, 2016-01-28 07:05:55]

On 2016/01/28 06:39:02, Ken Rockot wrote:
> On 2016/01/28 at 06:33:15, haraken wrote:
> > On 2016/01/28 04:39:53, Ken Rockot wrote:
> > > Hey folks, here's a partial CL which uses the new layout test mojo
bindings
> to
> > > mock a service consumed by web API. I think this will be a great way in
> general
> > > to mock out new service implementations going forward.
> > > 
> > > One major concern I have is how to deal with the generated JS bindings. I
> see
> > > three OK options:
> > > 
> > >   1. Build them into content resources so they're always available by name
> to
> > > the module system
> > > 
> > >   2. Have a standard generated output location to load them from the test
> > > runner; add build rules to copy them there.
> > > 
> > >   3. Do what this CL does and keep them in version control next to the
> tests,
> > > optionally with build rules to copy them in place on bindings update.
> > > 
> > > 
> > > I don't like #3 because it means committing generated resources (ick); but
> it's
> > > probably the fastest path forward.
> > > 
> > > #2 may be doable but I'm not sure what precedent there is for layout tests
> > > depending on generated output files in the build. seems like there's
> potential
> > > for some ugliness here.
> > > 
> > > #1 seems OK if we restrict the bindings content_shell only, and simplify
the
> > > process of adding new JS bindings -- e.g. we could add a build rule that
is
> just
> > > a list of mojom targets, and a tool that slurps them all into a generated
> > > resource file.
> > > 
> > > WDYT?
> > 
> > I think #1 sounds reasonable. Blink generates various bindings into
> out/{Release,Debug}/gen/blink/*.
> > 
> > Just to confirm: We're aiming at the following design, right?
> > 
> >   Mojo services (in the browser process) <== Mojo ==> Mojo clients (in the
> renderer process; written in C++ in Blink's platform/)
> >                                   ^========= Mojo ==> Tests of the Mojo
> services (in the renderer process; written in JS in LayoutTests)
> > 
> > Would you help me understand why we're not aiming at the following design?
> (I'm not objecting to the above design; just want to understand the
background.)
> > 
> >   Mojo services (in the browser process) <== Mojo ==> Mojo clients (in the
> renderer process; written in C++ in Blink's platform/) <== Mojo ==> Tests of
the
> Mojo clients (in the renderer process; written in C++ in platform/'s
unittests)
> 
> There isn't much background, and we could really go either way here. :)
> 
> In my opinion since all layout tests will be written in JS already, writing
> their corresponding mock services in JS will allow developers to move more
> quickly. All the test code is in one place and there's no need to switch
between
> the different bindings styles of C++ and JS.
> 
> Do you see any particular advantages to writing the mocks alongside platform
> unittests? If it's likely that these unittests will also consume the mock
> services in many cases, then that makes sense to me. But perhaps that won't be
> the case.
> 
> Of course a third option is that we can make it really easy for developers to
do
> either one and choose the approach that works best for their specific needs.

Does option #1 mean that you're going to generate the JS bindings into a single
root directory?  Then, I don't see much difference between #1 and #2.  If
generated JS bindings will be stored in a single root directory and they will be
indexed in the code search (like out/Debug/gen/blink/), I love it.