Enable Mojo on iOS

Enable Mojo on iOS

This CL enables Mojo on iOS. The following are not supported:
- Multiprocess-related files
- OpenGL-related code
- Apps
- Shell
- JS

Some of these, particularly JS, will likely be supported in the future.

[viettrungluu, 2014-07-01 16:37:22]

mojo lgtm, but you'll still need someone to approve the content changes

[blundell, 2014-07-01 16:50:43]

+jam@ for //content

[jamesr, 2014-07-01 19:40:22]

This is really invasive to the gyp files, which is sadmaking.  What bits do you
want to use on iOS?

[blundell, 2014-07-01 19:43:46]

On 2014/07/01 19:40:22, jamesr wrote:
> This is really invasive to the gyp files, which is sadmaking.  What bits do
you
> want to use on iOS?

The goal in this CL is to support C++-C++ communication via Mojo on iOS. I built
up a simple test case and found that the targets enabled here were the ones that
I needed to get that support working.

[jamesr, 2014-07-02 02:04:08]

On 2014/07/01 19:43:46, blundell wrote:
> On 2014/07/01 19:40:22, jamesr wrote:
> > This is really invasive to the gyp files, which is sadmaking.  What bits do
> you
> > want to use on iOS?
> 
> The goal in this CL is to support C++-C++ communication via Mojo on iOS. I
built
> up a simple test case and found that the targets enabled here were the ones
that
> I needed to get that support working.

Can't you simply depend on only the targets you want in your iOS code?  The
'mojo' meta-target is not something you will depend on in code so it shouldn't
impact you and I don't see any reason to cut up unit tests just for iOS - just
pick the ones that you want to run.

[blundell, 2014-07-02 08:14:30]

On 2014/07/02 02:04:08, jamesr wrote:
> On 2014/07/01 19:43:46, blundell wrote:
> > On 2014/07/01 19:40:22, jamesr wrote:
> > > This is really invasive to the gyp files, which is sadmaking.  What bits
do
> > you
> > > want to use on iOS?
> > 
> > The goal in this CL is to support C++-C++ communication via Mojo on iOS. I
> built
> > up a simple test case and found that the targets enabled here were the ones
> that
> > I needed to get that support working.
> 
> Can't you simply depend on only the targets you want in your iOS code?  The
> 'mojo' meta-target is not something you will depend on in code so it shouldn't
> impact you and I don't see any reason to cut up unit tests just for iOS - just
> pick the ones that you want to run.

When gyp processes a gypfile, it processes all the targets in that file. The
targets that are put in the !iOS blocks here cause gyp to choke on iOS because
it ends up looking for v8.gyp, which doesn't exist on iOS because iOS has None-d
out v8 in DEPS.

Additionally, I want to have these test suites running on iOS on the buildbots.

[blundell, 2014-07-02 08:19:16]

On 2014/07/02 08:14:30, blundell wrote:
> On 2014/07/02 02:04:08, jamesr wrote:
> > On 2014/07/01 19:43:46, blundell wrote:
> > > On 2014/07/01 19:40:22, jamesr wrote:
> > > > This is really invasive to the gyp files, which is sadmaking.  What bits
> do
> > > you
> > > > want to use on iOS?
> > > 
> > > The goal in this CL is to support C++-C++ communication via Mojo on iOS. I
> > built
> > > up a simple test case and found that the targets enabled here were the
ones
> > that
> > > I needed to get that support working.
> > 
> > Can't you simply depend on only the targets you want in your iOS code?  The
> > 'mojo' meta-target is not something you will depend on in code so it
shouldn't
> > impact you and I don't see any reason to cut up unit tests just for iOS -
just
> > pick the ones that you want to run.
> 
> When gyp processes a gypfile, it processes all the targets in that file. The
> targets that are put in the !iOS blocks here cause gyp to choke on iOS because
> it ends up looking for v8.gyp, which doesn't exist on iOS because iOS has
None-d
> out v8 in DEPS.
> 
> Additionally, I want to have these test suites running on iOS on the
buildbots.

An alternative way to make the changes re: which targets are included in
mojo.gyp would be to carve more .gypi files out of mojo.gyp, dividing them into
ones that are entirely included on iOS and ones that have no targets included on
iOS (then the condition is just on which gypi files get included). If that's
appealing, I'm happy to structure the changes that way.

[blundell, 2014-07-02 08:30:19]

On 2014/07/02 08:19:16, blundell wrote:
> On 2014/07/02 08:14:30, blundell wrote:
> > On 2014/07/02 02:04:08, jamesr wrote:
> > > On 2014/07/01 19:43:46, blundell wrote:
> > > > On 2014/07/01 19:40:22, jamesr wrote:
> > > > > This is really invasive to the gyp files, which is sadmaking.  What
bits
> > do
> > > > you
> > > > > want to use on iOS?
> > > > 
> > > > The goal in this CL is to support C++-C++ communication via Mojo on iOS.
I
> > > built
> > > > up a simple test case and found that the targets enabled here were the
> ones
> > > that
> > > > I needed to get that support working.
> > > 
> > > Can't you simply depend on only the targets you want in your iOS code? 
The
> > > 'mojo' meta-target is not something you will depend on in code so it
> shouldn't
> > > impact you and I don't see any reason to cut up unit tests just for iOS -
> just
> > > pick the ones that you want to run.
> > 
> > When gyp processes a gypfile, it processes all the targets in that file. The
> > targets that are put in the !iOS blocks here cause gyp to choke on iOS
because
> > it ends up looking for v8.gyp, which doesn't exist on iOS because iOS has
> None-d
> > out v8 in DEPS.
> > 
> > Additionally, I want to have these test suites running on iOS on the
> buildbots.
> 
> An alternative way to make the changes re: which targets are included in
> mojo.gyp would be to carve more .gypi files out of mojo.gyp, dividing them
into
> ones that are entirely included on iOS and ones that have no targets included
on
> iOS (then the condition is just on which gypi files get included). If that's
> appealing, I'm happy to structure the changes that way.

To be clear, this would only make sense to the extent that there are sensible
ways to structure the gypi files. For at least some of the targets in question,
that seems to be true (e.g., we could create mojo_shell.gypi).

[abarth-chromium, 2014-07-02 14:28:11]

On 2014/07/02 at 08:14:30, blundell wrote:
> When gyp processes a gypfile, it processes all the targets in that file.

Maybe we should fix that bug.

[blundell, 2014-07-02 15:28:51]

On 2014/07/02 14:28:11, abarth wrote:
> On 2014/07/02 at 08:14:30, blundell wrote:
> > When gyp processes a gypfile, it processes all the targets in that file.
> 
> Maybe we should fix that bug.

+mento

Mark, would changing gyp so that it only processes the targets in a given
gypfile that are actually referenced via dependencies from other gypfiles be
feasible? Is the current behavior intentional?

Thanks,

Colin

[jamesr, 2014-07-02 15:45:59]

On 2014/07/02 08:14:30, blundell wrote:
> which doesn't exist on iOS because iOS has None-d out v8 in DEPS.

An even simpler fix would be to fix this.  This is an optimization/hack to speed
up checkouts for iOS developers since those libraries aren't used, but it costs
a large amount of complexity in the gyp project of everything else in the build.
 Every other chrome developers checks out some dependencies that aren't used on
their development platform.  Disk space is very cheap, bandwidth is very cheap
and syncing deps is a highly parallel operation so the marginal cost of another
dep is even cheaper.  Developer time is not and our build system is a very large
source of lost developer time.  The current setup provides a very small speedup
for a very small minority of chrome developers at a heavy cost to every single
chrome developer.  That's a really bad tradeoff.  I don't think it's at all fair
for a small team like the iOS team to impose this cost on everyone else.

[jam, 2014-07-02 15:56:50]

On 2014/07/02 15:45:59, jamesr wrote:
> On 2014/07/02 08:14:30, blundell wrote:
> > which doesn't exist on iOS because iOS has None-d out v8 in DEPS.
> 
> An even simpler fix would be to fix this.  This is an optimization/hack to
speed
> up checkouts for iOS developers since those libraries aren't used, but it
costs
> a large amount of complexity in the gyp project of everything else in the
build.
>  Every other chrome developers checks out some dependencies that aren't used
on
> their development platform.  Disk space is very cheap, bandwidth is very cheap
> and syncing deps is a highly parallel operation so the marginal cost of
another
> dep is even cheaper.  Developer time is not and our build system is a very
large
> source of lost developer time.  The current setup provides a very small
speedup
> for a very small minority of chrome developers at a heavy cost to every single
> chrome developer.  That's a really bad tradeoff.  I don't think it's at all
fair
> for a small team like the iOS team to impose this cost on everyone else.

I'm not familiar with this in detail, but if James' characterization is correct,
I couldn't agree more.

[blundell, 2014-07-02 15:58:17]

On 2014/07/02 15:45:59, jamesr wrote:
> On 2014/07/02 08:14:30, blundell wrote:
> > which doesn't exist on iOS because iOS has None-d out v8 in DEPS.
> 
> An even simpler fix would be to fix this.  This is an optimization/hack to
speed
> up checkouts for iOS developers since those libraries aren't used, but it
costs
> a large amount of complexity in the gyp project of everything else in the
build.
>  Every other chrome developers checks out some dependencies that aren't used
on
> their development platform.  Disk space is very cheap, bandwidth is very cheap
> and syncing deps is a highly parallel operation so the marginal cost of
another
> dep is even cheaper.  Developer time is not and our build system is a very
large
> source of lost developer time.  The current setup provides a very small
speedup
> for a very small minority of chrome developers at a heavy cost to every single
> chrome developer.  That's a really bad tradeoff.  I don't think it's at all
fair
> for a small team like the iOS team to impose this cost on everyone else.

That wouldn't actually solve the problem here. A call to "ninja -C out/Debug"
(which is what the bots build I believe) would build these targets, which would
break on iOS due to their dependencies and assumptions that aren't buildable on
iOS (gin, v8, multiprocess, ...).

[blundell, 2014-07-02 15:59:49]

On 2014/07/02 15:58:17, blundell wrote:
> On 2014/07/02 15:45:59, jamesr wrote:
> > On 2014/07/02 08:14:30, blundell wrote:
> > > which doesn't exist on iOS because iOS has None-d out v8 in DEPS.
> > 
> > An even simpler fix would be to fix this.  This is an optimization/hack to
> speed
> > up checkouts for iOS developers since those libraries aren't used, but it
> costs
> > a large amount of complexity in the gyp project of everything else in the
> build.
> >  Every other chrome developers checks out some dependencies that aren't used
> on
> > their development platform.  Disk space is very cheap, bandwidth is very
cheap
> > and syncing deps is a highly parallel operation so the marginal cost of
> another
> > dep is even cheaper.  Developer time is not and our build system is a very
> large
> > source of lost developer time.  The current setup provides a very small
> speedup
> > for a very small minority of chrome developers at a heavy cost to every
single
> > chrome developer.  That's a really bad tradeoff.  I don't think it's at all
> fair
> > for a small team like the iOS team to impose this cost on everyone else.
> 
> That wouldn't actually solve the problem here. A call to "ninja -C out/Debug"
> (which is what the bots build I believe) would build these targets, which
would
> break on iOS due to their dependencies and assumptions that aren't buildable
on
> iOS (gin, v8, multiprocess, ...).

That's actually a response to Adam's point as well.

[jamesr, 2014-07-02 16:04:18]

On 2014/07/02 15:58:17, blundell wrote:
> On 2014/07/02 15:45:59, jamesr wrote:
> > On 2014/07/02 08:14:30, blundell wrote:
> > > which doesn't exist on iOS because iOS has None-d out v8 in DEPS.
> > 
> > An even simpler fix would be to fix this.  This is an optimization/hack to
> speed
> > up checkouts for iOS developers since those libraries aren't used, but it
> costs
> > a large amount of complexity in the gyp project of everything else in the
> build.
> >  Every other chrome developers checks out some dependencies that aren't used
> on
> > their development platform.  Disk space is very cheap, bandwidth is very
cheap
> > and syncing deps is a highly parallel operation so the marginal cost of
> another
> > dep is even cheaper.  Developer time is not and our build system is a very
> large
> > source of lost developer time.  The current setup provides a very small
> speedup
> > for a very small minority of chrome developers at a heavy cost to every
single
> > chrome developer.  That's a really bad tradeoff.  I don't think it's at all
> fair
> > for a small team like the iOS team to impose this cost on everyone else.
> 
> That wouldn't actually solve the problem here. A call to "ninja -C out/Debug"
> (which is what the bots build I believe) would build these targets, which
would
> break on iOS due to their dependencies and assumptions that aren't buildable
on
> iOS (gin, v8, multiprocess, ...).

You could configure the bots to build only targets relevant to the iOS build.

[blundell, 2014-07-02 16:08:36]

On 2014/07/02 16:04:18, jamesr wrote:
> On 2014/07/02 15:58:17, blundell wrote:
> > On 2014/07/02 15:45:59, jamesr wrote:
> > > On 2014/07/02 08:14:30, blundell wrote:
> > > > which doesn't exist on iOS because iOS has None-d out v8 in DEPS.
> > > 
> > > An even simpler fix would be to fix this.  This is an optimization/hack to
> > speed
> > > up checkouts for iOS developers since those libraries aren't used, but it
> > costs
> > > a large amount of complexity in the gyp project of everything else in the
> > build.
> > >  Every other chrome developers checks out some dependencies that aren't
used
> > on
> > > their development platform.  Disk space is very cheap, bandwidth is very
> cheap
> > > and syncing deps is a highly parallel operation so the marginal cost of
> > another
> > > dep is even cheaper.  Developer time is not and our build system is a very
> > large
> > > source of lost developer time.  The current setup provides a very small
> > speedup
> > > for a very small minority of chrome developers at a heavy cost to every
> single
> > > chrome developer.  That's a really bad tradeoff.  I don't think it's at
all
> > fair
> > > for a small team like the iOS team to impose this cost on everyone else.
> > 
> > That wouldn't actually solve the problem here. A call to "ninja -C
out/Debug"
> > (which is what the bots build I believe) would build these targets, which
> would
> > break on iOS due to their dependencies and assumptions that aren't buildable
> on
> > iOS (gin, v8, multiprocess, ...).
> 
> You could configure the bots to build only targets relevant to the iOS build.

I personally don't think it's viable to have platforms where "ninja -C
out/Debug" fails.

Note that the None-ing out of v8 on iOS is in response to the issue that v8
won't build (and can't be shipped due to Apple restrictions) on iOS. Removing
the None-ing out wouldn't change the fundamental issue that targets that are
built on iOS can't use v8, and that targets that use v8 can't be built on iOS.

[Mark Mentovai, 2014-07-02 16:13:09]

On 2014/07/02 14:28:11, abarth wrote:
> On 2014/07/02 at 08:14:30, blundell wrote:
> > When gyp processes a gypfile, it processes all the targets in that file.
> 
> Maybe we should fix that bug.

It’s not a bug.

[stuartmorgan, 2014-07-02 16:18:53]

On 2014/07/02 16:08:36, blundell wrote:
> I personally don't think it's viable to have platforms where "ninja -C
> out/Debug" fails.

+1. I'm not aware of any platform where it's accepted to say that developers
must carry the cognitive load of knowing which targets can and can't build, nor
does it seem at all reasonable that a core, standard ninja command should be
broken for anyone working on one specific platform.

jamesr: you said, we shouldn't impose heavy costs on parts of the team. Taking
information that belongs *in* the build system—which targets can and can't be
built—and externalizing that to any developer who ever needs to work on iOS, is
a very heavy cost.

[jamesr, 2014-07-02 16:58:27]

We have many platforms that use different parts of the system. None of the other
platforms require making gyp files unreadable like this.  Not lgtm.

[jamesr, 2014-07-02 23:40:45]

On 2014/07/02 15:58:17, blundell wrote:
> On 2014/07/02 15:45:59, jamesr wrote:
> > On 2014/07/02 08:14:30, blundell wrote:
> > > which doesn't exist on iOS because iOS has None-d out v8 in DEPS.
> > 
> > An even simpler fix would be to fix this.  This is an optimization/hack to
> speed
> > up checkouts for iOS developers since those libraries aren't used, but it
> costs
> > a large amount of complexity in the gyp project of everything else in the
> build.
> >  Every other chrome developers checks out some dependencies that aren't used
> on
> > their development platform.  Disk space is very cheap, bandwidth is very
cheap
> > and syncing deps is a highly parallel operation so the marginal cost of
> another
> > dep is even cheaper.  Developer time is not and our build system is a very
> large
> > source of lost developer time.  The current setup provides a very small
> speedup
> > for a very small minority of chrome developers at a heavy cost to every
single
> > chrome developer.  That's a really bad tradeoff.  I don't think it's at all
> fair
> > for a small team like the iOS team to impose this cost on everyone else.
> 
> That wouldn't actually solve the problem here. A call to "ninja -C out/Debug"
> (which is what the bots build I believe) would build these targets, 

This is also not accurate in two ways.  First, the bots don't do that.  Second,
humans don't and this doesn't work on all platforms.  We have a build/all.gyp
that defines the 'All' target (note the capitalization) and a number of other
targets that various buildbots are configured to compile.  That file is full of
conditionals, including for ios, so I suspect it's used on the ios bots too
(although I don't know how to check).  If it's not, it's easy to fix so that it
is.

Second, it's definitely not the case that building the 'all' target (the
implicit one, again note the capitalization) is expected to work everywhere.  On
android many targets do not work in the 'all' set such as 'content_shell' (you
build 'content_shell' instead) and this has been the case as long as I've been
aware (so well before open sourcing, at a minimum).  This is not ideal but the
alternative would make the gyp files pretty nasty and as far as I know the
android devs seem to be getting on.  I've also observed that on linux sometimes
the 'all' target doesn't build, which isn't too surprising for anything without
buildbot coverage.

[blundell, 2014-07-03 12:20:36]

On 2014/07/02 23:40:45, jamesr wrote:
> On 2014/07/02 15:58:17, blundell wrote:
> > On 2014/07/02 15:45:59, jamesr wrote:
> > > On 2014/07/02 08:14:30, blundell wrote:
> > > > which doesn't exist on iOS because iOS has None-d out v8 in DEPS.
> > > 
> > > An even simpler fix would be to fix this.  This is an optimization/hack to
> > speed
> > > up checkouts for iOS developers since those libraries aren't used, but it
> > costs
> > > a large amount of complexity in the gyp project of everything else in the
> > build.
> > >  Every other chrome developers checks out some dependencies that aren't
used
> > on
> > > their development platform.  Disk space is very cheap, bandwidth is very
> cheap
> > > and syncing deps is a highly parallel operation so the marginal cost of
> > another
> > > dep is even cheaper.  Developer time is not and our build system is a very
> > large
> > > source of lost developer time.  The current setup provides a very small
> > speedup
> > > for a very small minority of chrome developers at a heavy cost to every
> single
> > > chrome developer.  That's a really bad tradeoff.  I don't think it's at
all
> > fair
> > > for a small team like the iOS team to impose this cost on everyone else.
> > 
> > That wouldn't actually solve the problem here. A call to "ninja -C
out/Debug"
> > (which is what the bots build I believe) would build these targets, 
> 
> This is also not accurate in two ways.  First, the bots don't do that. 
Second,
> humans don't and this doesn't work on all platforms.  We have a build/all.gyp
> that defines the 'All' target (note the capitalization) and a number of other
> targets that various buildbots are configured to compile.  That file is full
of
> conditionals, including for ios, so I suspect it's used on the ios bots too
> (although I don't know how to check).  If it's not, it's easy to fix so that
it
> is.
> 
> Second, it's definitely not the case that building the 'all' target (the
> implicit one, again note the capitalization) is expected to work everywhere. 
On
> android many targets do not work in the 'all' set such as 'content_shell' (you
> build 'content_shell' instead) and this has been the case as long as I've been
> aware (so well before open sourcing, at a minimum).  This is not ideal but the
> alternative would make the gyp files pretty nasty and as far as I know the
> android devs seem to be getting on.  I've also observed that on linux
sometimes
> the 'all' target doesn't build, which isn't too surprising for anything
without
> buildbot coverage.

Hi James,

Thanks for the corrections here. I verified that the iOS bots build "All"
explicitly. I typically use the implicit "all" target and hadn't been aware that
it's not particularly supported, although it makes sense given that we have the
explicit "All" target. That leaves the question of pulling v8.

The reason that we don't pull v8 on iOS is not for performance; rather, it's to
ensure that no v8 dependencies creep into the iOS build (since Apple could
reject the app for those reasons). If needed, I can imagine other ways of
ensuring that (e.g., adding a step on the iOS bots that checks that no v8 object
files are generated), but I want to run an alternative proposal by you first to
see what you think.

I was able to remove the conditional defining of targets in mojo_public.gypi
(see the most recent patch). That leaves mojo.gyp. The targets that I'm looking
to build on iOS out of there are the following:

'mojo_common_lib', 'mojo_common_unittests', 'mojo_message_generator',
'mojo_service_manager', 'mojo_service_manager_unittests', 'mojo_system',
'mojo_system_impl', 'mojo_system_unittests', 'mojo_environment_chromium',
'mojo_environment_chromium_impl', 'mojo_common_test_support',
'mojo_run_all_unittests', 'mojo_test_support',       'mojo_test_support_impl'.

Does it seem reasonable to pull these targets out into a separate gypfile named
(e.g.) mojo_core.gyp that mojo.gyp would then depend on? IIUC we will also be
moving mojo_public.gypi to mojo_public.gyp (per Chris' email to mojo-dev). iOS
would then use targets from mojo_public.gyp and mojo_core.gyp, but not from
mojo.gyp; no targets would need to be conditionally defined. If it seems like it
would increase clarity, mojo.gyp could then be renamed or further split up
(e.g., mojo_shell.gyp, mojo_apps.gyp, ...). The targets mentioned above could of
course also be split across multiple gypfiles if that seems like a better
organization.

Thanks,

Colin

[blundell, 2014-07-09 09:58:34]

james: ping. Thanks!

[jamesr, 2014-07-10 00:58:37]

I don't see how that helps.  If those are the targets you want to use in ios,
then depend on those targets from ios targets.  It seems weird to split the
different language bindings and other up into 'core' and 'non-core' based on
what one consumer will use - it's not hard to imagine another consumer wanting
only the js bindings and not the chromium environment or other mixes.