Pull in native_client and build tests.

Pull in native_client and build tests.

NaCl has not been added to the default developer workflow, yet.  You must point
your .gclient file at DEPS.nacl and pass --nacl to mojob.py.

BUG=https://code.google.com/p/chromium/issues/detail?id=401761
R=jamesr@chromium.org

Committed: https://chromium.googlesource.com/external/mojo/+/e7561b4b207cf94608edca0364f6edf60aed975d

[Nick Bray (chromium), 2015-01-06 23:20:40]

ncbray@chromium.org changed reviewers:
+ jamesr@chromium.org

[Nick Bray (chromium), 2015-01-06 23:20:40]

Before landing:
* Requires Chrome to be rolled into Mojo (updated
src/build/download_nacl_toolchains.py).
* Requires a patch for Android to be landed in the NaCl repo and the DEPS in
this CL bumped up.

But the CL itself should be reviewable.

After landing, I'll follow up with a CL to run the tests being build by this CL
- but that will require changing mojob.py to be aware of the architecture it's
building for.

[Nick Bray (chromium), 2015-01-07 23:09:49]

NaCl-side changes rolled in. Good to go modulo Chrome roll.  Will do the roll if
it hasn't been done by the time this review is complete.

[jamesr, 2015-01-07 23:21:50]

Can you make pulling the nacl toolchain optional, like the dart syncing?  It's
very slow and the majority of developers won't care about about it.

[Nick Bray (chromium), 2015-01-08 01:29:32]

On 2015/01/07 23:21:50, jamesr wrote:
> Can you make pulling the nacl toolchain optional, like the dart syncing?  It's
> very slow and the majority of developers won't care about about it.

Do you have a citation for dart syncing?  A naive look in the DEPS file reveals
nothing.  My gut reaction is "things didn't work out so well for Dart."

Trying to think through the implications.  Reducing impact on developers until
it's ready for wider use, that's good.  The thing I worry about are regressions
for an untested configuration.  (Android is almost in the same boat, I had to
spend some extra effort testing and making sure I didn't break the Android
build.)  Do we have bots, yet?  If there were bots that always built with NaCl
enabled and a CQ, that would go a long way.  If there is no such beast, I would
be scared.

There's two possible flavors of cutting things down:
No //native_client/toolchain
No //native_client

If //native_client/toolchain disappears, it will be impossible to run any NaCl
tests (no untrusted binaries), but compiling the NaCl runtime for mojo_shell
would still detect and prevent trusted build breakages.  It might be possible to
can some untrusted binaries for tests, but that's brittle and I'd rather avoid
it.

Gating out the entire NaCl repo entirely would make system-level changes
somewhat dicey, but I suppose people working on the system would be idiots to
disable NaCl and the problem would be self correcting.  (Or self correcting in
light of a judiciously wielded revert hammer.)

There would need to be a mechanism to propagate disabling nacl all the way in to
mojob / gn.

Opt-in vs opt-out?  If there were appropriate bots, it would make sense to start
opt-in, with some way to eventually migrate to opt-out when things have jelled.

To be concrete about impact, the toolchain download will be faster than the
Chrome (I've cut things down quite a bit), but it'll still be ~120MB / 60
seconds on corp when the toolchain changes - maybe once a month?  Maybe more
often while I am trying to bring things up.

[jamesr, 2015-01-08 01:37:53]

We have bots and trybots for Android.  Test coverage on those bots on android is
a bit lacking but for the most part seeing if a mojo patch works for android or
not is just a matter of doing "git cl upload && git cl try".  Dart isn't run on
bots yet as it doesn't fully work yet (iiuc) and is mostly only being used by
the people actively working on it.  I feel like for now while things are being
brought up the same will be true for NaCL, and given that the toolchain download
etc is quite slow relative to the rest of the sync and it's flaky I don't really
want to impose that on everyone.

I'd prefer having it be opt-in at first, getting it working, adding bots that do
the opt-in, then when it's not going to change much and it's stable on the bots
talking about flipping the bit.

You can see how dart works in DEPS.dart and the --with-dart parameter to
mojob.py.  I haven't tried it myself, but given that the dart bindings are still
being developed and reviewed at least two people have had success with them.

[Nick Bray (chromium), 2015-01-08 21:39:49]

PTAL
I deviated slightly from dart by calling the flag --nacl because I anticipate a
--no-nacl someday.

> etc is quite slow relative to the rest of the sync and it's flaky I don't
really
> want to impose that on everyone.

FYI the main reason for flake is cloud storage flaking out.  If you're worried
about toolchain downloads flake, the same applies to pretty much every binary
that will be downloaded.

[jamesr, 2015-01-08 21:41:51]

lgtm

I haven't observed as much google cloud storage flake for smaller files, but
maybe we've just been lucky (or observer bias)

[Nick Bray (chromium), 2015-01-09 18:54:24]

Committed patchset #5 (id:80001) manually as
e7561b4b207cf94608edca0364f6edf60aed975d (presubmit successful).