Add script tests for `gn check`, `gyp_flag_compare` to GN bot configs.

Add script tests for `gn check`, `gyp_flag_compare` to GN bot configs.

This adds the tests on the GN bots that are currently configured
recipe-side in the chromium_gn recipe to //testing/scripts, so
that they can be configured repo-side and we will need to modify
the recipe less often.

R=phajdan.jr@chromium.org
BUG=481693
Committed: https://crrev.com/7edb74bac6fc21fa06c78b9596a41322a4e6c078
Cr-Commit-Position: refs/heads/master@{#330862}

[Dirk Pranke, 2015-05-15 00:59:41]

I think this is roughly what's needed to move the `gn check` and `gyp flag
compare` steps over to repo-side script steps.

We need to do this (I think) before we can switch the GN bots to using the
chromium and chromium_trybot recipes.

However, when I tried to test this (see
http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_...
), it didn't run the script test steps. Did I miss something, or do I need to
modify the chromium_gn recipe to be script-test aware?

(It did run the old recipe-side `gn check` step, so don't be confused by that).

[Dirk Pranke, 2015-05-15 01:00:54]

https://codereview.chromium.org/1145513005/diff/1/testing/scripts/gyp_flag_co...
File testing/scripts/gyp_flag_compare.py (right):

https://codereview.chromium.org/1145513005/diff/1/testing/scripts/gyp_flag_co...
testing/scripts/gyp_flag_compare.py:27: 'gyp_flag_compare.py')] + args.args)
gyp_flag_compare.py takes a list of build targets to look at. 

It would be nice if I could get main_compile_targets() to echo the same list of
build targets back, so that we could be sure that we were compiling them, but
from what i can tell, main_compile_targets doesn't actually get the list of args
passed to it? Am I wrong?

[Paweł Hajdan Jr., 2015-05-15 10:14:07]

https://codereview.chromium.org/1145513005/diff/1/testing/scripts/gn_check.py
File testing/scripts/gn_check.py (right):

https://codereview.chromium.org/1145513005/diff/1/testing/scripts/gn_check.py...
testing/scripts/gn_check.py:41: 'failures': [],
Do you intend this to always pass? If there are any failures they should be
added to this list.

https://codereview.chromium.org/1145513005/diff/1/testing/scripts/gyp_flag_co...
File testing/scripts/gyp_flag_compare.py (right):

https://codereview.chromium.org/1145513005/diff/1/testing/scripts/gyp_flag_co...
testing/scripts/gyp_flag_compare.py:27: 'gyp_flag_compare.py')] + args.args)
On 2015/05/15 at 01:00:54, Dirk Pranke wrote:
> gyp_flag_compare.py takes a list of build targets to look at. 

FWIW I don't see any args being passed to this script in this CL.

> It would be nice if I could get main_compile_targets() to echo the same list
of build targets back, so that we could be sure that we were compiling them, but
from what i can tell, main_compile_targets doesn't actually get the list of args
passed to it? Am I wrong?

Yeah, I think it isn't passed currently and we'd need to change ScriptTest for
that. Feel free to make such a change (it seems non-trivial though).

https://codereview.chromium.org/1145513005/diff/1/testing/scripts/gyp_flag_co...
testing/scripts/gyp_flag_compare.py:31: 'failures': [],
Do you intend this to always pass? If there are any failures they should be
added to this list.

[Dirk Pranke, 2015-05-15 15:09:40]

https://codereview.chromium.org/1145513005/diff/1/testing/scripts/gn_check.py
File testing/scripts/gn_check.py (right):

https://codereview.chromium.org/1145513005/diff/1/testing/scripts/gn_check.py...
testing/scripts/gn_check.py:41: 'failures': [],
On 2015/05/15 10:14:07, Paweł Hajdan Jr. wrote:
> Do you intend this to always pass? If there are any failures they should be
> added to this list.

I expect the step to fail if rc is nonzero. I don't expect there to be a useful
list of failures (the command is a binary pass/fail, one big test). Do I need to
create some sort of placeholder value anyway?

https://codereview.chromium.org/1145513005/diff/1/testing/scripts/gyp_flag_co...
File testing/scripts/gyp_flag_compare.py (right):

https://codereview.chromium.org/1145513005/diff/1/testing/scripts/gyp_flag_co...
testing/scripts/gyp_flag_compare.py:27: 'gyp_flag_compare.py')] + args.args)
On 2015/05/15 10:14:07, Paweł Hajdan Jr. wrote:
> On 2015/05/15 at 01:00:54, Dirk Pranke wrote:
> > gyp_flag_compare.py takes a list of build targets to look at. 
> 
> FWIW I don't see any args being passed to this script in this CL.

Yeah, I didn't add the step at all to the json file, apparently.

> 
> > It would be nice if I could get main_compile_targets() to echo the same list
> of build targets back, so that we could be sure that we were compiling them,
but
> from what i can tell, main_compile_targets doesn't actually get the list of
args
> passed to it? Am I wrong?
> 
> Yeah, I think it isn't passed currently and we'd need to change ScriptTest for
> that. Feel free to make such a change (it seems non-trivial though).

It looked a bit difficult to me, too. I'll think about it some more.

https://codereview.chromium.org/1145513005/diff/1/testing/scripts/gyp_flag_co...
testing/scripts/gyp_flag_compare.py:31: 'failures': [],
On 2015/05/15 10:14:07, Paweł Hajdan Jr. wrote:
> Do you intend this to always pass? If there are any failures they should be
> added to this list.

same answer as the other one. eventually we might be able to do something
smarter but I don't want to bother with it yet.

[Paweł Hajdan Jr., 2015-05-15 15:41:14]

https://codereview.chromium.org/1145513005/diff/1/testing/buildbot/tryserver....
File testing/buildbot/tryserver.chromium.linux.json (right):

https://codereview.chromium.org/1145513005/diff/1/testing/buildbot/tryserver....
testing/buildbot/tryserver.chromium.linux.json:15: "scripts": [
Oops, note that chromium/chromium_trybot recipes which support scripts just
don't read this file. You'd need to find some other way to test this - likely to
use this from chromium/chromium_trybot recipes. I'd much rather not add new
functionality e.g. to chromium_trybot_legacy.

https://codereview.chromium.org/1145513005/diff/1/testing/scripts/gn_check.py
File testing/scripts/gn_check.py (right):

https://codereview.chromium.org/1145513005/diff/1/testing/scripts/gn_check.py...
testing/scripts/gn_check.py:41: 'failures': [],
On 2015/05/15 15:09:40, Dirk Pranke wrote:
> On 2015/05/15 10:14:07, Paweł Hajdan Jr. wrote:
> > Do you intend this to always pass? If there are any failures they should be
> > added to this list.
> 
> I expect the step to fail if rc is nonzero. I don't expect there to be a
useful
> list of failures (the command is a binary pass/fail, one big test). Do I need
to
> create some sort of placeholder value anyway?

It's expected by recipes that if rc is nonzero there are failures.

We can put some dummy value here for now, but with a TODO to make it real.

[Paweł Hajdan Jr., 2015-05-15 16:03:26]

LGTM

[Dirk Pranke, 2015-05-20 22:55:40]

The CQ bit was checked by dpranke@chromium.org

[commit-bot: I haz the power, 2015-05-20 22:57:12]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1145513005/80001

[commit-bot: I haz the power, 2015-05-21 00:37:27]

Committed patchset #5 (id:80001)

[commit-bot: I haz the power, 2015-05-21 00:38:18]

Patchset 5 (id:??) landed as
https://crrev.com/7edb74bac6fc21fa06c78b9596a41322a4e6c078
Cr-Commit-Position: refs/heads/master@{#330862}