Adds ability to filter the set of tests that are run by a bot

Its not enabled for any bots yet. Enabling is done by way of
chromium_trybot.json.

BUG=109173
TEST=covered by tests

Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=287778

[sky, 2014-07-30 01:00:27]

https://codereview.chromium.org/427073003/diff/40001/scripts/slave/recipe_mod...
File scripts/slave/recipe_modules/filter/api.py (left):

https://codereview.chromium.org/427073003/diff/40001/scripts/slave/recipe_mod...
scripts/slave/recipe_modules/filter/api.py:64: step_test_data=lambda:
For the life of me I couldn't get step_test_data to work from
chromium_trybot.json. I ended up giving up on it.

[iannucci, 2014-07-30 01:20:37]

lgtm w/ comments

https://codereview.chromium.org/427073003/diff/40001/scripts/slave/recipe_mod...
File scripts/slave/recipe_modules/filter/api.py (left):

https://codereview.chromium.org/427073003/diff/40001/scripts/slave/recipe_mod...
scripts/slave/recipe_modules/filter/api.py:64: step_test_data=lambda:
On 2014/07/30 01:00:26, sky wrote:
> For the life of me I couldn't get step_test_data to work from
> chromium_trybot.json. I ended up giving up on it.

did you try duplicating the raw_io/test_api.py:stream_output method to
json/test_api.py ?

https://codereview.chromium.org/427073003/diff/40001/scripts/slave/recipe_mod...
File scripts/slave/recipe_modules/filter/example.py (right):

https://codereview.chromium.org/427073003/diff/40001/scripts/slave/recipe_mod...
scripts/slave/recipe_modules/filter/example.py:51:
stdout=api.json.output({'status': 'No dependency'})) +
just wondering... there's not a way to actually have the analyze step write to a
file, is there? that would let you actually have debugging/log statements on
stdout.

also be aware that stdout/stderr are not tee'd, so unless you write to stderr,
there will be no output from your analyze step until it's complete (and then the
output will only show up on the json.output link, I think).

https://codereview.chromium.org/427073003/diff/40001/scripts/slave/recipes/ch...
File
scripts/slave/recipes/chromium_trybot.expected/compile_because_of_analyze.json
(right):

https://codereview.chromium.org/427073003/diff/40001/scripts/slave/recipes/ch...
scripts/slave/recipes/chromium_trybot.expected/compile_because_of_analyze.json:99:
"--analyzer2",
is the analyzer2 syntax extensible enough that you will get some additional
milage out of it without having to resort to bumping this version number up
frequently? just curious :)

https://codereview.chromium.org/427073003/diff/40001/scripts/slave/recipes/ch...
File scripts/slave/recipes/chromium_trybot.py (right):

https://codereview.chromium.org/427073003/diff/40001/scripts/slave/recipes/ch...
scripts/slave/recipes/chromium_trybot.py:432: def filter_tests(possible_tests,
needed_tests):
nit: 2 spaces between module-level definitions

[sky, 2014-07-30 21:14:16]

Take another look?

https://codereview.chromium.org/427073003/diff/40001/scripts/slave/recipe_mod...
File scripts/slave/recipe_modules/filter/example.py (right):

https://codereview.chromium.org/427073003/diff/40001/scripts/slave/recipe_mod...
scripts/slave/recipe_modules/filter/example.py:51:
stdout=api.json.output({'status': 'No dependency'})) +
On 2014/07/30 01:20:37, iannucci wrote:
> just wondering... there's not a way to actually have the analyze step write to
a
> file, is there? that would let you actually have debugging/log statements on
> stdout.

I changed the analyze code to do that. It simplified things and I was able to
add use override_step_data again and have default step_test_data.

> also be aware that stdout/stderr are not tee'd, so unless you write to stderr,
> there will be no output from your analyze step until it's complete (and then
the
> output will only show up on the json.output link, I think).

The output is written at the very end, so it shouldn't be an issue.

https://codereview.chromium.org/427073003/diff/40001/scripts/slave/recipes/ch...
File
scripts/slave/recipes/chromium_trybot.expected/compile_because_of_analyze.json
(right):

https://codereview.chromium.org/427073003/diff/40001/scripts/slave/recipes/ch...
scripts/slave/recipes/chromium_trybot.expected/compile_because_of_analyze.json:99:
"--analyzer2",
On 2014/07/30 01:20:37, iannucci wrote:
> is the analyzer2 syntax extensible enough that you will get some additional
> milage out of it without having to resort to bumping this version number up
> frequently? just curious :)

The reason I went with analyzer2 here is to support the old format/args while
transitioning to the new. I need to do the following:
. land this
. make --analyzer the same as --analyzer2
. update this recipe to use --analyzer
. remove --analyzer2
oy!

Now that I'm passing around JSON objects to configure what the analyze step does
hopefully I won't have to do this again. Instead I just add new things to the
JSON object.

https://codereview.chromium.org/427073003/diff/40001/scripts/slave/recipes/ch...
File scripts/slave/recipes/chromium_trybot.py (right):

https://codereview.chromium.org/427073003/diff/40001/scripts/slave/recipes/ch...
scripts/slave/recipes/chromium_trybot.py:432: def filter_tests(possible_tests,
needed_tests):
On 2014/07/30 01:20:37, iannucci wrote:
> nit: 2 spaces between module-level definitions

I assume you mean 2 newlines.

[sky, 2014-08-05 18:54:24]

ping

[iannucci, 2014-08-05 18:59:24]

lgtm

https://codereview.chromium.org/427073003/diff/60001/scripts/slave/recipe_mod...
File scripts/slave/recipe_modules/filter/api.py (right):

https://codereview.chromium.org/427073003/diff/60001/scripts/slave/recipe_mod...
scripts/slave/recipe_modules/filter/api.py:73: test_output = {'status': 'No
dependency', 'targets': []}
I'd inline this

[sky, 2014-08-06 16:17:56]

The CQ bit was checked by sky@chromium.org

[commit-bot: I haz the power, 2014-08-06 16:18:54]

CQ is trying da patch. Follow status at
 https://chromium-status.appspot.com/cq/sky@chromium.org/427073003/80001

[commit-bot: I haz the power, 2014-08-06 16:20:47]

Change committed as 287778