Do not filter using analyze step if layout tests are modified.

Do not filter using analyze step if layout tests are modified.

This may be one possible option for making it so that layout tests
are run by tryserver.blink bots when just layout tests are modified
but Blink source itself is not modified.

Pros:
 - This is a very small, simple change
 - It is generally true that analyze doesn't work well for determining
   that layout tests should be run when layout tests are modified,
   so analyze should perhaps be ignored in this case?
 - This should also make linux_layout_tests_slimming_paint_v2 work as expected as long as the CL modifies some files that are in the exclusion list

Cons:
 - This may cause some unwanted tests to be run on some builders
   for CLs that modify layout tests?

BUG=631014

[qyearsley, 2016-07-28 23:29:36]

Description was changed from

==========
Do not filter using analyze step if layout tests are modified.

This may be one possible option for making it so that layout tests
are run by tryserver.blink bots when just layout tests are modified
but Blink source itself is not modified.

Pros:
 - This is a very small, simple change
 - It is generally true that analyze doesn't work well for determining
   that layout tests should be run when layout tests are modified,
   so analyze should perhaps be ignored in this case?

Cons:
 - This may cause some unwanted tests to be run on some builders
   for CLs that modify layout tests?

BUG=631014
==========

to

==========
Do not filter using analyze step if layout tests are modified.

This may be one possible option for making it so that layout tests
are run by tryserver.blink bots when just layout tests are modified
but Blink source itself is not modified.

Pros:
 - This is a very small, simple change
 - It is generally true that analyze doesn't work well for determining
   that layout tests should be run when layout tests are modified,
   so analyze should perhaps be ignored in this case?
 - This should also make linux_layout_tests_slimming_paint_v2 work as expected
as long as the CL modifies some files that are in the exclusion list

Cons:
 - This may cause some unwanted tests to be run on some builders
   for CLs that modify layout tests?

BUG=631014
==========

[qyearsley, 2016-07-28 23:33:29]

qyearsley@chromium.org changed reviewers:
+ dpranke@chromium.org

[qyearsley, 2016-07-28 23:33:29]

https://codereview.chromium.org/2190163003/diff/1/testing/buildbot/trybot_ana...
File testing/buildbot/trybot_analyze_config.json (right):

https://codereview.chromium.org/2190163003/diff/1/testing/buildbot/trybot_ana...
testing/buildbot/trybot_analyze_config.json:28:
"third_party/WebKit/LayoutTests/.*",
Maybe this should be "third_party/WebKit/.*" -- then the analyze step would be
ignored for all CLs that modify anything in Blink, I believe?

As far as I understand, the purpose of the "analyze" step is to pare down the
list of targets to build (and tests to run) to avoid wasting resources -- so if
this exclusion list is too broad, then the negative effect would be potentially
running some unnecessary tests, but it would never fail to run a test should be
run, right?

[qyearsley, 2016-07-29 16:18:03]

The CQ bit was checked by qyearsley@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-07-29 16:18:23]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-07-29 17:11:47]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-07-29 17:11:48]

Dry run: This issue passed the CQ dry run.

[Dirk Pranke, 2016-07-29 23:58:55]

https://codereview.chromium.org/2190163003/diff/1/testing/buildbot/trybot_ana...
File testing/buildbot/trybot_analyze_config.json (right):

https://codereview.chromium.org/2190163003/diff/1/testing/buildbot/trybot_ana...
testing/buildbot/trybot_analyze_config.json:28:
"third_party/WebKit/LayoutTests/.*",
On 2016/07/28 23:33:29, qyearsley wrote:
> Maybe this should be "third_party/WebKit/.*" -- then the analyze step would be
> ignored for all CLs that modify anything in Blink, I believe?
> 
> As far as I understand, the purpose of the "analyze" step is to pare down the
> list of targets to build (and tests to run) to avoid wasting resources -- so
if
> this exclusion list is too broad, then the negative effect would be
potentially
> running some unnecessary tests, but it would never fail to run a test should
be
> run, right?

In GN, you can actually set a "data" attribute on a build target, and if
something listed in that target changes, then the target will be considered
affected.

I.e., instead of doing this, you should add `data = [
"//third_party/WebKit/LayoutTests/", "//third_party/WebKit/Tools/Scripts/"]` to
the blink_tests target in //BUILD.gn

[qyearsley, 2016-08-01 16:58:55]

Closing in favor of http://crrev.com/2195223002