Expand Mash Browser Test Coverage

Based on running of the tests locally, along with on the trybots, the filter for mash_browser_tests have been updated with more tests

TEST=mash_browser_tests
BUG=678687
Review-Url: https://codereview.chromium.org/2786583003
Cr-Commit-Position: refs/heads/master@{#464015}
Committed: https://chromium.googlesource.com/chromium/src/+/0190af327299091a9bdd376da2093cd48fe1f993

[jonross, 2017-04-11 15:57:35]

Description was changed from

==========
WIP Expand Mash Browser Test Coverage
Try ALL the tests

BUG=678687
==========

to

==========
Based on running of the tests locally, along with on the trybots, the filter for
mash_browser_tests have been updated with more tests

TEST=mash_browser_tests
BUG=678687
==========

[jonross, 2017-04-11 15:58:31]

jonross@chromium.org changed reviewers:
+ sky@chromium.org

[jonross, 2017-04-11 15:58:32]

Hey sky@

Here is the expanded mash_browser_tests filter that I mentioned. It is following
the black list style from before.

I plan to dig further into the failures offline and file bugs accordingly.

PTAL

Thanks,
Jon

[sky, 2017-04-11 16:32:17]

I'm worried that given that large number of failures almost any newly added test
will need to be excluded, which is painful for all developers. How about using
"+foo" to explicitly add only the tests that work?

https://codereview.chromium.org/2786583003/diff/180001/testing/buildbot/filte...
File testing/buildbot/filters/mash.browser_tests.filter (right):

https://codereview.chromium.org/2786583003/diff/180001/testing/buildbot/filte...
testing/buildbot/filters/mash.browser_tests.filter:537:
-PowerPolicyLoginScreenBrowserTest.&
Is there really a test with an & in the name?

[jonross, 2017-04-11 17:08:40]

On 2017/04/11 16:32:17, sky wrote:
> I'm worried that given that large number of failures almost any newly added
test
> will need to be excluded, which is painful for all developers. How about using
> "+foo" to explicitly add only the tests that work?
> 
>
https://codereview.chromium.org/2786583003/diff/180001/testing/buildbot/filte...
> File testing/buildbot/filters/mash.browser_tests.filter (right):
> 
>
https://codereview.chromium.org/2786583003/diff/180001/testing/buildbot/filte...
> testing/buildbot/filters/mash.browser_tests.filter:537:
> -PowerPolicyLoginScreenBrowserTest.&
> Is there really a test with an & in the name?

Yeah I had hoped that the list of failures not be too large. So I built it up to
begin categorizing failures.

I can switch to more of a mixed filter model, with "+foo" and "-foo,flaky" for
flakes.
I'll record the failures in bugs.

[sky, 2017-04-11 17:22:01]

SG

On Tue, Apr 11, 2017 at 10:08 AM, <jonross@chromium.org> wrote:

> On 2017/04/11 16:32:17, sky wrote:
> > I'm worried that given that large number of failures almost any newly
> added
> test
> > will need to be excluded, which is painful for all developers. How about
> using
> > "+foo" to explicitly add only the tests that work?
> >
> >
> https://codereview.chromium.org/2786583003/diff/180001/
> testing/buildbot/filters/mash.browser_tests.filter
> > File testing/buildbot/filters/mash.browser_tests.filter (right):
> >
> >
> https://codereview.chromium.org/2786583003/diff/180001/
> testing/buildbot/filters/mash.browser_tests.filter#newcode537
> > testing/buildbot/filters/mash.browser_tests.filter:537:
> > -PowerPolicyLoginScreenBrowserTest.&
> > Is there really a test with an & in the name?
>
> Yeah I had hoped that the list of failures not be too large. So I built it
> up to
> begin categorizing failures.
>
> I can switch to more of a mixed filter model, with "+foo" and "-foo,flaky"
> for
> flakes.
> I'll record the failures in bugs.
>
> https://codereview.chromium.org/2786583003/
>

-- 
You received this message because you are subscribed to the Google Groups
"Chromium-reviews" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to chromium-reviews+unsubscribe@chromium.org.

[jonross, 2017-04-11 21:38:20]

Patchset #11 (id:200001) has been deleted

[jonross, 2017-04-11 22:26:30]

I've switched to the whitelist method mentioned. With the mixed blacklist for
recent flakes.

PTAL

[sky, 2017-04-12 03:39:07]

LGTM

[jonross, 2017-04-12 13:17:51]

The CQ bit was checked by jonross@chromium.org

[commit-bot: I haz the power, 2017-04-12 13:18:09]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-04-12 14:18:26]

CQ is committing da patch.

Bot data: {"patchset_id": 220001, "attempt_start_ts": 1492003071614560,
"parent_rev": "1c913ae1fbb09dc8575d840dd1707ed207734bc1", "commit_rev":
"0190af327299091a9bdd376da2093cd48fe1f993"}

[commit-bot: I haz the power, 2017-04-12 14:19:30]

Description was changed from

==========
Based on running of the tests locally, along with on the trybots, the filter for
mash_browser_tests have been updated with more tests

TEST=mash_browser_tests
BUG=678687
==========

to

==========
Based on running of the tests locally, along with on the trybots, the filter for
mash_browser_tests have been updated with more tests

TEST=mash_browser_tests
BUG=678687

Review-Url: https://codereview.chromium.org/2786583003
Cr-Commit-Position: refs/heads/master@{#464015}
Committed:
https://chromium.googlesource.com/chromium/src/+/0190af327299091a9bdd376da209...
==========

[commit-bot: I haz the power, 2017-04-12 14:19:30]

Committed patchset #11 (id:220001) as
https://chromium.googlesource.com/chromium/src/+/0190af327299091a9bdd376da209...

[tkent, 2017-04-13 08:20:45]

A revert of this CL (patchset #11 id:220001) has been created in
https://codereview.chromium.org/2820483002/ by tkent@chromium.org.

The reason for reverting is: Some tests are very flaky on ozone try bot.

https://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium...
.