[Tests] Add @RetryOnFailure to tests

[Tests] Add @RetryOnFailure to tests

Based on running chrome_public_test_apk on android_n5x_swarming_rel test results, currently need to add @RetryOnFailure to tests:
OverviewListLayoutTest#testCanSwipeClosed
PaymentRequestJourneyLoggerTest#testNumberOfSuggestionsShown_ShippingAddress_Completed
OmniboxTest#testRequestZeroSuggestTypeAndBackspace
ToastHWATest#testOpenedInBackgroundToast

BUG=731759
Review-Url: https://codereview.chromium.org/2991433002
Cr-Commit-Position: refs/heads/master@{#489044}
Committed: https://chromium.googlesource.com/chromium/src/+/b21cd269550a2a5cb229d7bf055d1039811e32a2

[shenghuazhang, 2017-07-21 20:04:39]

shenghuazhang@chromium.org changed reviewers:
+ tedchoc@chromium.org

[Ted C, 2017-07-21 20:13:34]

https://codereview.chromium.org/2991433002/diff/1/chrome/android/javatests/sr...
File
chrome/android/javatests/src/org/chromium/chrome/browser/payments/PaymentRequestJourneyLoggerTest.java
(right):

https://codereview.chromium.org/2991433002/diff/1/chrome/android/javatests/sr...
chrome/android/javatests/src/org/chromium/chrome/browser/payments/PaymentRequestJourneyLoggerTest.java:78:
@DisabledTest(message = "crbug.com/47201")
I feel this bug might not be the right one.

https://codereview.chromium.org/2991433002/diff/1/chrome/android/javatests/sr...
File
chrome/android/javatests/src/org/chromium/chrome/browser/widget/OverviewListLayoutTest.java
(right):

https://codereview.chromium.org/2991433002/diff/1/chrome/android/javatests/sr...
chrome/android/javatests/src/org/chromium/chrome/browser/widget/OverviewListLayoutTest.java:238:
@DisabledTest(message = "crbug.com/747134")
why not retryonfailure here?

[shenghuazhang, 2017-07-21 21:57:59]

https://codereview.chromium.org/2991433002/diff/1/chrome/android/javatests/sr...
File
chrome/android/javatests/src/org/chromium/chrome/browser/payments/PaymentRequestJourneyLoggerTest.java
(right):

https://codereview.chromium.org/2991433002/diff/1/chrome/android/javatests/sr...
chrome/android/javatests/src/org/chromium/chrome/browser/payments/PaymentRequestJourneyLoggerTest.java:78:
@DisabledTest(message = "crbug.com/47201")
On 2017/07/21 20:13:34, Ted C wrote:
> I feel this bug might not be the right one.

Sorry I entered a wrong one! :P Should be crbug.com/747201 here.

https://codereview.chromium.org/2991433002/diff/1/chrome/android/javatests/sr...
File
chrome/android/javatests/src/org/chromium/chrome/browser/widget/OverviewListLayoutTest.java
(right):

https://codereview.chromium.org/2991433002/diff/1/chrome/android/javatests/sr...
chrome/android/javatests/src/org/chromium/chrome/browser/widget/OverviewListLayoutTest.java:238:
@DisabledTest(message = "crbug.com/747134")
On 2017/07/21 20:13:34, Ted C wrote:
> why not retryonfailure here?

Oh I added @DisabledTest to it because the flakiness shows consistently failures
on bot. The frequency is each couple days: 
https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType...

I think this is consistent while not too frequently though... Which one do you
think is more proper on this test, @DisabledTest or @RetryOnFailure?

[jbudorick, 2017-07-21 21:59:15]

jbudorick@chromium.org changed reviewers:
+ jbudorick@chromium.org

[jbudorick, 2017-07-21 21:59:15]

https://codereview.chromium.org/2991433002/diff/1/chrome/android/javatests/sr...
File
chrome/android/javatests/src/org/chromium/chrome/browser/widget/OverviewListLayoutTest.java
(right):

https://codereview.chromium.org/2991433002/diff/1/chrome/android/javatests/sr...
chrome/android/javatests/src/org/chromium/chrome/browser/widget/OverviewListLayoutTest.java:238:
@DisabledTest(message = "crbug.com/747134")
On 2017/07/21 21:57:58, shenghuazhang wrote:
> On 2017/07/21 20:13:34, Ted C wrote:
> > why not retryonfailure here?
> 
> Oh I added @DisabledTest to it because the flakiness shows consistently
failures
> on bot. The frequency is each couple days: 
>
https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType...
> 
> I think this is consistent while not too frequently though... Which one do you
> think is more proper on this test, @DisabledTest or @RetryOnFailure?

@RetryOnFailure first.

[shenghuazhang, 2017-07-21 22:06:13]

Changed both @DisabledTest to @RetryOnFailure.

https://codereview.chromium.org/2991433002/diff/1/chrome/android/javatests/sr...
File
chrome/android/javatests/src/org/chromium/chrome/browser/payments/PaymentRequestJourneyLoggerTest.java
(right):

https://codereview.chromium.org/2991433002/diff/1/chrome/android/javatests/sr...
chrome/android/javatests/src/org/chromium/chrome/browser/payments/PaymentRequestJourneyLoggerTest.java:78:
@DisabledTest(message = "crbug.com/47201")
On 2017/07/21 21:57:58, shenghuazhang wrote:
> On 2017/07/21 20:13:34, Ted C wrote:
> > I feel this bug might not be the right one.
> 
> Sorry I entered a wrong one! :P Should be crbug.com/747201 here.

Will mark @RetryOnFailure here first as well.

https://codereview.chromium.org/2991433002/diff/1/chrome/android/javatests/sr...
File
chrome/android/javatests/src/org/chromium/chrome/browser/widget/OverviewListLayoutTest.java
(right):

https://codereview.chromium.org/2991433002/diff/1/chrome/android/javatests/sr...
chrome/android/javatests/src/org/chromium/chrome/browser/widget/OverviewListLayoutTest.java:238:
@DisabledTest(message = "crbug.com/747134")
On 2017/07/21 21:59:15, jbudorick wrote:
> On 2017/07/21 21:57:58, shenghuazhang wrote:
> > On 2017/07/21 20:13:34, Ted C wrote:
> > > why not retryonfailure here?
> > 
> > Oh I added @DisabledTest to it because the flakiness shows consistently
> failures
> > on bot. The frequency is each couple days: 
> >
>
https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType...
> > 
> > I think this is consistent while not too frequently though... Which one do
you
> > think is more proper on this test, @DisabledTest or @RetryOnFailure?
> 
> @RetryOnFailure first.

I see. Thanks!

[Ted C, 2017-07-21 22:08:23]

lgtm, please update the title of the bug now as well

[shenghuazhang, 2017-07-21 22:11:27]

Description was changed from

==========
[Tests] Add @DisabledTest and @RetryOnFailure to tests

Based on running chrome_public_test_apk on android_n5x_swarming_rel test
results, currently need to add @DisabledTest to tests:
OverviewListLayoutTest#testCanSwipeClosed
PaymentRequestJourneyLoggerTest#testNumberOfSuggestionsShown_ShippingAddress_Completed
and add @RetryOnFailure to tests:
OmniboxTest#testRequestZeroSuggestTypeAndBackspace
ToastHWATest#testOpenedInBackgroundToast

BUG=731759
==========

to

==========
[Tests] Add @RetryOnFailure to tests

Based on running chrome_public_test_apk on android_n5x_swarming_rel test
results, currently need to add @RetryOnFailure to tests:
OverviewListLayoutTest#testCanSwipeClosed
PaymentRequestJourneyLoggerTest#testNumberOfSuggestionsShown_ShippingAddress_Completed
OmniboxTest#testRequestZeroSuggestTypeAndBackspace
ToastHWATest#testOpenedInBackgroundToast

BUG=731759
==========

[shenghuazhang, 2017-07-21 22:12:36]

On 2017/07/21 22:08:23, Ted C wrote:
> lgtm, please update the title of the bug now as well

Thanks! Updated the tile of the CL and the bug.

[shenghuazhang, 2017-07-21 22:12:45]

The CQ bit was checked by shenghuazhang@chromium.org

[commit-bot: I haz the power, 2017-07-21 22:12:55]

CQ is trying da patch.

Follow status at:
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-07-21 22:35:15]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-07-21 22:35:16]

Try jobs failed on following builders:
  android_clang_dbg_recipe on master.tryserver.chromium.android (JOB_FAILED,
https://build.chromium.org/p/tryserver.chromium.android/builders/android_clan...)

[shenghuazhang, 2017-07-24 18:44:08]

The CQ bit was checked by shenghuazhang@chromium.org

[shenghuazhang, 2017-07-24 18:44:08]

The patchset sent to the CQ was uploaded after l-g-t-m from tedchoc@chromium.org
Link to the patchset: https://codereview.chromium.org/2991433002/#ps60001
(title: "add RetryOnFailure package")

[commit-bot: I haz the power, 2017-07-24 18:44:15]

CQ is trying da patch.

Follow status at:
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-07-24 19:17:07]

CQ is committing da patch.

Bot data: {"patchset_id": 60001, "attempt_start_ts": 1500921848429770,
"parent_rev": "5c626f896305f75baf70b6a8dd141fd12bca4345", "commit_rev":
"75c8d8b24ba7902f45d26e52e15f6c11f64db959"}

[commit-bot: I haz the power, 2017-07-24 19:17:36]

CQ is committing da patch.

Bot data: {"patchset_id": 60001, "attempt_start_ts": 1500921848429770,
"parent_rev": "607cb1475d625041f23200d3002c448e06239570", "commit_rev":
"b21cd269550a2a5cb229d7bf055d1039811e32a2"}

[commit-bot: I haz the power, 2017-07-24 19:17:48]

Description was changed from

==========
[Tests] Add @RetryOnFailure to tests

Based on running chrome_public_test_apk on android_n5x_swarming_rel test
results, currently need to add @RetryOnFailure to tests:
OverviewListLayoutTest#testCanSwipeClosed
PaymentRequestJourneyLoggerTest#testNumberOfSuggestionsShown_ShippingAddress_Completed
OmniboxTest#testRequestZeroSuggestTypeAndBackspace
ToastHWATest#testOpenedInBackgroundToast

BUG=731759
==========

to

==========
[Tests] Add @RetryOnFailure to tests

Based on running chrome_public_test_apk on android_n5x_swarming_rel test
results, currently need to add @RetryOnFailure to tests:
OverviewListLayoutTest#testCanSwipeClosed
PaymentRequestJourneyLoggerTest#testNumberOfSuggestionsShown_ShippingAddress_Completed
OmniboxTest#testRequestZeroSuggestTypeAndBackspace
ToastHWATest#testOpenedInBackgroundToast

BUG=731759

Review-Url: https://codereview.chromium.org/2991433002
Cr-Commit-Position: refs/heads/master@{#489044}
Committed:
https://chromium.googlesource.com/chromium/src/+/b21cd269550a2a5cb229d7bf055d...
==========

[commit-bot: I haz the power, 2017-07-24 19:17:50]

Committed patchset #4 (id:60001) as
https://chromium.googlesource.com/chromium/src/+/b21cd269550a2a5cb229d7bf055d...