Improve Android Test Size Annotation support in test_runner.py

Improve Android Test Size Annotation support in test_runner.py

- Add explicit support for EnormousTest
- Warn at runtime if a test size annotation isn't recognized
  (see: http://www.chromium.org/developers/testing/android-tests)
- Remove EnormousTest annotation from a test that wasn't actually using it
  (it was effectively using the SmallTest timeout value at runtime).
- Add SmallTest annotation to InstallerDelegateTest (it was the only
  set of tests that did not include these annotations).

TEST=ChromeShellTest instrumentation tests
BUG=392297
Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=283684

[mckev, 2014-07-08 22:36:43]

Reviewers, please review this patch as follows:

jbudorick: chrome/android/javatests/*
dfalcantara: build/android/pylib/instrumentation/test_runner.py

Thanks!

[jbudorick, 2014-07-09 01:31:53]

chrome/android/javatests/* lgtm

Thanks for the report & patch!

https://codereview.chromium.org/376873008/diff/1/build/android/pylib/instrume...
File build/android/pylib/instrumentation/test_runner.py (right):

https://codereview.chromium.org/376873008/diff/1/build/android/pylib/instrume...
build/android/pylib/instrumentation/test_runner.py:322: return 24 * 60 * 60
I think that 24 hours might be a little much. While EnormousTests should
absolutely have a longer timeout than LargeTests, I think something between 10
minutes and 1 hour would be more reasonable.

[gone, 2014-07-09 17:26:56]

yfriedman for test timeouts

https://codereview.chromium.org/376873008/diff/1/build/android/pylib/instrume...
File build/android/pylib/instrumentation/test_runner.py (right):

https://codereview.chromium.org/376873008/diff/1/build/android/pylib/instrume...
build/android/pylib/instrumentation/test_runner.py:322: return 24 * 60 * 60
On 2014/07/09 01:31:53, jbudorick wrote:
> I think that 24 hours might be a little much. While EnormousTests should
> absolutely have a longer timeout than LargeTests, I think something between 10
> minutes and 1 hour would be more reasonable.

Setting a single test timeout to 24 hours would result in that bot being
completely useless; too many CLs land in a day to do any bisecting on test
regressions.  I'd suggest 10-30 minutes, max.

[Yaron, 2014-07-09 20:47:41]

https://codereview.chromium.org/376873008/diff/1/build/android/pylib/instrume...
File build/android/pylib/instrumentation/test_runner.py (right):

https://codereview.chromium.org/376873008/diff/1/build/android/pylib/instrume...
build/android/pylib/instrumentation/test_runner.py:322: return 24 * 60 * 60
On 2014/07/09 17:26:56, dfalcantara wrote:
> On 2014/07/09 01:31:53, jbudorick wrote:
> > I think that 24 hours might be a little much. While EnormousTests should
> > absolutely have a longer timeout than LargeTests, I think something between
10
> > minutes and 1 hour would be more reasonable.
> 
> Setting a single test timeout to 24 hours would result in that bot being
> completely useless; too many CLs land in a day to do any bisecting on test
> regressions.  I'd suggest 10-30 minutes, max.

Agreed. Did you have a test in mind that needs longer than 10 mins?

[mckev, 2014-07-09 20:55:14]

On 2014/07/09 20:47:41, Yaron wrote:
>
https://codereview.chromium.org/376873008/diff/1/build/android/pylib/instrume...
> File build/android/pylib/instrumentation/test_runner.py (right):
> 
>
https://codereview.chromium.org/376873008/diff/1/build/android/pylib/instrume...
> build/android/pylib/instrumentation/test_runner.py:322: return 24 * 60 * 60
> On 2014/07/09 17:26:56, dfalcantara wrote:
> > On 2014/07/09 01:31:53, jbudorick wrote:
> > > I think that 24 hours might be a little much. While EnormousTests should
> > > absolutely have a longer timeout than LargeTests, I think something
between
> 10
> > > minutes and 1 hour would be more reasonable.
> > 
> > Setting a single test timeout to 24 hours would result in that bot being
> > completely useless; too many CLs land in a day to do any bisecting on test
> > regressions.  I'd suggest 10-30 minutes, max.
> 
> Agreed. Did you have a test in mind that needs longer than 10 mins?

Also agreed.  24 hours is quite extreme - I believe the original use case was
long-running perf tests, but we can start with 10 minutes and change it later if
necessary.  The primary objective with this CL is for better coverage of the
attributes.

[mckev, 2014-07-09 20:59:10]

Updated based on feedback.  If all looks good, can someone submit to the CQ?

[jbudorick, 2014-07-16 23:28:12]

On 2014/07/09 20:59:10, mckev wrote:
> Updated based on feedback.  If all looks good, can someone submit to the CQ?

still lgtm to me. Yaron, are you ok with this?

[Yaron, 2014-07-17 01:21:00]

On 2014/07/16 23:28:12, jbudorick wrote:
> On 2014/07/09 20:59:10, mckev wrote:
> > Updated based on feedback.  If all looks good, can someone submit to the CQ?
> 
> still lgtm to me. Yaron, are you ok with this?

lgtm

[jbudorick, 2014-07-17 01:41:38]

The CQ bit was checked by jbudorick@chromium.org

[commit-bot: I haz the power, 2014-07-17 01:45:08]

CQ is trying da patch. Follow status at
 https://chromium-status.appspot.com/cq/mckev@amazon.com/376873008/20001

[commit-bot: I haz the power, 2014-07-17 07:17:41]

Change committed as 283684