[Android] Split DeviceUtils tests and run them in presubmit.

[Android] Split DeviceUtils tests and run them in presubmit.

As requested by frankf:
  https://codereview.chromium.org/351603002/#msg2
  https://codereview.chromium.org/358993003/#msg9
  https://codereview.chromium.org/371813005/#msg8

BUG=267773
Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=282480

[frankf, 2014-07-10 17:45:09]

https://codereview.chromium.org/377373004/diff/1/build/android/PRESUBMIT.py
File build/android/PRESUBMIT.py (right):

https://codereview.chromium.org/377373004/diff/1/build/android/PRESUBMIT.py#n...
build/android/PRESUBMIT.py:59:
output.extend(input_api.canned_checks.RunUnitTests(
Why not run all pylib unit tests on the bots? These tests are likely going to
get larger in the future, not sure it makes sense to run them during presubmit.
I think tools like telemetry run their unittests on the bots, right?

[jbudorick, 2014-07-10 18:08:49]

https://codereview.chromium.org/377373004/diff/1/build/android/PRESUBMIT.py
File build/android/PRESUBMIT.py (right):

https://codereview.chromium.org/377373004/diff/1/build/android/PRESUBMIT.py#n...
build/android/PRESUBMIT.py:59:
output.extend(input_api.canned_checks.RunUnitTests(
On 2014/07/10 17:45:09, frankf wrote:
> Why not run all pylib unit tests on the bots? These tests are likely going to
> get larger in the future, not sure it makes sense to run them during
presubmit.
> I think tools like telemetry run their unittests on the bots, right?

I think that should be our goal. Even with that, though, I don't particularly
see a problem in running tests that are "fast enough" -- an admittedly slippery
distinction -- in presubmit.

(fwiw, the DeviceUtils tests currently run in ~1/3s, and I don't anticipate them
taking longer than 1s when finished.)

[jbudorick, 2014-07-10 18:12:48]

On 2014/07/10 18:08:49, jbudorick wrote:
> https://codereview.chromium.org/377373004/diff/1/build/android/PRESUBMIT.py
> File build/android/PRESUBMIT.py (right):
> 
>
https://codereview.chromium.org/377373004/diff/1/build/android/PRESUBMIT.py#n...
> build/android/PRESUBMIT.py:59:
> output.extend(input_api.canned_checks.RunUnitTests(
> On 2014/07/10 17:45:09, frankf wrote:
> > Why not run all pylib unit tests on the bots? These tests are likely going
to
> > get larger in the future, not sure it makes sense to run them during
> presubmit.
> > I think tools like telemetry run their unittests on the bots, right?
> 
> I think that should be our goal. Even with that, though, I don't particularly
> see a problem in running tests that are "fast enough" -- an admittedly
slippery
> distinction -- in presubmit.
> 
> (fwiw, the DeviceUtils tests currently run in ~1/3s, and I don't anticipate
them
> taking longer than 1s when finished.)

https://code.google.com/p/chromium/issues/detail?id=392911

[frankf, 2014-07-10 18:16:48]

On 2014/07/10 18:08:49, jbudorick wrote:
> https://codereview.chromium.org/377373004/diff/1/build/android/PRESUBMIT.py
> File build/android/PRESUBMIT.py (right):
> 
>
https://codereview.chromium.org/377373004/diff/1/build/android/PRESUBMIT.py#n...
> build/android/PRESUBMIT.py:59:
> output.extend(input_api.canned_checks.RunUnitTests(
> On 2014/07/10 17:45:09, frankf wrote:
> > Why not run all pylib unit tests on the bots? These tests are likely going
to
> > get larger in the future, not sure it makes sense to run them during
> presubmit.
> > I think tools like telemetry run their unittests on the bots, right?
> 
> I think that should be our goal. Even with that, though, I don't particularly
> see a problem in running tests that are "fast enough" -- an admittedly
slippery
> distinction -- in presubmit.
> 
> (fwiw, the DeviceUtils tests currently run in ~1/3s, and I don't anticipate
them
> taking longer than 1s when finished.)

Yea, it's a slippery slope. 

lgtm to get some coverage immediately, but all unit tests should be moved to the
bots. My impression is that presumbit is more for static analysis tools like
lint so this is a slippery slope indeed.

[jbudorick, 2014-07-10 18:19:19]

On 2014/07/10 18:16:48, frankf wrote:
> On 2014/07/10 18:08:49, jbudorick wrote:
> > https://codereview.chromium.org/377373004/diff/1/build/android/PRESUBMIT.py
> > File build/android/PRESUBMIT.py (right):
> > 
> >
>
https://codereview.chromium.org/377373004/diff/1/build/android/PRESUBMIT.py#n...
> > build/android/PRESUBMIT.py:59:
> > output.extend(input_api.canned_checks.RunUnitTests(
> > On 2014/07/10 17:45:09, frankf wrote:
> > > Why not run all pylib unit tests on the bots? These tests are likely going
> to
> > > get larger in the future, not sure it makes sense to run them during
> > presubmit.
> > > I think tools like telemetry run their unittests on the bots, right?
> > 
> > I think that should be our goal. Even with that, though, I don't
particularly
> > see a problem in running tests that are "fast enough" -- an admittedly
> slippery
> > distinction -- in presubmit.
> > 
> > (fwiw, the DeviceUtils tests currently run in ~1/3s, and I don't anticipate
> them
> > taking longer than 1s when finished.)
> 
> Yea, it's a slippery slope. 
> 
> lgtm to get some coverage immediately, but all unit tests should be moved to
the
> bots. My impression is that presumbit is more for static analysis tools like
> lint so this is a slippery slope indeed.

We do run the buildbot tests in presubmit. That doesn't mean it's necessarily
right to do, but we have a precedent for it.

I do plan to monitor how long these tests take; if they get too long, I'll
remove them from presubmit.

[jbudorick, 2014-07-10 18:20:02]

The CQ bit was checked by jbudorick@chromium.org

[commit-bot: I haz the power, 2014-07-10 18:21:18]

CQ is trying da patch. Follow status at
 https://chromium-status.appspot.com/cq/jbudorick@chromium.org/377373004/1

[commit-bot: I haz the power, 2014-07-11 00:20:40]

Change committed as 282480