[Android] Configurable instrumentation test runner + test SDK levels.

[Android] Configurable instrumentation test runner + test SDK levels.

This patch:
  - extracts the instrumentation test runner from the
    AndroidManifest.xml file in the apk.
  - allows test authors to specify a minimum SDK level for
    instrumentation test classes via the MinAndroidSdkLevel annotation.
  - filters the list of instrumentation tests according to the sdk
    levels of all attached devices.
  - speeds up our proguard function by dumping to a temporary file.

BUG=

Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=286014

[klundberg, 2014-07-23 16:11:57]

Hi John,

Have you tested this by various calls?

For example:
- using test size annotation - run all small tests
- using test name - run just a specific test
- run all tests

I took a somewhat quick look and this looks good but just want to make sure
everything is tested.

Thanks,
Karin

[jbudorick, 2014-07-23 23:44:30]

On 2014/07/23 16:11:57, klundberg wrote:
> Hi John,
> 
> Have you tested this by various calls?
> 
> For example:
> - using test size annotation - run all small tests

yes

> - using test name - run just a specific test

definitely yes

> - run all tests

also yes

> 
> I took a somewhat quick look and this looks good but just want to make sure
> everything is tested.
> 
> Thanks,
> Karin

I've also tested this on both K and L.

[klundberg, 2014-07-24 12:50:23]

lgtm

I think Craig should review the Python changes more closely but the approach
looks good to me.

[craigdh, 2014-07-24 15:19:31]

Python lgtm.

[Maria, 2014-07-25 23:14:34]

https://codereview.chromium.org/415463002/diff/40001/build/android/pylib/inst...
File build/android/pylib/instrumentation/test_jar.py (right):

https://codereview.chromium.org/415463002/diff/40001/build/android/pylib/inst...
build/android/pylib/instrumentation/test_jar.py:153: test_clazzez = ((n, i) for
n, i in clazzez.items() if n.endswith('Test'))
I thought list comprehensions were usually in [], not in ()

https://codereview.chromium.org/415463002/diff/60001/build/android/pylib/cons...
File build/android/pylib/constants.py (right):

https://codereview.chromium.org/415463002/diff/60001/build/android/pylib/cons...
build/android/pylib/constants.py:137: ANDROID_L_SDK_VERSION = 20
this is not correct, but also it doesn't seem to be used anywhere, could you
remove this?

[frankf, 2014-07-25 23:20:04]

Is it possible to use existing Restriction annotation:

https://code.google.com/p/chromium/codesearch#chromium/src/base/test/android/...

[jbudorick, 2014-07-25 23:25:40]

https://codereview.chromium.org/415463002/diff/40001/build/android/pylib/inst...
File build/android/pylib/instrumentation/test_jar.py (right):

https://codereview.chromium.org/415463002/diff/40001/build/android/pylib/inst...
build/android/pylib/instrumentation/test_jar.py:153: test_clazzez = ((n, i) for
n, i in clazzez.items() if n.endswith('Test'))
On 2014/07/25 23:14:33, Maria wrote:
> I thought list comprehensions were usually in [], not in ()

They are. () is a generator expression. Similar idea, but it doesn't create the
whole list in memory.

https://codereview.chromium.org/415463002/diff/60001/build/android/pylib/cons...
File build/android/pylib/constants.py (right):

https://codereview.chromium.org/415463002/diff/60001/build/android/pylib/cons...
build/android/pylib/constants.py:137: ANDROID_L_SDK_VERSION = 20
On 2014/07/25 23:14:33, Maria wrote:
> this is not correct, but also it doesn't seem to be used anywhere, could you
> remove this?

This is a relic from a previous revision. Removed.

[jbudorick, 2014-07-25 23:26:42]

On 2014/07/25 23:20:04, frankf wrote:
> Is it possible to use existing Restriction annotation:
> 
>
https://code.google.com/p/chromium/codesearch#chromium/src/base/test/android/...

It'll take some nontrivial voodoo to support this in the proguard processing
logic, but I'll try it out.

[jbudorick, 2014-07-28 15:46:10]

On 2014/07/25 23:26:42, jbudorick wrote:
> On 2014/07/25 23:20:04, frankf wrote:
> > Is it possible to use existing Restriction annotation:
> > 
> >
>
https://code.google.com/p/chromium/codesearch#chromium/src/base/test/android/...
> 
> It'll take some nontrivial voodoo to support this in the proguard processing
> logic, but I'll try it out.

This actually requires changing quite a bit:
 - Restriction currently only really supports boolean flags, not integer values.
 - Restriction currently only annotates methods; MinAndroidSdkLevel is primarily
designed to annotate base classes.
 - aforementioned proguard voodoo

Given that, the minimal upside doesn't seem worth it at the moment. I'll move
MinAndroidSdkLevel over to that directory + package, though.

[Maria, 2014-07-28 17:28:47]

lgtm

[jbudorick, 2014-07-28 17:36:14]

+Yaron for java OWNERS

[Yaron, 2014-07-28 17:59:59]

lgtm

https://codereview.chromium.org/415463002/diff/100001/base/test/android/javat...
File
base/test/android/javatests/src/org/chromium/base/test/util/MinAndroidSdkLevel.java
(right):

https://codereview.chromium.org/415463002/diff/100001/base/test/android/javat...
base/test/android/javatests/src/org/chromium/base/test/util/MinAndroidSdkLevel.java:16:
public @interface MinAndroidSdkLevel {
I don't see a MaxAndroidSdkLevel. I also agree that we don't want it. Please
update CL description.

[jbudorick, 2014-07-28 18:45:15]

On 2014/07/28 17:59:59, Yaron wrote:
> lgtm
> 
>
https://codereview.chromium.org/415463002/diff/100001/base/test/android/javat...
> File
>
base/test/android/javatests/src/org/chromium/base/test/util/MinAndroidSdkLevel.java
> (right):
> 
>
https://codereview.chromium.org/415463002/diff/100001/base/test/android/javat...
>
base/test/android/javatests/src/org/chromium/base/test/util/MinAndroidSdkLevel.java:16:
> public @interface MinAndroidSdkLevel {
> I don't see a MaxAndroidSdkLevel. I also agree that we don't want it. Please
> update CL description.

Done.

[jbudorick, 2014-07-28 18:45:19]

The CQ bit was checked by jbudorick@chromium.org

[commit-bot: I haz the power, 2014-07-28 18:46:17]

CQ is trying da patch. Follow status at
 https://chromium-status.appspot.com/cq/jbudorick@chromium.org/415463002/100001

[commit-bot: I haz the power, 2014-07-28 18:47:47]

FYI, CQ is re-trying this CL (attempt #1).
The failing builders are:
  linux_gpu on tryserver.chromium.gpu
(http://build.chromium.org/p/tryserver.chromium.gpu/builders/linux_gpu/builds/...)
  mac_gpu on tryserver.chromium.gpu
(http://build.chromium.org/p/tryserver.chromium.gpu/builders/mac_gpu/builds/34202)
  win_gpu on tryserver.chromium.gpu
(http://build.chromium.org/p/tryserver.chromium.gpu/builders/win_gpu/builds/39209)
  android_chromium_gn_compile_rel on tryserver.chromium.linux
(http://build.chromium.org/p/tryserver.chromium.linux/builders/android_chromiu...)
  chromium_presubmit on tryserver.chromium.linux
(http://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presub...)
  linux_chromium_chromeos_clang_dbg on tryserver.chromium.linux
(http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_...)
  linux_chromium_chromeos_rel on tryserver.chromium.linux
(http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_...)
  linux_chromium_clang_dbg on tryserver.chromium.linux
(http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_...)
  linux_chromium_gn_rel on tryserver.chromium.linux
(http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_...)
  linux_chromium_rel_swarming on tryserver.chromium.linux
(http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_...)
  ios_dbg_simulator on tryserver.chromium.mac
(http://build.chromium.org/p/tryserver.chromium.mac/builders/ios_dbg_simulator...)
  ios_rel_device on tryserver.chromium.mac
(http://build.chromium.org/p/tryserver.chromium.mac/builders/ios_rel_device/bu...)
  ios_rel_device_ninja on tryserver.chromium.mac
(http://build.chromium.org/p/tryserver.chromium.mac/builders/ios_rel_device_ni...)
  mac_chromium_compile_dbg on tryserver.chromium.mac
(http://build.chromium.org/p/tryserver.chromium.mac/builders/mac_chromium_comp...)
  mac_chromium_rel on tryserver.chromium.mac
(http://build.chromium.org/p/tryserver.chromium.mac/builders/mac_chromium_rel/...)
  win8_chromium_rel on tryserver.chromium.win
(http://build.chromium.org/p/tryserver.chromium.win/builders/win8_chromium_rel...)
  win_chromium_compile_dbg on tryserver.chromium.win
(http://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_comp...)
  win_chromium_rel on tryserver.chromium.win
(http://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_rel/...)
  win_chromium_x64_rel on tryserver.chromium.win
(http://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_x64_...)

[commit-bot: I haz the power, 2014-07-28 18:48:39]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2014-07-28 18:48:40]

Try jobs failed on following builders:
  linux_gpu on tryserver.chromium.gpu
(http://build.chromium.org/p/tryserver.chromium.gpu/builders/linux_gpu/builds/...)
  mac_gpu on tryserver.chromium.gpu
(http://build.chromium.org/p/tryserver.chromium.gpu/builders/mac_gpu/builds/34203)
  win_gpu on tryserver.chromium.gpu
(http://build.chromium.org/p/tryserver.chromium.gpu/builders/win_gpu/builds/39210)
  android_chromium_gn_compile_rel on tryserver.chromium.linux
(http://build.chromium.org/p/tryserver.chromium.linux/builders/android_chromiu...)
  chromium_presubmit on tryserver.chromium.linux
(http://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presub...)
  linux_chromium_chromeos_clang_dbg on tryserver.chromium.linux
(http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_...)
  linux_chromium_chromeos_rel on tryserver.chromium.linux
(http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_...)
  linux_chromium_clang_dbg on tryserver.chromium.linux
(http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_...)
  linux_chromium_gn_rel on tryserver.chromium.linux
(http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_...)
  linux_chromium_rel_swarming on tryserver.chromium.linux
(http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_...)
  ios_dbg_simulator on tryserver.chromium.mac
(http://build.chromium.org/p/tryserver.chromium.mac/builders/ios_dbg_simulator...)
  ios_rel_device on tryserver.chromium.mac
(http://build.chromium.org/p/tryserver.chromium.mac/builders/ios_rel_device/bu...)
  ios_rel_device_ninja on tryserver.chromium.mac
(http://build.chromium.org/p/tryserver.chromium.mac/builders/ios_rel_device_ni...)
  mac_chromium_compile_dbg on tryserver.chromium.mac
(http://build.chromium.org/p/tryserver.chromium.mac/builders/mac_chromium_comp...)
  mac_chromium_rel on tryserver.chromium.mac
(http://build.chromium.org/p/tryserver.chromium.mac/builders/mac_chromium_rel/...)
  win8_chromium_rel on tryserver.chromium.win
(http://build.chromium.org/p/tryserver.chromium.win/builders/win8_chromium_rel...)
  win_chromium_rel on tryserver.chromium.win
(http://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_rel/...)
  win_chromium_x64_rel on tryserver.chromium.win
(http://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_x64_...)

[jbudorick, 2014-07-28 20:39:47]

The CQ bit was checked by jbudorick@chromium.org

[commit-bot: I haz the power, 2014-07-28 20:44:29]

CQ is trying da patch. Follow status at
 https://chromium-status.appspot.com/cq/jbudorick@chromium.org/415463002/100001

[commit-bot: I haz the power, 2014-07-28 21:39:11]

FYI, CQ is re-trying this CL (attempt #1).
The failing builders are:
  linux_gpu on tryserver.chromium.gpu
(http://build.chromium.org/p/tryserver.chromium.gpu/builders/linux_gpu/builds/...)
  linux_chromium_chromeos_rel on tryserver.chromium.linux
(http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_...)

[commit-bot: I haz the power, 2014-07-28 23:06:21]

Change committed as 286014