Add new Android test runner command to handle linker tests.

Add new Android test runner command to handle linker tests.

This adds a new test runner command to run the content linker
tests on Android. This requires first the CL at
https://chromiumcodereview.appspot.com/23717023/
which defines a new build target named 'content_linker_test_apk'.

To launch the linker tests, simply do:

  ninja -C out/Debug content_linker_tests_apk
  build/android/test_runner.py linker

See test_case.py for details of what the test does.

BUG=287739
R=bulach@chromium.org, craigdh@chromium.org, frankf@chromium.org, yfriedman@chromium.org

Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=226739

[frankf, 2013-10-02 00:29:58]

On 2013/10/01 19:44:01, digit1 wrote:

I see 'error: old chunk mismatch' on sxs diffs, could you re-upload?

[digit1, 2013-10-02 09:47:31]

Weird, a new upload after a trivial rebase created a new issue. I've deleted it
and used "git cl issue" to correct this. Please take a look at Patch set 2.
Thanks.

[bulach, 2013-10-02 11:16:50]

lgtm thanks david! frank is far more familiar with the test harness, so please
wait for him...

some nits and suggestions, specially regarding watching adb logcat below:

https://codereview.chromium.org/25525003/diff/18001/build/android/pylib/linke...
File build/android/pylib/linker/test_case.py (right):

https://codereview.chromium.org/25525003/diff/18001/build/android/pylib/linke...
build/android/pylib/linker/test_case.py:68: BASE_ROOT = 'src' + os.sep
is this equivalent of:
from pylib import constants

constants.DIR_SOURCE_ROOT ?

https://codereview.chromium.org/25525003/diff/18001/build/android/pylib/linke...
build/android/pylib/linker/test_case.py:88: Args:
truly nit: the Args: and Returns: are normally aligned with the first " of """
above, i.e., unindented.

https://codereview.chromium.org/25525003/diff/18001/build/android/pylib/linke...
build/android/pylib/linker/test_case.py:119: A tuple, result[0] is True is there
is a complete match, then
nit: s/is there is/if there is/

https://codereview.chromium.org/25525003/diff/18001/build/android/pylib/linke...
build/android/pylib/linker/test_case.py:133: return (False, None)
nit: , None so it always returns a 3-ary tuple?

https://codereview.chromium.org/25525003/diff/18001/build/android/pylib/linke...
build/android/pylib/linker/test_case.py:168: print "[ %-*s ] %s" % (margin,
"RUN", self.tagged_name)
nit: s/"/'/ everywhere

https://codereview.chromium.org/25525003/diff/18001/build/android/pylib/linke...
build/android/pylib/linker/test_case.py:196: adb.GetCurrentRecordedLogcat())
hmm, would adb.StartMonitoringLogCat() (before StartActivity) and
adb.WaitForLogMatch() do the trick?


also... not sure how quick these messages would be print, but we had issues in
the past since logcat timestamp is non-asymptotic: it generated out-of-order log
entries... :(
we solved by writing to a fifo in the device and watching that instead, see
_WatchFifo in build/android/pylib/gtest/test_package_apk.py..

if the messages are far apart, it shouldn't be a problem, in which case
WaitForLogMatch() would probably be a better fit?

https://codereview.chromium.org/25525003/diff/18001/build/android/test_runner.py
File build/android/test_runner.py (right):

https://codereview.chromium.org/25525003/diff/18001/build/android/test_runner...
build/android/test_runner.py:117: 
nit: another \n here and after this method

[digit1, 2013-10-02 14:01:51]

https://codereview.chromium.org/25525003/diff/18001/build/android/pylib/linke...
File build/android/pylib/linker/test_case.py (right):

https://codereview.chromium.org/25525003/diff/18001/build/android/pylib/linke...
build/android/pylib/linker/test_case.py:68: BASE_ROOT = 'src' + os.sep
Actually, this came from pylib/host_driven/test_case.py which I used as the
original source for this file. This is no longer needed here so I've removed
this line.

https://codereview.chromium.org/25525003/diff/18001/build/android/pylib/linke...
build/android/pylib/linker/test_case.py:88: Args:
Oops, I've fixed this. Thanks.

https://codereview.chromium.org/25525003/diff/18001/build/android/pylib/linke...
build/android/pylib/linker/test_case.py:119: A tuple, result[0] is True is there
is a complete match, then
On 2013/10/02 11:16:50, bulach wrote:
> nit: s/is there is/if there is/

Done.

https://codereview.chromium.org/25525003/diff/18001/build/android/pylib/linke...
build/android/pylib/linker/test_case.py:133: return (False, None)
On 2013/10/02 11:16:50, bulach wrote:
> nit: , None so it always returns a 3-ary tuple?

Done.

https://codereview.chromium.org/25525003/diff/18001/build/android/pylib/linke...
build/android/pylib/linker/test_case.py:168: print "[ %-*s ] %s" % (margin,
"RUN", self.tagged_name)
On 2013/10/02 11:16:50, bulach wrote:
> nit: s/"/'/ everywhere

Done.

https://codereview.chromium.org/25525003/diff/18001/build/android/pylib/linke...
build/android/pylib/linker/test_case.py:196: adb.GetCurrentRecordedLogcat())
adb.WaitForLogMatch() isn't a good match here because I need to stop once I've
received _two_ distinct status lines in the logcat, something that cannot be
achieved with the function's current interface. I didn't want to modify the
function because, well, it looks pretty subtle :-)

Also, in this specific case, out-of-order entries are completely OK.

https://codereview.chromium.org/25525003/diff/18001/build/android/test_runner.py
File build/android/test_runner.py (right):

https://codereview.chromium.org/25525003/diff/18001/build/android/test_runner...
build/android/test_runner.py:117: 
On 2013/10/02 11:16:50, bulach wrote:
> nit: another \n here and after this method

Done.

[frankf, 2013-10-02 18:16:02]

lgtm w/ nits

https://codereview.chromium.org/25525003/diff/17001/build/android/pylib/andro...
File build/android/pylib/android_commands.py (right):

https://codereview.chromium.org/25525003/diff/17001/build/android/pylib/andro...
build/android/pylib/android_commands.py:1266: """Return the current content of
the logcat being recorded by
nit: first sentence on 1 line.

https://codereview.chromium.org/25525003/diff/17001/build/android/pylib/andro...
build/android/pylib/android_commands.py:1268: polling/parsing. Do not call after
StopRecordingLogcat()."""
Add Returns section in docstring and document it can return None

https://codereview.chromium.org/25525003/diff/17001/build/android/pylib/linke...
File build/android/pylib/linker/setup.py (right):

https://codereview.chromium.org/25525003/diff/17001/build/android/pylib/linke...
build/android/pylib/linker/setup.py:29: logging.debug('All available tests: ' +
str(
I believe test_dispatcher.py already logs this

https://codereview.chromium.org/25525003/diff/17001/build/android/pylib/linke...
File build/android/pylib/linker/test_case.py (right):

https://codereview.chromium.org/25525003/diff/17001/build/android/pylib/linke...
build/android/pylib/linker/test_case.py:79: def _CheckPrefixedTestStatus(logcat,
prefix):
Are you not using regex in case logcat is very large?

https://codereview.chromium.org/25525003/diff/17001/build/android/pylib/linke...
build/android/pylib/linker/test_case.py:88: A tuple, result[0] is True is a line
was found, then
is -> if

https://codereview.chromium.org/25525003/diff/17001/build/android/pylib/linke...
build/android/pylib/linker/test_case.py:125: return (False, None, None)
duplicate of line 128

https://codereview.chromium.org/25525003/diff/17001/build/android/pylib/linke...
build/android/pylib/linker/test_case.py:131: def
_CreateCommandLineFileOnDevice(adb, cmd_line):
you can use flag_changer.py

https://codereview.chromium.org/25525003/diff/17001/build/android/pylib/linke...
build/android/pylib/linker/test_case.py:157: # tests in the overall results.
isn't class_name sufficient for this. Note the linker tests would run as a
separate step on the bots since they have a different invocations
(test_runner.py linker ...), so there wouldn't be a mix up.

https://codereview.chromium.org/25525003/diff/17001/build/android/pylib/linke...
build/android/pylib/linker/test_case.py:163: print "[ %-*s ] %s" % (margin,
'RUN', self.tagged_name)
single quotes for consistancy

https://codereview.chromium.org/25525003/diff/17001/build/android/pylib/linke...
File build/android/pylib/linker/test_runner.py (right):

https://codereview.chromium.org/25525003/diff/17001/build/android/pylib/linke...
build/android/pylib/linker/test_runner.py:22: _PACKAGE_NAME =
"ContentLinkerTest"
single quotes

https://codereview.chromium.org/25525003/diff/17001/build/android/pylib/linke...
build/android/pylib/linker/test_runner.py:43: log=str(exc_type) + ' ' + log_msg)
use string formating '%s %s'

https://codereview.chromium.org/25525003/diff/17001/build/android/pylib/linke...
build/android/pylib/linker/test_runner.py:55: cleanup_test_files):
I think this fist on a single line

[digit1, 2013-10-03 09:16:00]

https://codereview.chromium.org/25525003/diff/17001/build/android/pylib/andro...
File build/android/pylib/android_commands.py (right):

https://codereview.chromium.org/25525003/diff/17001/build/android/pylib/andro...
build/android/pylib/android_commands.py:1266: """Return the current content of
the logcat being recorded by
On 2013/10/02 18:16:02, frankf wrote:
> nit: first sentence on 1 line.

Done.

https://codereview.chromium.org/25525003/diff/17001/build/android/pylib/andro...
build/android/pylib/android_commands.py:1268: polling/parsing. Do not call after
StopRecordingLogcat()."""
On 2013/10/02 18:16:02, frankf wrote:
> Add Returns section in docstring and document it can return None

Done.

https://codereview.chromium.org/25525003/diff/17001/build/android/pylib/linke...
File build/android/pylib/linker/setup.py (right):

https://codereview.chromium.org/25525003/diff/17001/build/android/pylib/linke...
build/android/pylib/linker/setup.py:29: logging.debug('All available tests: ' +
str(
This came directly from InstrumentationSetup which was used as the source for
this class, but I've removed it.

https://codereview.chromium.org/25525003/diff/17001/build/android/pylib/linke...
File build/android/pylib/linker/test_case.py (right):

https://codereview.chromium.org/25525003/diff/17001/build/android/pylib/linke...
build/android/pylib/linker/test_case.py:79: def _CheckPrefixedTestStatus(logcat,
prefix):
The logcat won't be very large, due to the filters being used here and the fact
that this only collects traces at program startup. I've changed it to a regex
though.

https://codereview.chromium.org/25525003/diff/17001/build/android/pylib/linke...
build/android/pylib/linker/test_case.py:88: A tuple, result[0] is True is a line
was found, then
On 2013/10/02 18:16:02, frankf wrote:
> is -> if

Done.

https://codereview.chromium.org/25525003/diff/17001/build/android/pylib/linke...
build/android/pylib/linker/test_case.py:125: return (False, None, None)
On 2013/10/02 18:16:02, frankf wrote:
> duplicate of line 128

Done.

https://codereview.chromium.org/25525003/diff/17001/build/android/pylib/linke...
build/android/pylib/linker/test_case.py:131: def
_CreateCommandLineFileOnDevice(adb, cmd_line):
On 2013/10/02 18:16:02, frankf wrote:
> you can use flag_changer.py 

Done.

https://codereview.chromium.org/25525003/diff/17001/build/android/pylib/linke...
build/android/pylib/linker/test_case.py:157: # tests in the overall results.
On 2013/10/02 18:16:02, frankf wrote:
> isn't class_name sufficient for this. Note the linker tests would run as a
> separate step on the bots since they have a different invocations
> (test_runner.py linker ...), so there wouldn't be a mix up.

The class name itself won't be sufficient, but self.qualified name can be. I've
changed the code to use that instead. (FWIW, this part also came originally from
host_driven.TestCase).

https://codereview.chromium.org/25525003/diff/17001/build/android/pylib/linke...
build/android/pylib/linker/test_case.py:163: print "[ %-*s ] %s" % (margin,
'RUN', self.tagged_name)
On 2013/10/02 18:16:02, frankf wrote:
> single quotes for consistancy

Done.

https://codereview.chromium.org/25525003/diff/17001/build/android/pylib/linke...
File build/android/pylib/linker/test_runner.py (right):

https://codereview.chromium.org/25525003/diff/17001/build/android/pylib/linke...
build/android/pylib/linker/test_runner.py:22: _PACKAGE_NAME =
"ContentLinkerTest"
On 2013/10/02 18:16:02, frankf wrote:
> single quotes

Done.

https://codereview.chromium.org/25525003/diff/17001/build/android/pylib/linke...
build/android/pylib/linker/test_runner.py:43: log=str(exc_type) + ' ' + log_msg)
On 2013/10/02 18:16:02, frankf wrote:
> use string formating '%s %s'

Done.

https://codereview.chromium.org/25525003/diff/17001/build/android/pylib/linke...
build/android/pylib/linker/test_runner.py:55: cleanup_test_files):
On 2013/10/02 18:16:02, frankf wrote:
> I think this fist on a single line

Done.

[commit-bot: I haz the power, 2013-10-03 09:49:10]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/digit@chromium.org/25525003/34002

[commit-bot: I haz the power, 2013-10-03 11:56:52]

Change committed as 226739