[android] Split top-level scripts and libraries from build/android.

[android] Split top-level scripts and libraries from build/android.

- Top-level scripts are kept under build/android.
- Utility libraries have been moved to build/android/pylib.
- Fixes all imports and headers.

This is in preparation for landing the "instrumentation" (java-based) tests,
which will reuse several of these components.

BUG=
TEST=existing android tests


Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=145653

[Satish, 2012-07-06 13:21:36]

http://codereview.chromium.org/10693110/diff/4001/build/android/enable_assert...
File build/android/enable_asserts.py (right):

http://codereview.chromium.org/10693110/diff/4001/build/android/enable_assert...
build/android/enable_asserts.py:1: #!/usr/bin/env python
newline after this

http://codereview.chromium.org/10693110/diff/4001/build/android/enable_assert...
build/android/enable_asserts.py:6: """Enables dalvik vm asserts in the android
device."""
Is this functionality used by any of the build tools? If not, seems like
build/android is not the right place. I see there are other run/debug utilities
under build/android and perhaps all of them should move under
build/android/tools etc. so it is clear that build/android is about scripts to
build & run tests?

http://codereview.chromium.org/10693110/diff/4001/testing/android/generate_na...
File testing/android/generate_native_test.py (right):

http://codereview.chromium.org/10693110/diff/4001/testing/android/generate_na...
testing/android/generate_native_test.py:1: #!/usr/bin/python
switch to new format?

[bulach, 2012-07-06 13:42:09]

thanks satish! comments addressed and one clarification below, another look
please?

http://codereview.chromium.org/10693110/diff/4001/build/android/enable_assert...
File build/android/enable_asserts.py (right):

http://codereview.chromium.org/10693110/diff/4001/build/android/enable_assert...
build/android/enable_asserts.py:1: #!/usr/bin/env python
On 2012/07/06 13:21:36, Satish wrote:
> newline after this

Done.

http://codereview.chromium.org/10693110/diff/4001/build/android/enable_assert...
build/android/enable_asserts.py:6: """Enables dalvik vm asserts in the android
device."""
On 2012/07/06 13:21:36, Satish wrote:
> Is this functionality used by any of the build tools? If not, seems like
> build/android is not the right place. I see there are other run/debug
utilities
> under build/android and perhaps all of them should move under
> build/android/tools etc. so it is clear that build/android is about scripts to
> build & run tests?

not sure I understood...

build/android is about top-level executable scripts, such as this one... this
functionality used to be in android_commands.py,
http://codereview.chromium.org/10693110/diff/4001/build/android/android_comma...

there was a TODO (in the downstream repo only) to remove such functionality from
the library, which is precisely what this is doing: a thin wrapper on top of the
now pure and reusable pylib.android_commands..

specifically: this is used by developers to tweak the assertion levels in their
device.. the test runners themselves also do that internally, using
android_commands directly..

is this ok?

http://codereview.chromium.org/10693110/diff/4001/testing/android/generate_na...
File testing/android/generate_native_test.py (right):

http://codereview.chromium.org/10693110/diff/4001/testing/android/generate_na...
testing/android/generate_native_test.py:1: #!/usr/bin/python
On 2012/07/06 13:21:36, Satish wrote:
> switch to new format?

Done.

[Satish, 2012-07-06 13:47:39]

http://codereview.chromium.org/10693110/diff/4001/build/android/enable_assert...
File build/android/enable_asserts.py (right):

http://codereview.chromium.org/10693110/diff/4001/build/android/enable_assert...
build/android/enable_asserts.py:6: """Enables dalvik vm asserts in the android
device."""
On 2012/07/06 13:42:09, bulach wrote:
> On 2012/07/06 13:21:36, Satish wrote:
> > Is this functionality used by any of the build tools? If not, seems like
> > build/android is not the right place. I see there are other run/debug
> utilities
> > under build/android and perhaps all of them should move under
> > build/android/tools etc. so it is clear that build/android is about scripts
to
> > build & run tests?
> 
> not sure I understood...
> 
> build/android is about top-level executable scripts, such as this one... this
> functionality used to be in android_commands.py,
>
http://codereview.chromium.org/10693110/diff/4001/build/android/android_comma...
> 
> there was a TODO (in the downstream repo only) to remove such functionality
from
> the library, which is precisely what this is doing: a thin wrapper on top of
the
> now pure and reusable pylib.android_commands..
> 
> specifically: this is used by developers to tweak the assertion levels in
their
> device.. the test runners themselves also do that internally, using
> android_commands directly..
> 
> is this ok?

I'm thinking build/android should contain only scripts for building and
running/testing the targets. Scripts like this one (and others such as
'adb_logcat_monitor.py' that already exists) seem like tools which are
infrequently used by individual developers during debugging. My suggestion was
to move them to a subdirectory such as 'tools' so that build/android can be only
about building targets and running/testing them. Would that be ok?

[bulach, 2012-07-06 14:02:27]

On 2012/07/06 13:47:39, Satish wrote:
>
http://codereview.chromium.org/10693110/diff/4001/build/android/enable_assert...
> File build/android/enable_asserts.py (right):
> 
>
http://codereview.chromium.org/10693110/diff/4001/build/android/enable_assert...
> build/android/enable_asserts.py:6: """Enables dalvik vm asserts in the android
> device."""
> On 2012/07/06 13:42:09, bulach wrote:
> > On 2012/07/06 13:21:36, Satish wrote:
> > > Is this functionality used by any of the build tools? If not, seems like
> > > build/android is not the right place. I see there are other run/debug
> > utilities
> > > under build/android and perhaps all of them should move under
> > > build/android/tools etc. so it is clear that build/android is about
scripts
> to
> > > build & run tests?
> > 
> > not sure I understood...
> > 
> > build/android is about top-level executable scripts, such as this one...
this
> > functionality used to be in android_commands.py,
> >
>
http://codereview.chromium.org/10693110/diff/4001/build/android/android_comma...
> > 
> > there was a TODO (in the downstream repo only) to remove such functionality
> from
> > the library, which is precisely what this is doing: a thin wrapper on top of
> the
> > now pure and reusable pylib.android_commands..
> > 
> > specifically: this is used by developers to tweak the assertion levels in
> their
> > device.. the test runners themselves also do that internally, using
> > android_commands directly..
> > 
> > is this ok?
> 
> I'm thinking build/android should contain only scripts for building and
> running/testing the targets. Scripts like this one (and others such as
> 'adb_logcat_monitor.py' that already exists) seem like tools which are
> infrequently used by individual developers during debugging. My suggestion was
> to move them to a subdirectory such as 'tools' so that build/android can be
only
> about building targets and running/testing them. Would that be ok?

I'm still not quite clear.. :)

The enable_asserts.py is used by developers for testing/debugging (agree, less
frequently..).
adb_logcat_monitor.py is used by the bots for building/testing..

The reason for this patch is to make the components under pylib/ more reusable,
so that we'll be able to add the instrumentation tests soon.. That is, the
"external interface" provided by build/android remains the same, I'm just
changing its implementation and moving to build/android/pylib...

I agree we could further split the remaining top-level scripts, but it's not
quite clear what to do there. for instance:

- by usage frequency seems very hard to track and would just add more
complication ("is script X in foo/ or in bar/ ?")

- by bots versus developers is also tricky, as there's hopefully a vast
overlap..

- by "building" and "testing": this may be clearer in the future with the
builders x testers split, but right now there's again an overlap of things we do
for building, test setup and test runners...

[Satish, 2012-07-06 14:12:47]

Cool, didn't realise adb_logcat_monitor.py was used by bots. The criteria I have
is simple - if it is used by a bot, build/android is a great place. So all test
runners and associated scripts reside here. If it is used only manually then
lets move out of build/android top level directory. This is probably similar to
the /bin directory that was present downstream.

However if you think there isn't going to much value in such a distinction I'm
ok landing this as is now and moving later when we see more need.

[Satish, 2012-07-06 14:12:55]

lgtm

[commit-bot: I haz the power, 2012-07-06 14:39:05]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/bulach@chromium.org/10693110/38

[commit-bot: I haz the power, 2012-07-06 14:39:17]

Presubmit check for 10693110-38 failed and returned exit status 1.

Running presubmit commit checks ...

** Presubmit Messages **
If this change has an associated bug, add BUG=[bug number].

** Presubmit Warnings **
License must match:
.*? Copyright \(c\) 2012 The Chromium Authors\. All rights reserved\.\n.*? Use
of this source code is governed by a BSD-style license that can be\n.*? found in
the LICENSE file\.(?: \*/)?\n
Found a bad license header in these files:
  build/android/pylib/__init__.py

Presubmit checks took 2.3s to calculate.

[bulach, 2012-07-06 14:42:25]

On 2012/07/06 14:12:47, Satish wrote:
> Cool, didn't realise adb_logcat_monitor.py was used by bots. The criteria I
have
> is simple - if it is used by a bot, build/android is a great place. So all
test
> runners and associated scripts reside here. If it is used only manually then
> lets move out of build/android top level directory. This is probably similar
to
> the /bin directory that was present downstream.
> 
> However if you think there isn't going to much value in such a distinction I'm
> ok landing this as is now and moving later when we see more need.

hmm... there were quite a few examples of things that moved from "manual only"
to "also on bots" and "bots only" to "also manual": test runners themselves
being one of them, but also things like "copy and deploy apk", "list all
devices", etc.. :)

the bin/ downstream has also some bash-scripts that replicate the functionality
already available in python..

my hope is that this pylib/ will encourage us to cleanup and reuse more of these
components, but anyways... this is still very early, let's see how it evolves
with the instrumentation tests, I'm always happy to move things around... :)

[commit-bot: I haz the power, 2012-07-09 08:07:47]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/bulach@chromium.org/10693110/12001

[commit-bot: I haz the power, 2012-07-09 09:12:01]

Change committed as 145653