GN: Android perf/tests

GN: Android perf/tests

Just to take inventory of which old problems still linger and which are now moot, I've gone out of my way to have this stand alone.  All of gn_android_flavor's logic is self contained, without any dependency on the platform_tools scripts.  

The tricky bits turn out to be, copying directories containing symlinks---or really any copying involving more than one file---and getting the exit code back from adb shell.  Luckily the ADB I've got on my desktop and my Nexus 5x seems to handle this all without the awkward workarounds you see here, so there's hope that One Day Soon the weird parts (basically, anything with inline python) can go away.  Once we've got these bots landed green, I'll go see whether the fixes are due to ADB updates, Android updates, or perhaps something else like hardware.

The parts marked TEMPORARY are a nod to the fact that the devices are used by gn_android_flavor and android_flavor both today.  It's mostly about not stepping on each other's toes or leaving anything laying around that might confuse each other.  The marked parts can go away when bots are either gn_ or non-gn_ but not both.

I have omitted a few steps that may be important, but which are easy independent follow-ups:
  - running as root
  - locking clocks
  - waiting on battery levels
  - fancier wait-for-ready than adb wait-for-usb-device
It'd be nice to, e.g., reaffirm that locking clocks helps perf stability, and that we're locking to the best policy.  I've tried to keep this CL as trim as possible, leaving any of these vaguely optional steps for later.

As of PS 41 or so, it looks like the trybots are all behaving as expected.
We should expect no new images in Gold.  Can we see trybots in Perf yet?

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002

CQ_INCLUDE_TRYBOTS=master.client.skia.android:Perf-Android-Clang-AndroidOne-CPU-MT6582-arm-Debug-GN_Android-Trybot,Perf-Android-Clang-AndroidOne-CPU-MT6582-arm-Release-GN_Android-Trybot,Perf-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Debug-GN_Android-Trybot,Perf-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Release-GN_Android-Trybot,Test-Android-Clang-AndroidOne-CPU-MT6582-arm-Debug-GN_Android-Trybot,Test-Android-Clang-AndroidOne-CPU-MT6582-arm-Release-GN_Android-Trybot,Test-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Debug-GN_Android-Trybot,Test-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Release-GN_Android-Trybot

Committed: https://skia.googlesource.com/skia/+/df5df81084a9b8652e6ed0987c2454c62d787188

[mtklein_C, 2016-09-08 17:15:21]

Description was changed from

==========
GN: Android perf/tests

BUG=skia:
==========

to

==========
GN: Android perf/tests

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002
==========

[mtklein_C, 2016-09-09 21:20:34]

Description was changed from

==========
GN: Android perf/tests

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002
==========

to

==========
GN: Android perf/tests

Just to take inventory of what old problems are still problems and what's now
moot, I've gone out of my way to have this flavor stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002
==========

[mtklein_C, 2016-09-09 21:21:53]

Description was changed from

==========
GN: Android perf/tests

Just to take inventory of what old problems are still problems and what's now
moot, I've gone out of my way to have this flavor stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002
==========

to

==========
GN: Android perf/tests

Just to take inventory of what old problems are still problems and what's now
moot, I've gone out of my way to have this flavor stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.

The parts marked TEMPORARY are a nod to the fact that the bots are used by
gn_android_flavor and android_flavor both today.  It's mostly about not stepping
on each other's toes, and can go away when bots are either gn_ or non-gn_ but
not both.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002
==========

[mtklein_C, 2016-09-09 21:22:26]

Description was changed from

==========
GN: Android perf/tests

Just to take inventory of what old problems are still problems and what's now
moot, I've gone out of my way to have this flavor stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.

The parts marked TEMPORARY are a nod to the fact that the bots are used by
gn_android_flavor and android_flavor both today.  It's mostly about not stepping
on each other's toes, and can go away when bots are either gn_ or non-gn_ but
not both.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002
==========

to

==========
GN: Android perf/tests

Just to take inventory of what old problems are still problems and what's now
moot, I've gone out of my way to have this flavor stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.

The parts marked TEMPORARY are a nod to the fact that the devices are used by
gn_android_flavor and android_flavor both today.  It's mostly about not stepping
on each other's toes, and can go away when bots are either gn_ or non-gn_ but
not both.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002
==========

[mtklein_C, 2016-09-09 21:26:41]

Description was changed from

==========
GN: Android perf/tests

Just to take inventory of what old problems are still problems and what's now
moot, I've gone out of my way to have this flavor stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.

The parts marked TEMPORARY are a nod to the fact that the devices are used by
gn_android_flavor and android_flavor both today.  It's mostly about not stepping
on each other's toes, and can go away when bots are either gn_ or non-gn_ but
not both.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002
==========

to

==========
GN: Android perf/tests

Just to take inventory of what old problems are still problems and what's now
moot, I've gone out of my way to have this flavor stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.

The parts marked TEMPORARY are a nod to the fact that the devices are used by
gn_android_flavor and android_flavor both today.  It's mostly about not stepping
on each other's toes, and can go away when bots are either gn_ or non-gn_ but
not both.

I have left out a few steps that may or may not be important, but which are easy
to follow up with independently:
  - running as root
  - locking clocks
  - waiting on battery levels
  - fancier waits than adb wait-for-usb-device
It'd be nice to, e.g., reaffirm that locking clocks helps perf stability, and
that we're locking to the best policy.  I was thinking it helps to leave
tweaking these knobs to follow ups, to keep this as trim as possible.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002
==========

[mtklein_C, 2016-09-09 21:28:29]

Description was changed from

==========
GN: Android perf/tests

Just to take inventory of what old problems are still problems and what's now
moot, I've gone out of my way to have this flavor stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.

The parts marked TEMPORARY are a nod to the fact that the devices are used by
gn_android_flavor and android_flavor both today.  It's mostly about not stepping
on each other's toes, and can go away when bots are either gn_ or non-gn_ but
not both.

I have left out a few steps that may or may not be important, but which are easy
to follow up with independently:
  - running as root
  - locking clocks
  - waiting on battery levels
  - fancier waits than adb wait-for-usb-device
It'd be nice to, e.g., reaffirm that locking clocks helps perf stability, and
that we're locking to the best policy.  I was thinking it helps to leave
tweaking these knobs to follow ups, to keep this as trim as possible.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002
==========

to

==========
GN: Android perf/tests

Just to take inventory of what old problems are still problems and what's now
moot, I've gone out of my way to have this flavor stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.

The parts marked TEMPORARY are a nod to the fact that the devices are used by
gn_android_flavor and android_flavor both today.  It's mostly about not stepping
on each other's toes, and can go away when bots are either gn_ or non-gn_ but
not both.

I have left out a few steps that may or may not be important, but which are easy
to follow up with independently:
  - running as root
  - locking clocks
  - waiting on battery levels
  - fancier waits than adb wait-for-usb-device
It'd be nice to, e.g., reaffirm that locking clocks helps perf stability, and
that we're locking to the best policy.  I was thinking it helps to leave
tweaking these knobs to follow ups, to keep this as trim as possible.

As of PS 41 or so, it looks like my trybots are all behaving as expected.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002
==========

[mtklein_C, 2016-09-09 21:32:08]

Description was changed from

==========
GN: Android perf/tests

Just to take inventory of what old problems are still problems and what's now
moot, I've gone out of my way to have this flavor stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.

The parts marked TEMPORARY are a nod to the fact that the devices are used by
gn_android_flavor and android_flavor both today.  It's mostly about not stepping
on each other's toes, and can go away when bots are either gn_ or non-gn_ but
not both.

I have left out a few steps that may or may not be important, but which are easy
to follow up with independently:
  - running as root
  - locking clocks
  - waiting on battery levels
  - fancier waits than adb wait-for-usb-device
It'd be nice to, e.g., reaffirm that locking clocks helps perf stability, and
that we're locking to the best policy.  I was thinking it helps to leave
tweaking these knobs to follow ups, to keep this as trim as possible.

As of PS 41 or so, it looks like my trybots are all behaving as expected.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002
==========

to

==========
GN: Android perf/tests

Just to take inventory of which old problems still linger and which are now
moot, I've gone out of my way to have this stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.

The parts marked TEMPORARY are a nod to the fact that the devices are used by
gn_android_flavor and android_flavor both today.  It's mostly about not stepping
on each other's toes, and can go away when bots are either gn_ or non-gn_ but
not both.

I have omitted a few steps that may be important, but which are easy independent
follow-ups:
  - running as root
  - locking clocks
  - waiting on battery levels
  - fancier wait-for-ready than adb wait-for-usb-device
It'd be nice to, e.g., reaffirm that locking clocks helps perf stability, and
that we're locking to the best policy.  I've tried to keep this skeleton as trim
as possible, leaving any of these vaguely optional steps for later.

As of PS 41 or so, it looks like the trybots are all behaving as expected.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002
==========

[mtklein_C, 2016-09-09 21:32:41]

Description was changed from

==========
GN: Android perf/tests

Just to take inventory of which old problems still linger and which are now
moot, I've gone out of my way to have this stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.

The parts marked TEMPORARY are a nod to the fact that the devices are used by
gn_android_flavor and android_flavor both today.  It's mostly about not stepping
on each other's toes, and can go away when bots are either gn_ or non-gn_ but
not both.

I have omitted a few steps that may be important, but which are easy independent
follow-ups:
  - running as root
  - locking clocks
  - waiting on battery levels
  - fancier wait-for-ready than adb wait-for-usb-device
It'd be nice to, e.g., reaffirm that locking clocks helps perf stability, and
that we're locking to the best policy.  I've tried to keep this skeleton as trim
as possible, leaving any of these vaguely optional steps for later.

As of PS 41 or so, it looks like the trybots are all behaving as expected.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002
==========

to

==========
GN: Android perf/tests

Just to take inventory of which old problems still linger and which are now
moot, I've gone out of my way to have this stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.

The parts marked TEMPORARY are a nod to the fact that the devices are used by
gn_android_flavor and android_flavor both today.  It's mostly about not stepping
on each other's toes, and can go away when bots are either gn_ or non-gn_ but
not both.

I have omitted a few steps that may be important, but which are easy independent
follow-ups:
  - running as root
  - locking clocks
  - waiting on battery levels
  - fancier wait-for-ready than adb wait-for-usb-device
It'd be nice to, e.g., reaffirm that locking clocks helps perf stability, and
that we're locking to the best policy.  I've tried to keep this skeleton as trim
as possible, leaving any of these vaguely optional steps for later.

As of PS 41 or so, it looks like the trybots are all behaving as expected.
We should expect no new images in Gold.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002
==========

[mtklein_C, 2016-09-09 21:43:00]

Description was changed from

==========
GN: Android perf/tests

Just to take inventory of which old problems still linger and which are now
moot, I've gone out of my way to have this stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.

The parts marked TEMPORARY are a nod to the fact that the devices are used by
gn_android_flavor and android_flavor both today.  It's mostly about not stepping
on each other's toes, and can go away when bots are either gn_ or non-gn_ but
not both.

I have omitted a few steps that may be important, but which are easy independent
follow-ups:
  - running as root
  - locking clocks
  - waiting on battery levels
  - fancier wait-for-ready than adb wait-for-usb-device
It'd be nice to, e.g., reaffirm that locking clocks helps perf stability, and
that we're locking to the best policy.  I've tried to keep this skeleton as trim
as possible, leaving any of these vaguely optional steps for later.

As of PS 41 or so, it looks like the trybots are all behaving as expected.
We should expect no new images in Gold.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002
==========

to

==========
GN: Android perf/tests

Just to take inventory of which old problems still linger and which are now
moot, I've gone out of my way to have this stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.  

The tricky bits turn out to be, copying many files at once, and getting the exit
code back from adb shell.  Luckily the ADB I've got on my machine seems to
handle both of these without the awkward workarounds you see here, so there's
hope that One Day Soon the weird bits in here (anywhere with inline python) can
go away.

The parts marked TEMPORARY are a nod to the fact that the devices are used by
gn_android_flavor and android_flavor both today.  It's mostly about not stepping
on each other's toes, and can go away when bots are either gn_ or non-gn_ but
not both.

I have omitted a few steps that may be important, but which are easy independent
follow-ups:
  - running as root
  - locking clocks
  - waiting on battery levels
  - fancier wait-for-ready than adb wait-for-usb-device
It'd be nice to, e.g., reaffirm that locking clocks helps perf stability, and
that we're locking to the best policy.  I've tried to keep this skeleton as trim
as possible, leaving any of these vaguely optional steps for later.

As of PS 41 or so, it looks like the trybots are all behaving as expected.
We should expect no new images in Gold.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002
==========

[mtklein_C, 2016-09-09 21:44:11]

Description was changed from

==========
GN: Android perf/tests

Just to take inventory of which old problems still linger and which are now
moot, I've gone out of my way to have this stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.  

The tricky bits turn out to be, copying many files at once, and getting the exit
code back from adb shell.  Luckily the ADB I've got on my machine seems to
handle both of these without the awkward workarounds you see here, so there's
hope that One Day Soon the weird bits in here (anywhere with inline python) can
go away.

The parts marked TEMPORARY are a nod to the fact that the devices are used by
gn_android_flavor and android_flavor both today.  It's mostly about not stepping
on each other's toes, and can go away when bots are either gn_ or non-gn_ but
not both.

I have omitted a few steps that may be important, but which are easy independent
follow-ups:
  - running as root
  - locking clocks
  - waiting on battery levels
  - fancier wait-for-ready than adb wait-for-usb-device
It'd be nice to, e.g., reaffirm that locking clocks helps perf stability, and
that we're locking to the best policy.  I've tried to keep this skeleton as trim
as possible, leaving any of these vaguely optional steps for later.

As of PS 41 or so, it looks like the trybots are all behaving as expected.
We should expect no new images in Gold.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002
==========

to

==========
GN: Android perf/tests

Just to take inventory of which old problems still linger and which are now
moot, I've gone out of my way to have this stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.  

The tricky bits turn out to be, copying many files at once, and getting the exit
code back from adb shell.  Luckily the ADB I've got on my machine seems to
handle both of these with my Nexus 5x without the awkward workarounds you see
here, so there's hope that One Day Soon the weird bits in here (anywhere with
inline python) can go away.  Once we've got these bots landed green, I'll go see
whether the fixes are due to ADB updates, Android updates, or perhaps hardware.

The parts marked TEMPORARY are a nod to the fact that the devices are used by
gn_android_flavor and android_flavor both today.  It's mostly about not stepping
on each other's toes, and can go away when bots are either gn_ or non-gn_ but
not both.

I have omitted a few steps that may be important, but which are easy independent
follow-ups:
  - running as root
  - locking clocks
  - waiting on battery levels
  - fancier wait-for-ready than adb wait-for-usb-device
It'd be nice to, e.g., reaffirm that locking clocks helps perf stability, and
that we're locking to the best policy.  I've tried to keep this skeleton as trim
as possible, leaving any of these vaguely optional steps for later.

As of PS 41 or so, it looks like the trybots are all behaving as expected.
We should expect no new images in Gold.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002
==========

[mtklein_C, 2016-09-09 21:45:16]

Description was changed from

==========
GN: Android perf/tests

Just to take inventory of which old problems still linger and which are now
moot, I've gone out of my way to have this stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.  

The tricky bits turn out to be, copying many files at once, and getting the exit
code back from adb shell.  Luckily the ADB I've got on my machine seems to
handle both of these with my Nexus 5x without the awkward workarounds you see
here, so there's hope that One Day Soon the weird bits in here (anywhere with
inline python) can go away.  Once we've got these bots landed green, I'll go see
whether the fixes are due to ADB updates, Android updates, or perhaps hardware.

The parts marked TEMPORARY are a nod to the fact that the devices are used by
gn_android_flavor and android_flavor both today.  It's mostly about not stepping
on each other's toes, and can go away when bots are either gn_ or non-gn_ but
not both.

I have omitted a few steps that may be important, but which are easy independent
follow-ups:
  - running as root
  - locking clocks
  - waiting on battery levels
  - fancier wait-for-ready than adb wait-for-usb-device
It'd be nice to, e.g., reaffirm that locking clocks helps perf stability, and
that we're locking to the best policy.  I've tried to keep this skeleton as trim
as possible, leaving any of these vaguely optional steps for later.

As of PS 41 or so, it looks like the trybots are all behaving as expected.
We should expect no new images in Gold.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002
==========

to

==========
GN: Android perf/tests

Just to take inventory of which old problems still linger and which are now
moot, I've gone out of my way to have this stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.  

The tricky bits turn out to be, copying directories containing symlinks---or
really any copying involving more than one file---and getting the exit code back
from adb shell.  Luckily the ADB I've got on my machine seems to handle both of
these with my Nexus 5x without the awkward workarounds you see here, so there's
hope that One Day Soon the weird bits in here (anywhere with inline python) can
go away.  Once we've got these bots landed green, I'll go see whether the fixes
are due to ADB updates, Android updates, or perhaps hardware.

The parts marked TEMPORARY are a nod to the fact that the devices are used by
gn_android_flavor and android_flavor both today.  It's mostly about not stepping
on each other's toes, and can go away when bots are either gn_ or non-gn_ but
not both.

I have omitted a few steps that may be important, but which are easy independent
follow-ups:
  - running as root
  - locking clocks
  - waiting on battery levels
  - fancier wait-for-ready than adb wait-for-usb-device
It'd be nice to, e.g., reaffirm that locking clocks helps perf stability, and
that we're locking to the best policy.  I've tried to keep this skeleton as trim
as possible, leaving any of these vaguely optional steps for later.

As of PS 41 or so, it looks like the trybots are all behaving as expected.
We should expect no new images in Gold.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002
==========

[mtklein_C, 2016-09-09 21:46:19]

Description was changed from

==========
GN: Android perf/tests

Just to take inventory of which old problems still linger and which are now
moot, I've gone out of my way to have this stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.  

The tricky bits turn out to be, copying directories containing symlinks---or
really any copying involving more than one file---and getting the exit code back
from adb shell.  Luckily the ADB I've got on my machine seems to handle both of
these with my Nexus 5x without the awkward workarounds you see here, so there's
hope that One Day Soon the weird bits in here (anywhere with inline python) can
go away.  Once we've got these bots landed green, I'll go see whether the fixes
are due to ADB updates, Android updates, or perhaps hardware.

The parts marked TEMPORARY are a nod to the fact that the devices are used by
gn_android_flavor and android_flavor both today.  It's mostly about not stepping
on each other's toes, and can go away when bots are either gn_ or non-gn_ but
not both.

I have omitted a few steps that may be important, but which are easy independent
follow-ups:
  - running as root
  - locking clocks
  - waiting on battery levels
  - fancier wait-for-ready than adb wait-for-usb-device
It'd be nice to, e.g., reaffirm that locking clocks helps perf stability, and
that we're locking to the best policy.  I've tried to keep this skeleton as trim
as possible, leaving any of these vaguely optional steps for later.

As of PS 41 or so, it looks like the trybots are all behaving as expected.
We should expect no new images in Gold.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002
==========

to

==========
GN: Android perf/tests

Just to take inventory of which old problems still linger and which are now
moot, I've gone out of my way to have this stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.  

The tricky bits turn out to be, copying directories containing symlinks---or
really any copying involving more than one file---and getting the exit code back
from adb shell.  Luckily the ADB I've got on my desktop and my Nexus 5x seems to
handle this all without the awkward workarounds you see here, so there's hope
that One Day Soon the weird parts (basically, anything with inline python) can
go away.  Once we've got these bots landed green, I'll go see whether the fixes
are due to ADB updates, Android updates, or perhaps something else like
hardware.

The parts marked TEMPORARY are a nod to the fact that the devices are used by
gn_android_flavor and android_flavor both today.  It's mostly about not stepping
on each other's toes, and can go away when bots are either gn_ or non-gn_ but
not both.

I have omitted a few steps that may be important, but which are easy independent
follow-ups:
  - running as root
  - locking clocks
  - waiting on battery levels
  - fancier wait-for-ready than adb wait-for-usb-device
It'd be nice to, e.g., reaffirm that locking clocks helps perf stability, and
that we're locking to the best policy.  I've tried to keep this skeleton as trim
as possible, leaving any of these vaguely optional steps for later.

As of PS 41 or so, it looks like the trybots are all behaving as expected.
We should expect no new images in Gold.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002
==========

[mtklein_C, 2016-09-09 21:47:09]

Description was changed from

==========
GN: Android perf/tests

Just to take inventory of which old problems still linger and which are now
moot, I've gone out of my way to have this stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.  

The tricky bits turn out to be, copying directories containing symlinks---or
really any copying involving more than one file---and getting the exit code back
from adb shell.  Luckily the ADB I've got on my desktop and my Nexus 5x seems to
handle this all without the awkward workarounds you see here, so there's hope
that One Day Soon the weird parts (basically, anything with inline python) can
go away.  Once we've got these bots landed green, I'll go see whether the fixes
are due to ADB updates, Android updates, or perhaps something else like
hardware.

The parts marked TEMPORARY are a nod to the fact that the devices are used by
gn_android_flavor and android_flavor both today.  It's mostly about not stepping
on each other's toes, and can go away when bots are either gn_ or non-gn_ but
not both.

I have omitted a few steps that may be important, but which are easy independent
follow-ups:
  - running as root
  - locking clocks
  - waiting on battery levels
  - fancier wait-for-ready than adb wait-for-usb-device
It'd be nice to, e.g., reaffirm that locking clocks helps perf stability, and
that we're locking to the best policy.  I've tried to keep this skeleton as trim
as possible, leaving any of these vaguely optional steps for later.

As of PS 41 or so, it looks like the trybots are all behaving as expected.
We should expect no new images in Gold.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002
==========

to

==========
GN: Android perf/tests

Just to take inventory of which old problems still linger and which are now
moot, I've gone out of my way to have this stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.  

The tricky bits turn out to be, copying directories containing symlinks---or
really any copying involving more than one file---and getting the exit code back
from adb shell.  Luckily the ADB I've got on my desktop and my Nexus 5x seems to
handle this all without the awkward workarounds you see here, so there's hope
that One Day Soon the weird parts (basically, anything with inline python) can
go away.  Once we've got these bots landed green, I'll go see whether the fixes
are due to ADB updates, Android updates, or perhaps something else like
hardware.

The parts marked TEMPORARY are a nod to the fact that the devices are used by
gn_android_flavor and android_flavor both today.  It's mostly about not stepping
on each other's toes or leaving anything laying around that might confuse each
other.  The marked parts can go away when bots are either gn_ or non-gn_ but not
both.

I have omitted a few steps that may be important, but which are easy independent
follow-ups:
  - running as root
  - locking clocks
  - waiting on battery levels
  - fancier wait-for-ready than adb wait-for-usb-device
It'd be nice to, e.g., reaffirm that locking clocks helps perf stability, and
that we're locking to the best policy.  I've tried to keep this skeleton as trim
as possible, leaving any of these vaguely optional steps for later.

As of PS 41 or so, it looks like the trybots are all behaving as expected.
We should expect no new images in Gold.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002
==========

[mtklein_C, 2016-09-09 21:48:51]

Description was changed from

==========
GN: Android perf/tests

Just to take inventory of which old problems still linger and which are now
moot, I've gone out of my way to have this stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.  

The tricky bits turn out to be, copying directories containing symlinks---or
really any copying involving more than one file---and getting the exit code back
from adb shell.  Luckily the ADB I've got on my desktop and my Nexus 5x seems to
handle this all without the awkward workarounds you see here, so there's hope
that One Day Soon the weird parts (basically, anything with inline python) can
go away.  Once we've got these bots landed green, I'll go see whether the fixes
are due to ADB updates, Android updates, or perhaps something else like
hardware.

The parts marked TEMPORARY are a nod to the fact that the devices are used by
gn_android_flavor and android_flavor both today.  It's mostly about not stepping
on each other's toes or leaving anything laying around that might confuse each
other.  The marked parts can go away when bots are either gn_ or non-gn_ but not
both.

I have omitted a few steps that may be important, but which are easy independent
follow-ups:
  - running as root
  - locking clocks
  - waiting on battery levels
  - fancier wait-for-ready than adb wait-for-usb-device
It'd be nice to, e.g., reaffirm that locking clocks helps perf stability, and
that we're locking to the best policy.  I've tried to keep this skeleton as trim
as possible, leaving any of these vaguely optional steps for later.

As of PS 41 or so, it looks like the trybots are all behaving as expected.
We should expect no new images in Gold.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002
==========

to

==========
GN: Android perf/tests

Just to take inventory of which old problems still linger and which are now
moot, I've gone out of my way to have this stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.  

The tricky bits turn out to be, copying directories containing symlinks---or
really any copying involving more than one file---and getting the exit code back
from adb shell.  Luckily the ADB I've got on my desktop and my Nexus 5x seems to
handle this all without the awkward workarounds you see here, so there's hope
that One Day Soon the weird parts (basically, anything with inline python) can
go away.  Once we've got these bots landed green, I'll go see whether the fixes
are due to ADB updates, Android updates, or perhaps something else like
hardware.

The parts marked TEMPORARY are a nod to the fact that the devices are used by
gn_android_flavor and android_flavor both today.  It's mostly about not stepping
on each other's toes or leaving anything laying around that might confuse each
other.  The marked parts can go away when bots are either gn_ or non-gn_ but not
both.

I have omitted a few steps that may be important, but which are easy independent
follow-ups:
  - running as root
  - locking clocks
  - waiting on battery levels
  - fancier wait-for-ready than adb wait-for-usb-device
It'd be nice to, e.g., reaffirm that locking clocks helps perf stability, and
that we're locking to the best policy.  I've tried to keep this CL as trim as
possible, leaving any of these vaguely optional steps for later.

As of PS 41 or so, it looks like the trybots are all behaving as expected.
We should expect no new images in Gold.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002
==========

[mtklein_C, 2016-09-09 21:49:54]

Description was changed from

==========
GN: Android perf/tests

Just to take inventory of which old problems still linger and which are now
moot, I've gone out of my way to have this stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.  

The tricky bits turn out to be, copying directories containing symlinks---or
really any copying involving more than one file---and getting the exit code back
from adb shell.  Luckily the ADB I've got on my desktop and my Nexus 5x seems to
handle this all without the awkward workarounds you see here, so there's hope
that One Day Soon the weird parts (basically, anything with inline python) can
go away.  Once we've got these bots landed green, I'll go see whether the fixes
are due to ADB updates, Android updates, or perhaps something else like
hardware.

The parts marked TEMPORARY are a nod to the fact that the devices are used by
gn_android_flavor and android_flavor both today.  It's mostly about not stepping
on each other's toes or leaving anything laying around that might confuse each
other.  The marked parts can go away when bots are either gn_ or non-gn_ but not
both.

I have omitted a few steps that may be important, but which are easy independent
follow-ups:
  - running as root
  - locking clocks
  - waiting on battery levels
  - fancier wait-for-ready than adb wait-for-usb-device
It'd be nice to, e.g., reaffirm that locking clocks helps perf stability, and
that we're locking to the best policy.  I've tried to keep this CL as trim as
possible, leaving any of these vaguely optional steps for later.

As of PS 41 or so, it looks like the trybots are all behaving as expected.
We should expect no new images in Gold.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002
==========

to

==========
GN: Android perf/tests

Just to take inventory of which old problems still linger and which are now
moot, I've gone out of my way to have this stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.  

The tricky bits turn out to be, copying directories containing symlinks---or
really any copying involving more than one file---and getting the exit code back
from adb shell.  Luckily the ADB I've got on my desktop and my Nexus 5x seems to
handle this all without the awkward workarounds you see here, so there's hope
that One Day Soon the weird parts (basically, anything with inline python) can
go away.  Once we've got these bots landed green, I'll go see whether the fixes
are due to ADB updates, Android updates, or perhaps something else like
hardware.

The parts marked TEMPORARY are a nod to the fact that the devices are used by
gn_android_flavor and android_flavor both today.  It's mostly about not stepping
on each other's toes or leaving anything laying around that might confuse each
other.  The marked parts can go away when bots are either gn_ or non-gn_ but not
both.

I have omitted a few steps that may be important, but which are easy independent
follow-ups:
  - running as root
  - locking clocks
  - waiting on battery levels
  - fancier wait-for-ready than adb wait-for-usb-device
It'd be nice to, e.g., reaffirm that locking clocks helps perf stability, and
that we're locking to the best policy.  I've tried to keep this CL as trim as
possible, leaving any of these vaguely optional steps for later.

As of PS 41 or so, it looks like the trybots are all behaving as expected.
We should expect no new images in Gold.  Can we see trybots in perf yet?

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002
==========

[mtklein_C, 2016-09-09 21:50:02]

Description was changed from

==========
GN: Android perf/tests

Just to take inventory of which old problems still linger and which are now
moot, I've gone out of my way to have this stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.  

The tricky bits turn out to be, copying directories containing symlinks---or
really any copying involving more than one file---and getting the exit code back
from adb shell.  Luckily the ADB I've got on my desktop and my Nexus 5x seems to
handle this all without the awkward workarounds you see here, so there's hope
that One Day Soon the weird parts (basically, anything with inline python) can
go away.  Once we've got these bots landed green, I'll go see whether the fixes
are due to ADB updates, Android updates, or perhaps something else like
hardware.

The parts marked TEMPORARY are a nod to the fact that the devices are used by
gn_android_flavor and android_flavor both today.  It's mostly about not stepping
on each other's toes or leaving anything laying around that might confuse each
other.  The marked parts can go away when bots are either gn_ or non-gn_ but not
both.

I have omitted a few steps that may be important, but which are easy independent
follow-ups:
  - running as root
  - locking clocks
  - waiting on battery levels
  - fancier wait-for-ready than adb wait-for-usb-device
It'd be nice to, e.g., reaffirm that locking clocks helps perf stability, and
that we're locking to the best policy.  I've tried to keep this CL as trim as
possible, leaving any of these vaguely optional steps for later.

As of PS 41 or so, it looks like the trybots are all behaving as expected.
We should expect no new images in Gold.  Can we see trybots in perf yet?

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002
==========

to

==========
GN: Android perf/tests

Just to take inventory of which old problems still linger and which are now
moot, I've gone out of my way to have this stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.  

The tricky bits turn out to be, copying directories containing symlinks---or
really any copying involving more than one file---and getting the exit code back
from adb shell.  Luckily the ADB I've got on my desktop and my Nexus 5x seems to
handle this all without the awkward workarounds you see here, so there's hope
that One Day Soon the weird parts (basically, anything with inline python) can
go away.  Once we've got these bots landed green, I'll go see whether the fixes
are due to ADB updates, Android updates, or perhaps something else like
hardware.

The parts marked TEMPORARY are a nod to the fact that the devices are used by
gn_android_flavor and android_flavor both today.  It's mostly about not stepping
on each other's toes or leaving anything laying around that might confuse each
other.  The marked parts can go away when bots are either gn_ or non-gn_ but not
both.

I have omitted a few steps that may be important, but which are easy independent
follow-ups:
  - running as root
  - locking clocks
  - waiting on battery levels
  - fancier wait-for-ready than adb wait-for-usb-device
It'd be nice to, e.g., reaffirm that locking clocks helps perf stability, and
that we're locking to the best policy.  I've tried to keep this CL as trim as
possible, leaving any of these vaguely optional steps for later.

As of PS 41 or so, it looks like the trybots are all behaving as expected.
We should expect no new images in Gold.  Can we see trybots in Perf yet?

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002
==========

[mtklein_C, 2016-09-09 21:50:14]

mtklein@chromium.org changed reviewers:
+ borenet@google.com, jcgregorio@google.com

[mtklein_C, 2016-09-10 13:54:17]

Description was changed from

==========
GN: Android perf/tests

Just to take inventory of which old problems still linger and which are now
moot, I've gone out of my way to have this stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.  

The tricky bits turn out to be, copying directories containing symlinks---or
really any copying involving more than one file---and getting the exit code back
from adb shell.  Luckily the ADB I've got on my desktop and my Nexus 5x seems to
handle this all without the awkward workarounds you see here, so there's hope
that One Day Soon the weird parts (basically, anything with inline python) can
go away.  Once we've got these bots landed green, I'll go see whether the fixes
are due to ADB updates, Android updates, or perhaps something else like
hardware.

The parts marked TEMPORARY are a nod to the fact that the devices are used by
gn_android_flavor and android_flavor both today.  It's mostly about not stepping
on each other's toes or leaving anything laying around that might confuse each
other.  The marked parts can go away when bots are either gn_ or non-gn_ but not
both.

I have omitted a few steps that may be important, but which are easy independent
follow-ups:
  - running as root
  - locking clocks
  - waiting on battery levels
  - fancier wait-for-ready than adb wait-for-usb-device
It'd be nice to, e.g., reaffirm that locking clocks helps perf stability, and
that we're locking to the best policy.  I've tried to keep this CL as trim as
possible, leaving any of these vaguely optional steps for later.

As of PS 41 or so, it looks like the trybots are all behaving as expected.
We should expect no new images in Gold.  Can we see trybots in Perf yet?

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002
==========

to

==========
GN: Android perf/tests

Just to take inventory of which old problems still linger and which are now
moot, I've gone out of my way to have this stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.  

The tricky bits turn out to be, copying directories containing symlinks---or
really any copying involving more than one file---and getting the exit code back
from adb shell.  Luckily the ADB I've got on my desktop and my Nexus 5x seems to
handle this all without the awkward workarounds you see here, so there's hope
that One Day Soon the weird parts (basically, anything with inline python) can
go away.  Once we've got these bots landed green, I'll go see whether the fixes
are due to ADB updates, Android updates, or perhaps something else like
hardware.

The parts marked TEMPORARY are a nod to the fact that the devices are used by
gn_android_flavor and android_flavor both today.  It's mostly about not stepping
on each other's toes or leaving anything laying around that might confuse each
other.  The marked parts can go away when bots are either gn_ or non-gn_ but not
both.

I have omitted a few steps that may be important, but which are easy independent
follow-ups:
  - running as root
  - locking clocks
  - waiting on battery levels
  - fancier wait-for-ready than adb wait-for-usb-device
It'd be nice to, e.g., reaffirm that locking clocks helps perf stability, and
that we're locking to the best policy.  I've tried to keep this CL as trim as
possible, leaving any of these vaguely optional steps for later.

As of PS 41 or so, it looks like the trybots are all behaving as expected.
We should expect no new images in Gold.  Can we see trybots in Perf yet?

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002

CQ_INCLUDE_TRYBOTS=master.client.skia.android:Perf-Android-Clang-AndroidOne-CPU-MT6582-arm-Debug-GN_Android-Trybot,Perf-Android-Clang-AndroidOne-CPU-MT6582-arm-Release-GN_Android-Trybot,Perf-Android-Clang-AndroidOne-CPU-MT6582-arm-Release-GN_Android-Trybot,Perf-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Debug-GN_Android-Trybot,Perf-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Release-GN_Android-Trybot,Test-Android-Clang-AndroidOne-CPU-MT6582-arm-Debug-GN_Android-Trybot,Test-Android-Clang-AndroidOne-CPU-MT6582-arm-Release-GN_Android-Trybot,Test-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Debug-GN_Android-Trybot,Test-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Release-GN_Android-Trybot
==========

[mtklein_C, 2016-09-10 13:54:32]

The CQ bit was checked by mtklein@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-09-10 13:54:39]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-09-10 13:54:42]

There were warnings when CQ was processing your CL:
 * CQ is not running the
Perf-Android-Clang-AndroidOne-CPU-MT6582-arm-Release-GN_Android-Trybot trybot,
per your CQ_INCLUDE_TRYBOTS flag request. That bot was already specified, with
these tests: ['defaulttests']

[mtklein_C, 2016-09-10 13:56:16]

Description was changed from

==========
GN: Android perf/tests

Just to take inventory of which old problems still linger and which are now
moot, I've gone out of my way to have this stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.  

The tricky bits turn out to be, copying directories containing symlinks---or
really any copying involving more than one file---and getting the exit code back
from adb shell.  Luckily the ADB I've got on my desktop and my Nexus 5x seems to
handle this all without the awkward workarounds you see here, so there's hope
that One Day Soon the weird parts (basically, anything with inline python) can
go away.  Once we've got these bots landed green, I'll go see whether the fixes
are due to ADB updates, Android updates, or perhaps something else like
hardware.

The parts marked TEMPORARY are a nod to the fact that the devices are used by
gn_android_flavor and android_flavor both today.  It's mostly about not stepping
on each other's toes or leaving anything laying around that might confuse each
other.  The marked parts can go away when bots are either gn_ or non-gn_ but not
both.

I have omitted a few steps that may be important, but which are easy independent
follow-ups:
  - running as root
  - locking clocks
  - waiting on battery levels
  - fancier wait-for-ready than adb wait-for-usb-device
It'd be nice to, e.g., reaffirm that locking clocks helps perf stability, and
that we're locking to the best policy.  I've tried to keep this CL as trim as
possible, leaving any of these vaguely optional steps for later.

As of PS 41 or so, it looks like the trybots are all behaving as expected.
We should expect no new images in Gold.  Can we see trybots in Perf yet?

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002

CQ_INCLUDE_TRYBOTS=master.client.skia.android:Perf-Android-Clang-AndroidOne-CPU-MT6582-arm-Debug-GN_Android-Trybot,Perf-Android-Clang-AndroidOne-CPU-MT6582-arm-Release-GN_Android-Trybot,Perf-Android-Clang-AndroidOne-CPU-MT6582-arm-Release-GN_Android-Trybot,Perf-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Debug-GN_Android-Trybot,Perf-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Release-GN_Android-Trybot,Test-Android-Clang-AndroidOne-CPU-MT6582-arm-Debug-GN_Android-Trybot,Test-Android-Clang-AndroidOne-CPU-MT6582-arm-Release-GN_Android-Trybot,Test-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Debug-GN_Android-Trybot,Test-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Release-GN_Android-Trybot
==========

to

==========
GN: Android perf/tests

Just to take inventory of which old problems still linger and which are now
moot, I've gone out of my way to have this stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.  

The tricky bits turn out to be, copying directories containing symlinks---or
really any copying involving more than one file---and getting the exit code back
from adb shell.  Luckily the ADB I've got on my desktop and my Nexus 5x seems to
handle this all without the awkward workarounds you see here, so there's hope
that One Day Soon the weird parts (basically, anything with inline python) can
go away.  Once we've got these bots landed green, I'll go see whether the fixes
are due to ADB updates, Android updates, or perhaps something else like
hardware.

The parts marked TEMPORARY are a nod to the fact that the devices are used by
gn_android_flavor and android_flavor both today.  It's mostly about not stepping
on each other's toes or leaving anything laying around that might confuse each
other.  The marked parts can go away when bots are either gn_ or non-gn_ but not
both.

I have omitted a few steps that may be important, but which are easy independent
follow-ups:
  - running as root
  - locking clocks
  - waiting on battery levels
  - fancier wait-for-ready than adb wait-for-usb-device
It'd be nice to, e.g., reaffirm that locking clocks helps perf stability, and
that we're locking to the best policy.  I've tried to keep this CL as trim as
possible, leaving any of these vaguely optional steps for later.

As of PS 41 or so, it looks like the trybots are all behaving as expected.
We should expect no new images in Gold.  Can we see trybots in Perf yet?

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002

CQ_INCLUDE_TRYBOTS=master.client.skia.android:Perf-Android-Clang-AndroidOne-CPU-MT6582-arm-Debug-GN_Android-Trybot,Perf-Android-Clang-AndroidOne-CPU-MT6582-arm-Release-GN_Android-Trybot,Perf-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Debug-GN_Android-Trybot,Perf-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Release-GN_Android-Trybot,Test-Android-Clang-AndroidOne-CPU-MT6582-arm-Debug-GN_Android-Trybot,Test-Android-Clang-AndroidOne-CPU-MT6582-arm-Release-GN_Android-Trybot,Test-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Debug-GN_Android-Trybot,Test-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Release-GN_Android-Trybot
==========

[commit-bot: I haz the power, 2016-09-10 14:00:09]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-09-10 14:00:10]

Dry run: Try jobs failed on following builders:
  Test-Ubuntu-GCC-ShuttleA-GPU-GTX660-x86_64-Release-GN-Trybot on
master.client.skia (JOB_FAILED,
http://build.chromium.org/p/client.skia/builders/Test-Ubuntu-GCC-ShuttleA-GPU...)

[borenet, 2016-09-12 13:10:13]

https://codereview.chromium.org/2320153002/diff/820001/infra/bots/recipe_modu...
File infra/bots/recipe_modules/flavor/gn_android_flavor.py (right):

https://codereview.chromium.org/2320153002/diff/820001/infra/bots/recipe_modu...
infra/bots/recipe_modules/flavor/gn_android_flavor.py:29: title =
subprocess.list2cmdline(cmd)
This generates bad step names, eg. containing symbols that buildbot/etc don't
like.  I recommend just using the first part of the command.

https://codereview.chromium.org/2320153002/diff/820001/infra/bots/recipe_modu...
infra/bots/recipe_modules/flavor/gn_android_flavor.py:32: if k in ['PATH']}
Why do we need to overwrite the default_env here?  IMO we should avoid modifying
anything in self.m.vars.

https://codereview.chromium.org/2320153002/diff/820001/infra/bots/recipe_modu...
infra/bots/recipe_modules/flavor/gn_android_flavor.py:64: self._adb('shell',
'rm', '-rf', '/data/local/tmp/*')    # TEMPORARY
Do we really need this?  It's going to require us to re-copy all of the SKPs,
SkImages, etc which is very slow.

https://codereview.chromium.org/2320153002/diff/820001/infra/bots/recipes/swa...
File infra/bots/recipes/swarm_perf.py (right):

https://codereview.chromium.org/2320153002/diff/820001/infra/bots/recipes/swa...
infra/bots/recipes/swarm_perf.py:241: api.flavor.cleanup_steps()
Need try/finally here.

[mtklein, 2016-09-12 14:12:10]

mtklein@google.com changed reviewers:
+ mtklein@google.com

[mtklein, 2016-09-12 14:12:12]

https://codereview.chromium.org/2320153002/diff/820001/infra/bots/recipe_modu...
File infra/bots/recipe_modules/flavor/gn_android_flavor.py (right):

https://codereview.chromium.org/2320153002/diff/820001/infra/bots/recipe_modu...
infra/bots/recipe_modules/flavor/gn_android_flavor.py:29: title =
subprocess.list2cmdline(cmd)
On 2016/09/12 13:10:12, borenet wrote:
> This generates bad step names, eg. containing symbols that buildbot/etc don't
> like.  I recommend just using the first part of the command.

Huh.  What error did you see?

But, done... updated everything to use an explicit name.

https://codereview.chromium.org/2320153002/diff/820001/infra/bots/recipe_modu...
infra/bots/recipe_modules/flavor/gn_android_flavor.py:32: if k in ['PATH']}
On 2016/09/12 13:10:12, borenet wrote:
> Why do we need to overwrite the default_env here?  IMO we should avoid
modifying
> anything in self.m.vars.

Same deal as in gn_flavor.py and this file previously.  Everything in the
environment is there for GYP or GYP-built binaries, so it's confusing to see
that environment in a GN compile or run.  It's mostly harmless, but by removing
it from the environment we enforce that it's harmless.

I too would like to be able to do this without modifying self.m.vars, but it's
injected into the environment outside my control here.  Can we rework that so
that it's threaded through as env={...} from the outset and then I just sanitize
my own copy?

https://codereview.chromium.org/2320153002/diff/820001/infra/bots/recipe_modu...
infra/bots/recipe_modules/flavor/gn_android_flavor.py:64: self._adb('shell',
'rm', '-rf', '/data/local/tmp/*')    # TEMPORARY
On 2016/09/12 13:10:12, borenet wrote:
> Do we really need this?  It's going to require us to re-copy all of the SKPs,
> SkImages, etc which is very slow.

Not sure, but we definitely don't need it in the long term.

The file copying doesn't really take too long compared to the run times of these
bots.  You have to have really good timing to catch the resources or skps
copying, but yeah, the images can take a couple minutes.  

Let's land this as-is then follow up with a separate CL to re-enable the
caching?  That way it's independently revertible.

https://codereview.chromium.org/2320153002/diff/820001/infra/bots/recipes/swa...
File infra/bots/recipes/swarm_perf.py (right):

https://codereview.chromium.org/2320153002/diff/820001/infra/bots/recipes/swa...
infra/bots/recipes/swarm_perf.py:241: api.flavor.cleanup_steps()
On 2016/09/12 13:10:12, borenet wrote:
> Need try/finally here.

Done.

[mtklein_C, 2016-09-12 14:12:21]

The CQ bit was checked by mtklein@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-09-12 14:12:26]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[borenet, 2016-09-12 14:32:00]

https://codereview.chromium.org/2320153002/diff/820001/infra/bots/recipe_modu...
File infra/bots/recipe_modules/flavor/gn_android_flavor.py (right):

https://codereview.chromium.org/2320153002/diff/820001/infra/bots/recipe_modu...
infra/bots/recipe_modules/flavor/gn_android_flavor.py:29: title =
subprocess.list2cmdline(cmd)
On 2016/09/12 14:12:12, mtklein wrote:
> On 2016/09/12 13:10:12, borenet wrote:
> > This generates bad step names, eg. containing symbols that buildbot/etc
don't
> > like.  I recommend just using the first part of the command.
> 
> Huh.  What error did you see?
> 
> But, done... updated everything to use an explicit name.

No error here, but the master refuses to create log links for any steps with
"unusual" symbols in the name.

https://codereview.chromium.org/2320153002/diff/820001/infra/bots/recipe_modu...
infra/bots/recipe_modules/flavor/gn_android_flavor.py:32: if k in ['PATH']}
On 2016/09/12 14:12:12, mtklein wrote:
> On 2016/09/12 13:10:12, borenet wrote:
> > Why do we need to overwrite the default_env here?  IMO we should avoid
> modifying
> > anything in self.m.vars.
> 
> Same deal as in gn_flavor.py and this file previously.  Everything in the
> environment is there for GYP or GYP-built binaries, so it's confusing to see
> that environment in a GN compile or run.  It's mostly harmless, but by
removing
> it from the environment we enforce that it's harmless.
> 
> I too would like to be able to do this without modifying self.m.vars, but it's
> injected into the environment outside my control here.  Can we rework that so
> that it's threaded through as env={...} from the outset and then I just
sanitize
> my own copy?

This is fine for now, but the assumption for default_env should be that it
applies for all steps, and this may cause unexpected behavior if something
important and non-GYP-related were added.

https://codereview.chromium.org/2320153002/diff/820001/infra/bots/recipe_modu...
infra/bots/recipe_modules/flavor/gn_android_flavor.py:64: self._adb('shell',
'rm', '-rf', '/data/local/tmp/*')    # TEMPORARY
On 2016/09/12 14:12:12, mtklein wrote:
> On 2016/09/12 13:10:12, borenet wrote:
> > Do we really need this?  It's going to require us to re-copy all of the
SKPs,
> > SkImages, etc which is very slow.
> 
> Not sure, but we definitely don't need it in the long term.
> 
> The file copying doesn't really take too long compared to the run times of
these
> bots.  You have to have really good timing to catch the resources or skps
> copying, but yeah, the images can take a couple minutes.  
> 
> Let's land this as-is then follow up with a separate CL to re-enable the
> caching?  That way it's independently revertible.

SGTM

https://codereview.chromium.org/2320153002/diff/840001/infra/bots/recipes/swa...
File
infra/bots/recipes/swarm_test.expected/Test-Android-Clang-AndroidOne-CPU-MT6582-arm-Release-GN_Android.json
(right):

https://codereview.chromium.org/2320153002/diff/840001/infra/bots/recipes/swa...
infra/bots/recipes/swarm_test.expected/Test-Android-Clang-AndroidOne-CPU-MT6582-arm-Release-GN_Android.json:27:
"name": "TEMPORARY clear /data/local/tmp"
Still contains '/' which breaks log links.

[borenet, 2016-09-12 15:11:10]

LGTM per live conversation

[mtklein_C, 2016-09-12 15:14:23]

The CQ bit was unchecked by mtklein@chromium.org

[mtklein_C, 2016-09-12 15:14:24]

The CQ bit was checked by mtklein@chromium.org

[commit-bot: I haz the power, 2016-09-12 15:14:38]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-09-12 15:45:24]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-09-12 15:45:25]

Try jobs failed on following builders:
  Perf-Android-Clang-AndroidOne-CPU-MT6582-arm-Debug-GN_Android-Trybot on
master.client.skia.android (JOB_FAILED,
http://build.chromium.org/p/client.skia.android/builders/Perf-Android-Clang-A...)

[mtklein_C, 2016-09-12 15:51:44]

The CQ bit was checked by mtklein@chromium.org

[commit-bot: I haz the power, 2016-09-12 15:51:49]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-09-12 16:07:11]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-09-12 16:07:12]

Try jobs failed on following builders:
  Perf-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Debug-GN_Android-Trybot on
master.client.skia.android (JOB_FAILED,
http://build.chromium.org/p/client.skia.android/builders/Perf-Android-Clang-A...)
  Test-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Debug-GN_Android-Trybot on
master.client.skia.android (JOB_FAILED,
http://build.chromium.org/p/client.skia.android/builders/Test-Android-Clang-A...)

[mtklein_C, 2016-09-12 16:17:24]

The CQ bit was checked by mtklein@chromium.org

[commit-bot: I haz the power, 2016-09-12 16:17:33]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-09-12 16:26:38]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-09-12 16:26:40]

Try jobs failed on following builders:
  Test-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Release-GN_Android-Trybot on
master.client.skia.android (JOB_FAILED,
http://build.chromium.org/p/client.skia.android/builders/Test-Android-Clang-A...)

[mtklein_C, 2016-09-12 16:32:39]

The CQ bit was checked by mtklein@chromium.org

[commit-bot: I haz the power, 2016-09-12 16:32:43]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-09-12 16:34:13]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-09-12 16:34:15]

Try jobs failed on following builders:
  Test-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Debug-GN_Android-Trybot on
master.client.skia.android (JOB_FAILED,
http://build.chromium.org/p/client.skia.android/builders/Test-Android-Clang-A...)

[mtklein_C, 2016-09-12 16:38:32]

The CQ bit was checked by mtklein@chromium.org

[commit-bot: I haz the power, 2016-09-12 16:38:37]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-09-12 16:53:44]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-09-12 16:53:45]

Try jobs failed on following builders:
  Test-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Debug-GN_Android-Trybot on
master.client.skia.android (JOB_FAILED,
http://build.chromium.org/p/client.skia.android/builders/Test-Android-Clang-A...)

[mtklein_C, 2016-09-12 16:54:26]

The CQ bit was checked by mtklein@chromium.org

[commit-bot: I haz the power, 2016-09-12 16:54:36]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-09-12 17:09:10]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-09-12 17:09:12]

Try jobs failed on following builders:
  Test-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Debug-GN_Android-Trybot on
master.client.skia.android (JOB_FAILED,
http://build.chromium.org/p/client.skia.android/builders/Test-Android-Clang-A...)

[mtklein_C, 2016-09-12 17:10:44]

The CQ bit was checked by mtklein@chromium.org

[commit-bot: I haz the power, 2016-09-12 17:10:49]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-09-12 17:17:38]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-09-12 17:17:39]

Try jobs failed on following builders:
  Perf-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Release-GN_Android-Trybot on
master.client.skia.android (JOB_FAILED,
http://build.chromium.org/p/client.skia.android/builders/Perf-Android-Clang-A...)

[mtklein_C, 2016-09-12 17:19:05]

The CQ bit was checked by mtklein@chromium.org

[commit-bot: I haz the power, 2016-09-12 17:19:18]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-09-12 19:24:41]

Description was changed from

==========
GN: Android perf/tests

Just to take inventory of which old problems still linger and which are now
moot, I've gone out of my way to have this stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.  

The tricky bits turn out to be, copying directories containing symlinks---or
really any copying involving more than one file---and getting the exit code back
from adb shell.  Luckily the ADB I've got on my desktop and my Nexus 5x seems to
handle this all without the awkward workarounds you see here, so there's hope
that One Day Soon the weird parts (basically, anything with inline python) can
go away.  Once we've got these bots landed green, I'll go see whether the fixes
are due to ADB updates, Android updates, or perhaps something else like
hardware.

The parts marked TEMPORARY are a nod to the fact that the devices are used by
gn_android_flavor and android_flavor both today.  It's mostly about not stepping
on each other's toes or leaving anything laying around that might confuse each
other.  The marked parts can go away when bots are either gn_ or non-gn_ but not
both.

I have omitted a few steps that may be important, but which are easy independent
follow-ups:
  - running as root
  - locking clocks
  - waiting on battery levels
  - fancier wait-for-ready than adb wait-for-usb-device
It'd be nice to, e.g., reaffirm that locking clocks helps perf stability, and
that we're locking to the best policy.  I've tried to keep this CL as trim as
possible, leaving any of these vaguely optional steps for later.

As of PS 41 or so, it looks like the trybots are all behaving as expected.
We should expect no new images in Gold.  Can we see trybots in Perf yet?

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002

CQ_INCLUDE_TRYBOTS=master.client.skia.android:Perf-Android-Clang-AndroidOne-CPU-MT6582-arm-Debug-GN_Android-Trybot,Perf-Android-Clang-AndroidOne-CPU-MT6582-arm-Release-GN_Android-Trybot,Perf-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Debug-GN_Android-Trybot,Perf-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Release-GN_Android-Trybot,Test-Android-Clang-AndroidOne-CPU-MT6582-arm-Debug-GN_Android-Trybot,Test-Android-Clang-AndroidOne-CPU-MT6582-arm-Release-GN_Android-Trybot,Test-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Debug-GN_Android-Trybot,Test-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Release-GN_Android-Trybot
==========

to

==========
GN: Android perf/tests

Just to take inventory of which old problems still linger and which are now
moot, I've gone out of my way to have this stand alone.  All of
gn_android_flavor's logic is self contained, without any dependency on the
platform_tools scripts.  

The tricky bits turn out to be, copying directories containing symlinks---or
really any copying involving more than one file---and getting the exit code back
from adb shell.  Luckily the ADB I've got on my desktop and my Nexus 5x seems to
handle this all without the awkward workarounds you see here, so there's hope
that One Day Soon the weird parts (basically, anything with inline python) can
go away.  Once we've got these bots landed green, I'll go see whether the fixes
are due to ADB updates, Android updates, or perhaps something else like
hardware.

The parts marked TEMPORARY are a nod to the fact that the devices are used by
gn_android_flavor and android_flavor both today.  It's mostly about not stepping
on each other's toes or leaving anything laying around that might confuse each
other.  The marked parts can go away when bots are either gn_ or non-gn_ but not
both.

I have omitted a few steps that may be important, but which are easy independent
follow-ups:
  - running as root
  - locking clocks
  - waiting on battery levels
  - fancier wait-for-ready than adb wait-for-usb-device
It'd be nice to, e.g., reaffirm that locking clocks helps perf stability, and
that we're locking to the best policy.  I've tried to keep this CL as trim as
possible, leaving any of these vaguely optional steps for later.

As of PS 41 or so, it looks like the trybots are all behaving as expected.
We should expect no new images in Gold.  Can we see trybots in Perf yet?

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002

CQ_INCLUDE_TRYBOTS=master.client.skia.android:Perf-Android-Clang-AndroidOne-CPU-MT6582-arm-Debug-GN_Android-Trybot,Perf-Android-Clang-AndroidOne-CPU-MT6582-arm-Release-GN_Android-Trybot,Perf-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Debug-GN_Android-Trybot,Perf-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Release-GN_Android-Trybot,Test-Android-Clang-AndroidOne-CPU-MT6582-arm-Debug-GN_Android-Trybot,Test-Android-Clang-AndroidOne-CPU-MT6582-arm-Release-GN_Android-Trybot,Test-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Debug-GN_Android-Trybot,Test-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Release-GN_Android-Trybot

Committed:
https://skia.googlesource.com/skia/+/df5df81084a9b8652e6ed0987c2454c62d787188
==========

[commit-bot: I haz the power, 2016-09-12 19:24:42]

Committed patchset #45 (id:840001) as
https://skia.googlesource.com/skia/+/df5df81084a9b8652e6ed0987c2454c62d787188