Allow pyauto performance tests in perf.py to run as an autotest.

Allow pyauto performance tests in perf.py to run as an autotest.

This change adds autotest control files and an associated pyauto suite
for running the tests in perf.py as an autotest.  This change only
allows execution as an autotest, but does not enable the tests to run
continuously in the autotest lab.  That will be done in a later CL.

I also modified perf.py so that the v8 benchmark test only runs for a
single iteration, and for the other tests, the number of iterations
can optionally be tweaked by setting an environment variable (useful when
running the perf tests manually).

BUG=chromium-os:18187, chromium-os:18185, chromium-os:18458
TEST=None

Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=96461

[Nirnimesh, 2011-08-11 18:26:36]

http://codereview.chromium.org/7617014/diff/6/chrome/test/chromeos/autotest/f...
File
chrome/test/chromeos/autotest/files/client/site_tests/desktopui_PyAutoPerfTests/desktopui_PyAutoPerfTests.py
(right):

http://codereview.chromium.org/7617014/diff/6/chrome/test/chromeos/autotest/f...
chrome/test/chromeos/autotest/files/client/site_tests/desktopui_PyAutoPerfTests/desktopui_PyAutoPerfTests.py:5:
import dbus
unused

http://codereview.chromium.org/7617014/diff/6/chrome/test/functional/perf.py
File chrome/test/functional/perf.py (right):

http://codereview.chromium.org/7617014/diff/6/chrome/test/functional/perf.py#...
chrome/test/functional/perf.py:30: """Performs necessary setup work before
running each test."""
So if it's not defined in the environment, the test will not run anything?
That doesn't sound right. It should do the default 50, when nothing has been
provided, and then the environment var can be used to tweak it.

http://codereview.chromium.org/7617014/diff/6/chrome/test/functional/perf.py#...
chrome/test/functional/perf.py:34: self._num_trials =
int(os.environ['PYAUTO_PERF_TEST_NUM_TRIALS'])
That's a env var no one will remember. Use NUM_ITERATIONS

[dennis_jeffrey, 2011-08-11 19:29:31]

http://codereview.chromium.org/7617014/diff/6/chrome/test/chromeos/autotest/f...
File
chrome/test/chromeos/autotest/files/client/site_tests/desktopui_PyAutoPerfTests/desktopui_PyAutoPerfTests.py
(right):

http://codereview.chromium.org/7617014/diff/6/chrome/test/chromeos/autotest/f...
chrome/test/chromeos/autotest/files/client/site_tests/desktopui_PyAutoPerfTests/desktopui_PyAutoPerfTests.py:5:
import dbus
On 2011/08/11 18:26:36, Nirnimesh wrote:
> unused

Done.

http://codereview.chromium.org/7617014/diff/6/chrome/test/functional/perf.py
File chrome/test/functional/perf.py (right):

http://codereview.chromium.org/7617014/diff/6/chrome/test/functional/perf.py#...
chrome/test/functional/perf.py:30: """Performs necessary setup work before
running each test."""
On 2011/08/11 18:26:36, Nirnimesh wrote:
> So if it's not defined in the environment, the test will not run anything?

No, if that variable is not defined in the environment, it would have run a
single trial only (the default behavior).

> That doesn't sound right. It should do the default 50, when nothing has been
> provided, and then the environment var can be used to tweak it.

Changed the code based on our offline discussion about how many iterations each
test should run in the default case.

http://codereview.chromium.org/7617014/diff/6/chrome/test/functional/perf.py#...
chrome/test/functional/perf.py:34: self._num_trials =
int(os.environ['PYAUTO_PERF_TEST_NUM_TRIALS'])
On 2011/08/11 18:26:36, Nirnimesh wrote:
> That's a env var no one will remember. Use NUM_ITERATIONS

Done.

[dennis_jeffrey, 2011-08-11 20:42:51]

I found one minor bug after uploading my previous comments, which I have fixed
in the latest patch.  Sorry about catching it late.

[Nirnimesh, 2011-08-11 21:21:23]

LGTM

http://codereview.chromium.org/7617014/diff/8001/chrome/test/functional/perf.py
File chrome/test/functional/perf.py (right):

http://codereview.chromium.org/7617014/diff/8001/chrome/test/functional/perf....
chrome/test/functional/perf.py:150: self.WaitUntil(
wrap this inside assertTrue()

[dennis_jeffrey, 2011-08-11 21:39:51]

Thank you for reviewing this code.  I addressed the last comment and will submit
now.

http://codereview.chromium.org/7617014/diff/8001/chrome/test/functional/perf.py
File chrome/test/functional/perf.py (right):

http://codereview.chromium.org/7617014/diff/8001/chrome/test/functional/perf....
chrome/test/functional/perf.py:150: self.WaitUntil(
On 2011/08/11 21:21:23, Nirnimesh wrote:
> wrap this inside assertTrue()

Done.