Migrate cluster telemetry benchmarks to tools/perf/contrib/cluster_telemetry/

Migrate cluster telemetry benchmarks to tools/perf/contrib/cluster_telemetry/ 


*Notes to reviewers:
1) Main files to review: tools/perf/cluster_telemetry/OWNERS
2) Logical move includes:
i) Straight move of multipage_skpicture_printer.py, 
rasterize_and_record_micro.py, repaint.py, repaint_unittest.py, 
repaint_helpers.py, skpicture_printer.py to contrib/cluster_telemetry/
ii) Move part of loading.py contrib/cluster_telemetry/loading_ct.py

Note: this also include removes referencing to repaint module in 
benchmark_smoke_unittest. This is actually irrelevant since the repaint module
no longer contains locally runnable benchmarks after https://codereview.chromium.org/2874043003/

BUG=715704
Review-Url: https://codereview.chromium.org/2876073003
Cr-Commit-Position: refs/heads/master@{#471431}
Committed: https://chromium.googlesource.com/chromium/src/+/59b0d3260589cd8a1328e7d533752619687fb6fa

[nednguyen, 2017-05-12 15:17:59]

nednguyen@google.com changed reviewers:
+ rmistry@chromium.org, wkorman@chromium.org

[nednguyen, 2017-05-12 15:18:35]

On 2017/05/12 15:17:59, nednguyen wrote:

Note that I remove all the @Disabled('all') decorators because benchmarks in
contrib/ are not scheduled on perf waterfall by default.

[nednguyen, 2017-05-12 15:21:49]

Description was changed from

==========
Migrate cluster telemetry benchmarks to tools/perf/contrib/ directory

BUG=715704
==========

to

==========
Migrate cluster telemetry benchmarks to tools/perf/contrib/cluster_telemetry/ 


*Notes to reviewers:
1) Main files to review: tools/perf/cluster_telemetry/OWNERS
2) Logical move includes:
i) Straight move of multipage_skpicture_printer.py,
rasterize_and_record_micro.py, repaint.py, repaint_unittest.py,
repaint_helpers.py, skpicture_printer.py to contrib/cluster_telemetry/
ii) Move part of loading.py contrib/cluster_telemetry/loading_ct.py

BUG=715704
==========

[nednguyen, 2017-05-12 15:22:07]

Description was changed from

==========
Migrate cluster telemetry benchmarks to tools/perf/contrib/cluster_telemetry/ 


*Notes to reviewers:
1) Main files to review: tools/perf/cluster_telemetry/OWNERS
2) Logical move includes:
i) Straight move of multipage_skpicture_printer.py,
rasterize_and_record_micro.py, repaint.py, repaint_unittest.py,
repaint_helpers.py, skpicture_printer.py to contrib/cluster_telemetry/
ii) Move part of loading.py contrib/cluster_telemetry/loading_ct.py

BUG=715704
==========

to

==========
Migrate cluster telemetry benchmarks to tools/perf/contrib/cluster_telemetry/ 


*Notes to reviewers:
1) Main files to review: tools/perf/cluster_telemetry/OWNERS
2) Logical move includes:
i) Straight move of multipage_skpicture_printer.py, 
rasterize_and_record_micro.py, repaint.py, repaint_unittest.py, 
repaint_helpers.py, skpicture_printer.py to contrib/cluster_telemetry/
ii) Move part of loading.py contrib/cluster_telemetry/loading_ct.py

BUG=715704
==========

[nednguyen, 2017-05-12 15:27:30]

The CQ bit was checked by nednguyen@google.com to run a CQ dry run

[commit-bot: I haz the power, 2017-05-12 15:28:20]

Dry run: CQ is trying da patch.

Follow status at:
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[rmistry, 2017-05-12 16:27:43]

rmistry@google.com changed reviewers:
+ rmistry@google.com

[rmistry, 2017-05-12 16:27:44]

lgtm


Here is an example command run by CT:

PYTHONPATH=/b/storage/page_sets/Dummy1k:/b/storage/chromium/src/tools/perf:/b/storage/chromium/src/tools/telemetry:/b/storage/chromium/src/third_party/catapult:$PYTHONPATH
DISPLAY=:0 
python /b/storage/chromium/src/tools/perf/run_benchmark
rasterize_and_record_micro_ct \
  --also-run-disabled-tests \
  --user-agent=mobile \
  --urls-list=http://www.chase.com \
  --archive-data-file=/b/storage/webpage_archives/Dummy1k/101.json \
  --output-dir=/b/storage/tasks/benchmark_runs/rmistry-20170511175847/101 \
  --browser=exact \
 
--browser-executable=/b/storage/chromium_builds/try-dedb6d42c8dfc3-a9d9ab368318cd-rmistry-20170511175847-withpatch/chrome
\
  --device=desktop \
  --output-format=csv-pivot-table \
  --pageset-repeat=1

Does anything need to change after this lands?

[nednguyen, 2017-05-12 16:29:58]

On 2017/05/12 16:27:44, rmistry wrote:
> lgtm
> 
> 
> Here is an example command run by CT:
> 
>
PYTHONPATH=/b/storage/page_sets/Dummy1k:/b/storage/chromium/src/tools/perf:/b/storage/chromium/src/tools/telemetry:/b/storage/chromium/src/third_party/catapult:$PYTHONPATH
> DISPLAY=:0 
> python /b/storage/chromium/src/tools/perf/run_benchmark
> rasterize_and_record_micro_ct \
>   --also-run-disabled-tests \
>   --user-agent=mobile \
>   --urls-list=http://www.chase.com \
>   --archive-data-file=/b/storage/webpage_archives/Dummy1k/101.json \
>   --output-dir=/b/storage/tasks/benchmark_runs/rmistry-20170511175847/101 \
>   --browser=exact \
>  
>
--browser-executable=/b/storage/chromium_builds/try-dedb6d42c8dfc3-a9d9ab368318cd-rmistry-20170511175847-withpatch/chrome
> \
>   --device=desktop \
>   --output-format=csv-pivot-table \
>   --pageset-repeat=1
> 
> Does anything need to change after this lands?

Since the command was referencing absolute paths, I think there is nothing need
to change after this land except you can remove "--also-run-disabled-tests" flag
now.

[rmistry, 2017-05-12 16:32:53]

On 2017/05/12 16:29:58, nednguyen wrote:
> On 2017/05/12 16:27:44, rmistry wrote:
> > lgtm
> > 
> > 
> > Here is an example command run by CT:
> > 
> >
>
PYTHONPATH=/b/storage/page_sets/Dummy1k:/b/storage/chromium/src/tools/perf:/b/storage/chromium/src/tools/telemetry:/b/storage/chromium/src/third_party/catapult:$PYTHONPATH
> > DISPLAY=:0 
> > python /b/storage/chromium/src/tools/perf/run_benchmark
> > rasterize_and_record_micro_ct \
> >   --also-run-disabled-tests \
> >   --user-agent=mobile \
> >   --urls-list=http://www.chase.com \
> >   --archive-data-file=/b/storage/webpage_archives/Dummy1k/101.json \
> >   --output-dir=/b/storage/tasks/benchmark_runs/rmistry-20170511175847/101 \
> >   --browser=exact \
> >  
> >
>
--browser-executable=/b/storage/chromium_builds/try-dedb6d42c8dfc3-a9d9ab368318cd-rmistry-20170511175847-withpatch/chrome
> > \
> >   --device=desktop \
> >   --output-format=csv-pivot-table \
> >   --pageset-repeat=1
> > 
> > Does anything need to change after this lands?
> 
> Since the command was referencing absolute paths, I think there is nothing
need
> to change after this land except you can remove "--also-run-disabled-tests"
flag
> now.

lgtm'er

[commit-bot: I haz the power, 2017-05-12 16:44:50]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-05-12 16:44:51]

Dry run: This issue passed the CQ dry run.

[wkorman, 2017-05-12 17:44:24]

lgtm

https://codereview.chromium.org/2876073003/diff/40001/tools/perf/benchmarks/b...
File tools/perf/benchmarks/benchmark_smoke_unittest.py (left):

https://codereview.chromium.org/2876073003/diff/40001/tools/perf/benchmarks/b...
tools/perf/benchmarks/benchmark_smoke_unittest.py:96: repaint,  # Often fails &
takes long time to timeout on cq bot.
This seems worth noting in the CL description. If we fix the flakiness bug
http://crbug.com/715962 would we want to restore the benchmark to run as a smoke
test to catch regression?

https://codereview.chromium.org/2876073003/diff/40001/tools/perf/contrib/clus...
File tools/perf/contrib/cluster_telemetry/repaint.py (right):

https://codereview.chromium.org/2876073003/diff/40001/tools/perf/contrib/clus...
tools/perf/contrib/cluster_telemetry/repaint.py:6: from
contrib.cluster_telemetry import page_set
why rename to singular? Just curious.

[nednguyen, 2017-05-12 20:17:03]

https://codereview.chromium.org/2876073003/diff/40001/tools/perf/benchmarks/b...
File tools/perf/benchmarks/benchmark_smoke_unittest.py (left):

https://codereview.chromium.org/2876073003/diff/40001/tools/perf/benchmarks/b...
tools/perf/benchmarks/benchmark_smoke_unittest.py:96: repaint,  # Often fails &
takes long time to timeout on cq bot.
On 2017/05/12 17:44:23, wkorman wrote:
> This seems worth noting in the CL description. If we fix the flakiness bug
> http://crbug.com/715962 would we want to restore the benchmark to run as a
smoke
> test to catch regression?

Oh, this is smoke test of repaint benchmarks that are run on perf waterfall,
which is no longer relevant after your removal of them in
https://codereview.chromium.org/2874043003/

https://codereview.chromium.org/2876073003/diff/40001/tools/perf/contrib/clus...
File tools/perf/contrib/cluster_telemetry/repaint.py (right):

https://codereview.chromium.org/2876073003/diff/40001/tools/perf/contrib/clus...
tools/perf/contrib/cluster_telemetry/repaint.py:6: from
contrib.cluster_telemetry import page_set
On 2017/05/12 17:44:23, wkorman wrote:
> why rename to singular? Just curious.

Ah, back then page_sets mean to refer to
https://cs.chromium.org/chromium/src/tools/perf/page_sets/ which host many
page_set. Now it's a single module that hosts only cluster_telemetry page_set

https://codereview.chromium.org/2876073003/diff/40001/tools/perf/contrib/clus...
File tools/perf/contrib/cluster_telemetry/repaint_unittest.py (right):

https://codereview.chromium.org/2876073003/diff/40001/tools/perf/contrib/clus...
tools/perf/contrib/cluster_telemetry/repaint_unittest.py:40:
@decorators.Disabled("all") # crbug.com/715962
Walter: this is the actual smoke coverage of repaint now. Though you may want to
look at crbug.com/715962

[nednguyen, 2017-05-12 20:20:30]

On 2017/05/12 20:17:03, nednguyen wrote:
>
https://codereview.chromium.org/2876073003/diff/40001/tools/perf/benchmarks/b...
> File tools/perf/benchmarks/benchmark_smoke_unittest.py (left):
> 
>
https://codereview.chromium.org/2876073003/diff/40001/tools/perf/benchmarks/b...
> tools/perf/benchmarks/benchmark_smoke_unittest.py:96: repaint,  # Often fails
&
> takes long time to timeout on cq bot.
> On 2017/05/12 17:44:23, wkorman wrote:
> > This seems worth noting in the CL description. If we fix the flakiness bug
> > http://crbug.com/715962 would we want to restore the benchmark to run as a
> smoke
> > test to catch regression?
> 
> Oh, this is smoke test of repaint benchmarks that are run on perf waterfall,
> which is no longer relevant after your removal of them in
> https://codereview.chromium.org/2874043003/
> 
>
https://codereview.chromium.org/2876073003/diff/40001/tools/perf/contrib/clus...
> File tools/perf/contrib/cluster_telemetry/repaint.py (right):
> 
>
https://codereview.chromium.org/2876073003/diff/40001/tools/perf/contrib/clus...
> tools/perf/contrib/cluster_telemetry/repaint.py:6: from
> contrib.cluster_telemetry import page_set
> On 2017/05/12 17:44:23, wkorman wrote:
> > why rename to singular? Just curious.
> 
> Ah, back then page_sets mean to refer to
> https://cs.chromium.org/chromium/src/tools/perf/page_sets/ which host many
> page_set. Now it's a single module that hosts only cluster_telemetry page_set
> 
>
https://codereview.chromium.org/2876073003/diff/40001/tools/perf/contrib/clus...
> File tools/perf/contrib/cluster_telemetry/repaint_unittest.py (right):
> 
>
https://codereview.chromium.org/2876073003/diff/40001/tools/perf/contrib/clus...
> tools/perf/contrib/cluster_telemetry/repaint_unittest.py:40:
> @decorators.Disabled("all") # crbug.com/715962
> Walter: this is the actual smoke coverage of repaint now. Though you may want
to
> look at crbug.com/715962

Since the CT experiment run seems to succeeded (produced data in
https://storage.cloud.google.com/cluster-telemetry/tasks/benchmark_runs/rmist...),
I am landing this CL now.

[nednguyen, 2017-05-12 20:21:42]

Description was changed from

==========
Migrate cluster telemetry benchmarks to tools/perf/contrib/cluster_telemetry/ 


*Notes to reviewers:
1) Main files to review: tools/perf/cluster_telemetry/OWNERS
2) Logical move includes:
i) Straight move of multipage_skpicture_printer.py, 
rasterize_and_record_micro.py, repaint.py, repaint_unittest.py, 
repaint_helpers.py, skpicture_printer.py to contrib/cluster_telemetry/
ii) Move part of loading.py contrib/cluster_telemetry/loading_ct.py

BUG=715704
==========

to

==========
Migrate cluster telemetry benchmarks to tools/perf/contrib/cluster_telemetry/ 


*Notes to reviewers:
1) Main files to review: tools/perf/cluster_telemetry/OWNERS
2) Logical move includes:
i) Straight move of multipage_skpicture_printer.py, 
rasterize_and_record_micro.py, repaint.py, repaint_unittest.py, 
repaint_helpers.py, skpicture_printer.py to contrib/cluster_telemetry/
ii) Move part of loading.py contrib/cluster_telemetry/loading_ct.py

Note: this also include removes referencing to repaint module in 
benchmark_smoke_unittest. This is actually irrelevant since the repaint module
no longer contains locally runnable benchmarks after
https://codereview.chromium.org/2874043003/

BUG=715704
==========

[nednguyen, 2017-05-12 20:21:54]

The CQ bit was checked by nednguyen@google.com

[commit-bot: I haz the power, 2017-05-12 20:22:31]

CQ is trying da patch.

Follow status at:
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-05-12 20:28:38]

CQ is committing da patch.

Bot data: {"patchset_id": 40001, "attempt_start_ts": 1494620514499470,
"parent_rev": "143d0a7278a1f2b6ff9173ec78e6755b7100704c", "commit_rev":
"59b0d3260589cd8a1328e7d533752619687fb6fa"}

[commit-bot: I haz the power, 2017-05-12 20:28:50]

Description was changed from

==========
Migrate cluster telemetry benchmarks to tools/perf/contrib/cluster_telemetry/ 


*Notes to reviewers:
1) Main files to review: tools/perf/cluster_telemetry/OWNERS
2) Logical move includes:
i) Straight move of multipage_skpicture_printer.py, 
rasterize_and_record_micro.py, repaint.py, repaint_unittest.py, 
repaint_helpers.py, skpicture_printer.py to contrib/cluster_telemetry/
ii) Move part of loading.py contrib/cluster_telemetry/loading_ct.py

Note: this also include removes referencing to repaint module in 
benchmark_smoke_unittest. This is actually irrelevant since the repaint module
no longer contains locally runnable benchmarks after
https://codereview.chromium.org/2874043003/

BUG=715704
==========

to

==========
Migrate cluster telemetry benchmarks to tools/perf/contrib/cluster_telemetry/ 


*Notes to reviewers:
1) Main files to review: tools/perf/cluster_telemetry/OWNERS
2) Logical move includes:
i) Straight move of multipage_skpicture_printer.py, 
rasterize_and_record_micro.py, repaint.py, repaint_unittest.py, 
repaint_helpers.py, skpicture_printer.py to contrib/cluster_telemetry/
ii) Move part of loading.py contrib/cluster_telemetry/loading_ct.py

Note: this also include removes referencing to repaint module in 
benchmark_smoke_unittest. This is actually irrelevant since the repaint module
no longer contains locally runnable benchmarks after
https://codereview.chromium.org/2874043003/

BUG=715704

Review-Url: https://codereview.chromium.org/2876073003
Cr-Commit-Position: refs/heads/master@{#471431}
Committed:
https://chromium.googlesource.com/chromium/src/+/59b0d3260589cd8a1328e7d53375...
==========

[commit-bot: I haz the power, 2017-05-12 20:28:52]

Committed patchset #3 (id:40001) as
https://chromium.googlesource.com/chromium/src/+/59b0d3260589cd8a1328e7d53375...