Backend for new run_chromium_analysis CT task

Backend for new run_chromium_analysis CT task.

Tested with:
$ go run go/worker_scripts/run_chromium_analysis/main.go --logtostderr --start_range=1 --num=3 --pageset_type=Dummy1k --chromium_build=try-1e224f6-f1036b2-rmistry-20160206122201-withpatch --run_id=testing --benchmark_name=rasterize_and_record_micro_ct --benchmark_extra_args="--output-format=csv-pivot-table" --browser_extra_args="--disable-setuid-sandbox --enable-threaded-compositing --enable-impl-side-painting" --local

$ go run go/master_scripts/run_chromium_analysis_on_workers/main.go --logtostderr --emails="rmistry@google.com" --pageset_type="Dummy1k" --run_id="testing" --benchmark_name="rasterize_and_record_micro_ct" --benchmark_extra_args="--output-format=csv-pivot-table" --browser_extra_args="--disable-setuid-sandbox --enable-threaded-compositing --enable-impl-side-painting"

Results are available here: https://storage.cloud.google.com/cluster-telemetry/tasks/benchmark_runs/testing/consolidated_outputs/testing.output


BUG=skia:5316
Committed: https://skia.googlesource.com/buildbot/+/c5c7a6e7a57fb98634eedbd38909ca4a470a77e8

[rmistry, 2016-05-19 17:54:12]

Patchset #6 (id:100001) has been deleted

[rmistry, 2016-05-19 18:00:00]

Description was changed from

==========
Backend for new run_chromium_analysis CT task

BUG=skia:5316
==========

to

==========
Backend for new run_chromium_analysis CT task.

Tested with:
$ go run go/worker_scripts/run_chromium_analysis/main.go --logtostderr
--start_range=1 --num=3 --pageset_type=Dummy1k
--chromium_build=try-1e224f6-f1036b2-rmistry-20160206122201-withpatch
--run_id=testing --benchmark_name=rasterize_and_record_micro_ct
--benchmark_extra_args="--output-format=csv-pivot-table"
--browser_extra_args="--disable-setuid-sandbox --enable-threaded-compositing
--enable-impl-side-painting" --local

$ go run go/master_scripts/run_chromium_analysis_on_workers/main.go
--logtostderr --emails="rmistry@google.com" --pageset_type="Dummy1k"
--run_id="testing" --benchmark_name="rasterize_and_record_micro_ct"
--benchmark_extra_args="--output-format=csv-pivot-table"
--browser_extra_args="--disable-setuid-sandbox --enable-threaded-compositing
--enable-impl-side-painting"

This created the CSV output here:



BUG=skia:5316
==========

[rmistry, 2016-05-19 18:02:51]

Description was changed from

==========
Backend for new run_chromium_analysis CT task.

Tested with:
$ go run go/worker_scripts/run_chromium_analysis/main.go --logtostderr
--start_range=1 --num=3 --pageset_type=Dummy1k
--chromium_build=try-1e224f6-f1036b2-rmistry-20160206122201-withpatch
--run_id=testing --benchmark_name=rasterize_and_record_micro_ct
--benchmark_extra_args="--output-format=csv-pivot-table"
--browser_extra_args="--disable-setuid-sandbox --enable-threaded-compositing
--enable-impl-side-painting" --local

$ go run go/master_scripts/run_chromium_analysis_on_workers/main.go
--logtostderr --emails="rmistry@google.com" --pageset_type="Dummy1k"
--run_id="testing" --benchmark_name="rasterize_and_record_micro_ct"
--benchmark_extra_args="--output-format=csv-pivot-table"
--browser_extra_args="--disable-setuid-sandbox --enable-threaded-compositing
--enable-impl-side-painting"

This created the CSV output here:



BUG=skia:5316
==========

to

==========
Backend for new run_chromium_analysis CT task.

Tested with:
$ go run go/worker_scripts/run_chromium_analysis/main.go --logtostderr
--start_range=1 --num=3 --pageset_type=Dummy1k
--chromium_build=try-1e224f6-f1036b2-rmistry-20160206122201-withpatch
--run_id=testing --benchmark_name=rasterize_and_record_micro_ct
--benchmark_extra_args="--output-format=csv-pivot-table"
--browser_extra_args="--disable-setuid-sandbox --enable-threaded-compositing
--enable-impl-side-painting" --local

$ go run go/master_scripts/run_chromium_analysis_on_workers/main.go
--logtostderr --emails="rmistry@google.com" --pageset_type="Dummy1k"
--run_id="testing" --benchmark_name="rasterize_and_record_micro_ct"
--benchmark_extra_args="--output-format=csv-pivot-table"
--browser_extra_args="--disable-setuid-sandbox --enable-threaded-compositing
--enable-impl-side-painting"


BUG=skia:5316
==========

[rmistry, 2016-05-19 18:08:35]

rmistry@google.com changed reviewers:
+ benjaminwagner@google.com

[rmistry, 2016-05-19 18:08:36]

util.go looks scarier than it is. It contains methods extracted out of
run_chromium_perf and then slightly modified. Eventually when run_chromium_perf
is on swarming it will use these new methods.
To make reviewing them easy I uploaded patchsets with no changes. The following
links will show the right diff for those methods.

MergeUploadCSVFiles:
https://codereview.chromium.org/1995023002/diff2/20001:40001/ct/go/util/util.go

RunBenchmark:
https://codereview.chromium.org/1995023002/diff2/60001:80001/ct/go/util/util.go

MergeUploadCSVFilesOnWorkers, getRowsFromCSV, writeRowsToCSV:
https://codereview.chromium.org/1995023002/diff2/80001:120001/ct/go/util/util.go

[rmistry, 2016-05-19 18:08:50]

The CQ bit was checked by rmistry@google.com to run a CQ dry run

[commit-bot: I haz the power, 2016-05-19 18:08:56]

Dry run: CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1995023002/120001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1995023002/120001

[commit-bot: I haz the power, 2016-05-19 18:14:23]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-05-19 18:14:24]

Dry run: Try jobs failed on following builders:
  Infra-PerCommit-Trybot on client.skia.fyi (JOB_FAILED,
http://build.chromium.org/p/client.skia.fyi/builders/Infra-PerCommit-Trybot/b...)

[rmistry, 2016-05-19 18:15:35]

Description was changed from

==========
Backend for new run_chromium_analysis CT task.

Tested with:
$ go run go/worker_scripts/run_chromium_analysis/main.go --logtostderr
--start_range=1 --num=3 --pageset_type=Dummy1k
--chromium_build=try-1e224f6-f1036b2-rmistry-20160206122201-withpatch
--run_id=testing --benchmark_name=rasterize_and_record_micro_ct
--benchmark_extra_args="--output-format=csv-pivot-table"
--browser_extra_args="--disable-setuid-sandbox --enable-threaded-compositing
--enable-impl-side-painting" --local

$ go run go/master_scripts/run_chromium_analysis_on_workers/main.go
--logtostderr --emails="rmistry@google.com" --pageset_type="Dummy1k"
--run_id="testing" --benchmark_name="rasterize_and_record_micro_ct"
--benchmark_extra_args="--output-format=csv-pivot-table"
--browser_extra_args="--disable-setuid-sandbox --enable-threaded-compositing
--enable-impl-side-painting"


BUG=skia:5316
==========

to

==========
Backend for new run_chromium_analysis CT task.

Tested with:
$ go run go/worker_scripts/run_chromium_analysis/main.go --logtostderr
--start_range=1 --num=3 --pageset_type=Dummy1k
--chromium_build=try-1e224f6-f1036b2-rmistry-20160206122201-withpatch
--run_id=testing --benchmark_name=rasterize_and_record_micro_ct
--benchmark_extra_args="--output-format=csv-pivot-table"
--browser_extra_args="--disable-setuid-sandbox --enable-threaded-compositing
--enable-impl-side-painting" --local

$ go run go/master_scripts/run_chromium_analysis_on_workers/main.go
--logtostderr --emails="rmistry@google.com" --pageset_type="Dummy1k"
--run_id="testing" --benchmark_name="rasterize_and_record_micro_ct"
--benchmark_extra_args="--output-format=csv-pivot-table"
--browser_extra_args="--disable-setuid-sandbox --enable-threaded-compositing
--enable-impl-side-painting"

Results are available here:
https://storage.cloud.google.com/cluster-telemetry/tasks/benchmark_runs/testi...


BUG=skia:5316
==========

[rmistry, 2016-05-19 18:17:07]

The CQ bit was checked by rmistry@google.com to run a CQ dry run

[commit-bot: I haz the power, 2016-05-19 18:17:12]

Dry run: CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1995023002/140001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1995023002/140001

[commit-bot: I haz the power, 2016-05-19 18:21:43]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-05-19 18:21:44]

Dry run: This issue passed the CQ dry run.

[dogben, 2016-05-19 21:06:41]

lgtm

Only ignorable comments. Hooray!

I only reviewed the diffs you pointed to for util.go.

https://codereview.chromium.org/1995023002/diff/160001/ct/go/master_scripts/r...
File ct/go/master_scripts/run_chromium_analysis_on_workers/main.go (right):

https://codereview.chromium.org/1995023002/diff/160001/ct/go/master_scripts/r...
ct/go/master_scripts/run_chromium_analysis_on_workers/main.go:30: gaeTaskID     
    = flag.Int64("gae_task_id", -1, "The key of the App Engine task. This task
will be updated when the task is completed.")
Should probably update the doc string, if not the name.

https://codereview.chromium.org/1995023002/diff/160001/ct/go/master_scripts/r...
ct/go/master_scripts/run_chromium_analysis_on_workers/main.go:65: The chromium
perf %s benchmark task on %s pageset has completed.<br/>
s/perf/analysis/

https://codereview.chromium.org/1995023002/diff/160001/ct/go/master_scripts/r...
ct/go/master_scripts/run_chromium_analysis_on_workers/main.go:112: defer
skutil.RemoveAll(filepath.Join(util.StorageDir, util.ChromiumPerfRunsDir))
ChromiumPerfRunsDir is correct?

https://codereview.chromium.org/1995023002/diff/160001/ct/go/worker_scripts/r...
File ct/go/worker_scripts/run_chromium_analysis/main.go (right):

https://codereview.chromium.org/1995023002/diff/160001/ct/go/worker_scripts/r...
ct/go/worker_scripts/run_chromium_analysis/main.go:32: pagesetType        =
flag.String("pageset_type", util.PAGESET_TYPE_MOBILE_10k, "The type of pagesets
to create from the Alexa CSV list. Eg: 10k, Mobile10k, All.")
s/to create from the Alexa CSV list/to analyze/

https://codereview.chromium.org/1995023002/diff/160001/ct/go/worker_scripts/r...
ct/go/worker_scripts/run_chromium_analysis/main.go:44: defer
util.TimeTrack(time.Now(), "Running Chromium Perf")
s/Perf/Analysis/

https://codereview.chromium.org/1995023002/diff/160001/ct/go/worker_scripts/r...
ct/go/worker_scripts/run_chromium_analysis/main.go:81: benchmarkPatchLocalPath
:= filepath.Join(os.TempDir(), benchmarkPatchName)
nit: ioutil.TempDir or ioutil.TempFile

https://codereview.chromium.org/1995023002/diff/160001/ct/go/worker_scripts/r...
ct/go/worker_scripts/run_chromium_analysis/main.go:95: if err :=
ioutil.WriteFile(benchmarkPatchLocalPath, buf.Bytes(), 0666); err != nil {
nit: os.Create + io.Copy is probably slightly better.

https://codereview.chromium.org/1995023002/diff/160001/ct/go/worker_scripts/r...
ct/go/worker_scripts/run_chromium_analysis/main.go:101: if buf.Len() > 10 {
(io.Copy returns the number of bytes copied)

https://codereview.chromium.org/1995023002/diff/160001/ct/go/worker_scripts/r...
ct/go/worker_scripts/run_chromium_analysis/main.go:167: // Create and run a
goroutine closure that captures SKPs.
nit: update comment

[rmistry, 2016-05-20 11:07:16]

https://codereview.chromium.org/1995023002/diff/160001/ct/go/master_scripts/r...
File ct/go/master_scripts/run_chromium_analysis_on_workers/main.go (right):

https://codereview.chromium.org/1995023002/diff/160001/ct/go/master_scripts/r...
ct/go/master_scripts/run_chromium_analysis_on_workers/main.go:30: gaeTaskID     
    = flag.Int64("gae_task_id", -1, "The key of the App Engine task. This task
will be updated when the task is completed.")
On 2016/05/19 21:06:41, Ben Wagner wrote:
> Should probably update the doc string, if not the name.

Ah looks like all master scripts have this. Will make a note to mass update them
later. Updating doc string only for now.

https://codereview.chromium.org/1995023002/diff/160001/ct/go/master_scripts/r...
ct/go/master_scripts/run_chromium_analysis_on_workers/main.go:65: The chromium
perf %s benchmark task on %s pageset has completed.<br/>
On 2016/05/19 21:06:40, Ben Wagner wrote:
> s/perf/analysis/

Done.

https://codereview.chromium.org/1995023002/diff/160001/ct/go/master_scripts/r...
ct/go/master_scripts/run_chromium_analysis_on_workers/main.go:112: defer
skutil.RemoveAll(filepath.Join(util.StorageDir, util.ChromiumPerfRunsDir))
On 2016/05/19 21:06:41, Ben Wagner wrote:
> ChromiumPerfRunsDir is correct?

Changed. Done.

https://codereview.chromium.org/1995023002/diff/160001/ct/go/worker_scripts/r...
File ct/go/worker_scripts/run_chromium_analysis/main.go (right):

https://codereview.chromium.org/1995023002/diff/160001/ct/go/worker_scripts/r...
ct/go/worker_scripts/run_chromium_analysis/main.go:32: pagesetType        =
flag.String("pageset_type", util.PAGESET_TYPE_MOBILE_10k, "The type of pagesets
to create from the Alexa CSV list. Eg: 10k, Mobile10k, All.")
On 2016/05/19 21:06:41, Ben Wagner wrote:
> s/to create from the Alexa CSV list/to analyze/

Done.

https://codereview.chromium.org/1995023002/diff/160001/ct/go/worker_scripts/r...
ct/go/worker_scripts/run_chromium_analysis/main.go:44: defer
util.TimeTrack(time.Now(), "Running Chromium Perf")
On 2016/05/19 21:06:41, Ben Wagner wrote:
> s/Perf/Analysis/

Done.

https://codereview.chromium.org/1995023002/diff/160001/ct/go/worker_scripts/r...
ct/go/worker_scripts/run_chromium_analysis/main.go:81: benchmarkPatchLocalPath
:= filepath.Join(os.TempDir(), benchmarkPatchName)
On 2016/05/19 21:06:41, Ben Wagner wrote:
> nit: ioutil.TempDir or ioutil.TempFile

Done.

https://codereview.chromium.org/1995023002/diff/160001/ct/go/worker_scripts/r...
ct/go/worker_scripts/run_chromium_analysis/main.go:95: if err :=
ioutil.WriteFile(benchmarkPatchLocalPath, buf.Bytes(), 0666); err != nil {
On 2016/05/19 21:06:41, Ben Wagner wrote:
> nit: os.Create + io.Copy is probably slightly better.

This is also how the poller creates local patches, leaving it the same to be
slightly consistent.

https://codereview.chromium.org/1995023002/diff/160001/ct/go/worker_scripts/r...
ct/go/worker_scripts/run_chromium_analysis/main.go:101: if buf.Len() > 10 {
On 2016/05/19 21:06:41, Ben Wagner wrote:
> (io.Copy returns the number of bytes copied)

Acknowledged.

https://codereview.chromium.org/1995023002/diff/160001/ct/go/worker_scripts/r...
ct/go/worker_scripts/run_chromium_analysis/main.go:167: // Create and run a
goroutine closure that captures SKPs.
On 2016/05/19 21:06:41, Ben Wagner wrote:
> nit: update comment

Done.

[dogben, 2016-05-20 12:45:34]

lgtm

Still LGTM.

https://codereview.chromium.org/1995023002/diff/160001/ct/go/master_scripts/r...
File ct/go/master_scripts/run_chromium_analysis_on_workers/main.go (right):

https://codereview.chromium.org/1995023002/diff/160001/ct/go/master_scripts/r...
ct/go/master_scripts/run_chromium_analysis_on_workers/main.go:112: defer
skutil.RemoveAll(filepath.Join(util.StorageDir, util.ChromiumPerfRunsDir))
On 2016/05/20 11:07:16, rmistry wrote:
> On 2016/05/19 21:06:41, Ben Wagner wrote:
> > ChromiumPerfRunsDir is correct?
> 
> Changed. Done.

Actually, is there anything that gets written here?

[rmistry, 2016-05-20 13:00:31]

https://codereview.chromium.org/1995023002/diff/160001/ct/go/master_scripts/r...
File ct/go/master_scripts/run_chromium_analysis_on_workers/main.go (right):

https://codereview.chromium.org/1995023002/diff/160001/ct/go/master_scripts/r...
ct/go/master_scripts/run_chromium_analysis_on_workers/main.go:112: defer
skutil.RemoveAll(filepath.Join(util.StorageDir, util.ChromiumPerfRunsDir))
On 2016/05/20 12:45:34, Ben Wagner wrote:
> On 2016/05/20 11:07:16, rmistry wrote:
> > On 2016/05/19 21:06:41, Ben Wagner wrote:
> > > ChromiumPerfRunsDir is correct?
> > 
> > Changed. Done.
> 
> Actually, is there anything that gets written here?

Looks like the local dir is not used. Removed this line. The remote dir is still
constructed by using util.ChromiumAnalysisRunsDir.

[rmistry, 2016-05-20 13:00:57]

The CQ bit was checked by rmistry@google.com

[rmistry, 2016-05-20 13:00:57]

The patchset sent to the CQ was uploaded after l-g-t-m from
benjaminwagner@google.com
Link to the patchset: https://codereview.chromium.org/1995023002/#ps200001
(title: "Local ChromiumAnalysisRunsDir is not used")

[commit-bot: I haz the power, 2016-05-20 13:01:06]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1995023002/200001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1995023002/200001

[commit-bot: I haz the power, 2016-05-20 13:04:24]

Description was changed from

==========
Backend for new run_chromium_analysis CT task.

Tested with:
$ go run go/worker_scripts/run_chromium_analysis/main.go --logtostderr
--start_range=1 --num=3 --pageset_type=Dummy1k
--chromium_build=try-1e224f6-f1036b2-rmistry-20160206122201-withpatch
--run_id=testing --benchmark_name=rasterize_and_record_micro_ct
--benchmark_extra_args="--output-format=csv-pivot-table"
--browser_extra_args="--disable-setuid-sandbox --enable-threaded-compositing
--enable-impl-side-painting" --local

$ go run go/master_scripts/run_chromium_analysis_on_workers/main.go
--logtostderr --emails="rmistry@google.com" --pageset_type="Dummy1k"
--run_id="testing" --benchmark_name="rasterize_and_record_micro_ct"
--benchmark_extra_args="--output-format=csv-pivot-table"
--browser_extra_args="--disable-setuid-sandbox --enable-threaded-compositing
--enable-impl-side-painting"

Results are available here:
https://storage.cloud.google.com/cluster-telemetry/tasks/benchmark_runs/testi...


BUG=skia:5316
==========

to

==========
Backend for new run_chromium_analysis CT task.

Tested with:
$ go run go/worker_scripts/run_chromium_analysis/main.go --logtostderr
--start_range=1 --num=3 --pageset_type=Dummy1k
--chromium_build=try-1e224f6-f1036b2-rmistry-20160206122201-withpatch
--run_id=testing --benchmark_name=rasterize_and_record_micro_ct
--benchmark_extra_args="--output-format=csv-pivot-table"
--browser_extra_args="--disable-setuid-sandbox --enable-threaded-compositing
--enable-impl-side-painting" --local

$ go run go/master_scripts/run_chromium_analysis_on_workers/main.go
--logtostderr --emails="rmistry@google.com" --pageset_type="Dummy1k"
--run_id="testing" --benchmark_name="rasterize_and_record_micro_ct"
--benchmark_extra_args="--output-format=csv-pivot-table"
--browser_extra_args="--disable-setuid-sandbox --enable-threaded-compositing
--enable-impl-side-painting"

Results are available here:
https://storage.cloud.google.com/cluster-telemetry/tasks/benchmark_runs/testi...


BUG=skia:5316

Committed:
https://skia.googlesource.com/buildbot/+/c5c7a6e7a57fb98634eedbd38909ca4a470a...
==========

[commit-bot: I haz the power, 2016-05-20 13:04:25]

Committed patchset #10 (id:200001) as
https://skia.googlesource.com/buildbot/+/c5c7a6e7a57fb98634eedbd38909ca4a470a...