Use swarming in run_lua CT task

Use swarming in run_lua CT task


Tested run_lua with:

$ go run go/worker_scripts/run_lua/main.go --logtostderr --start_range=1 --num=10 --pageset_type=Dummy1k --chromium_build=try-1e224f6-f1036b2-rmistry-20160206122201-withpatch --run_id=testing --local

$ gsutil cp gs://cluster-telemetry/tasks/lua_runs/testing/scripts/* /tmp/
$ go run go/master_scripts/run_lua_on_workers/main.go --logtostderr --emails=”rmistry@google.com” --pageset_type=Dummy1k --chromium_build=try-1e224f6-f1036b2-rmistry-20160206122201-withpatch --run_id=testing


BUG=skia:5320
Committed: https://skia.googlesource.com/buildbot/+/e315dd306c6c406e4b2afc7e83b4ea07a91eebdf

[rmistry, 2016-05-18 18:28:11]

Description was changed from

==========
Use swarming in run_lua CT task

Tested run_lua with:
$
$

BUG=skia:5320
==========

to

==========
Use swarming in run_lua CT task

Tested run_lua with:
$ go run go/worker_scripts/run_lua/main.go --logtostderr --start_range=1
--num=10 --pageset_type=Dummy1k
--chromium_build=try-1e224f6-f1036b2-rmistry-20160206122201-withpatch
--run_id=testing
$ go run go/master_scripts/run_lua_on_workers/main.go --logtostderr
--emails=”rmistry@google.com” --pageset_type=Dummy1k
--chromium_build=try-1e224f6-f1036b2-rmistry-20160206122201-withpatch
--run_id=testing


BUG=skia:5320
==========

[rmistry, 2016-05-18 18:29:17]

Description was changed from

==========
Use swarming in run_lua CT task

Tested run_lua with:
$ go run go/worker_scripts/run_lua/main.go --logtostderr --start_range=1
--num=10 --pageset_type=Dummy1k
--chromium_build=try-1e224f6-f1036b2-rmistry-20160206122201-withpatch
--run_id=testing
$ go run go/master_scripts/run_lua_on_workers/main.go --logtostderr
--emails=”rmistry@google.com” --pageset_type=Dummy1k
--chromium_build=try-1e224f6-f1036b2-rmistry-20160206122201-withpatch
--run_id=testing


BUG=skia:5320
==========

to

==========
Use swarming in run_lua CT task

Tested run_lua with:
$ go run go/worker_scripts/run_lua/main.go --logtostderr --start_range=1
--num=10 --pageset_type=Dummy1k
--chromium_build=try-1e224f6-f1036b2-rmistry-20160206122201-withpatch
--run_id=testing --local
$ go run go/master_scripts/run_lua_on_workers/main.go --logtostderr
--emails=”rmistry@google.com” --pageset_type=Dummy1k
--chromium_build=try-1e224f6-f1036b2-rmistry-20160206122201-withpatch
--run_id=testing


BUG=skia:5320
==========

[rmistry, 2016-05-18 18:54:37]

Description was changed from

==========
Use swarming in run_lua CT task

Tested run_lua with:
$ go run go/worker_scripts/run_lua/main.go --logtostderr --start_range=1
--num=10 --pageset_type=Dummy1k
--chromium_build=try-1e224f6-f1036b2-rmistry-20160206122201-withpatch
--run_id=testing --local
$ go run go/master_scripts/run_lua_on_workers/main.go --logtostderr
--emails=”rmistry@google.com” --pageset_type=Dummy1k
--chromium_build=try-1e224f6-f1036b2-rmistry-20160206122201-withpatch
--run_id=testing


BUG=skia:5320
==========

to

==========
Use swarming in run_lua CT task


Tested run_lua with:

$ go run go/worker_scripts/run_lua/main.go --logtostderr --start_range=1
--num=10 --pageset_type=Dummy1k
--chromium_build=try-1e224f6-f1036b2-rmistry-20160206122201-withpatch
--run_id=testing --local

$ gsutil cp gs://cluster-telemetry/tasks/lua_runs/testing/scripts/* /tmp/
$ go run go/master_scripts/run_lua_on_workers/main.go --logtostderr
--emails=”rmistry@google.com” --pageset_type=Dummy1k
--chromium_build=try-1e224f6-f1036b2-rmistry-20160206122201-withpatch
--run_id=testing


BUG=skia:5320
==========

[rmistry, 2016-05-18 18:57:39]

rmistry@google.com changed reviewers:
+ benjaminwagner@google.com

[rmistry, 2016-05-18 18:58:39]

The CQ bit was checked by rmistry@google.com to run a CQ dry run

[commit-bot: I haz the power, 2016-05-18 18:58:47]

Dry run: CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1991943002/20001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1991943002/20001

[commit-bot: I haz the power, 2016-05-18 19:01:55]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-05-18 19:01:56]

Dry run: This issue passed the CQ dry run.

[rmistry, 2016-05-19 16:36:28]

Ready for review

[dogben, 2016-05-19 17:02:42]

https://codereview.chromium.org/1991943002/diff/40001/ct/go/master_scripts/ru...
File ct/go/master_scripts/run_lua_on_workers/main.go (right):

https://codereview.chromium.org/1991943002/diff/40001/ct/go/master_scripts/ru...
ct/go/master_scripts/run_lua_on_workers/main.go:138: gsBaseDir :=
filepath.Join(util.SWARMING_DIR_NAME, filepath.Join(util.LuaRunsDir, *runID),
*pagesetType)
I would hope that runID is unique and therefore this dir wouldn't already
exist...

(Probably could/should have made this comment on other CLs.)

https://codereview.chromium.org/1991943002/diff/40001/ct/go/master_scripts/ru...
ct/go/master_scripts/run_lua_on_workers/main.go:154: // If the file already
exists it could be that there is another lua task running on this machine.
Please just use ioutil.TempDir instead of os.TempDir to avoid this issue.

[rmistry, 2016-05-19 17:08:19]

https://codereview.chromium.org/1991943002/diff/40001/ct/go/master_scripts/ru...
File ct/go/master_scripts/run_lua_on_workers/main.go (right):

https://codereview.chromium.org/1991943002/diff/40001/ct/go/master_scripts/ru...
ct/go/master_scripts/run_lua_on_workers/main.go:138: gsBaseDir :=
filepath.Join(util.SWARMING_DIR_NAME, filepath.Join(util.LuaRunsDir, *runID),
*pagesetType)
On 2016/05/19 17:02:41, Ben Wagner wrote:
> I would hope that runID is unique and therefore this dir wouldn't already
> exist...
> 
> (Probably could/should have made this comment on other CLs.)

The other CLs IIRC did not use a runID here.
But yes this dir would normally not already exist, but it is especially useful
for testing. I can keep using the same fake runID for local runs.

https://codereview.chromium.org/1991943002/diff/40001/ct/go/master_scripts/ru...
ct/go/master_scripts/run_lua_on_workers/main.go:154: // If the file already
exists it could be that there is another lua task running on this machine.
On 2016/05/19 17:02:41, Ben Wagner wrote:
> Please just use ioutil.TempDir instead of os.TempDir to avoid this issue.

Unfortunately the file has to be in /tmp/lua-output. That is where all lua
aggregator scripts expect the output to reside. I could probably document that
lua aggregator scripts should instead include some unique string (eg:
__OUTPUT_FILE__) which can be replaced here. But for now it expects output in
/tmp/lua-output.

[dogben, 2016-05-19 17:15:38]

lgtm

https://codereview.chromium.org/1991943002/diff/40001/ct/go/master_scripts/ru...
File ct/go/master_scripts/run_lua_on_workers/main.go (right):

https://codereview.chromium.org/1991943002/diff/40001/ct/go/master_scripts/ru...
ct/go/master_scripts/run_lua_on_workers/main.go:154: // If the file already
exists it could be that there is another lua task running on this machine.
On 2016/05/19 17:08:19, rmistry wrote:
> On 2016/05/19 17:02:41, Ben Wagner wrote:
> > Please just use ioutil.TempDir instead of os.TempDir to avoid this issue.
> 
> Unfortunately the file has to be in /tmp/lua-output. That is where all lua
> aggregator scripts expect the output to reside. I could probably document that
> lua aggregator scripts should instead include some unique string (eg:
> __OUTPUT_FILE__) which can be replaced here. But for now it expects output in
> /tmp/lua-output.

Oh, that is unfortunate.

[rmistry, 2016-05-19 17:23:55]

The CQ bit was checked by rmistry@google.com

[commit-bot: I haz the power, 2016-05-19 17:23:58]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1991943002/40001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1991943002/40001

[commit-bot: I haz the power, 2016-05-19 17:27:16]

Description was changed from

==========
Use swarming in run_lua CT task


Tested run_lua with:

$ go run go/worker_scripts/run_lua/main.go --logtostderr --start_range=1
--num=10 --pageset_type=Dummy1k
--chromium_build=try-1e224f6-f1036b2-rmistry-20160206122201-withpatch
--run_id=testing --local

$ gsutil cp gs://cluster-telemetry/tasks/lua_runs/testing/scripts/* /tmp/
$ go run go/master_scripts/run_lua_on_workers/main.go --logtostderr
--emails=”rmistry@google.com” --pageset_type=Dummy1k
--chromium_build=try-1e224f6-f1036b2-rmistry-20160206122201-withpatch
--run_id=testing


BUG=skia:5320
==========

to

==========
Use swarming in run_lua CT task


Tested run_lua with:

$ go run go/worker_scripts/run_lua/main.go --logtostderr --start_range=1
--num=10 --pageset_type=Dummy1k
--chromium_build=try-1e224f6-f1036b2-rmistry-20160206122201-withpatch
--run_id=testing --local

$ gsutil cp gs://cluster-telemetry/tasks/lua_runs/testing/scripts/* /tmp/
$ go run go/master_scripts/run_lua_on_workers/main.go --logtostderr
--emails=”rmistry@google.com” --pageset_type=Dummy1k
--chromium_build=try-1e224f6-f1036b2-rmistry-20160206122201-withpatch
--run_id=testing


BUG=skia:5320

Committed:
https://skia.googlesource.com/buildbot/+/e315dd306c6c406e4b2afc7e83b4ea07a91e...
==========

[commit-bot: I haz the power, 2016-05-19 17:27:17]

Committed patchset #3 (id:40001) as
https://skia.googlesource.com/buildbot/+/e315dd306c6c406e4b2afc7e83b4ea07a91e...