Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(142)

Issue 2330133002: Updating the SwarmingIsolatedScriptTest to upload chartjson results to the (Closed)

Created:
4 years, 3 months ago by eyaich1
Modified:
4 years, 2 months ago
CC:
chromium-reviews, infra-reviews+build_chromium.org, kjellander-cc_chromium.org
Target Ref:
refs/heads/master
Project:
build
Visibility:
Public.

Description

Updating the SwarmingIsolatedScriptTest to upload chartjson results to the perf dashboard when present. I am trying to replicate how we currently do this for perf tests in scripts/slave/runtest.py, but I am not sure how close I am. I will iterate once this recipe is running on the fyi waterfall. It *shouldn't* break any existing behavior given that these code paths will not execute unless chartjson is present in the results. BUG=chromium:633253 Committed: https://chromium.googlesource.com/chromium/tools/build/+/d068176dade2dfc687b0625dd976b92737956843

Patch Set 1 #

Patch Set 2 : Removing stale code #

Total comments: 4

Patch Set 3 : Updating steps.py to utilize new swarming api #

Total comments: 6

Patch Set 4 : responding to review comments #

Patch Set 5 : Updating api to check for enabled benchmark #

Total comments: 16

Patch Set 6 : Responding to review comments #

Patch Set 7 : Expecations #

Total comments: 4

Patch Set 8 : Responding to review comments #

Patch Set 9 : Adding an expectation #

Patch Set 10 : Moving results dashboard call to new script #

Patch Set 11 : Adding additional test coverage #

Patch Set 12 : Updating slave utils #

Patch Set 13 : Fixing presubmit issues #

Unified diffs Side-by-side diffs Delta from patch set Stats (+492 lines, -244 lines) Patch
M scripts/slave/recipe_modules/chromium_tests/api.py View 1 2 3 4 5 6 1 chunk +3 lines, -3 lines 0 comments Download
M scripts/slave/recipe_modules/chromium_tests/steps.py View 1 2 3 4 5 6 7 8 9 10 3 chunks +40 lines, -0 lines 0 comments Download
M scripts/slave/recipe_modules/test_utils/test_api.py View 1 2 3 4 5 6 2 chunks +3 lines, -1 line 0 comments Download
M scripts/slave/recipes/chromium.py View 1 2 3 4 5 6 7 8 9 10 2 chunks +50 lines, -5 lines 0 comments Download
A + scripts/slave/recipes/chromium.expected/dynamic_swarmed_sharded_isolated_chartjson_test_disabled.json View 1 2 3 4 5 6 7 8 1 chunk +2 lines, -2 lines 0 comments Download
M scripts/slave/recipes/chromium.expected/dynamic_swarmed_sharded_isolated_chartjson_test_harness_failure.json View 1 2 3 4 5 6 1 chunk +3 lines, -3 lines 0 comments Download
M scripts/slave/recipes/chromium.expected/dynamic_swarmed_sharded_isolated_chartjson_test_missing_shard.json View 1 2 3 4 5 6 1 chunk +1 line, -1 line 0 comments Download
M scripts/slave/recipes/chromium.expected/dynamic_swarmed_sharded_passed_isolated_script_test.json View 1 2 3 4 5 6 7 8 9 10 8 chunks +43 lines, -10 lines 0 comments Download
M scripts/slave/runtest.py View 1 2 3 4 5 6 7 8 9 10 11 12 5 chunks +15 lines, -152 lines 0 comments Download
M scripts/slave/slave_utils.py View 1 2 3 4 5 6 7 8 9 10 11 12 3 chunks +176 lines, -0 lines 0 comments Download
M scripts/slave/unittests/runtest_test.py View 1 2 3 4 3 chunks +0 lines, -67 lines 0 comments Download
M scripts/slave/unittests/slave_utils_test.py View 1 2 3 4 5 2 chunks +73 lines, -0 lines 0 comments Download
A scripts/slave/upload_perf_dashboard_results.py View 1 2 3 4 5 6 7 8 9 10 11 12 1 chunk +83 lines, -0 lines 0 comments Download

Messages

Total messages: 49 (19 generated)
Ken Russell (switch to Gerrit)
Adding dpranke@ for opinions on this. https://codereview.chromium.org/2330133002/diff/20001/scripts/slave/recipe_modules/chromium_tests/steps.py File scripts/slave/recipe_modules/chromium_tests/steps.py (right): https://codereview.chromium.org/2330133002/diff/20001/scripts/slave/recipe_modules/chromium_tests/steps.py#newcode1087 scripts/slave/recipe_modules/chromium_tests/steps.py:1087: if 'chartjson' in ...
4 years, 3 months ago (2016-09-12 19:35:41 UTC) #2
eyaich1
This CL is now dependent on https://codereview.chromium.org/2336293002/ which updates the swarming api, but will not ...
4 years, 3 months ago (2016-09-13 16:52:59 UTC) #3
Ken Russell (switch to Gerrit)
Looks like a better direction. A few questions, mainly about how this will be tested. ...
4 years, 3 months ago (2016-09-13 19:41:00 UTC) #4
eyaich1
https://codereview.chromium.org/2330133002/diff/40001/scripts/slave/recipe_modules/chromium_tests/steps.py File scripts/slave/recipe_modules/chromium_tests/steps.py (right): https://codereview.chromium.org/2330133002/diff/40001/scripts/slave/recipe_modules/chromium_tests/steps.py#newcode1088 scripts/slave/recipe_modules/chromium_tests/steps.py:1088: self._output_charjson_results_if_present(api, step_result) On 2016/09/13 19:41:00, Ken Russell wrote: > ...
4 years, 3 months ago (2016-09-14 18:42:30 UTC) #5
Dirk Pranke
I defer to kbr@ here, so lgtm when he's happy. (Unless kbr@ really wants me ...
4 years, 3 months ago (2016-09-14 19:37:19 UTC) #6
Ken Russell (switch to Gerrit)
Since I gather this is dependent on https://codereview.chromium.org/2336293002/, let's defer re-review until that's sorted out.
4 years, 3 months ago (2016-09-17 03:34:44 UTC) #7
eyaich1
On 2016/09/17 03:34:44, Ken Russell wrote: > Since I gather this is dependent on https://codereview.chromium.org/2336293002/, ...
4 years, 3 months ago (2016-09-20 18:01:18 UTC) #8
Ken Russell (switch to Gerrit)
On 2016/09/20 18:01:18, eyaich1 wrote: > On 2016/09/17 03:34:44, Ken Russell wrote: > > Since ...
4 years, 3 months ago (2016-09-21 00:26:46 UTC) #9
eyaich1
On 2016/09/21 00:26:46, Ken Russell wrote: > On 2016/09/20 18:01:18, eyaich1 wrote: > > On ...
4 years, 2 months ago (2016-09-27 14:22:12 UTC) #10
Ken Russell (switch to Gerrit)
Looks good overall. A few questions. Could you update the Subject and the commit message ...
4 years, 2 months ago (2016-09-28 20:34:26 UTC) #11
eyaich1
I responded to most of your comments with TODOs as I would like to limit ...
4 years, 2 months ago (2016-09-29 12:43:35 UTC) #12
Ken Russell (switch to Gerrit)
TODOs and later cleanups are fine. Sorry if I missed earlier comments about this intending ...
4 years, 2 months ago (2016-09-30 15:03:48 UTC) #13
commit-bot: I haz the power
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.org/2330133002/100001
4 years, 2 months ago (2016-09-30 15:09:33 UTC) #16
commit-bot: I haz the power
Try jobs failed on following builders: Build Presubmit on luci.infra.try (JOB_FAILED, https://luci-milo.appspot.com/swarming/task/3194769d24e4a210)
4 years, 2 months ago (2016-09-30 15:16:31 UTC) #18
eyaich1
I forgot to update the expectations and coverage. I had to add no cover for ...
4 years, 2 months ago (2016-09-30 17:50:25 UTC) #19
Ken Russell (switch to Gerrit)
LGTM again with a couple of comments. https://codereview.chromium.org/2330133002/diff/120001/scripts/slave/recipe_modules/chromium_tests/steps.py File scripts/slave/recipe_modules/chromium_tests/steps.py (right): https://codereview.chromium.org/2330133002/diff/120001/scripts/slave/recipe_modules/chromium_tests/steps.py#newcode1129 scripts/slave/recipe_modules/chromium_tests/steps.py:1129: if dashboard_json: ...
4 years, 2 months ago (2016-09-30 21:45:32 UTC) #20
eyaich1
Thanks! https://codereview.chromium.org/2330133002/diff/120001/scripts/slave/recipe_modules/chromium_tests/steps.py File scripts/slave/recipe_modules/chromium_tests/steps.py (right): https://codereview.chromium.org/2330133002/diff/120001/scripts/slave/recipe_modules/chromium_tests/steps.py#newcode1129 scripts/slave/recipe_modules/chromium_tests/steps.py:1129: if dashboard_json: # pragma: no cover On 2016/09/30 ...
4 years, 2 months ago (2016-10-03 13:32:56 UTC) #21
commit-bot: I haz the power
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.org/2330133002/140001
4 years, 2 months ago (2016-10-03 13:33:11 UTC) #24
commit-bot: I haz the power
Try jobs failed on following builders: Build Presubmit on luci.infra.try (JOB_FAILED, https://luci-milo.appspot.com/swarming/task/31a3913c8f8c1210)
4 years, 2 months ago (2016-10-03 13:41:20 UTC) #26
commit-bot: I haz the power
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.org/2330133002/160001
4 years, 2 months ago (2016-10-03 14:05:59 UTC) #29
commit-bot: I haz the power
Try jobs failed on following builders: Build Presubmit on luci.infra.try (JOB_FAILED, https://luci-milo.appspot.com/swarming/task/31a3af56718ed910)
4 years, 2 months ago (2016-10-03 14:14:09 UTC) #31
eyaich1
On 2016/10/03 14:14:09, commit-bot: I haz the power wrote: > Try jobs failed on following ...
4 years, 2 months ago (2016-10-04 15:55:52 UTC) #32
Ken Russell (switch to Gerrit)
On 2016/10/04 15:55:52, eyaich1 wrote: > On 2016/10/03 14:14:09, commit-bot: I haz the power wrote: ...
4 years, 2 months ago (2016-10-05 01:07:21 UTC) #33
commit-bot: I haz the power
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.org/2330133002/200001
4 years, 2 months ago (2016-10-05 13:44:24 UTC) #36
commit-bot: I haz the power
Try jobs failed on following builders: Build Presubmit on luci.infra.try (JOB_FAILED, https://luci-milo.appspot.com/swarming/task/31ade83d86f78b10)
4 years, 2 months ago (2016-10-05 13:52:53 UTC) #38
commit-bot: I haz the power
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.org/2330133002/220001
4 years, 2 months ago (2016-10-05 13:57:53 UTC) #41
commit-bot: I haz the power
Try jobs failed on following builders: Build Presubmit on luci.infra.try (JOB_FAILED, https://luci-milo.appspot.com/swarming/task/31adf8433a2f9910)
4 years, 2 months ago (2016-10-05 14:06:27 UTC) #43
commit-bot: I haz the power
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.org/2330133002/240001
4 years, 2 months ago (2016-10-05 14:25:47 UTC) #46
commit-bot: I haz the power
Committed patchset #13 (id:240001) as https://chromium.googlesource.com/chromium/tools/build/+/d068176dade2dfc687b0625dd976b92737956843
4 years, 2 months ago (2016-10-05 14:32:34 UTC) #48
eyaich1
4 years, 2 months ago (2016-10-05 18:42:04 UTC) #49
Message was sent while issue was closed.
A revert of this CL (patchset #13 id:240001) has been created in
https://codereview.chromium.org/2391353002/ by eyaich@chromium.org.

The reason for reverting is: Broke the cronet bots

https://build.chromium.org/p/chromium.android/waterfall?builder=Android+Crone...

See crbug.com/653167

Will break up this CL into the refactor of slave utils and run test (which is
what I think broke the existing waterfall since it doesn't run through the new
code in chromium_tests).

Powered by Google App Engine
This is Rietveld 408576698