Make swarming dimensions for GPU bots explicit

Make swarming dimensions for GPU bots explicit

BUG=507402, 579640

[Paweł Hajdan Jr., 2016-02-22 10:27:29]

phajdan.jr@chromium.org changed reviewers:
+ kbr@chromium.org

[Paweł Hajdan Jr., 2016-02-22 10:27:33]

The CQ bit was checked by phajdan.jr@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-02-22 10:27:39]

Dry run: CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1722573002/1
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1722573002/1

[commit-bot: I haz the power, 2016-02-22 10:30:29]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-02-22 10:30:30]

Dry run: This issue passed the CQ dry run.

[Ken Russell (switch to Gerrit), 2016-02-22 22:39:39]

I'm not sure this is the best place for these changes. What tests are failing to
specify these in the swarming dimensions? Is there a reason they can't be added
to the machine descriptions in src/content/test/gpu/generate_buildbot_json.py in
the Chromium workspace? (The OS specifier already seems to be there for these
configurations.)

[Paweł Hajdan Jr., 2016-02-23 18:14:10]

On 2016/02/22 at 22:39:39, kbr wrote:
> I'm not sure this is the best place for these changes. What tests are failing
to specify these in the swarming dimensions? Is there a reason they can't be
added to the machine descriptions in
src/content/test/gpu/generate_buildbot_json.py in the Chromium workspace? (The
OS specifier already seems to be there for these configurations.)

Please see expectations changing with this CL.

Note that anything requiring src-side files will NOT be reflected in these
expectations.

Is there anything actively wrong with this change?

FWIW, I don't have to make it. It just looked suspicious to me our expectations
would show dimensions not matching the builder name, and dimensions not
specified for these bots as other bots do. Please consider looking into this.

[Ken Russell (switch to Gerrit), 2016-02-23 21:26:32]

On 2016/02/23 18:14:10, Paweł Hajdan Jr. wrote:
> On 2016/02/22 at 22:39:39, kbr wrote:
> > I'm not sure this is the best place for these changes. What tests are
failing
> to specify these in the swarming dimensions? Is there a reason they can't be
> added to the machine descriptions in
> src/content/test/gpu/generate_buildbot_json.py in the Chromium workspace? (The
> OS specifier already seems to be there for these configurations.)
> 
> Please see expectations changing with this CL.
> 
> Note that anything requiring src-side files will NOT be reflected in these
> expectations.

That's the problem -- the swarming dimensions for the GPU tests are all defined
src-side, and there are currently no mocks of these tests in the recipe
simulation tests, so the recipe expectations don't reflect how the targets on
the GPU bots are invoked.

> Is there anything actively wrong with this change?

There is no test covering whether the swarming dimensions that are read from the
src-side JSON files will be merged properly with the dimensions being specified
here. That is the risk with making this change recipe-side. While it will
probably work as expected, at least one test is needed. See for example the
dynamic_swarmed_gtest_mac_gpu test in scripts/slave/recipes/chromium.py. Perhaps
you could create a new test that would mock the build results from "GPU Mac
Builder" and fake a run of gl_tests on one of these 10.10 GPU bots. (That sort
of simulation test was beyond my ability, by the way, which is why I didn't add
one when converting the GPU bots to the Chromium recipe.) Ideally, the fake data
for the chromium recipe's tests would be updated to produce JSON files that are
similar to those that are checked in to the Chromium repo. That would actually
eliminate the need for the changes to chromium_gpu.py and chromium_gpu_fyi.py in
this CL.

> FWIW, I don't have to make it. It just looked suspicious to me our
expectations
> would show dimensions not matching the builder name, and dimensions not
> specified for these bots as other bots do. Please consider looking into this.

I agree with your assessment that the recipe expectations are bogus, but I don't
think that making this change improves the situation. See above for my
suggestion of how to improve the simulation tests for the GPU bots.