Change specifier string of WebKit Mac10.11 builders to "Mac10.11".

Change specifier string of WebKit Mac10.11 builders to "Mac10.11".

BUG=624020
Committed: https://crrev.com/983ca232e5b4df5c19e5f669d32d294968c6b2c5
Cr-Commit-Position: refs/heads/master@{#406411}

[qyearsley, 2016-06-27 23:09:07]

qyearsley@chromium.org changed reviewers:
+ dpranke@chromium.org

[Dirk Pranke, 2016-06-27 23:10:19]

dpranke@chromium.org changed reviewers:
+ ojan@chromium.org

[Dirk Pranke, 2016-06-27 23:10:20]

I've forgotten ... I thought we actually wanted to change this to "10.11",
"10.10", etc, and so maybe this was intentional.

Ojan?

[ojan, 2016-06-28 04:27:32]

We had a lot of redundant "Mac" prefixes on this and doing 10.10 is more
consistent with how we do other platforms (XP, Precise, etc). So, that's what
makes sense to me. We should either do 10.10 and XP or Mac10.10 and WinXP for
consistency?

[ojan, 2016-06-28 04:28:32]

Although, I see that the windows ones are Win prefixed now.

I don't really care which direction we go. We should just do something
consistent across platforms.

[qyearsley, 2016-06-28 16:48:50]

On 2016/06/28 at 04:28:32, ojan wrote:
> Although, I see that the windows ones are Win prefixed now.
> 
> I don't really care which direction we go. We should just do something
consistent across platforms.

Alright -- since these specifiers are used in TestExpectations files, I think it
might be clearer if the specifiers all contain OS name. For example, [ Precise ]
might potentially be less clear than [ LinuxPrecise ]. So we'd change:

  Retina -> MacRetina
  Precise -> LinuxPrecise
  Trusty -> LinuxTrusty
  10.11 -> Mac10.11

Does that sound good?

[wkorman, 2016-06-28 16:55:45]

On 2016/06/28 at 16:48:50, qyearsley wrote:
> On 2016/06/28 at 04:28:32, ojan wrote:
> > Although, I see that the windows ones are Win prefixed now.
> > 
> > I don't really care which direction we go. We should just do something
consistent across platforms.
> 
> Alright -- since these specifiers are used in TestExpectations files, I think
it might be clearer if the specifiers all contain OS name. For example, [
Precise ] might potentially be less clear than [ LinuxPrecise ]. So we'd change:
> 
>   Retina -> MacRetina
>   Precise -> LinuxPrecise
>   Trusty -> LinuxTrusty
>   10.11 -> Mac10.11
> 
> Does that sound good?

It sounds good to me, do we need to update the actual contents of
TestExpectations to match in the same change then?

Also, what have the existing expectations using the
we-now-seem-to-realize-is-incorrect Mac10.11 name been doing all this time? :)

Does our presubmit not validate that every specifier is a known and supported
one?

[Dirk Pranke, 2016-06-28 17:02:57]

On 2016/06/28 16:48:50, qyearsley wrote:
> On 2016/06/28 at 04:28:32, ojan wrote:
> > Although, I see that the windows ones are Win prefixed now.
> > 
> > I don't really care which direction we go. We should just do something
> consistent across platforms.
> 
> Alright -- since these specifiers are used in TestExpectations files, I think
it
> might be clearer if the specifiers all contain OS name. For example, [ Precise
]
> might potentially be less clear than [ LinuxPrecise ]. So we'd change:
> 
>   Retina -> MacRetina
>   Precise -> LinuxPrecise
>   Trusty -> LinuxTrusty
>   10.11 -> Mac10.11
> 
> Does that sound good?

I think the existing names are clear enough and I have a bias towards shorter 
names, but I don't really care too much about this, so if you wanted to use
the longer names that's fine, too.

[qyearsley, 2016-06-28 17:33:37]

On 2016/06/28 at 16:55:45, wkorman wrote:
> On 2016/06/28 at 16:48:50, qyearsley wrote:
> > On 2016/06/28 at 04:28:32, ojan wrote:
> > > Although, I see that the windows ones are Win prefixed now.
> > > 
> > > I don't really care which direction we go. We should just do something
consistent across platforms.
> > 
> > Alright -- since these specifiers are used in TestExpectations files, I
think it might be clearer if the specifiers all contain OS name. For example, [
Precise ] might potentially be less clear than [ LinuxPrecise ]. So we'd change:
> > 
> >   Retina -> MacRetina
> >   Precise -> LinuxPrecise
> >   Trusty -> LinuxTrusty
> >   10.11 -> Mac10.11
> > 
> > Does that sound good?
> 
> It sounds good to me, do we need to update the actual contents of
TestExpectations to match in the same change then?
> 
> Also, what have the existing expectations using the
we-now-seem-to-realize-is-incorrect Mac10.11 name been doing all this time? :)

I think that for lines currently marked
  [ Mac10.11 ] sometest [ Failure ]
If that test fails on Mac10.11, it should still be an unexpected failure, since
the specifier for the Mac 10.11 bot is "10.11".

If this hypothesis is true, then the current expectations for Mac10.11 are bogus
and I should be able to remove all "Mac10.11" specifiers and there should still
be no unexpected results on the WebKit Mac 10.11 builder.

Although right now I couldn't test this hypothesis with try bots -- might it be
too troublesome to try to make another CL and submit it in order to test this
now? As long as the CL description clearly states that it might cause failures
on that builder and I want to revert it if it does, then it should be OK, right?


> Does our presubmit not validate that every specifier is a known and supported
one?

It looks like it doesn't -- when parsing the test expectations file, if it sees
an unknown configuration, it adds it anyway.
https://cs.chromium.org/chromium/src/third_party/WebKit/Tools/Scripts/webkitp...

Just made a separate CL for this issue: http://crrev.com/2106893002


On 2016/06/28 at 17:02:57, dpranke wrote: 
> I think the existing names are clear enough and I have a bias towards shorter 
> names, but I don't really care too much about this, so if you wanted to use
> the longer names that's fine, too.

Shorter names are also OK, although it's harder to abbreviate Win7 and Win10
("7" and "10" seem more ambiguous), and so to be consistent with the Windows
naming, it's probably easier to change the other names.

[qyearsley, 2016-06-28 17:40:13]

Description was changed from

==========
Change specifier string of WebKit Mac10.11 builders to "Mac10.11".
==========

to

==========
Change specifier string of WebKit Mac10.11 builders to "Mac10.11".

BUG=624020
==========

[qyearsley, 2016-07-07 18:39:39]

The CQ bit was checked by qyearsley@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-07-07 18:40:16]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-07-07 20:16:01]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-07-07 20:16:03]

Dry run: This issue passed the CQ dry run.

[qyearsley, 2016-07-19 16:48:16]

On 2016/07/07 at 20:16:03, commit-bot wrote:
> Dry run: This issue passed the CQ dry run.

In http://crrev.com/2106903002 I found that the lines in TestExpectations do
indeed have an effect (they are used as the expectations for the current port
when running on the WebKit Mac10.11 bot). I believe that this is because when
the tests are run on that bot, the full port name is still determined to be
mac-mac10.11.

As far as I can tell, the specifiers list associated with the builders just
affects BotTestExpectations.

In any case, does it sound OK to first commit this, and then maybe follow up by
renaming OS version specifiers to be of the form <OS><Version, e.g.
Linux-Trusty, Linux-Precise, Mac-Retina?

[qyearsley, 2016-07-19 16:48:36]

qyearsley@chromium.org changed reviewers:
+ wkorman@chromium.org

[qyearsley, 2016-07-19 16:49:33]

qyearsley@chromium.org changed reviewers:
- ojan@chromium.org

[wkorman, 2016-07-19 21:37:10]

On 2016/07/19 at 16:48:16, qyearsley wrote:
> On 2016/07/07 at 20:16:03, commit-bot wrote:
> > Dry run: This issue passed the CQ dry run.
> 
> In http://crrev.com/2106903002 I found that the lines in TestExpectations do
indeed have an effect (they are used as the expectations for the current port
when running on the WebKit Mac10.11 bot). I believe that this is because when
the tests are run on that bot, the full port name is still determined to be
mac-mac10.11.
> 
> As far as I can tell, the specifiers list associated with the builders just
affects BotTestExpectations.
> 
> In any case, does it sound OK to first commit this, and then maybe follow up
by renaming OS version specifiers to be of the form <OS><Version, e.g.
Linux-Trusty, Linux-Precise, Mac-Retina?

Sounds ok to me.

[wkorman, 2016-07-19 21:37:14]

lgtm

[qyearsley, 2016-07-19 22:02:59]

The CQ bit was checked by qyearsley@chromium.org

[commit-bot: I haz the power, 2016-07-19 22:03:32]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-07-19 23:26:46]

Committed patchset #1 (id:1)

[commit-bot: I haz the power, 2016-07-19 23:27:08]

CQ bit was unchecked.

[commit-bot: I haz the power, 2016-07-19 23:28:15]

Description was changed from

==========
Change specifier string of WebKit Mac10.11 builders to "Mac10.11".

BUG=624020
==========

to

==========
Change specifier string of WebKit Mac10.11 builders to "Mac10.11".

BUG=624020

Committed: https://crrev.com/983ca232e5b4df5c19e5f669d32d294968c6b2c5
Cr-Commit-Position: refs/heads/master@{#406411}
==========

[commit-bot: I haz the power, 2016-07-19 23:28:16]

Patchset 1 (id:??) landed as
https://crrev.com/983ca232e5b4df5c19e5f669d32d294968c6b2c5
Cr-Commit-Position: refs/heads/master@{#406411}