slave_hosts_cfg: Use new Windows GCE instances, use fewer, 16-core GCE instances

slave_hosts_cfg: Use new Windows GCE instances, use fewer, 16-core GCE instances

BUG=skia:2755
R=mtklein@google.com, rmistry@google.com

Committed: https://skia.googlesource.com/buildbot/+/c28a700

[borenet, 2014-07-09 15:06:28]

Blocked on Windows GCE setup.

https://codereview.chromium.org/380733003/diff/20001/site_config/slave_hosts_...
File site_config/slave_hosts_cfg.py (right):

https://codereview.chromium.org/380733003/diff/20001/site_config/slave_hosts_...
site_config/slave_hosts_cfg.py:253: 'skia-vm-001': {
Why do we start the count at 1 instead of 0?

[mtklein, 2014-07-09 16:26:41]

lgtm as is, with a few ideas for bonus work

https://codereview.chromium.org/380733003/diff/20001/master/slaves.cfg
File master/slaves.cfg (right):

https://codereview.chromium.org/380733003/diff/20001/master/slaves.cfg#newcod...
master/slaves.cfg:544: 'Build-Win8-VS2012-x86-Debug',
Is it still useful to distinguish Win7 and Win8 compile bots?  Wasn't the key
distinction the VSyyyy they were using, which is now all the same?

https://codereview.chromium.org/380733003/diff/20001/master/slaves.cfg#newcod...
master/slaves.cfg:589: 'hostname': 'skiabot-shuttle-ubuntu13-003',
Move this guy to VM too?

https://codereview.chromium.org/380733003/diff/20001/site_config/slave_hosts_...
File site_config/slave_hosts_cfg.py (right):

https://codereview.chromium.org/380733003/diff/20001/site_config/slave_hosts_...
site_config/slave_hosts_cfg.py:253: 'skia-vm-001': {
On 2014/07/09 15:06:28, borenet wrote:
> Why do we start the count at 1 instead of 0?

Doesn't that match the last octet of the IP?

https://codereview.chromium.org/380733003/diff/20001/site_config/slave_hosts_...
site_config/slave_hosts_cfg.py:253: 'skia-vm-001': {
Might be nice to give this file the list comprehension treatment you've done in
slaves.cfg

[rmistry, 2014-07-10 12:16:27]

https://codereview.chromium.org/380733003/diff/20001/site_config/slave_hosts_...
File site_config/slave_hosts_cfg.py (right):

https://codereview.chromium.org/380733003/diff/20001/site_config/slave_hosts_...
site_config/slave_hosts_cfg.py:253: 'skia-vm-001': {
On 2014/07/09 15:06:28, borenet wrote:
> Why do we start the count at 1 instead of 0?

The IP range we have is 0-127, in the old days 0 was the Cluster Telemetry
master, after the move to bare-metal machines I kept it around to test
complicated CT changes, which is why buildbots start from 1.

https://codereview.chromium.org/380733003/diff/20001/site_config/slave_hosts_...
site_config/slave_hosts_cfg.py:253: 'skia-vm-001': {
On 2014/07/09 16:26:41, mtklein wrote:
> On 2014/07/09 15:06:28, borenet wrote:
> > Why do we start the count at 1 instead of 0?
> 
> Doesn't that match the last octet of the IP?

Acknowledged.

(Just testing the 'Ack' button in Rietveld I added yesterday)

[rmistry, 2014-07-16 11:47:04]

Just noticed that this has 32 instances * 16 cores = 512 cores = our limit.
Lets leave atleast 16 cores (or more) lying around for experimentation. Maybe
take out 2 linux compilers? (or something else that you prefer).

[borenet, 2014-07-16 12:35:31]

On 2014/07/16 11:47:04, rmistry wrote:
> Just noticed that this has 32 instances * 16 cores = 512 cores = our limit.
> Lets leave atleast 16 cores (or more) lying around for experimentation. Maybe
> take out 2 linux compilers? (or something else that you prefer).

That's by design.  If we want to experiment, we can take down one of the linux
compile bots, but there's no reason to let those cores go unused.

[rmistry, 2014-07-16 12:45:05]

On 2014/07/16 12:35:31, borenet wrote:
> On 2014/07/16 11:47:04, rmistry wrote:
> > Just noticed that this has 32 instances * 16 cores = 512 cores = our limit.
> > Lets leave atleast 16 cores (or more) lying around for experimentation.
Maybe
> > take out 2 linux compilers? (or something else that you prefer).
> 
> That's by design.  If we want to experiment, we can take down one of the linux
> compile bots, but there's no reason to let those cores go unused.

I disagree, I do frequent experiments and I do not want to take down one or more
compile bots and then bring them back up everytime.
In the original email I asked for 3x16 cores be unused, my new ask is 2x16
unused cores taken from wherever you seem appropriate.

[borenet, 2014-07-16 13:01:40]

Done in patch set 2.

https://codereview.chromium.org/380733003/diff/20001/master/slaves.cfg
File master/slaves.cfg (right):

https://codereview.chromium.org/380733003/diff/20001/master/slaves.cfg#newcod...
master/slaves.cfg:544: 'Build-Win8-VS2012-x86-Debug',
On 2014/07/09 16:26:41, mtklein wrote:
> Is it still useful to distinguish Win7 and Win8 compile bots?  Wasn't the key
> distinction the VSyyyy they were using, which is now all the same?

Yeah, probably not.  Should I go ahead and remove the Win8 compile bots?

https://codereview.chromium.org/380733003/diff/20001/master/slaves.cfg#newcod...
master/slaves.cfg:589: 'hostname': 'skiabot-shuttle-ubuntu13-003',
On 2014/07/09 16:26:41, mtklein wrote:
> Move this guy to VM too?

Needs to be in the Skia lab so that it can SSH in to all of the other lab
machines.

[mtklein, 2014-07-16 13:07:45]

https://codereview.chromium.org/380733003/diff/20001/master/slaves.cfg
File master/slaves.cfg (right):

https://codereview.chromium.org/380733003/diff/20001/master/slaves.cfg#newcod...
master/slaves.cfg:544: 'Build-Win8-VS2012-x86-Debug',
On 2014/07/16 13:01:40, borenet wrote:
> On 2014/07/09 16:26:41, mtklein wrote:
> > Is it still useful to distinguish Win7 and Win8 compile bots?  Wasn't the
key
> > distinction the VSyyyy they were using, which is now all the same?
> 
> Yeah, probably not.  Should I go ahead and remove the Win8 compile bots?

Yup, think so.

https://codereview.chromium.org/380733003/diff/20001/master/slaves.cfg#newcod...
master/slaves.cfg:589: 'hostname': 'skiabot-shuttle-ubuntu13-003',
On 2014/07/16 13:01:40, borenet wrote:
> On 2014/07/09 16:26:41, mtklein wrote:
> > Move this guy to VM too?
> 
> Needs to be in the Skia lab so that it can SSH in to all of the other lab
> machines.

Gotcha.

[rmistry, 2014-07-16 13:20:04]

LGTM

If everything looks ok with Windows GCE instances today then I will create all
the 16 core Linux instances tomorrow and we can submit this CL on Friday
morning. After that I will create the Windows GCE instances so that the master
does not think they are Linux compile bots.

https://codereview.chromium.org/380733003/diff/40001/site_config/slave_hosts_...
File site_config/slave_hosts_cfg.py (right):

https://codereview.chromium.org/380733003/diff/40001/site_config/slave_hosts_...
site_config/slave_hosts_cfg.py:425: 'skia-vm-014': {
Not sequential anymore but understandable since it will be a pain to move things
around again.

[borenet, 2014-07-16 13:28:52]

Uploaded patch set 4.

https://codereview.chromium.org/380733003/diff/20001/master/slaves.cfg
File master/slaves.cfg (right):

https://codereview.chromium.org/380733003/diff/20001/master/slaves.cfg#newcod...
master/slaves.cfg:544: 'Build-Win8-VS2012-x86-Debug',
On 2014/07/16 13:07:45, mtklein wrote:
> On 2014/07/16 13:01:40, borenet wrote:
> > On 2014/07/09 16:26:41, mtklein wrote:
> > > Is it still useful to distinguish Win7 and Win8 compile bots?  Wasn't the
> key
> > > distinction the VSyyyy they were using, which is now all the same?
> > 
> > Yeah, probably not.  Should I go ahead and remove the Win8 compile bots?
> 
> Yup, think so.

Done.

[mtklein, 2014-07-16 13:33:18]

LGTM

https://codereview.chromium.org/380733003/diff/60001/master/master_compile_bu...
File master/master_compile_builders_cfg.py (right):

https://codereview.chromium.org/380733003/diff/60001/master/master_compile_bu...
master/master_compile_builders_cfg.py:59: ('Win',         'VS2013', 'Debug',  
'x86',    None,          GYP_WIN7,  True,  f_factory, WIN32,  S_PERCOMMIT, {}),
I'd be OK with even just 'VS' here, and that only to distinguish it from
possible future Windows/Clang bots.  If this setup stands the test of time, one
day the compilers will update and "2013" will be out of date.  :)

[borenet, 2014-07-17 11:51:23]

On 2014/07/16 13:33:18, mtklein wrote:
> LGTM
> 
>
https://codereview.chromium.org/380733003/diff/60001/master/master_compile_bu...
> File master/master_compile_builders_cfg.py (right):
> 
>
https://codereview.chromium.org/380733003/diff/60001/master/master_compile_bu...
> master/master_compile_builders_cfg.py:59: ('Win',         'VS2013', 'Debug',  
> 'x86',    None,          GYP_WIN7,  True,  f_factory, WIN32,  S_PERCOMMIT,
{}),
> I'd be OK with even just 'VS' here, and that only to distinguish it from
> possible future Windows/Clang bots.  If this setup stands the test of time,
one
> day the compilers will update and "2013" will be out of date.  :)

I'm okay with having to update it, since that won't be too often.

[borenet, 2014-07-17 11:52:29]

Committed patchset #5 manually as rc28a700 (presubmit successful).