Change builder names

master/master_builders_cfg.py

 # Copyright (c) 2012 The Chromium Authors. All rights reserved.

[epoger, 2013/04/09 17:38:36]

regarding builder name length, whether to use abbreviations, etc.:

I think the thing we need to know first is: where do these builder names show
up?  And in those places, what are the nice-to-haves/need-to-haves about the
builder names?  (including length constraints)

I do not think it's a good idea to use nonobvious abbreviations in user-facing
places (e.g. buildbot pages and rietveld tryjobs popup).  In those places, I
think developers will be frustrated unless the names are self-contained (i.e.,
they can be mostly understood WITHOUT referencing a separate lookup table).  I
think we can have *some* stuff in a lookup table, like "MacMiniA" vs "MacMiniB"
to refer to different revisions of MacMini hardware.

[borenet, 2013/04/09 19:11:45]

1. Trybots on Rietveld - we want to see the minimal amount of information to
identify the platforms we want while reducing the giant size of the list. 
Unfortunately, we can't insert information from a lookup table here.
2. Trybots from submit_try - same as #1 except that we can get around too-long
or short and uninformative builder names through the use of filters and lists.
3. Bench dashboard - Related to #1 and #2 - Really, it's better to obtain and
use the relevant information, either from the builder name or using a lookup,
than to display the builder name itself.  For example, choose data by OS and
hardware rather than picking from a list of builder names.
4. Buildbot pages - I don't think it's as bad to have very short names here,
since the webpages themselves contain more information (OS, hardware, etc)

 # Use of this source code is governed by a BSD-style license that can be
 # found in the LICENSE file.
 
 # Sets up all the builders we want this buildbot master to run.
 
 from skia_master_scripts import factory as skia_factory
 from skia_master_scripts import utils
 from skia_master_scripts.utils import MakeBuilderSet, \
                                       MakeAndroidBuilderSet, \
                                       MakeChromeOSBuilderSet, \
@@ skipping 36 matching lines @@
   #
   # Don't put spaces or 'funny characters' within the builder names, so that
   # we can safely use the builder name as part of a filepath.
   #
   do_upload_results = active_master.is_production_host
 
   # Linux (Ubuntu12) on Shuttle with ATI5770 graphics card
   defaults['category'] = 'Linux'
   MakeBuilderSet(
       helper=helper,
-      builder_base_name='Skia_Shuttle_Ubuntu12_ATI5770_Float_%s_64',
+      os='Linux',
+      version='Ubuntu12',

[borenet, 2013/04/09 16:31:46]

Do we need "Linux" here?  Or could we just say os='Ubuntu' and version='12'?

[epoger, 2013/04/09 17:38:36]

I think either one is fine.  I would say that "Ubuntu" is a reasonable way of
abbreviating "Ubuntu Linux".

[borenet, 2013/04/09 19:11:45]

Agreed.  I just want to be consistent - if we use Ubuntu here, we shouldn't use
Linux anywhere else.

+      arch_width='32',
+      hardware='ATI5770',
+      do_upload_results=do_upload_results,
+      target_platform=skia_factory.TARGET_PLATFORM_LINUX,
+      environment_variables=
+          {'GYP_DEFINES': 'skia_scalar=float skia_mesa=0 skia_arch_width=32'},
+      gm_image_subdir='base-shuttle_ubuntu12_ati5770',
+      perf_output_basedir=perf_output_basedir_linux)
+  MakeBuilderSet(
+      helper=helper,
+      os='Linux',
+      version='Ubuntu12',
+      arch_width='64',
+      hardware='ATI5770',
       do_upload_results=do_upload_results,
       target_platform=skia_factory.TARGET_PLATFORM_LINUX,
       environment_variables=
           {'GYP_DEFINES': 'skia_scalar=float skia_mesa=0 skia_arch_width=64'},
       gm_image_subdir='base-shuttle_ubuntu12_ati5770',
       perf_output_basedir=perf_output_basedir_linux)
-  MakeBuilderSet(
-      helper=helper,
-      builder_base_name='Skia_Shuttle_Ubuntu12_ATI5770_Float_%s_32',
-      do_upload_results=do_upload_results,
-      target_platform=skia_factory.TARGET_PLATFORM_LINUX,
-      environment_variables=
-          {'GYP_DEFINES': 'skia_scalar=float skia_mesa=0 skia_arch_width=32'},
-      gm_image_subdir='base-shuttle_ubuntu12_ati5770',
-      perf_output_basedir=perf_output_basedir_linux)
 
   # Android (runs on a Linux buildbot slave)...
   defaults['category'] = 'Android'
   MakeAndroidBuilderSet(
       helper=helper,
-      builder_base_name='Skia_NexusS_4-1_Float_%s_32',
+      os='Android',

[borenet, 2013/04/09 16:31:46]

We can probably infer from the device name that the OS is Android, but I think
it's simpler to include it here.

[epoger, 2013/04/09 17:38:36]

Agreed.

+      version='4.1',

[borenet, 2013/04/09 16:31:46]

Does the '.' break the "no funny characters" rule?  We can use an underscore
instead if necessary.

[epoger, 2013/04/09 17:38:36]

It's often handy to use dots within version numbers, so I think underscore is
good as a field separator and then allow dots within any field.

+      arch_width='32',
+      hardware='NexusS',
       device='nexus_s',
       do_upload_results=do_upload_results,
       target_platform=skia_factory.TARGET_PLATFORM_LINUX,
       environment_variables={'GYP_DEFINES': 'skia_scalar=float skia_mesa=0'},
       gm_image_subdir='base-android-nexus-s',
       perf_output_basedir=perf_output_basedir_linux)
   MakeAndroidBuilderSet(
       helper=helper,
-      builder_base_name='Skia_Xoom_4-1_Float_%s_32',
+      os='Android',
+      version='4.1',

[borenet, 2013/04/09 16:31:46]

We keep our Android devices pretty up-to-date.  Only a couple of them are still
running 4.1.  I don't think we want to keep updating this number when we flash
new builds to devices.

[epoger, 2013/04/09 17:38:36]

If we don't intend to update this number, we should probably just leave the
field blank for these builders.  (Or use a "blank equivalent" value, like "x")

[borenet, 2013/04/09 19:11:45]

My only problem with this is that it wastes characters.  However, leaving the
field out will make parsing harder, since we'll have to guess which fields are
filled in.

[epoger, 2013/04/09 19:16:12]

Yeah, the extra 2 characters ("-x-" instead of "-") seems like a small price to
pay.

[borenet, 2013/04/09 19:30:09]

My ideal is to take a dictionary or object like this:

{ 'os': 'Ubuntu',
  'version': '12',
  'arch_width': '32',
  'gpu': 'ATI5770' }

And convert that into a builder name which contains all of that information in a
human-readable but concise form.  However, the question of what to do with blank
fields remains.  Here are a couple of options I can think of:
1. Use a single-character placeholder "x"
2. Use an empty string, eg. "Android--32-Xoom"
3. Include the keys of the dictionary somehow, eg
"os_Ubuntu-v_12-arch_64-gpu_ATI5770-role_Debug".  This adds more characters than
I'd like, but it would guarantee that we could parse back into dictionary form
even if we left out or re-ordered fields.

[epoger, 2013/04/09 19:40:13]

I like the idea of keeping the information as a directory, and then expressing
it as directly as possible in the builder name.

I think including the keys in the builder name will be really unwieldy for
humans.  I think option #1 is the best.

+      arch_width='32',
+      hardware='Xoom',
       device='xoom',
       do_upload_results=do_upload_results,
       target_platform=skia_factory.TARGET_PLATFORM_LINUX,
       environment_variables={'GYP_DEFINES': 'skia_scalar=float skia_mesa=0'},
       gm_image_subdir='base-android-xoom',
       perf_output_basedir=perf_output_basedir_linux)
   MakeAndroidBuilderSet(
       helper=helper,
-      builder_base_name='Skia_GalaxyNexus_4-1_Float_%s_32',
+      os='Android',
+      version='4.1',
+      arch_width='32',
+      hardware='GalaxyNexus',
       device='galaxy_nexus',
       do_upload_results=do_upload_results,
       target_platform=skia_factory.TARGET_PLATFORM_LINUX,
       environment_variables={'GYP_DEFINES': 'skia_scalar=float skia_mesa=0'},
       gm_image_subdir='base-android-galaxy-nexus',
       perf_output_basedir=perf_output_basedir_linux)
   MakeAndroidBuilderSet(
       helper=helper,
-      builder_base_name='Skia_Nexus4_4-1_Float_%s_32',
+      os='Android',
+      version='4.1',
+      arch_width='32',
+      hardware='Nexus4',
       device='nexus_4',
       do_upload_results=do_upload_results,
       target_platform=skia_factory.TARGET_PLATFORM_LINUX,
       environment_variables={'GYP_DEFINES': 'skia_scalar=float skia_mesa=0'},
       gm_image_subdir='base-android-nexus-4',
       perf_output_basedir=perf_output_basedir_linux)
   MakeAndroidBuilderSet(
       helper=helper,
-      builder_base_name='Skia_Nexus7_4-1_Float_%s_32',
+      os='Android',
+      version='4.1',
+      arch_width='32',
+      hardware='Nexus7',
       device='nexus_7',
       do_upload_results=do_upload_results,
       target_platform=skia_factory.TARGET_PLATFORM_LINUX,
       environment_variables={'GYP_DEFINES': 'skia_scalar=float skia_mesa=0'},
       gm_image_subdir='base-android-nexus-7',
       perf_output_basedir=perf_output_basedir_linux)
   MakeAndroidBuilderSet(
       helper=helper,
-      builder_base_name='Skia_Nexus10_4-1_Float_%s_32',
+      os='Android',
+      version='4.1',
+      arch_width='32',
+      hardware='Nexus10',
       device='nexus_10',
       do_upload_results=do_upload_results,
       target_platform=skia_factory.TARGET_PLATFORM_LINUX,
       environment_variables={'GYP_DEFINES': 'skia_scalar=float skia_mesa=0'},
       gm_image_subdir='base-android-nexus-10',
       perf_output_basedir=perf_output_basedir_linux)
   MakeAndroidBuilderSet(
       helper=helper,
-      builder_base_name='Skia_RazrI_4-1_Float_%s_32',
+      os='Android',
+      version='4.1',
+      arch_width='32',
+      hardware='RazrI',
       device='x86',
       do_upload_results=do_upload_results,
       target_platform=skia_factory.TARGET_PLATFORM_LINUX,
       environment_variables={'GYP_DEFINES': 'skia_scalar=float skia_mesa=0'},
       gm_image_subdir='base-android-razr-i',
       perf_output_basedir=perf_output_basedir_linux)
 
   # Mac 10.6 (SnowLeopard) ...
   defaults['category'] = 'Mac-10.6'
   MakeBuilderSet(
       helper=helper,
-      builder_base_name='Skia_Mac_Float_%s_32',
+      os='OSX',
+      version='10.6',
+      arch_width='32',
+      hardware='MacMini',

[borenet, 2013/04/09 16:31:46]

We can probably be more specific than MacMini, right?  Like GPU vendor, etc?  Do
all MacMinis have the same GPU?

[epoger, 2013/04/09 17:38:36]

No, different revs of the MacMini will vary a bit.  I suggest "MacMiniA",
"MacMiniB", or some similar variations.

One idea could be to use the Machine Model IDs as listed in
http://en.wikipedia.org/wiki/Mac_Mini#Specifications_3 : "Macmini4,1", etc. 
Although there are even sometimes variations within a model ID...

       do_upload_results=do_upload_results,
       target_platform=skia_factory.TARGET_PLATFORM_MAC,
       environment_variables=
           {'GYP_DEFINES': ('skia_osx_sdkroot=macosx10.6 skia_scalar=float '
                            'skia_arch_width=32')},
       gm_image_subdir='base-macmini',
       perf_output_basedir=perf_output_basedir_mac)
   MakeBuilderSet(
       helper=helper,
-      builder_base_name='Skia_Mac_Float_%s_64',
+      os='OSX',
+      version='10.6',
+      arch_width='64',
+      hardware='MacMini',
       do_upload_results=do_upload_results,
       target_platform=skia_factory.TARGET_PLATFORM_MAC,
       environment_variables=
           {'GYP_DEFINES': ('skia_osx_sdkroot=macosx10.6 skia_scalar=float '
                            'skia_arch_width=64')},
       gm_image_subdir='base-macmini',
       perf_output_basedir=perf_output_basedir_mac,
       compile_bot_warnings_as_errors=False)
 
   # Mac 10.7 (Lion) ...
   defaults['category'] = 'Mac-10.7'
   MakeBuilderSet(
       helper=helper,
-      builder_base_name='Skia_MacMiniLion_Float_%s_32',
+      os='OSX',
+      version='10.7',
+      arch_width='32',
+      hardware='MacMini',
       do_upload_results=do_upload_results,
       target_platform=skia_factory.TARGET_PLATFORM_MAC,
       environment_variables=
           {'GYP_DEFINES': 'skia_scalar=float skia_mesa=1 skia_arch_width=32'},
       gm_image_subdir='base-macmini-lion-float',
       perf_output_basedir=perf_output_basedir_mac)
   MakeBuilderSet(
       helper=helper,
-      builder_base_name='Skia_MacMiniLion_Float_%s_64',
+      os='OSX',
+      version='10.7',
+      arch_width='64',
+      hardware='MacMini',
       do_upload_results=do_upload_results,
       target_platform=skia_factory.TARGET_PLATFORM_MAC,
       environment_variables=
           {'GYP_DEFINES': 'skia_scalar=float skia_mesa=1 skia_arch_width=64'},
       gm_image_subdir='base-macmini-lion-float',
       perf_output_basedir=perf_output_basedir_mac,
       compile_bot_warnings_as_errors=False)
 
   # Mac 10.8 (Mountain Lion) ...
   defaults['category'] = 'Mac-10.8'
   MakeBuilderSet(
       helper=helper,
-      builder_base_name='Skia_MacMini_10_8_Float_%s_32',
+      os='OSX',
+      version='10.8',
+      arch_width='32',
+      hardware='MacMini',
       do_upload_results=do_upload_results,
       target_platform=skia_factory.TARGET_PLATFORM_MAC,
       environment_variables=
           {'GYP_DEFINES': 'skia_scalar=float skia_arch_width=32'},
       gm_image_subdir='base-macmini-10_8',
       perf_output_basedir=perf_output_basedir_mac)
   MakeBuilderSet(
       helper=helper,
-      builder_base_name='Skia_MacMini_10_8_Float_%s_64',
+      os='OSX',
+      version='10.8',
+      arch_width='64',
+      hardware='MacMini',
       do_upload_results=do_upload_results,
       target_platform=skia_factory.TARGET_PLATFORM_MAC,
       environment_variables=
           {'GYP_DEFINES': 'skia_scalar=float skia_arch_width=64'},
       gm_image_subdir='base-macmini-10_8',
       perf_output_basedir=perf_output_basedir_mac,
       compile_bot_warnings_as_errors=False)
 
   # Windows7 running on Shuttle PC with Intel Core i7-2600 with on-CPU graphics
   defaults['category'] = 'Win7'
   MakeBuilderSet(
       helper=helper,
-      builder_base_name='Skia_Shuttle_Win7_Intel_Float_%s_32',
+      os='Win',
+      version='7',
+      arch_width='32',
+      hardware='Intel',

[borenet, 2013/04/09 16:31:46]

This refers to the Intel integrated graphics.  The meaning of "hardware" is a
little ambiguous to me; it could mean the form factor or computer type (MacMini)
OR it could be the type of GPU.  I hesitate to include both types of
information, since our machines are all either MacMini or Shuttle.  Maybe
"hardware" should just become "gpu".

[epoger, 2013/04/09 17:38:36]

I think it would be a good idea to split this into two fields: 1.GPU and
2.computer type (this would imply CPU, memory, and other chipset parts).

I think we *could* leave the "computer type" field blank for now, if you like
(because it's either MacMini or Shuttle in most cases).  Or, to avoid schema
changes in the future, we could use IDs like these: Macmini4,1 Macmini5,1 
MacPro3,1  ShuttleA  GCE  etc.

For our current Shuttle build ("ShuttleA" -> Intel core i7-2600, etc.), the GPU
field could read "integrated", or maybe "HD4000"...

       do_upload_results=do_upload_results,
       target_platform=skia_factory.TARGET_PLATFORM_WIN32,
       environment_variables=
           {'GYP_DEFINES': ('skia_scalar=float skia_arch_width=32 '
                            'skia_win_debuggers_path=c:/DbgHelp')},
       gm_image_subdir='base-shuttle-win7-intel-float',
       perf_output_basedir=perf_output_basedir_windows)
   MakeBuilderSet(
       helper=helper,
-      builder_base_name='Skia_Shuttle_Win7_Intel_Float_%s_64',
+      os='Win',
+      version='7',
+      arch_width='64',
+      hardware='Intel',
       do_upload_results=do_upload_results,
       target_platform=skia_factory.TARGET_PLATFORM_WIN32,
       environment_variables=
           {'GYP_DEFINES': ('skia_scalar=float skia_arch_width=64 '
                            'skia_win_debuggers_path=c:/DbgHelp')},
       gm_image_subdir='base-shuttle-win7-intel-float',
       perf_output_basedir=perf_output_basedir_windows,
       compile_bot_warnings_as_errors=False)
 
   # Special-purpose Win7 builders
   defaults['category'] = 'Win7-Special'
   MakeBuilderSet(
       helper=helper,
-      builder_base_name='Skia_Shuttle_Win7_Intel_Float_ANGLE_%s_32',
+      os='Win',
+      version='7',
+      arch_width='32',
+      hardware='Intel',
+      extra_config='ANGLE',

[borenet, 2013/04/09 16:31:46]

"extra_config" seemed like the best way to describe things like "ANGLE" and
"DirectWrite".  Thoughts?

[epoger, 2013/04/09 17:38:36]

Sure.

       do_upload_results=do_upload_results,
       target_platform=skia_factory.TARGET_PLATFORM_WIN32,
       environment_variables=
           {'GYP_DEFINES': ('skia_scalar=float skia_angle=1 skia_arch_width=32 '
                            'skia_win_debuggers_path=c:/DbgHelp')},
       gm_image_subdir='base-shuttle-win7-intel-angle',
       perf_output_basedir=perf_output_basedir_windows,
       gm_args=['--config', 'angle'],
       bench_args=['--config', 'ANGLE'],
       bench_pictures_cfg='angle')
   MakeBuilderSet(
       helper=helper,
-      builder_base_name='Skia_Shuttle_Win7_Intel_Float_DirectWrite_%s_32',
+      os='Win',
+      version='7',
+      arch_width='32',
+      hardware='Intel',
+      extra_config='DirectWrite',
       do_upload_results=do_upload_results,
       target_platform=skia_factory.TARGET_PLATFORM_WIN32,
       environment_variables=
           {'GYP_DEFINES':
            ('skia_scalar=float skia_directwrite=1 skia_arch_width=32 '
             'skia_win_debuggers_path=c:/DbgHelp')},
       gm_image_subdir='base-shuttle-win7-intel-directwrite',
       perf_output_basedir=perf_output_basedir_windows)
 
   defaults['category'] = 'iOS'
   MakeIOSBuilderSet(
       helper=helper,
-      builder_base_name='Skia_iOS_%s_32',
+      os='iOS',
+      version='6',
+      arch_width='32',
+      hardware='iPhone',
       do_upload_results=do_upload_results,
       target_platform=skia_factory.TARGET_PLATFORM_MAC,
       environment_variables={'GYP_DEFINES': 'skia_os=ios'},
       gm_image_subdir=None,
       perf_output_basedir=None,
       do_debug=False,
       do_release=False,
       do_bench=False)
 
   # House Keeping
   defaults['category'] = '  Housekeeping'
   MakeHousekeeperBuilderSet(
       helper=helper,
       do_trybots=True,
       do_upload_results=do_upload_results)
 
   # "Special" bots, running on Linux
   defaults['category'] = 'Linux-Special'
   MakeBuilderSet(
       helper=helper,
-      builder_base_name='Skia_Linux_NoGPU_%s_32',
+      os='Linux',
+      version='Ubuntu12',
+      arch_width='32',
+      hardware='NoGPU',

[borenet, 2013/04/09 16:31:46]

NoGPU is more of an extra_config than a hardware type, but it doesn't make much
sense to say "Intel" or "ATI5770" and then "NoGPU".

[epoger, 2013/04/09 17:38:36]

I think NoGPU should be listed as extra_config, because it describes some extra
configs that were passed to the build, NOT characteristics of the machine on
which the tests were run.

[borenet, 2013/04/09 19:11:45]

If a gpu field is added, that's where NoGPU should go.

[epoger, 2013/04/09 19:16:12]

I can see arguments either way (either gpu field or extra_config field).  Either
way should be fine.

[borenet, 2013/04/09 19:30:09]

I think this is an interesting distinction.  For the NoGPU bot, we *don't care*
what GPU the machine itself has.  For most bots, the builder corresponds pretty
directly to a specific machine setup, but for NoGPU and the Compile bots, it
doesn't matter.

[epoger, 2013/04/09 19:40:13]

Agreed on all points.

       do_upload_results=do_upload_results,
       target_platform=skia_factory.TARGET_PLATFORM_LINUX,
       environment_variables=
           {'GYP_DEFINES': 'skia_scalar=float skia_gpu=0 skia_arch_width=64'},
       gm_image_subdir='base-shuttle_ubuntu12_ati5770',
       perf_output_basedir=None, # no perf measurement for debug builds
       bench_pictures_cfg='no_gpu',
       do_release=False,
       do_bench=False)
 
   MakeNaClBuilderSet(
       helper=helper,
-      builder_base_name='Skia_Shuttle_Ubuntu12_NaCl_%s',
+      os='Linux',
+      version='Ubuntu12',
+      arch_width='32_64',
+      hardware='NaCl',

[borenet, 2013/04/09 16:31:46]

NaCl isn't hardware at all.  Maybe this is an extra_config too?

[epoger, 2013/04/09 17:38:36]

Yes, seems more like extra_config

       do_upload_results=do_upload_results,
       target_platform=skia_factory.TARGET_PLATFORM_LINUX,
       gm_image_subdir=None,
       perf_output_basedir=None,
       do_debug=False,
       do_release=False,
       do_bench=False)
 
   # Chrome OS
   defaults['category'] = 'ChromeOS'
   MakeChromeOSBuilderSet(
       helper=helper,
-      builder_base_name='Skia_ChromeOS_Alex_%s_32',
+      os='ChromeOS',
+      version='V',

[borenet, 2013/04/09 16:31:46]

AFAIK the ChromeOS version gets updated automatically.  I don't think we want to
have to change this number with each update.

[epoger, 2013/04/09 17:38:36]

Agreed.  Blank, or 'x', something like that.

+      arch_width='32',
+      hardware='Alex',
       do_upload_results=do_upload_results,
       target_platform=skia_factory.TARGET_PLATFORM_LINUX,
       environment_variables=
           {'GYP_DEFINES': 'skia_arch_width=32 skia_gpu=0'},
       gm_image_subdir=None,
       perf_output_basedir=perf_output_basedir_linux,
       bench_pictures_cfg='no_gpu')
 
   return helper.Update(c)