Modify Skia recipes to allow running XSAN on Swarming.

Modify Skia recipes to allow running XSAN on Swarming. TSAN seems to work, ASAN and MSAN are failing.

Depends on https://codereview.chromium.org/1859413003

Notes:
 - The XSAN flavor depends on llvm-build, pulled in from Chromium DEPS runhooks, for llvm_symbolizer, which allows suppressions based on symbol name.
 - MSAN depends on llvm being compiled to msan_out. To include this and llvm-build in the compile isolate, I added copy_extra_build_products to flavors.
 - I changed SkiaApi.swarming_out_dir to be a Path instead of a string.

BUG=skia:5157
Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=300439

[dogben, 2016-05-02 16:53:44]

Patchset #4 (id:60001) has been deleted

[dogben, 2016-05-02 16:53:47]

Patchset #3 (id:40001) has been deleted

[dogben, 2016-05-02 16:53:51]

Patchset #2 (id:20001) has been deleted

[dogben, 2016-05-02 16:53:54]

Patchset #1 (id:1) has been deleted

[dogben, 2016-05-02 17:15:03]

Patchset #1 (id:80001) has been deleted

[dogben, 2016-05-02 18:52:12]

Patchset #2 (id:120001) has been deleted

[dogben, 2016-05-02 18:52:16]

Patchset #1 (id:100001) has been deleted

[dogben, 2016-05-03 23:36:42]

Description was changed from

==========
Modify Skia recipes to allow running XSAN on Swarming.

BUG=
==========

to

==========
Modify Skia recipes to allow running XSAN on Swarming. TSAN seems to work, ASAN
and MSAN are failing.

Depends on https://codereview.chromium.org/1859413003

Notes:
 - The XSAN flavor depends on llvm-build, pulled in from Chromium DEPS runhooks,
for llvm_symbolizer, which allows suppressions based on symbol name.
 - MSAN depends on llvm being compiled to msan_out. To include this and
llvm-build in the compile isolate, I added copy_extra_build_products to flavors.
 - I changed SkiaApi.swarming_out_dir to be a Path instead of a string.

BUG=skia:5157
==========

[dogben, 2016-05-03 23:37:06]

Patchset #1 (id:140001) has been deleted

[dogben, 2016-05-03 23:39:28]

benjaminwagner@google.com changed reviewers:
+ borenet@google.com

[dogben, 2016-05-03 23:39:43]

The CQ bit was checked by benjaminwagner@google.com to run a CQ dry run

[commit-bot: I haz the power, 2016-05-03 23:39:46]

Dry run: CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1920283002/160001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1920283002/160001

[commit-bot: I haz the power, 2016-05-03 23:39:48]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-05-03 23:39:49]

Dry run: No L-G-T-M from a valid reviewer yet. 
CQ run can only be started by full committers or once the patch has
received an L-G-T-M from a full committer.
Even if an L-G-T-M may have been provided, it was from a non-committer,
_not_ a full super star committer.
Committers are members of the group "project-infra-committers".
Note that this has nothing to do with OWNERS files.

[borenet, 2016-05-04 11:15:55]

LGTM with a nit and a question.

https://codereview.chromium.org/1920283002/diff/160001/masters/master.client....
File masters/master.client.skia.fyi/builders.cfg (right):

https://codereview.chromium.org/1920283002/diff/160001/masters/master.client....
masters/master.client.skia.fyi/builders.cfg:55: { 'name':
'Test-Ubuntu-GCC-GCE-CPU-AVX2-x86_64-Debug-SwarmingMSAN',
Is this an important difference?

https://codereview.chromium.org/1920283002/diff/160001/scripts/slave/recipe_m...
File scripts/slave/recipe_modules/skia/xsan_flavor.py (right):

https://codereview.chromium.org/1920283002/diff/160001/scripts/slave/recipe_m...
scripts/slave/recipe_modules/skia/xsan_flavor.py:31: }[key]
I think I'd just do this:

}[self._skia_api.builder_cfg['extra_config'].strip('Swarming')]

[dogben, 2016-05-04 13:48:02]

https://codereview.chromium.org/1920283002/diff/160001/masters/master.client....
File masters/master.client.skia.fyi/builders.cfg (right):

https://codereview.chromium.org/1920283002/diff/160001/masters/master.client....
masters/master.client.skia.fyi/builders.cfg:55: { 'name':
'Test-Ubuntu-GCC-GCE-CPU-AVX2-x86_64-Debug-SwarmingMSAN',
On 2016/05/04 at 11:15:55, borenet wrote:
> Is this an important difference?

When I added these, I hadn't realized that the current ASAN and MSAN bots are
Debug but the TSAN bot is Release. I had copy-and-pasted TSAN for ASAN and MSAN.

There will definitely be a difference, because Debug will enable code/asserts
that might not be XSAN-clean. I don't know how important that is.

My preference is that we don't introduce more differences between the current
bot and the new bot.

https://codereview.chromium.org/1920283002/diff/160001/scripts/slave/recipe_m...
File scripts/slave/recipe_modules/skia/xsan_flavor.py (right):

https://codereview.chromium.org/1920283002/diff/160001/scripts/slave/recipe_m...
scripts/slave/recipe_modules/skia/xsan_flavor.py:31: }[key]
On 2016/05/04 at 11:15:55, borenet wrote:
> I think I'd just do this:
> 
> }[self._skia_api.builder_cfg['extra_config'].strip('Swarming')]

I can do that if you think it's best, but I want to make sure you know that the
argument to strip is treated as a set of characters, e.g.:
>>> 'swarmingtsan'.strip('swarming')
't'

[dogben, 2016-05-04 13:49:57]

https://codereview.chromium.org/1920283002/diff/160001/scripts/slave/recipe_m...
File scripts/slave/recipe_modules/skia/xsan_flavor.py (right):

https://codereview.chromium.org/1920283002/diff/160001/scripts/slave/recipe_m...
scripts/slave/recipe_modules/skia/xsan_flavor.py:31: }[key]
On 2016/05/04 at 13:48:02, Ben Wagner wrote:
> On 2016/05/04 at 11:15:55, borenet wrote:
> > I think I'd just do this:
> > 
> > }[self._skia_api.builder_cfg['extra_config'].strip('Swarming')]
> 
> I can do that if you think it's best, but I want to make sure you know that
the argument to strip is treated as a set of characters, e.g.:
> >>> 'swarmingtsan'.strip('swarming')
> 't'

I'll change it to self._skia_api.builder_cfg['extra_config'].replace('Swarming',
'')

[borenet, 2016-05-04 13:51:57]

https://codereview.chromium.org/1920283002/diff/160001/scripts/slave/recipe_m...
File scripts/slave/recipe_modules/skia/xsan_flavor.py (right):

https://codereview.chromium.org/1920283002/diff/160001/scripts/slave/recipe_m...
scripts/slave/recipe_modules/skia/xsan_flavor.py:31: }[key]
On 2016/05/04 13:49:56, Ben Wagner wrote:
> On 2016/05/04 at 13:48:02, Ben Wagner wrote:
> > On 2016/05/04 at 11:15:55, borenet wrote:
> > > I think I'd just do this:
> > > 
> > > }[self._skia_api.builder_cfg['extra_config'].strip('Swarming')]
> > 
> > I can do that if you think it's best, but I want to make sure you know that
> the argument to strip is treated as a set of characters, e.g.:
> > >>> 'swarmingtsan'.strip('swarming')
> > 't'
> 
> I'll change it to
self._skia_api.builder_cfg['extra_config'].replace('Swarming',
> '')

Oh, right. Replace SGTM, or you can leave it as-is.

[borenet, 2016-05-04 13:52:58]

https://codereview.chromium.org/1920283002/diff/160001/masters/master.client....
File masters/master.client.skia.fyi/builders.cfg (right):

https://codereview.chromium.org/1920283002/diff/160001/masters/master.client....
masters/master.client.skia.fyi/builders.cfg:55: { 'name':
'Test-Ubuntu-GCC-GCE-CPU-AVX2-x86_64-Debug-SwarmingMSAN',
On 2016/05/04 13:48:02, Ben Wagner wrote:
> On 2016/05/04 at 11:15:55, borenet wrote:
> > Is this an important difference?
> 
> When I added these, I hadn't realized that the current ASAN and MSAN bots are
> Debug but the TSAN bot is Release. I had copy-and-pasted TSAN for ASAN and
MSAN.
> 
> There will definitely be a difference, because Debug will enable code/asserts
> that might not be XSAN-clean. I don't know how important that is.
> 
> My preference is that we don't introduce more differences between the current
> bot and the new bot.

Agreed.

[dogben, 2016-05-04 13:54:43]

The CQ bit was checked by benjaminwagner@google.com

[dogben, 2016-05-04 13:54:43]

The patchset sent to the CQ was uploaded after l-g-t-m from borenet@google.com
Link to the patchset: https://codereview.chromium.org/1920283002/#ps180001
(title: "Changes based on code review.")

[commit-bot: I haz the power, 2016-05-04 13:54:51]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1920283002/180001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1920283002/180001

[commit-bot: I haz the power, 2016-05-04 13:54:52]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-05-04 13:54:53]

No L-G-T-M from a valid reviewer yet. 
CQ run can only be started by full committers or once the patch has
received an L-G-T-M from a full committer.
Even if an L-G-T-M may have been provided, it was from a non-committer,
_not_ a full super star committer.
Committers are members of the group "project-infra-committers".
Note that this has nothing to do with OWNERS files.

[borenet2, 2016-05-04 13:55:41]

On 2016/05/04 13:54:53, commit-bot: I haz the power wrote:
> No L-G-T-M from a valid reviewer yet. 
> CQ run can only be started by full committers or once the patch has
> received an L-G-T-M from a full committer.
> Even if an L-G-T-M may have been provided, it was from a non-committer,
> _not_ a full super star committer.
> Committers are members of the group "project-infra-committers".
> Note that this has nothing to do with OWNERS files.

LGTM

[borenet2, 2016-05-04 13:55:47]

The CQ bit was checked by borenet@chromium.org

[commit-bot: I haz the power, 2016-05-04 13:55:52]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1920283002/180001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1920283002/180001

[commit-bot: I haz the power, 2016-05-04 14:02:31]

Description was changed from

==========
Modify Skia recipes to allow running XSAN on Swarming. TSAN seems to work, ASAN
and MSAN are failing.

Depends on https://codereview.chromium.org/1859413003

Notes:
 - The XSAN flavor depends on llvm-build, pulled in from Chromium DEPS runhooks,
for llvm_symbolizer, which allows suppressions based on symbol name.
 - MSAN depends on llvm being compiled to msan_out. To include this and
llvm-build in the compile isolate, I added copy_extra_build_products to flavors.
 - I changed SkiaApi.swarming_out_dir to be a Path instead of a string.

BUG=skia:5157
==========

to

==========
Modify Skia recipes to allow running XSAN on Swarming. TSAN seems to work, ASAN
and MSAN are failing.

Depends on https://codereview.chromium.org/1859413003

Notes:
 - The XSAN flavor depends on llvm-build, pulled in from Chromium DEPS runhooks,
for llvm_symbolizer, which allows suppressions based on symbol name.
 - MSAN depends on llvm being compiled to msan_out. To include this and
llvm-build in the compile isolate, I added copy_extra_build_products to flavors.
 - I changed SkiaApi.swarming_out_dir to be a Path instead of a string.

BUG=skia:5157

Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=300439
==========

[commit-bot: I haz the power, 2016-05-04 14:02:32]

Committed patchset #2 (id:180001) as
http://src.chromium.org/viewvc/chrome?view=rev&revision=300439