Added GYP defines to dump linux symbols for gpu perf bots and trybots.

Added GYP defines to dump linux symbols for gpu perf bots and trybots.

R=agable@chromium.org, kbr@chromium.org
BUG=405623
Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=294706

[Ken Russell (switch to Gerrit), 2015-04-02 00:28:36]

kbr@chromium.org changed reviewers:
+ iannucci@chromium.org, phajdan.jr@chromium.org, sergiyb@chromium.org

[Ken Russell (switch to Gerrit), 2015-04-02 00:28:37]

Have you verified locally that fastbuild=1 linux_dump_symbols=1 still produces
reasonable stack traces? We should make sure this change does what's desired.

CC'ing other reviewers since this impacts a fair number of bots.

https://codereview.chromium.org/1058633004/diff/1/scripts/slave/recipe_module...
File scripts/slave/recipe_modules/chromium/config.py (right):

https://codereview.chromium.org/1058633004/diff/1/scripts/slave/recipe_module...
scripts/slave/recipe_modules/chromium/config.py:755: if c.TARGET_PLATFORM ==
'linux':
Given the large number of expectation changes in this CL, and the intended
impact, I think we should restrict this further. Can you figure out a way to
have these changes not apply to Android or Chrome OS?

[Sergiy Byelozyorov, 2015-04-02 09:22:18]

the change looks fine with me, but I'd also constrain affected bots further
since it may produce too much unnecessary log output

[Paweł Hajdan Jr., 2015-04-02 09:55:40]

LGTM

[David Yen, 2015-04-02 18:27:52]

I've verified that fastbuild=1 still produces the correct stack traces. I built
isolates with those settings and ran them on both Mac and Linux. Here are the
hashes:

Linux: c8c941c279e4270eb74c80346516eccde555fc6f
Mac: 6c5bec4c90919da3e8e8f392eeeaf0697d3c62da

https://codereview.chromium.org/1058633004/diff/1/scripts/slave/recipe_module...
File scripts/slave/recipe_modules/chromium/config.py (right):

https://codereview.chromium.org/1058633004/diff/1/scripts/slave/recipe_module...
scripts/slave/recipe_modules/chromium/config.py:755: if c.TARGET_PLATFORM ==
'linux':
On 2015/04/02 00:28:37, Ken Russell wrote:
> Given the large number of expectation changes in this CL, and the intended
> impact, I think we should restrict this further. Can you figure out a way to
> have these changes not apply to Android or Chrome OS?

For android and chrome OS targets TARGET_PLATFORM should be 'android' and
'chromeos'. It looks like this is true for most of the configurations (for
example, none of the blink android expectations were regenerated). The only
exception to this rule is for the "android tester" and
"chromium_linux_linux_chromium_chromeos". Are these configured incorrectly?
Perhaps someone from the infra team can chime in here.

[Ken Russell (switch to Gerrit), 2015-04-02 22:07:31]

On 2015/04/02 18:27:52, David Yen wrote:
> I've verified that fastbuild=1 still produces the correct stack traces. I
built
> isolates with those settings and ran them on both Mac and Linux. Here are the
> hashes:
> 
> Linux: c8c941c279e4270eb74c80346516eccde555fc6f
> Mac: 6c5bec4c90919da3e8e8f392eeeaf0697d3c62da
> 
>
https://codereview.chromium.org/1058633004/diff/1/scripts/slave/recipe_module...
> File scripts/slave/recipe_modules/chromium/config.py (right):
> 
>
https://codereview.chromium.org/1058633004/diff/1/scripts/slave/recipe_module...
> scripts/slave/recipe_modules/chromium/config.py:755: if c.TARGET_PLATFORM ==
> 'linux':
> On 2015/04/02 00:28:37, Ken Russell wrote:
> > Given the large number of expectation changes in this CL, and the intended
> > impact, I think we should restrict this further. Can you figure out a way to
> > have these changes not apply to Android or Chrome OS?
> 
> For android and chrome OS targets TARGET_PLATFORM should be 'android' and
> 'chromeos'. It looks like this is true for most of the configurations (for
> example, none of the blink android expectations were regenerated). The only
> exception to this rule is for the "android tester" and
> "chromium_linux_linux_chromium_chromeos". Are these configured incorrectly?
> Perhaps someone from the infra team can chime in here.

Yes, those test expectation changes are confusing -- at least the android one. I
can imagine that "linux_chromium_chromeos" is actually the Linux build of Chrome
in "Chrome OS" flavor -- which should probably be affected by your change here.

LGTM once you've got this figured out. Thanks for double-checking that
fastbuild=1 doesn't break the stack traces.

[David Yen, 2015-04-03 17:20:53]

On 2015/04/02 22:07:31, Ken Russell wrote:
> On 2015/04/02 18:27:52, David Yen wrote:
> > I've verified that fastbuild=1 still produces the correct stack traces. I
> built
> > isolates with those settings and ran them on both Mac and Linux. Here are
the
> > hashes:
> > 
> > Linux: c8c941c279e4270eb74c80346516eccde555fc6f
> > Mac: 6c5bec4c90919da3e8e8f392eeeaf0697d3c62da
> > 
> >
>
https://codereview.chromium.org/1058633004/diff/1/scripts/slave/recipe_module...
> > File scripts/slave/recipe_modules/chromium/config.py (right):
> > 
> >
>
https://codereview.chromium.org/1058633004/diff/1/scripts/slave/recipe_module...
> > scripts/slave/recipe_modules/chromium/config.py:755: if c.TARGET_PLATFORM ==
> > 'linux':
> > On 2015/04/02 00:28:37, Ken Russell wrote:
> > > Given the large number of expectation changes in this CL, and the intended
> > > impact, I think we should restrict this further. Can you figure out a way
to
> > > have these changes not apply to Android or Chrome OS?
> > 
> > For android and chrome OS targets TARGET_PLATFORM should be 'android' and
> > 'chromeos'. It looks like this is true for most of the configurations (for
> > example, none of the blink android expectations were regenerated). The only
> > exception to this rule is for the "android tester" and
> > "chromium_linux_linux_chromium_chromeos". Are these configured incorrectly?
> > Perhaps someone from the infra team can chime in here.
> 
> Yes, those test expectation changes are confusing -- at least the android one.
I
> can imagine that "linux_chromium_chromeos" is actually the Linux build of
Chrome
> in "Chrome OS" flavor -- which should probably be affected by your change
here.
> 
> LGTM once you've got this figured out. Thanks for double-checking that
> fastbuild=1 doesn't break the stack traces.

@phajdan.jr - I added the TARGET_PLATFORM configuration as "android" for the
android testers. This sounds like what we want. Could you take a look to be sure
I did it right?

@kbr - After setting the android configuration correctly for the android tester
recipes, the android expectations no longer change.

[Ken Russell (switch to Gerrit), 2015-04-03 21:07:13]

On 2015/04/03 17:20:53, David Yen wrote:
> On 2015/04/02 22:07:31, Ken Russell wrote:
> > On 2015/04/02 18:27:52, David Yen wrote:
> > > I've verified that fastbuild=1 still produces the correct stack traces. I
> > built
> > > isolates with those settings and ran them on both Mac and Linux. Here are
> the
> > > hashes:
> > > 
> > > Linux: c8c941c279e4270eb74c80346516eccde555fc6f
> > > Mac: 6c5bec4c90919da3e8e8f392eeeaf0697d3c62da
> > > 
> > >
> >
>
https://codereview.chromium.org/1058633004/diff/1/scripts/slave/recipe_module...
> > > File scripts/slave/recipe_modules/chromium/config.py (right):
> > > 
> > >
> >
>
https://codereview.chromium.org/1058633004/diff/1/scripts/slave/recipe_module...
> > > scripts/slave/recipe_modules/chromium/config.py:755: if c.TARGET_PLATFORM
==
> > > 'linux':
> > > On 2015/04/02 00:28:37, Ken Russell wrote:
> > > > Given the large number of expectation changes in this CL, and the
intended
> > > > impact, I think we should restrict this further. Can you figure out a
way
> to
> > > > have these changes not apply to Android or Chrome OS?
> > > 
> > > For android and chrome OS targets TARGET_PLATFORM should be 'android' and
> > > 'chromeos'. It looks like this is true for most of the configurations (for
> > > example, none of the blink android expectations were regenerated). The
only
> > > exception to this rule is for the "android tester" and
> > > "chromium_linux_linux_chromium_chromeos". Are these configured
incorrectly?
> > > Perhaps someone from the infra team can chime in here.
> > 
> > Yes, those test expectation changes are confusing -- at least the android
one.
> I
> > can imagine that "linux_chromium_chromeos" is actually the Linux build of
> Chrome
> > in "Chrome OS" flavor -- which should probably be affected by your change
> here.
> > 
> > LGTM once you've got this figured out. Thanks for double-checking that
> > fastbuild=1 doesn't break the stack traces.
> 
> @phajdan.jr - I added the TARGET_PLATFORM configuration as "android" for the
> android testers. This sounds like what we want. Could you take a look to be
sure
> I did it right?
> 
> @kbr - After setting the android configuration correctly for the android
tester
> recipes, the android expectations no longer change.

Nice. LGTM

[David Yen, 2015-04-07 21:15:39]

The CQ bit was checked by dyen@chromium.org

[David Yen, 2015-04-07 21:15:40]

The patchset sent to the CQ was uploaded after l-g-t-m from
phajdan.jr@chromium.org
Link to the patchset: https://codereview.chromium.org/1058633004/#ps20001
(title: "Configured android testers to target platform "android"")

[commit-bot: I haz the power, 2015-04-07 21:15:49]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1058633004/20001

[commit-bot: I haz the power, 2015-04-07 21:21:25]

Committed patchset #2 (id:20001) as
http://src.chromium.org/viewvc/chrome?view=rev&revision=294706

[Adrian Kuegel, 2015-04-08 14:02:51]

On 2015/04/07 21:21:25, I haz the power (commit-bot) wrote:
> Committed patchset #2 (id:20001) as
> http://src.chromium.org/viewvc/chrome?view=rev&revision=294706

This patch also affected internal recipes, so their expectations should have
been regenerated as well. I have a CL to add the updated expectations (just
mentioning it here in case someone else is working on that).

[David Yen, 2015-04-08 16:19:05]

On 2015/04/08 14:02:51, Adrian Kuegel wrote:
> On 2015/04/07 21:21:25, I haz the power (commit-bot) wrote:
> > Committed patchset #2 (id:20001) as
> > http://src.chromium.org/viewvc/chrome?view=rev&revision=294706
> 
> This patch also affected internal recipes, so their expectations should have
> been regenerated as well. I have a CL to add the updated expectations (just
> mentioning it here in case someone else is working on that).

Great, thanks for handling that for me!

[Vadim Sh., 2015-04-08 22:19:27]

On 2015/04/08 16:19:05, David Yen wrote:
> On 2015/04/08 14:02:51, Adrian Kuegel wrote:
> > On 2015/04/07 21:21:25, I haz the power (commit-bot) wrote:
> > > Committed patchset #2 (id:20001) as
> > > http://src.chromium.org/viewvc/chrome?view=rev&revision=294706
> > 
> > This patch also affected internal recipes, so their expectations should have
> > been regenerated as well. I have a CL to add the updated expectations (just
> > mentioning it here in case someone else is working on that).
> 
> Great, thanks for handling that for me!

We suspect this CL is causing many slaves to go offline due to "disk is full"
error.
Can somebody who knows about debug symbols estimate what their total size would
be? For example for "linux_chromium_gn_rel" bot?
I'm considering reverting it to stop bots to die until we figure out more
appropriate solution.

[Ken Russell (switch to Gerrit), 2015-04-08 22:58:40]

On 2015/04/08 22:19:27, Vadim Sh. wrote:
> On 2015/04/08 16:19:05, David Yen wrote:
> > On 2015/04/08 14:02:51, Adrian Kuegel wrote:
> > > On 2015/04/07 21:21:25, I haz the power (commit-bot) wrote:
> > > > Committed patchset #2 (id:20001) as
> > > > http://src.chromium.org/viewvc/chrome?view=rev&revision=294706
> > > 
> > > This patch also affected internal recipes, so their expectations should
have
> > > been regenerated as well. I have a CL to add the updated expectations
(just
> > > mentioning it here in case someone else is working on that).
> > 
> > Great, thanks for handling that for me!
> 
> We suspect this CL is causing many slaves to go offline due to "disk is full"
> error.
> Can somebody who knows about debug symbols estimate what their total size
would
> be? For example for "linux_chromium_gn_rel" bot?
> I'm considering reverting it to stop bots to die until we figure out more
> appropriate solution.

I don't know how much larger the output directory is when these symbols are
generated. I'll do side-by-side builds and update crbug.com/405623 . Please note
that other recipe expectations had to be updated as a consequence of this CL and
I don't know in which workspace they are.

[jam, 2015-04-13 23:50:12]

A revert of this CL (patchset #2 id:20001) has been created in
https://codereview.chromium.org/1081103005/ by jam@chromium.org.

The reason for reverting is: Looks like this is responsible for a 13x increase
in files that are being pushed to isolate servers, making the isolate step take
30 minutes on try runs and nearly doubling the run time of the linux & cros
trybots.

See "CQ stats for the week of Mar 30" thread on chrome-infra.