Collect failed "target"s from ninja in the compile step on Waterfall.

Collect failed "target"s from ninja in the compile step on Waterfall.

The "target"s here are actually the output nodes of the failed edges in ninja's build graph.
This CL scrapes the ninja output to extract the failed "target"s and set them as a build property so that it shows up in the build json data.

Simulation test passed for build_internal, thus no expectation change is needed there.

BUG=592715, 592808

[stgao, 2016-03-08 00:11:18]

Description was changed from

==========
Collect failed compile targets on Waterfall.

BUG=
==========

to

==========
Collect failed compile targets on Waterfall.

BUG=592715, 592808
==========

[stgao, 2016-03-08 00:13:27]

Description was changed from

==========
Collect failed compile targets on Waterfall.

BUG=592715, 592808
==========

to

==========
Collect failed "target" in the compile step on Waterfall.

BUG=592715, 592808
==========

[stgao, 2016-03-08 00:36:23]

Description was changed from

==========
Collect failed "target" in the compile step on Waterfall.

BUG=592715, 592808
==========

to

==========
Collect failed "target" in the compile step on Waterfall.

Note: this CL needs to hold until ninja 1.7 is released and used by default on
all bots.

BUG=592715, 592808
==========

[stgao, 2016-03-08 02:09:04]

Patchset #2 (id:20001) has been deleted

[stgao, 2016-03-08 02:09:15]

Patchset #2 (id:40001) has been deleted

[stgao, 2016-03-08 02:22:24]

Description was changed from

==========
Collect failed "target" in the compile step on Waterfall.

Note: this CL needs to hold until ninja 1.7 is released and used by default on
all bots.

BUG=592715, 592808
==========

to

==========
Collect failed "target" from ninja in the compile step on Waterfall.

Note: this CL needs to hold until ninja 1.7 is released and used by default on
all bots.

BUG=592715, 592808
==========

[stgao, 2016-03-10 01:05:40]

Patchset #2 (id:60001) has been deleted

[stgao, 2016-03-22 18:05:33]

Description was changed from

==========
Collect failed "target" from ninja in the compile step on Waterfall.

Note: this CL needs to hold until ninja 1.7 is released and used by default on
all bots.

BUG=592715, 592808
==========

to

==========
Collect failed "target" from ninja in the compile step on Waterfall.

The "target" here is actually the output nodes of the failed edges in ninja's
build graph.

Note: this CL needs to hold until ninja 1.7 is released and used by default on
all bots.

BUG=592715, 592808
==========

[stgao, 2016-03-23 20:30:49]

Patchset #3 (id:100001) has been deleted

[stgao, 2016-03-23 20:33:11]

Patchset #3 (id:120001) has been deleted

[stgao, 2016-03-23 21:03:18]

Description was changed from

==========
Collect failed "target" from ninja in the compile step on Waterfall.

The "target" here is actually the output nodes of the failed edges in ninja's
build graph.

Note: this CL needs to hold until ninja 1.7 is released and used by default on
all bots.

BUG=592715, 592808
==========

to

==========
Collect failed "target" from ninja in the compile step on Waterfall.

The "target" here is actually the output nodes of the failed edges in ninja's
build graph.

Note: this CL should be committed after ninja 1.7 is released and used by
default on all bots.

BUG=592715, 592808
==========

[stgao, 2016-03-23 21:07:28]

stgao@chromium.org changed reviewers:
+ agable@chromium.org, iannucci@chromium.org, thakis@chromium.org

[stgao, 2016-03-23 21:07:29]

PTAL when you have a cycle:)

I only included an example of recipe expectation change for reference in code
review, and will add all of them in the final patch.

thakis@: do you mind reviewing scripts/slave/compile.py which is related to
ninja?

iannucci@, agable@: do you mind reviewing the rest -- changes to recipes and
recipe modules?

[stgao, 2016-04-27 20:06:53]

Hi Nico,

As ninja 1.7 will be released soon, do you mind reviewing this CL so that I
could get it ready for commit around the same time as ninja release?

Thanks,
Shuotao Gao

[stgao, 2016-04-28 18:33:58]

Description was changed from

==========
Collect failed "target" from ninja in the compile step on Waterfall.

The "target" here is actually the output nodes of the failed edges in ninja's
build graph.

Note: this CL should be committed after ninja 1.7 is released and used by
default on all bots.

BUG=592715, 592808
==========

to

==========
Collect failed "target" from ninja in the compile step on Waterfall.

The "target" here is actually the output nodes of the failed edges in ninja's
build graph.

BUG=592715, 592808
==========

[stgao, 2016-04-28 19:03:51]

Description was changed from

==========
Collect failed "target" from ninja in the compile step on Waterfall.

The "target" here is actually the output nodes of the failed edges in ninja's
build graph.

BUG=592715, 592808
==========

to

==========
Collect failed "target" from ninja in the compile step on Waterfall.

The "target" here is actually the output nodes of the failed edges in ninja's
build graph.
Simulation test passed on build_internal, thus no expectation change is needed
there.

BUG=592715, 592808
==========

[stgao, 2016-04-28 19:06:57]

Patchset #4 (id:160001) has been deleted

[stgao, 2016-05-02 18:06:33]

Description was changed from

==========
Collect failed "target" from ninja in the compile step on Waterfall.

The "target" here is actually the output nodes of the failed edges in ninja's
build graph.
Simulation test passed on build_internal, thus no expectation change is needed
there.

BUG=592715, 592808
==========

to

==========
Collect failed "target"s from ninja in the compile step on Waterfall.

The "target"s here are actually the output nodes of the failed edges in ninja's
build graph.
This CL scrapes the ninja output to extract the failed "target"s and set them as
a build property so that it shows up in the build json data.

Simulation test passed for build_internal, thus no expectation change is needed
there.

BUG=592715, 592808
==========

[Nico, 2016-05-02 18:21:58]

The filtering looks ok, but I kind of expected this to happen after compile
instead of in compile.py. Does recipes have a mechanism by which scripts can
look at the stdout of a build step? (I was waiting for agable or iannucci to
comment for that reason -- they're more familiar with recipes)

https://codereview.chromium.org/1766873002/diff/200001/scripts/slave/compile.py
File scripts/slave/compile.py (right):

https://codereview.chromium.org/1766873002/diff/200001/scripts/slave/compile....
scripts/slave/compile.py:107: for node in
line[len(self.NINJA_FAILURE_PREFIX):].split(' '):
This is ambiguous for targets that contain spaces in their names (mostly
"Chromium Framework" on Mac). I guess that's ok?

[stgao, 2016-05-02 19:43:06]

https://codereview.chromium.org/1766873002/diff/200001/scripts/slave/compile.py
File scripts/slave/compile.py (right):

https://codereview.chromium.org/1766873002/diff/200001/scripts/slave/compile....
scripts/slave/compile.py:107: for node in
line[len(self.NINJA_FAILURE_PREFIX):].split(' '):
On 2016/05/02 18:21:58, Nico wrote:
> This is ambiguous for targets that contain spaces in their names (mostly
> "Chromium Framework" on Mac). I guess that's ok?

In such cases, does ninja quote the names in a format like this?
FAILED: "Chromium Framework target" some/other/target/name

If so, I could update the parser to handle such cases.
Otherwise, they might cause some trouble: ninja can't recognize the target if
not quoted as shown below. The target name is "chromedriver4t t4unittests".

~/chromium/src$ ninja -C out/gn_build_dir/ "chromedriver4t" "t4unittests"
ninja: Entering directory `out/gn_build_dir/'
ninja: no work to do.

~/chromium/src$ ninja -C out/gn_build_dir/ chromedriver4t t4unittests
ninja: Entering directory `out/gn_build_dir/'
ninja: no work to do.

~/chromium/src$ ninja -C out/gn_build_dir/ "chromedriver4t t4unittests"
ninja: Entering directory `out/gn_build_dir/'
[1/1] LINK ./chromedriver4t t4unittests

[stgao, 2016-05-03 23:36:19]

https://codereview.chromium.org/1766873002/diff/200001/scripts/slave/compile.py
File scripts/slave/compile.py (right):

https://codereview.chromium.org/1766873002/diff/200001/scripts/slave/compile....
scripts/slave/compile.py:107: for node in
line[len(self.NINJA_FAILURE_PREFIX):].split(' '):
I've verified that ninja 1.7.1 won't quote the names if they have space as shown
below. There is a space in "embedded automation_extension.cc".

Nico: should we handle "Chromium Framework" on Mac as special cases? Are there
other workarounds?


~/chromium/src$ ninja --version
1.7.1
~/chromium/src$ ninja -C out/gn_build_dir/ chromedriver
ninja: Entering directory `out/gn_build_dir/'
[1/11] ACTION
//chrome/test/chromedriver:embed_extension_in_cpp(//build/toolchain/linux:clang_x64)
FAILED: gen/chrome/test/chromedriver/chrome/embedded automation_extension.cc
gen/chrome/test/chromedriver/chrome/embedded_automation_extension.h 
python ../../chrome/test/chromedriver/embed_extension_in_cpp.py --directory
gen/chrome/test/chromedriver/chrome
../../chrome/test/chromedriver/extension/background.js
../../chrome/test/chromedriver/extension/manifest.json

[stgao, 2017-06-09 20:54:06]

Patchset #5 (id:200001) has been deleted