Add scripts for running LLVM coverage

Add scripts for running LLVM coverage

BUG=skia:2430
Committed: https://skia.googlesource.com/skia/+/11271fe3a40280a233c9a1c583497844d067088e

[borenet, 2015-07-01 20:06:50]

borenet@google.com changed reviewers:
+ jcgregorio@google.com, mtklein@google.com

[borenet, 2015-07-01 20:06:51]

This is a little rough right now, but it works.

https://codereview.chromium.org/1213063009/diff/20001/tools/llvm_coverage_run.py
File tools/llvm_coverage_run.py (right):

https://codereview.chromium.org/1213063009/diff/20001/tools/llvm_coverage_run...
tools/llvm_coverage_run.py:66: return filtered
The right solution here is to submit a change to LLVM which adds the option to
print the full path to the file.  For the moment, just skip the files we can't
find.

[mtklein, 2015-07-01 20:17:37]

https://codereview.chromium.org/1213063009/diff/20001/tools/llvm_coverage.sh
File tools/llvm_coverage.sh (right):

https://codereview.chromium.org/1213063009/diff/20001/tools/llvm_coverage.sh#...
tools/llvm_coverage.sh:15: tools/llvm_coverage_build $1
Any reason not to fold tools/llvm_coverage_build right into here?  Not sure it
ever makes sense to have build-only coverage bots.

https://codereview.chromium.org/1213063009/diff/20001/tools/llvm_coverage_build
File tools/llvm_coverage_build (right):

https://codereview.chromium.org/1213063009/diff/20001/tools/llvm_coverage_bui...
tools/llvm_coverage_build:26: export GYP_DEFINES="skia_warnings_as_errors=0
skia_clang_build=1"
Is needing to disable warnings-as-errors due to Clang or specifically coverage?

https://codereview.chromium.org/1213063009/diff/20001/tools/llvm_coverage_run.py
File tools/llvm_coverage_run.py (right):

https://codereview.chromium.org/1213063009/diff/20001/tools/llvm_coverage_run...
tools/llvm_coverage_run.py:66: return filtered
On 2015/07/01 20:06:51, borenet wrote:
> The right solution here is to submit a change to LLVM which adds the option to
> print the full path to the file.  For the moment, just skip the files we can't
> find.

I wonder how basename.startswith('Sk') works as our test.

https://codereview.chromium.org/1213063009/diff/20001/tools/llvm_coverage_run...
tools/llvm_coverage_run.py:102: for percent, f in res:
Can we get a line-by-line breakdown or some sort of explorable HTML thing?

[borenet, 2015-07-06 12:06:35]

https://codereview.chromium.org/1213063009/diff/20001/tools/llvm_coverage.sh
File tools/llvm_coverage.sh (right):

https://codereview.chromium.org/1213063009/diff/20001/tools/llvm_coverage.sh#...
tools/llvm_coverage.sh:15: tools/llvm_coverage_build $1
On 2015/07/01 20:17:37, mtklein wrote:
> Any reason not to fold tools/llvm_coverage_build right into here?  Not sure it
> ever makes sense to have build-only coverage bots.

My plan is to run this bot as a new "coverage" flavor of the same recipe we use
for all bots.  That requires that we build and run separately.

https://codereview.chromium.org/1213063009/diff/20001/tools/llvm_coverage_build
File tools/llvm_coverage_build (right):

https://codereview.chromium.org/1213063009/diff/20001/tools/llvm_coverage_bui...
tools/llvm_coverage_build:26: export GYP_DEFINES="skia_warnings_as_errors=0
skia_clang_build=1"
On 2015/07/01 20:17:37, mtklein wrote:
> Is needing to disable warnings-as-errors due to Clang or specifically
coverage?

Clang.  I have 3.6.0 on my machine and get these warnings:

../../src/core/SkRWBuffer.cpp:312:12: error: 'getPosition' overrides a member
function but is not marked 'override' [-Werror,-Winconsistent-missing-override]
    size_t getPosition() const {
           ^
../../include/core/SkStream.h:157:12: note: overridden virtual function is here
    size_t getPosition() const override = 0;
           ^
../../src/core/SkRWBuffer.cpp:316:10: error: 'seek' overrides a member function
but is not marked 'override' [-Werror,-Winconsistent-missing-override]
    bool seek(size_t position) {
         ^
../../include/core/SkStream.h:158:10: note: overridden virtual function is here
    bool seek(size_t position) override = 0;
         ^
../../src/core/SkRWBuffer.cpp:325:10: error: 'move' overrides a member function
but is not marked 'override' [-Werror,-Winconsistent-missing-override]
    bool move(long offset) {
         ^
../../include/core/SkStream.h:159:10: note: overridden virtual function is here
    bool move(long offset) override = 0;
         ^

https://codereview.chromium.org/1213063009/diff/20001/tools/llvm_coverage_run.py
File tools/llvm_coverage_run.py (right):

https://codereview.chromium.org/1213063009/diff/20001/tools/llvm_coverage_run...
tools/llvm_coverage_run.py:66: return filtered
On 2015/07/01 20:17:37, mtklein wrote:
> On 2015/07/01 20:06:51, borenet wrote:
> > The right solution here is to submit a change to LLVM which adds the option
to
> > print the full path to the file.  For the moment, just skip the files we
can't
> > find.
> 
> I wonder how basename.startswith('Sk') works as our test.

Maybe.  The biggest problem with that is we'll miss all of the files whose names
are longer than 22 characters.

https://codereview.chromium.org/1213063009/diff/20001/tools/llvm_coverage_run...
tools/llvm_coverage_run.py:102: for percent, f in res:
On 2015/07/01 20:17:37, mtklein wrote:
> Can we get a line-by-line breakdown or some sort of explorable HTML thing?

`llvm-cov show` generates a giant file which shows the code annotated with the
number of times each region was hit.  Unfortunately, it's a single file with
everything, and it's not nearly as nice to navigate as the HTML report generated
by genhtml.  I'd assumed I could find an equivalent but I haven't so far.

[mtklein, 2015-07-06 14:21:20]

lgtm

[borenet, 2015-07-06 14:36:57]

The CQ bit was checked by borenet@google.com

[commit-bot: I haz the power, 2015-07-06 14:37:00]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1213063009/20001

[commit-bot: I haz the power, 2015-07-06 14:44:01]

Committed patchset #2 (id:20001) as
https://skia.googlesource.com/skia/+/11271fe3a40280a233c9a1c583497844d067088e