Add AddressSanitizer to fuzzer analysis

Add AddressSanitizer to fuzzer analysis

Instead of running potential fuzzes against two executables (Debug, Release), this now runs them against 4 executables (Debug, Release) x (Clang, ASAN).  ASAN is short for a clang build with the AddressSanitizer flag on.

There are many new test cases that excercise the various combinations of ASAN output and Clang output.  For making a stacktrace, we look for the ASAN crash first, if that didn't crash, we use the Clang crash then. The ASAN stacktrace parsing requires 'llvm-symbolizer' to be on the path.  This typically involves a symbolic link.

To avoid duplicate code and functions with many parameters, I made some new structs in result.go, namely GCSPackage.  GCSPackage represents all the analysis information that is stored in Google Storage: The fuzz name and category, and the raw strings of the 6 files that are the output.  This allows reuse for uploading to and downloading from GCS.

A FuzzReport has changed from having a single HumanReadableFlags array to having one for each Debug and Release.  This allows for more symmetrical code and could allow for better filtering on the frontend.

There are two new flags for the backend, one that forces a re-analysis of the current fuzzes and one that toggles the outputs of builds.  The latter hopefully will make production logs easier to read.

BUG=skia:4438
Committed: https://skia.googlesource.com/buildbot/+/5610c848205e7d4b732bbafa646ab669bcb34e42

[kjlubick, 2016-02-04 18:01:22]

Description was changed from

==========
Add AddressSanitizer to fuzzer analysis

BUG=skia:4438
==========

to

==========
Add AddressSanitizer to fuzzer analysis

Instead of running potential fuzzes against two executables (Debug, Release),
this now runs them against 4 executables (Debug, Release) x (Clang, ASAN).  ASAN
is short for a clang build with the AddressSanitizer flag on.

There are many new test cases that excercise the various combinations of ASAN
output and Clang output.  For making a stacktrace, we look for the ASAN crash
first, if that didn't crash, we use the Clang crash then.  

To avoid duplicate code and functions with many parameters, I made some new
structs in result.go, namely GCSPackage.  GCSPackage represents all the analysis
information that is stored in Google Storage: The fuzz name and category, and
the raw strings of the 6 files that are the output.  This allows reuse for
uploading to and downloading from GCS.

A FuzzReport has changed from having a single HumanReadableFlags array to having
one for each Debug and Release.  This allows for more symmetrical code and could
allow for better filtering on the frontend.

There are two new flags for the backend, one that forces a re-analysis of the
current fuzzes and one that toggles the outputs of builds.  The latter hopefully
will make production logs easier to read.

BUG=skia:4438
==========

[kjlubick, 2016-02-04 18:01:23]

kjlubick@google.com changed reviewers:
+ jcgregorio@google.com

[kjlubick, 2016-02-04 18:19:52]

The CQ bit was checked by kjlubick@google.com to run a CQ dry run

[commit-bot: I haz the power, 2016-02-04 18:20:01]

Dry run: CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1668543004/20001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1668543004/20001

[kjlubick, 2016-02-04 18:21:56]

Description was changed from

==========
Add AddressSanitizer to fuzzer analysis

Instead of running potential fuzzes against two executables (Debug, Release),
this now runs them against 4 executables (Debug, Release) x (Clang, ASAN).  ASAN
is short for a clang build with the AddressSanitizer flag on.

There are many new test cases that excercise the various combinations of ASAN
output and Clang output.  For making a stacktrace, we look for the ASAN crash
first, if that didn't crash, we use the Clang crash then.  

To avoid duplicate code and functions with many parameters, I made some new
structs in result.go, namely GCSPackage.  GCSPackage represents all the analysis
information that is stored in Google Storage: The fuzz name and category, and
the raw strings of the 6 files that are the output.  This allows reuse for
uploading to and downloading from GCS.

A FuzzReport has changed from having a single HumanReadableFlags array to having
one for each Debug and Release.  This allows for more symmetrical code and could
allow for better filtering on the frontend.

There are two new flags for the backend, one that forces a re-analysis of the
current fuzzes and one that toggles the outputs of builds.  The latter hopefully
will make production logs easier to read.

BUG=skia:4438
==========

to

==========
Add AddressSanitizer to fuzzer analysis

Instead of running potential fuzzes against two executables (Debug, Release),
this now runs them against 4 executables (Debug, Release) x (Clang, ASAN).  ASAN
is short for a clang build with the AddressSanitizer flag on.

There are many new test cases that excercise the various combinations of ASAN
output and Clang output.  For making a stacktrace, we look for the ASAN crash
first, if that didn't crash, we use the Clang crash then. The ASAN stacktrace
parsing requires 'llvm-symbolizer' to be on the path.  This typically involves a
symbolic link.

To avoid duplicate code and functions with many parameters, I made some new
structs in result.go, namely GCSPackage.  GCSPackage represents all the analysis
information that is stored in Google Storage: The fuzz name and category, and
the raw strings of the 6 files that are the output.  This allows reuse for
uploading to and downloading from GCS.

A FuzzReport has changed from having a single HumanReadableFlags array to having
one for each Debug and Release.  This allows for more symmetrical code and could
allow for better filtering on the frontend.

There are two new flags for the backend, one that forces a re-analysis of the
current fuzzes and one that toggles the outputs of builds.  The latter hopefully
will make production logs easier to read.

BUG=skia:4438
==========

[commit-bot: I haz the power, 2016-02-04 18:23:02]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-02-04 18:23:02]

Dry run: This issue passed the CQ dry run.

[jcgregorio, 2016-02-05 20:37:19]

lgtm

https://codereview.chromium.org/1668543004/diff/60001/fuzzer/go/common/versio...
File fuzzer/go/common/version_watcher.go (right):

https://codereview.chromium.org/1668543004/diff/60001/fuzzer/go/common/versio...
fuzzer/go/common/version_watcher.go:86: config.Common.ForceReanalysis = false
Will this ever get turned back on?

[kjlubick, 2016-02-05 20:41:02]

https://codereview.chromium.org/1668543004/diff/60001/fuzzer/go/common/versio...
File fuzzer/go/common/version_watcher.go (right):

https://codereview.chromium.org/1668543004/diff/60001/fuzzer/go/common/versio...
fuzzer/go/common/version_watcher.go:86: config.Common.ForceReanalysis = false
On 2016/02/05 at 20:37:19, jcgregorio wrote:
> Will this ever get turned back on?

No.  It's only set by the command flag.

[kjlubick, 2016-02-08 13:38:24]

The CQ bit was checked by kjlubick@google.com

[commit-bot: I haz the power, 2016-02-08 13:38:30]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1668543004/60001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1668543004/60001

[commit-bot: I haz the power, 2016-02-08 13:42:04]

Description was changed from

==========
Add AddressSanitizer to fuzzer analysis

Instead of running potential fuzzes against two executables (Debug, Release),
this now runs them against 4 executables (Debug, Release) x (Clang, ASAN).  ASAN
is short for a clang build with the AddressSanitizer flag on.

There are many new test cases that excercise the various combinations of ASAN
output and Clang output.  For making a stacktrace, we look for the ASAN crash
first, if that didn't crash, we use the Clang crash then. The ASAN stacktrace
parsing requires 'llvm-symbolizer' to be on the path.  This typically involves a
symbolic link.

To avoid duplicate code and functions with many parameters, I made some new
structs in result.go, namely GCSPackage.  GCSPackage represents all the analysis
information that is stored in Google Storage: The fuzz name and category, and
the raw strings of the 6 files that are the output.  This allows reuse for
uploading to and downloading from GCS.

A FuzzReport has changed from having a single HumanReadableFlags array to having
one for each Debug and Release.  This allows for more symmetrical code and could
allow for better filtering on the frontend.

There are two new flags for the backend, one that forces a re-analysis of the
current fuzzes and one that toggles the outputs of builds.  The latter hopefully
will make production logs easier to read.

BUG=skia:4438
==========

to

==========
Add AddressSanitizer to fuzzer analysis

Instead of running potential fuzzes against two executables (Debug, Release),
this now runs them against 4 executables (Debug, Release) x (Clang, ASAN).  ASAN
is short for a clang build with the AddressSanitizer flag on.

There are many new test cases that excercise the various combinations of ASAN
output and Clang output.  For making a stacktrace, we look for the ASAN crash
first, if that didn't crash, we use the Clang crash then. The ASAN stacktrace
parsing requires 'llvm-symbolizer' to be on the path.  This typically involves a
symbolic link.

To avoid duplicate code and functions with many parameters, I made some new
structs in result.go, namely GCSPackage.  GCSPackage represents all the analysis
information that is stored in Google Storage: The fuzz name and category, and
the raw strings of the 6 files that are the output.  This allows reuse for
uploading to and downloading from GCS.

A FuzzReport has changed from having a single HumanReadableFlags array to having
one for each Debug and Release.  This allows for more symmetrical code and could
allow for better filtering on the frontend.

There are two new flags for the backend, one that forces a re-analysis of the
current fuzzes and one that toggles the outputs of builds.  The latter hopefully
will make production logs easier to read.

BUG=skia:4438

Committed:
https://skia.googlesource.com/buildbot/+/5610c848205e7d4b732bbafa646ab669bcb3...
==========

[commit-bot: I haz the power, 2016-02-08 13:42:05]

Committed patchset #4 (id:60001) as
https://skia.googlesource.com/buildbot/+/5610c848205e7d4b732bbafa646ab669bcb3...