cleanup and improve A/B fasta test harness

cleanup and improve A/B fasta test harness

* Update harness to show 99.9% best and worst case

* Moved compile summary functionality
  from pkg/front_end/tool/fasta/compile.dart
  into pkg/front_end/lib/src/fasta/outline.dart

* Modified outline.dart to output a JSON summary
  containing elapse execution times if -Dsummary=true

* Moved all statistical calculations into abcompile.dart

* Adjusted pkg/front_end/tool/fasta/abcompile.dart so that
  --verbose is not needed to capture numbers and progress

* Updated the A/B summary section to paste more easily
  into Google Sheets

* Address comments in
  https://codereview.chromium.org/2734863003/

R=paulberry@google.com

Committed: https://github.com/dart-lang/sdk/commit/516646735d9185455690314925e82c57686a8697

[danrubel, 2017-03-09 02:00:45]

danrubel@google.com changed reviewers:
+ paulberry@google.com

[Paul Berry, 2017-03-09 16:58:26]

lgtm

https://codereview.chromium.org/2739963002/diff/1/pkg/front_end/lib/src/fasta...
File pkg/front_end/lib/src/fasta/outline.dart (right):

https://codereview.chromium.org/2739963002/diff/1/pkg/front_end/lib/src/fasta...
pkg/front_end/lib/src/fasta/outline.dart:34: List<double> elapseTimes =
<double>[];
Nit: I think you mean "elapsedTimes"

https://codereview.chromium.org/2739963002/diff/1/pkg/front_end/lib/src/fasta...
pkg/front_end/lib/src/fasta/outline.dart:48: var json = JSON.encode(<String,
dynamic>{'elapseTimes': elapseTimes});
Here as well (in the string)

https://codereview.chromium.org/2739963002/diff/1/pkg/front_end/tool/fasta/ab...
File pkg/front_end/tool/fasta/abcompile.dart (right):

https://codereview.chromium.org/2739963002/diff/1/pkg/front_end/tool/fasta/ab...
pkg/front_end/tool/fasta/abcompile.dart:106: double coldStdDevMean =
sqrt(pow(aColdSDM, 2) + pow(bColdSDM, 2));
Nit: this variable should be called "coldUncertainty".  It's confusing to all it
"StdDevMean" because "standard deviation of the mean" is one of several
techniques for computing uncertainty, and it's not even the technique we're
using here (the technique we're using here is "uncertainty propagation").

Similar comment applies to warmStdDevMean, coldSDMPercent, and warmSDMPercent
below.

https://codereview.chromium.org/2739963002/diff/1/pkg/front_end/tool/fasta/ab...
pkg/front_end/tool/fasta/abcompile.dart:158: List<double> elapseTimes =
results['elapseTimes'];
This should be "elapsedTimes" as well IMHO (here and in lines 159, 160, 161,
163, and 165).

[danrubel, 2017-03-09 19:05:16]

https://codereview.chromium.org/2739963002/diff/1/pkg/front_end/lib/src/fasta...
File pkg/front_end/lib/src/fasta/outline.dart (right):

https://codereview.chromium.org/2739963002/diff/1/pkg/front_end/lib/src/fasta...
pkg/front_end/lib/src/fasta/outline.dart:34: List<double> elapseTimes =
<double>[];
On 2017/03/09 16:58:25, Paul Berry wrote:
> Nit: I think you mean "elapsedTimes"

Yes. Fixed.

https://codereview.chromium.org/2739963002/diff/1/pkg/front_end/lib/src/fasta...
pkg/front_end/lib/src/fasta/outline.dart:48: var json = JSON.encode(<String,
dynamic>{'elapseTimes': elapseTimes});
On 2017/03/09 16:58:25, Paul Berry wrote:
> Here as well (in the string)

Done.

https://codereview.chromium.org/2739963002/diff/1/pkg/front_end/tool/fasta/ab...
File pkg/front_end/tool/fasta/abcompile.dart (right):

https://codereview.chromium.org/2739963002/diff/1/pkg/front_end/tool/fasta/ab...
pkg/front_end/tool/fasta/abcompile.dart:106: double coldStdDevMean =
sqrt(pow(aColdSDM, 2) + pow(bColdSDM, 2));
On 2017/03/09 16:58:26, Paul Berry wrote:
> Nit: this variable should be called "coldUncertainty".  It's confusing to all
it
> "StdDevMean" because "standard deviation of the mean" is one of several
> techniques for computing uncertainty, and it's not even the technique we're
> using here (the technique we're using here is "uncertainty propagation").
> 
> Similar comment applies to warmStdDevMean, coldSDMPercent, and warmSDMPercent
> below.

Good point. Fixed.

https://codereview.chromium.org/2739963002/diff/1/pkg/front_end/tool/fasta/ab...
pkg/front_end/tool/fasta/abcompile.dart:158: List<double> elapseTimes =
results['elapseTimes'];
On 2017/03/09 16:58:25, Paul Berry wrote:
> This should be "elapsedTimes" as well IMHO (here and in lines 159, 160, 161,
> 163, and 165).

Done.

[danrubel, 2017-03-09 19:07:50]

Description was changed from

==========
cleanup and improve A/B fasta test harness

* Update harness to show 99.9% best and worst case

* Moved compile summary functionality
  from pkg/front_end/tool/fasta/compile.dart
  into pkg/front_end/lib/src/fasta/outline.dart

* Modified outline.dart to output a JSON summary
  containing elapse execution times if -Dsummary=true

* Moved all statistical calculations into abcompile.dart

* Adjusted pkg/front_end/tool/fasta/abcompile.dart so that
  --verbose is not needed to capture numbers and progress

* Updated the A/B summary section to paste more easily
  into Google Sheets

* Address comments in
  https://codereview.chromium.org/2734863003/
==========

to

==========
cleanup and improve A/B fasta test harness

* Update harness to show 99.9% best and worst case

* Moved compile summary functionality
  from pkg/front_end/tool/fasta/compile.dart
  into pkg/front_end/lib/src/fasta/outline.dart

* Modified outline.dart to output a JSON summary
  containing elapse execution times if -Dsummary=true

* Moved all statistical calculations into abcompile.dart

* Adjusted pkg/front_end/tool/fasta/abcompile.dart so that
  --verbose is not needed to capture numbers and progress

* Updated the A/B summary section to paste more easily
  into Google Sheets

* Address comments in
  https://codereview.chromium.org/2734863003/

R=paulberry@google.com

Committed:
https://github.com/dart-lang/sdk/commit/516646735d9185455690314925e82c57686a8697
==========

[danrubel, 2017-03-09 19:07:51]

Committed patchset #3 (id:40001) manually as
516646735d9185455690314925e82c57686a8697 (presubmit successful).