Teach mojo_benchmark to produce chart_data for the perf dashboard.

mojo/devtools/common/mojo_benchmark

Index: mojo/devtools/common/mojo_benchmark
diff --git a/mojo/devtools/common/mojo_benchmark b/mojo/devtools/common/mojo_benchmark
index 0f2126bceb7b57da1b960c2d70eaa0dd1ecead6a..f8931cfab868cac5e4cabede52c59ca9b0da2eee 100755
--- a/mojo/devtools/common/mojo_benchmark
+++ b/mojo/devtools/common/mojo_benchmark
@@ -14,6 +14,7 @@ import re
 
 from devtoolslib import shell_arguments
 from devtoolslib import shell_config
+from devtoolslib import performance_dashboard
 
 
 _DESCRIPTION = """Runner for Mojo application benchmarks.
@@ -175,6 +176,9 @@ def main():
                       help='a file listing benchmarks to run')
   parser.add_argument('--save-traces', action='store_true',
                       help='save the traces produced by benchmarks to disk')
+  parser.add_argument('--chart-data-output-file', type=argparse.FileType('w'),
+                      help='file to which chart data for the performance '

[viettrungluu, 2015/10/15 22:32:56]

Despite what snobs may think, "file to write chart data for the performance
dashboard to" is both shorter and clearer.

[ppi, 2015/10/15 23:13:14]

Done.

+                      'dashboard is to be written')
 
   # Common shell configuration arguments.
   shell_config.add_shell_arguments(parser)
@@ -191,6 +195,10 @@ def main():
   benchmark_list_params = {"target_os": target_os}
   exec script_args.benchmark_list_file in benchmark_list_params
 
+  chart_data_recorder = None
+  if script_args.chart_data_output_file:
+    chart_data_recorder = performance_dashboard.ChartDataRecorder()
+
   exit_code = 0
   for benchmark_spec in benchmark_list_params['benchmarks']:
     for variant_spec in _generate_benchmark_variants(benchmark_spec):
@@ -213,8 +221,13 @@ def main():
         # results and preserve the required order.
         for measurement_spec in measurements:
           if measurement_spec in measurement_results:
-            print '%s %s' % (measurement_spec,
-                             measurement_results[measurement_spec])
+            result = measurement_results[measurement_spec]
+            print '%s %s' % (measurement_spec, result)
+
+            if chart_data_recorder:
+              measurement_name = measurement_spec.replace('/', '-')

[viettrungluu, 2015/10/15 22:32:56]

Can't you align the way things are named/specified?

[ppi, 2015/10/15 23:13:14]

Mixing undescores with dashes in the spec seems not palatable.

I think we will eventually want to have some better format to spec the
measurements - maybe in the function notation, ie
"time_until(category:event_name)",
time_between(category1:event1,category2:event2),
percentile_duration(category:event_name, percentile)", but  this would perhaps
better be addressed separately.

Wdyt?

[viettrungluu, 2015/10/21 15:45:27]

OK

+              chart_data_recorder.record_scalar(name, measurement_name, 'ms',

[viettrungluu, 2015/10/15 22:32:56]

Are the results always milliseconds?

(Likewise here: It'd be nice if the measurement_results also contained the
units.)

[ppi, 2015/10/15 23:13:14]

For now, yes. I think we will want to have the benchmark app report results
along with the units and propagate this here. I'd suggest to to this when/if we
grow first measurement type that does not return milliseconds.

[viettrungluu, 2015/10/21 15:45:27]

Sure.

+                                                result)
           else:
             print '%s ?' % measurement_spec
             some_measurements_failed = True
@@ -230,6 +243,9 @@ def main():
         print '-' * 72
         exit_code = 1
 
+  if script_args.chart_data_output_file:
+    script_args.chart_data_output_file.write(chart_data_recorder.get_json())
+
   return exit_code
 
 if __name__ == '__main__':