Add support for multiple output formatters for telemetry results.

Add support for multiple output formatters for telemetry results using the following syntax:

./run_benchmark --output-format=json --output-format=html foo

BUG=

Committed: https://crrev.com/e3b499340f9e21bcc09636315b09a30ec1c5a7ab
Cr-Commit-Position: refs/heads/master@{#295149}

[ariblue, 2014-09-15 23:16:35]

Patchset #2 (id:20001) has been deleted

[ariblue, 2014-09-15 23:17:41]

ariblue@google.com changed reviewers:
+ chrishenry@google.com, nednguyen@google.com

[ariblue, 2014-09-15 23:20:14]

This approach seem reasonable (that is, side-stepping the multiple output
formatters with only one output_file flag by just throwing an Exception for
now)?

[chrishenry, 2014-09-16 00:40:09]

How about these tests:
https://code.google.com/p/chromium/codesearch#search/&type=cs&q=%5C.output_fo...

https://codereview.chromium.org/574593002/diff/40001/tools/telemetry/telemetr...
File tools/telemetry/telemetry/results/results_options.py (right):

https://codereview.chromium.org/574593002/diff/40001/tools/telemetry/telemetr...
tools/telemetry/telemetry/results/results_options.py:64: def
_GetProgressReporter(output_formats, suppress_gtest_report):
Don't pass the whole output_formats, just pass a bool.

[ariblue, 2014-09-16 17:28:14]

https://codereview.chromium.org/574593002/diff/40001/tools/telemetry/telemetr...
File tools/telemetry/telemetry/results/results_options.py (right):

https://codereview.chromium.org/574593002/diff/40001/tools/telemetry/telemetr...
tools/telemetry/telemetry/results/results_options.py:64: def
_GetProgressReporter(output_formats, suppress_gtest_report):
On 2014/09/16 00:40:09, chrishenry wrote:
> Don't pass the whole output_formats, just pass a bool.

Done.

[chrishenry, 2014-09-16 17:43:14]

lgtm

[nednguyen, 2014-09-16 17:43:36]

lgtm with nit

https://codereview.chromium.org/574593002/diff/60001/tools/telemetry/telemetr...
File tools/telemetry/telemetry/results/results_options.py (right):

https://codereview.chromium.org/574593002/diff/60001/tools/telemetry/telemetr...
tools/telemetry/telemetry/results/results_options.py:53: def
_GetOutputStream(output_format, output_file):
Seems like this doesn't accept 'none', 'gtest' and 'chartjson'. Can you add an
assert that list all possible kinds of output_format this function can accept?

[ariblue, 2014-09-16 18:26:14]

https://codereview.chromium.org/574593002/diff/60001/tools/telemetry/telemetr...
File tools/telemetry/telemetry/results/results_options.py (right):

https://codereview.chromium.org/574593002/diff/60001/tools/telemetry/telemetr...
tools/telemetry/telemetry/results/results_options.py:53: def
_GetOutputStream(output_format, output_file):
On 2014/09/16 17:43:35, nednguyen wrote:
> Seems like this doesn't accept 'none', 'gtest' and 'chartjson'. Can you add an
> assert that list all possible kinds of output_format this function can accept?

This should work with the 'chartjson' output_format. If an output_file is
provided then that is used, otherwise it will use stdout. I'm not certain how
the --chartjson flag will end up being used, so this doesn't handle any of that
right now.

I've added an assertion to make sure that the output_format is not 'none' or
'gtest', but in _OUTPUT_FORMAT_CHOICES. Seem reasonable?

[nednguyen, 2014-09-16 19:18:07]

https://codereview.chromium.org/574593002/diff/80001/tools/telemetry/telemetr...
File tools/telemetry/telemetry/results/results_options.py (right):

https://codereview.chromium.org/574593002/diff/80001/tools/telemetry/telemetr...
tools/telemetry/telemetry/results/results_options.py:92: if output_format ==
'none' or output_format == "gtest" or options.chartjson:
So the or condition on "options.chartjson" is dead code?

[ariblue, 2014-09-16 19:29:17]

https://codereview.chromium.org/574593002/diff/80001/tools/telemetry/telemetr...
File tools/telemetry/telemetry/results/results_options.py (right):

https://codereview.chromium.org/574593002/diff/80001/tools/telemetry/telemetr...
tools/telemetry/telemetry/results/results_options.py:92: if output_format ==
'none' or output_format == "gtest" or options.chartjson:
On 2014/09/16 19:18:07, nednguyen wrote:
> So the or condition on "options.chartjson" is dead code?

I believe so, I don't see it anywhere in perf/ or telemetry/. It was added here:
https://codereview.chromium.org/531973002 and removed from benchmark.py here:
https://codereview.chromium.org/551973002/

Regardless, I've moved the creation of the output_stream to after this if
statement.

[nednguyen, 2014-09-16 19:31:26]

On 2014/09/16 19:29:17, ariblue wrote:
>
https://codereview.chromium.org/574593002/diff/80001/tools/telemetry/telemetr...
> File tools/telemetry/telemetry/results/results_options.py (right):
> 
>
https://codereview.chromium.org/574593002/diff/80001/tools/telemetry/telemetr...
> tools/telemetry/telemetry/results/results_options.py:92: if output_format ==
> 'none' or output_format == "gtest" or options.chartjson:
> On 2014/09/16 19:18:07, nednguyen wrote:
> > So the or condition on "options.chartjson" is dead code?
> 
> I believe so, I don't see it anywhere in perf/ or telemetry/. It was added
here:
> https://codereview.chromium.org/531973002 and removed from benchmark.py here:
> https://codereview.chromium.org/551973002/
> 
> Regardless, I've moved the creation of the output_stream to after this if
> statement.

LGTM again.
If you can make another CL or remind me to delete that dead code, it would be
great :)

[ariblue, 2014-09-16 20:34:47]

The CQ bit was checked by ariblue@google.com

[commit-bot: I haz the power, 2014-09-16 20:35:09]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patchset/574593002/100001

[commit-bot: I haz the power, 2014-09-16 21:42:27]

Committed patchset #5 (id:100001) as aa934914ee7919399d8df43494b31f3c7bc1a0a6

[commit-bot: I haz the power, 2014-09-16 21:43:08]

Patchset 5 (id:??) landed as
https://crrev.com/e3b499340f9e21bcc09636315b09a30ec1c5a7ab
Cr-Commit-Position: refs/heads/master@{#295149}