Make JSON output formatter output to a file.

Make JSON output formatter output to a file.

This makes it so that --output-format=json will produce a file called
results.json by default.

Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=287786

[chrishenry, 2014-07-31 07:20:40]

https://codereview.chromium.org/429043002/diff/1/tools/telemetry/telemetry/re...
File tools/telemetry/telemetry/results/json_output_formatter.py (right):

https://codereview.chromium.org/429043002/diff/1/tools/telemetry/telemetry/re...
tools/telemetry/telemetry/results/json_output_formatter.py:9: def
ResultsAsDict(res):
s/res/page_test_results/ here and for _all_pages.

https://codereview.chromium.org/429043002/diff/1/tools/telemetry/telemetry/re...
tools/telemetry/telemetry/results/json_output_formatter.py:19: def
_all_pages(res):
_GetAllPages

[nednguyen, 2014-07-31 14:32:41]

lgtm

Looks like we don't have unittest that shows how the json results look like in
string form, can you add one?

[eakuefner, 2014-07-31 16:47:05]

On 2014/07/31 14:32:41, nednguyen wrote:
> lgtm
> 
> Looks like we don't have unittest that shows how the json results look like in
> string form, can you add one?

It's difficult to compare JSON objects against string literals because of the
nondeterministic nature of the json module -- keys may be output in any order,
etc. I think there are two components that make up for this: one, the JSON patch
that has already landed includes a unittest to check that the output of the
JSONOutputFormatter parses as JSON. Two, I'm working on a patch to include
FromDict and deserialization code, and once that's in I'll also be able to
comprehensively unittest that serialization preserves results object semantics.
Since the actual output as JSON is all delegated to the built-in json module,
this combination of a parse smoke test and a serialization/deserialization
equivalence unittest should be enough to give us a reasonable guarantee of
correctness.

[tonyg, 2014-07-31 17:03:21]

On 2014/07/31 16:47:05, eakuefner wrote:
> On 2014/07/31 14:32:41, nednguyen wrote:
> > lgtm
> > 
> > Looks like we don't have unittest that shows how the json results look like
in
> > string form, can you add one?
> 
> It's difficult to compare JSON objects against string literals because of the
> nondeterministic nature of the json module -- keys may be output in any order,

Does sort_keys=True help?
https://docs.python.org/2/library/json.html#json.dumps

ps. indent=2 also helps readability where that's useful.

> etc. I think there are two components that make up for this: one, the JSON
patch
> that has already landed includes a unittest to check that the output of the
> JSONOutputFormatter parses as JSON. Two, I'm working on a patch to include
> FromDict and deserialization code, and once that's in I'll also be able to
> comprehensively unittest that serialization preserves results object
semantics.
> Since the actual output as JSON is all delegated to the built-in json module,
> this combination of a parse smoke test and a serialization/deserialization
> equivalence unittest should be enough to give us a reasonable guarantee of
> correctness.

[nduca, 2014-07-31 17:08:40]

we shouldn't have a unit test for the final form. it means that any minor tweak
to the stack that produces the json will cause that test to fail. its bad
testnig.

[nduca, 2014-07-31 17:09:35]

lgtm

[nednguyen, 2014-07-31 18:09:19]

On 2014/07/31 17:09:35, nduca wrote:
> lgtm

Thanks, if we have deserializer, all is fine. I am a little paranoid by the fact
that some bot relying on specific order of items in list, etc...

Do we have any backward compatibility maintenance plan with the "format_version"
field in the json result? i.e: what changes to results will require us to update
the version?

[eakuefner, 2014-08-03 02:14:31]

The CQ bit was checked by eakuefner@chromium.org

[commit-bot: I haz the power, 2014-08-03 02:15:32]

CQ is trying da patch. Follow status at
 https://chromium-status.appspot.com/cq/eakuefner@chromium.org/429043002/1

[commit-bot: I haz the power, 2014-08-03 02:27:18]

FYI, CQ is re-trying this CL (attempt #1).
The failing builders are:
  linux_gpu on tryserver.chromium.gpu
(http://build.chromium.org/p/tryserver.chromium.gpu/builders/linux_gpu/builds/...)
  mac_gpu on tryserver.chromium.gpu
(http://build.chromium.org/p/tryserver.chromium.gpu/builders/mac_gpu/builds/36825)
  win_gpu on tryserver.chromium.gpu
(http://build.chromium.org/p/tryserver.chromium.gpu/builders/win_gpu/builds/41851)
  android_aosp on tryserver.chromium.linux
(http://build.chromium.org/p/tryserver.chromium.linux/builders/android_aosp/bu...)
  android_chromium_gn_compile_rel on tryserver.chromium.linux
(http://build.chromium.org/p/tryserver.chromium.linux/builders/android_chromiu...)
  android_clang_dbg on tryserver.chromium.linux
(http://build.chromium.org/p/tryserver.chromium.linux/builders/android_clang_d...)
  android_dbg on tryserver.chromium.linux
(http://build.chromium.org/p/tryserver.chromium.linux/builders/android_dbg/bui...)
  chromium_presubmit on tryserver.chromium.linux
(http://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presub...)
  linux_chromium_chromeos_rel_swarming on tryserver.chromium.linux
(http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_...)
  linux_chromium_gn_rel on tryserver.chromium.linux
(http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_...)
  linux_chromium_rel_swarming on tryserver.chromium.linux
(http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_...)
  ios_dbg_simulator on tryserver.chromium.mac
(http://build.chromium.org/p/tryserver.chromium.mac/builders/ios_dbg_simulator...)
  ios_rel_device on tryserver.chromium.mac
(http://build.chromium.org/p/tryserver.chromium.mac/builders/ios_rel_device/bu...)
  ios_rel_device_ninja on tryserver.chromium.mac
(http://build.chromium.org/p/tryserver.chromium.mac/builders/ios_rel_device_ni...)
  mac_chromium_compile_dbg on tryserver.chromium.mac
(http://build.chromium.org/p/tryserver.chromium.mac/builders/mac_chromium_comp...)
  mac_chromium_rel on tryserver.chromium.mac
(http://build.chromium.org/p/tryserver.chromium.mac/builders/mac_chromium_rel/...)
  win8_chromium_rel on tryserver.chromium.win
(http://build.chromium.org/p/tryserver.chromium.win/builders/win8_chromium_rel...)
  win_chromium_compile_dbg on tryserver.chromium.win
(http://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_comp...)
  win_chromium_rel on tryserver.chromium.win
(http://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_rel/...)
  win_chromium_x64_rel on tryserver.chromium.win
(http://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_x64_...)

[commit-bot: I haz the power, 2014-08-03 02:28:18]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2014-08-03 02:28:18]

Try jobs failed on following builders:
  mac_chromium_compile_dbg on tryserver.chromium.mac
(http://build.chromium.org/p/tryserver.chromium.mac/builders/mac_chromium_comp...)

[eakuefner, 2014-08-05 20:59:27]

The CQ bit was checked by eakuefner@chromium.org

[commit-bot: I haz the power, 2014-08-05 21:03:32]

CQ is trying da patch. Follow status at
 https://chromium-status.appspot.com/cq/eakuefner@chromium.org/429043002/20001

[commit-bot: I haz the power, 2014-08-06 01:19:37]

FYI, CQ is re-trying this CL (attempt #1).
The failing builders are:
  linux_chromium_chromeos_rel_swarming on tryserver.chromium.linux
(http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_...)
  linux_chromium_rel_swarming on tryserver.chromium.linux
(http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_...)
  mac_chromium_compile_dbg on tryserver.chromium.mac
(http://build.chromium.org/p/tryserver.chromium.mac/builders/mac_chromium_comp...)

[commit-bot: I haz the power, 2014-08-06 01:32:04]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2014-08-06 01:32:05]

Try jobs failed on following builders:
  mac_chromium_compile_dbg on tryserver.chromium.mac
(http://build.chromium.org/p/tryserver.chromium.mac/builders/mac_chromium_comp...)

[nednguyen, 2014-08-06 16:13:15]

The CQ bit was checked by nednguyen@google.com

[commit-bot: I haz the power, 2014-08-06 16:14:08]

CQ is trying da patch. Follow status at
 https://chromium-status.appspot.com/cq/eakuefner@chromium.org/429043002/20001

[commit-bot: I haz the power, 2014-08-06 16:43:30]

Change committed as 287786