Add new site parser based off on generate_test_report.

Add new site parser based off on generate_test_report.

This site parser is used to generate a JSON report of test failures, crashes,
and the associated logs for later consumption by an Email generator.

The parser uses the test report generator which comes bundled with the Chrome
OS source tree in order to maintain consistency. As well as not having to keep
track of any secondary failure white lists.

The path to the Chrome OS source tree is defined in global_config under the
CROS section as 'source_tree'.

Existing parse behavior is kept completely intact. If the site parser is not
configured it will print a debug message and exit after default parser is
called.

Relies on changes to generate_test_report here:

http://codereview.chromium.org/6731024/

Change-Id: Ib4696b88a22ca67b0822a72366d57709a5863ae5

BUG=chromium-os:12100, chromium-os:13586, chromium-os:13496
TEST=Ran parser on many sets of results.

Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=e5436f3

[pauldean_chromium, 2011-03-31 20:28:41]

I'm not familiar with the generate_test_report piece that this relies on, so I
can't comment much there. But I have poked around with the scheduler and tko
parser before, and those parts of this change LGTM.

Thanks,
paul

[ericli, 2011-03-31 21:04:24]

overall LGTM.

http://codereview.chromium.org/6758040/diff/1/tko/parsers/version_0.py
File tko/parsers/version_0.py (right):

http://codereview.chromium.org/6758040/diff/1/tko/parsers/version_0.py#newcod...
tko/parsers/version_0.py:267: kv = re.search(r"^(\w+)=(.+)", parts[part_index])
dont do this unless you want to upstream your change. It only give me trouble at
merge time but nothing else.

http://codereview.chromium.org/6758040/diff/1/tko/site_parse.py
File tko/site_parse.py (right):

http://codereview.chromium.org/6758040/diff/1/tko/site_parse.py#newcode24
tko/site_parse.py:24: import os, simplejson, sys
we are using json and simple json across the autotest codebase, better use only
one.

http://codereview.chromium.org/6758040/diff/1/tko/site_parse.py#newcode60
tko/site_parse.py:60: from generate_test_report import ResultCollector
I think eventually you want generate_test_report to call site_parser and migrate
its functions into it.

[Dale Curtis, 2011-03-31 21:09:58]

http://codereview.chromium.org/6758040/diff/1/tko/parsers/version_0.py
File tko/parsers/version_0.py (right):

http://codereview.chromium.org/6758040/diff/1/tko/parsers/version_0.py#newcod...
tko/parsers/version_0.py:267: kv = re.search(r"^(\w+)=(.+)", parts[part_index])
This is a local only change that I made a long time ago. I just realized it had
the wrong identing.

On 2011/03/31 21:04:24, ericli wrote:
> dont do this unless you want to upstream your change. It only give me trouble
at
> merge time but nothing else.

http://codereview.chromium.org/6758040/diff/1/tko/site_parse.py
File tko/site_parse.py (right):

http://codereview.chromium.org/6758040/diff/1/tko/site_parse.py#newcode24
tko/site_parse.py:24: import os, simplejson, sys
On 2011/03/31 21:04:24, ericli wrote:
> we are using json and simple json across the autotest codebase, better use
only
> one.

Done.