Update GraphingPageCyclerLogProcessor to work with updated Telemetry pagecyclers

Update GraphingPageCyclerLogProcessor to work with updated Telemetry pagecyclers

The GraphingPageCyclerLogProcessor previously expected that any array in perf
results would be in the form:
[<url1_iter1>, <url2_iter1>, <url1_iter2>, <url2_iter2>, ...]

The updated Telemetry page cyclers will continue to output the overall results
that way, but will also output results by URL. For example:
url1: [<iter1>, <iter2>, ...]
url2: [<iter1>, <iter2>, ...]

This patch causes the processor to process the by-url results in the same way
as other perf results instead of in the page cycler specific manner.

This fixes a bug which caused the process to fail with the following error when
the page cycler update was submitted the first time:
KeyError: 'blog.chromium.org'

The new unittest data in this patch was taken from an actual run of the updated
page cyclers and includes the by-url traces. Without this patch, it fails in the
same way mentioned above.

BUG=None
TEST=./process_log_test.py && ./runtest_annotator_test.py

Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=194202

[tonyg, 2013-04-12 19:16:10]

Marja,

Mike suggested I have you review this since you touched performance log
processing most recently. Would you mind taking a look?

Thanks,
Tony

[Mike Stip (use stip instead), 2013-04-12 19:19:38]

Tony, can you add a similar unit test in
scripts/slave/unittests/runtest_annotator_test.py? Other than that lg2m.

[tonyg, 2013-04-12 21:11:00]

On 2013/04/12 19:19:38, Mike Stipicevic wrote:
> Tony, can you add a similar unit test in
> scripts/slave/unittests/runtest_annotator_test.py? Other than that lg2m.

Done. I added the test. It is a shame there's not a better way to share code
there.

Also, a question for you. Once this land, do I need to get a restart of
chromium.perf, or do the page cyclers there only use the slave
process_log_utils?

[Mike Stip (use stip instead), 2013-04-12 21:15:52]

On 2013/04/12 21:11:00, tonyg wrote:
> Also, a question for you. Once this land, do I need to get a restart of
> chromium.perf, or do the page cyclers there only use the slave
> process_log_utils?

It will appear immediately on chromium.perf. The only masters using the old
master code are the NaCl and ChromeOS waterfalls, because they have a highly
unusual perf setup that hasn't been annotated yet. They should be annotated very
soon (within the next month).

[tonyg, 2013-04-12 21:25:14]

On 2013/04/12 21:15:52, Mike Stipicevic wrote:
> On 2013/04/12 21:11:00, tonyg wrote:
> > Also, a question for you. Once this land, do I need to get a restart of
> > chromium.perf, or do the page cyclers there only use the slave
> > process_log_utils?
> 
> It will appear immediately on chromium.perf. The only masters using the old
> master code are the NaCl and ChromeOS waterfalls, because they have a highly
> unusual perf setup that hasn't been annotated yet. They should be annotated
very
> soon (within the next month).

Thanks, that's really awesome!

I'll wait for Marja to give her lgtm.

[marja, 2013-04-15 09:09:26]

lgtm

Are "<iter1>, <iter2>, ..." the times for each individual iterations when we
repeatedly do the same page load?

So this doesn't change the way the HISTOGRAM lines are processed, since there is
only one HISTOGRAM per test? (Though, we should take the last histogram, not the
first histogram.)

[tonyg, 2013-04-15 16:27:43]

On 2013/04/15 09:09:26, marja wrote:
> lgtm
> 
> Are "<iter1>, <iter2>, ..." the times for each individual iterations when we
> repeatedly do the same page load?

Yes. This shows up when using the --pageset-repeat option.

> 
> So this doesn't change the way the HISTOGRAM lines are processed, since there
is
> only one HISTOGRAM per test? (Though, we should take the last histogram, not
the
> first histogram.)

Right. I don't expect this to change histograms. I'll file a bug for taking the
last instead of first histogram.

[Mike Stip (use stip instead), 2013-04-15 18:22:11]

lgtm

[tonyg, 2013-04-15 18:25:20]

Committed patchset #2 manually as r194202 (presubmit successful).