Added in framework to get more bench data

Added in framework to get more bench data

BUG=skia:
NOTRY=true

Committed: https://skia.googlesource.com/skia/+/fa1eaaa843367b5b732ed923dc5522e258e6b27a

[kelvinly, 2014-06-12 17:29:39]

Not nearly finished, but I wanted to see what you guys thought of how I was
getting additional data in the rebaseline stuff

[benchen, 2014-06-12 17:50:24]

Need some basic readability fixes - lines over 80 chars, etc.

Problem with using data from previous revisions is that the revisions themselves
may make the data different by nature. For instance a revision changed the SKP
version (we have this nightly, and only the "current" data points are calculated
with the new inputs) or improved/degraded performance or measurement.

I'm guessing that you just wanted to use historical data to evaluate variance
etc., but use data from one revision to establish the expected baseline? If so
that should be worth trying.

https://codereview.chromium.org/331683003/diff/1/bench/gen_bench_expectations.py
File bench/gen_bench_expectations.py (right):

https://codereview.chromium.org/331683003/diff/1/bench/gen_bench_expectations...
bench/gen_bench_expectations.py:103: def get_parent_commits(start_hash,
num_back):
Need method description here.

[kelvinly, 2014-06-12 17:59:23]

On 2014/06/12 17:50:24, benchen wrote:
> Need some basic readability fixes - lines over 80 chars, etc.
> 
> Problem with using data from previous revisions is that the revisions
themselves
> may make the data different by nature. For instance a revision changed the SKP
> version (we have this nightly, and only the "current" data points are
calculated
> with the new inputs) or improved/degraded performance or measurement.
> 
> I'm guessing that you just wanted to use historical data to evaluate variance
> etc., but use data from one revision to establish the expected baseline? If so
> that should be worth trying.
> 
>
https://codereview.chromium.org/331683003/diff/1/bench/gen_bench_expectations.py
> File bench/gen_bench_expectations.py (right):
> 
>
https://codereview.chromium.org/331683003/diff/1/bench/gen_bench_expectations...
> bench/gen_bench_expectations.py:103: def get_parent_commits(start_hash,
> num_back):
> Need method description here.

Yep, I was planning on at least having the data available for whatever
rebaselining analysis stuff we end up doing, without touching the algorithm yet.
Is how I'm getting the old benchmark data good? I'm worried it might be too
hackish

[kelvinly, 2014-06-12 18:02:00]

https://codereview.chromium.org/331683003/diff/1/bench/gen_bench_expectations.py
File bench/gen_bench_expectations.py (right):

https://codereview.chromium.org/331683003/diff/1/bench/gen_bench_expectations...
bench/gen_bench_expectations.py:103: def get_parent_commits(start_hash,
num_back):
On 2014/06/12 17:50:23, benchen wrote:
> Need method description here.

Done.

[benchen, 2014-06-12 18:12:54]

It does look hackish - better way would be to use apiary, but that might require
credential setup etc. on the bots. Could call gsutil by adding a flag to tell
bots to set gsutil binary location, but that has its own risks.
At this stage let's just try use the urls. But please add another TODO to
address this later.
lgtm but please add the TODO before submitting..

https://codereview.chromium.org/331683003/diff/20001/bench/gen_bench_expectat...
File bench/gen_bench_expectations.py (right):

https://codereview.chromium.org/331683003/diff/20001/bench/gen_bench_expectat...
bench/gen_bench_expectations.py:171: # TODO: Look for commit data by finding the
files on CS and parsing it
This TODO needs to be assigned (to you). By CS I think you meant GS.

[kelvinly, 2014-06-12 18:26:20]

https://codereview.chromium.org/331683003/diff/20001/bench/gen_bench_expectat...
File bench/gen_bench_expectations.py (right):

https://codereview.chromium.org/331683003/diff/20001/bench/gen_bench_expectat...
bench/gen_bench_expectations.py:171: # TODO: Look for commit data by finding the
files on CS and parsing it
On 2014/06/12 18:12:54, benchen wrote:
> This TODO needs to be assigned (to you). By CS I think you meant GS.

Ah, I forgot to get rid of this TODO actually. Fixed

[kelvinly, 2014-06-12 18:26:27]

The CQ bit was checked by kelvinly@google.com

[commit-bot: I haz the power, 2014-06-12 18:27:37]

CQ is trying da patch. Follow status at
 https://skia-tree-status.appspot.com/cq/kelvinly@google.com/331683003/40001

[commit-bot: I haz the power, 2014-06-12 18:27:45]

Change committed as fa1eaaa843367b5b732ed923dc5522e258e6b27a