Migrate from telemetry PageSet -> StorySet.

Migrate from telemetry PageSet -> StorySet.

BUG=439512
Committed: https://crrev.com/db17bf4c4d4bae6268eb2c73e9e3f908d85f1a2d
Cr-Commit-Position: refs/heads/master@{#337685}

[sullivan, 2015-07-07 18:14:38]

sullivan@chromium.org changed reviewers:
+ nednguyen@chromium.org, sclittle@chromium.org

[sullivan, 2015-07-07 18:14:39]

This CL gets the tests much farther, but most run_benchmark commands still fail
for me. Sample output below, any ideas? I'm running on a linux workstation.


c$ tools/chrome_proxy/run_benchmark chrome_proxy_benchmark.block_once.block_once
--browser=release
[ RUN      ] http://check.googlezip.net/blocksingle/
WARNING:root:2015-07-07 14:11:35,281:Could not find Flash at
./third_party/adobe/flash/binaries/ppapi/linux/libpepflashplayer.so. Continuing
without Flash.
To run with Flash, check it out via http://go/read-src-internal
Traceback (most recent call last):
  File "tools/chrome_proxy/../telemetry/telemetry/internal/story_runner.py",
line 84, in _RunStoryAndProcessErrorIfNeeded
    state.RunStory(results)
  File "tools/chrome_proxy/../telemetry/telemetry/page/shared_page_state.py",
line 287, in RunStory
    self._current_page, self._current_tab, results)
  File
"/usr/local/google/code/chromium/src/tools/chrome_proxy/common/chrome_proxy_measurements.py",
line 41, in ValidateAndMeasurePage
    self.AddResults(tab, results)
  File
"/usr/local/google/code/chromium/src/tools/chrome_proxy/integration_tests/chrome_proxy_measurements.py",
line 90, in AddResults
    self._metrics.AddResultsForBlockOnce(tab, results)
  File
"/usr/local/google/code/chromium/src/tools/chrome_proxy/integration_tests/chrome_proxy_metrics.py",
line 395, in AddResultsForBlockOnce
    resp.response.url)
ChromeProxyMetricException: Missing Chrome-Compression-Proxy Via
header.http://check.googlezip.net/image.png

[  FAILED  ] http://check.googlezip.net/blocksingle/ (2356 ms)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ]  http://check.googlezip.net/blocksingle/

1 FAILED TEST

Pages: []
RESULT telemetry_page_measurement_results: num_failed= 1 count
RESULT telemetry_page_measurement_results: num_errored= 0 count

View result at
file:///usr/local/google/code/chromium/src/tools/chrome_proxy/results.html

[nednguyen, 2015-07-07 18:34:45]

nednguyen@google.com changed reviewers:
+ nednguyen@google.com

[nednguyen, 2015-07-07 18:34:45]

https://codereview.chromium.org/1218073014/diff/20001/tools/chrome_proxy/inte...
File tools/chrome_proxy/integration_tests/chrome_proxy_pagesets/smoke.py
(right):

https://codereview.chromium.org/1218073014/diff/20001/tools/chrome_proxy/inte...
tools/chrome_proxy/integration_tests/chrome_proxy_pagesets/smoke.py:6: from
telemetry.story import story_set as story_set_module
this should be from "telemetry import story"

https://codereview.chromium.org/1218073014/diff/20001/tools/chrome_proxy/inte...
tools/chrome_proxy/integration_tests/chrome_proxy_pagesets/smoke.py:81: class
SmokeStorySet(story_set_module.StorySet):
story.StorySet

[sclittle, 2015-07-07 18:51:24]

On 2015/07/07 18:14:39, sullivan wrote:
> This CL gets the tests much farther, but most run_benchmark commands still
fail
> for me. Sample output below, any ideas? I'm running on a linux workstation.
> 
> 
> c$ tools/chrome_proxy/run_benchmark
chrome_proxy_benchmark.block_once.block_once
> --browser=release
> [ RUN      ] http://check.googlezip.net/blocksingle/
> WARNING:root:2015-07-07 14:11:35,281:Could not find Flash at
> ./third_party/adobe/flash/binaries/ppapi/linux/libpepflashplayer.so.
Continuing
> without Flash.
> To run with Flash, check it out via http://go/read-src-internal
> Traceback (most recent call last):
>   File "tools/chrome_proxy/../telemetry/telemetry/internal/story_runner.py",
> line 84, in _RunStoryAndProcessErrorIfNeeded
>     state.RunStory(results)
>   File "tools/chrome_proxy/../telemetry/telemetry/page/shared_page_state.py",
> line 287, in RunStory
>     self._current_page, self._current_tab, results)
>   File
>
"/usr/local/google/code/chromium/src/tools/chrome_proxy/common/chrome_proxy_measurements.py",
> line 41, in ValidateAndMeasurePage
>     self.AddResults(tab, results)
>   File
>
"/usr/local/google/code/chromium/src/tools/chrome_proxy/integration_tests/chrome_proxy_measurements.py",
> line 90, in AddResults
>     self._metrics.AddResultsForBlockOnce(tab, results)
>   File
>
"/usr/local/google/code/chromium/src/tools/chrome_proxy/integration_tests/chrome_proxy_metrics.py",
> line 395, in AddResultsForBlockOnce
>     resp.response.url)
> ChromeProxyMetricException: Missing Chrome-Compression-Proxy Via
> header.http://check.googlezip.net/image.png
> 
> [  FAILED  ] http://check.googlezip.net/blocksingle/ (2356 ms)
> [  PASSED  ] 0 tests.
> [  FAILED  ] 1 test, listed below:
> [  FAILED  ]  http://check.googlezip.net/blocksingle/
> 
> 1 FAILED TEST
> 
> Pages: []
> RESULT telemetry_page_measurement_results: num_failed= 1 count
> RESULT telemetry_page_measurement_results: num_errored= 0 count
> 
> View result at
> file:///usr/local/google/code/chromium/src/tools/chrome_proxy/results.html

The chrome_proxy tests are meant to be run against live sites, using the
--use_live_sites flag. That might fix the problem you're seeing.

[sullivan, 2015-07-07 19:32:37]

On 2015/07/07 18:51:24, sclittle wrote:
> On 2015/07/07 18:14:39, sullivan wrote:
> > This CL gets the tests much farther, but most run_benchmark commands still
> fail
> > for me. Sample output below, any ideas? I'm running on a linux workstation.
> > 
> > 
> > c$ tools/chrome_proxy/run_benchmark
> chrome_proxy_benchmark.block_once.block_once
> > --browser=release
> > [ RUN      ] http://check.googlezip.net/blocksingle/
> > WARNING:root:2015-07-07 14:11:35,281:Could not find Flash at
> > ./third_party/adobe/flash/binaries/ppapi/linux/libpepflashplayer.so.
> Continuing
> > without Flash.
> > To run with Flash, check it out via http://go/read-src-internal
> > Traceback (most recent call last):
> >   File "tools/chrome_proxy/../telemetry/telemetry/internal/story_runner.py",
> > line 84, in _RunStoryAndProcessErrorIfNeeded
> >     state.RunStory(results)
> >   File
"tools/chrome_proxy/../telemetry/telemetry/page/shared_page_state.py",
> > line 287, in RunStory
> >     self._current_page, self._current_tab, results)
> >   File
> >
>
"/usr/local/google/code/chromium/src/tools/chrome_proxy/common/chrome_proxy_measurements.py",
> > line 41, in ValidateAndMeasurePage
> >     self.AddResults(tab, results)
> >   File
> >
>
"/usr/local/google/code/chromium/src/tools/chrome_proxy/integration_tests/chrome_proxy_measurements.py",
> > line 90, in AddResults
> >     self._metrics.AddResultsForBlockOnce(tab, results)
> >   File
> >
>
"/usr/local/google/code/chromium/src/tools/chrome_proxy/integration_tests/chrome_proxy_metrics.py",
> > line 395, in AddResultsForBlockOnce
> >     resp.response.url)
> > ChromeProxyMetricException: Missing Chrome-Compression-Proxy Via
> > header.http://check.googlezip.net/image.png
> > 
> > [  FAILED  ] http://check.googlezip.net/blocksingle/ (2356 ms)
> > [  PASSED  ] 0 tests.
> > [  FAILED  ] 1 test, listed below:
> > [  FAILED  ]  http://check.googlezip.net/blocksingle/
> > 
> > 1 FAILED TEST
> > 
> > Pages: []
> > RESULT telemetry_page_measurement_results: num_failed= 1 count
> > RESULT telemetry_page_measurement_results: num_errored= 0 count
> > 
> > View result at
> > file:///usr/local/google/code/chromium/src/tools/chrome_proxy/results.html
> 
> The chrome_proxy tests are meant to be run against live sites, using the
> --use_live_sites flag. That might fix the problem you're seeing.

Thanks! I am still seeing the error after adding the --use-live-sites flag.
Maybe the smoke test failures are more representative?
$ tools/chrome_proxy/run_benchmark --use-live-sites --browser=release
chrome_proxy_benchmark.smoke.smoke
[ RUN      ] header validation
WARNING:root:2015-07-07 15:19:51,151:Could not find Flash at
./third_party/adobe/flash/binaries/ppapi/linux/libpepflashplayer.so. Continuing
without Flash.
To run with Flash, check it out via http://go/read-src-internal
Traceback (most recent call last):
  File "tools/chrome_proxy/../telemetry/telemetry/internal/story_runner.py",
line 84, in _RunStoryAndProcessErrorIfNeeded
    state.RunStory(results)
  File "tools/chrome_proxy/../telemetry/telemetry/page/shared_page_state.py",
line 287, in RunStory
    self._current_page, self._current_tab, results)
  File
"/usr/local/google/code/chromium/src/tools/chrome_proxy/common/chrome_proxy_measurements.py",
line 41, in ValidateAndMeasurePage
    self.AddResults(tab, results)
  File
"/usr/local/google/code/chromium/src/tools/chrome_proxy/integration_tests/chrome_proxy_measurements.py",
line 375, in AddResults
    add_result(tab, results)
  File
"/usr/local/google/code/chromium/src/tools/chrome_proxy/integration_tests/chrome_proxy_metrics.py",
line 70, in AddResultsForHeaderValidation
    r.url, r.GetHeader('Via'), r.GetHeader('Referer'), r.status))
ChromeProxyMetricException: http://check.googlezip.net/test.html: Via header
(None) is not valid (refer=None, status=200)

[  FAILED  ] header validation (1415 ms)
[ RUN      ] compression: image
Traceback (most recent call last):
  File "tools/chrome_proxy/../telemetry/telemetry/internal/story_runner.py",
line 84, in _RunStoryAndProcessErrorIfNeeded
    state.RunStory(results)
  File "tools/chrome_proxy/../telemetry/telemetry/page/shared_page_state.py",
line 287, in RunStory
    self._current_page, self._current_tab, results)
  File
"/usr/local/google/code/chromium/src/tools/chrome_proxy/common/chrome_proxy_measurements.py",
line 41, in ValidateAndMeasurePage
    self.AddResults(tab, results)
  File
"/usr/local/google/code/chromium/src/tools/chrome_proxy/integration_tests/chrome_proxy_measurements.py",
line 375, in AddResults
    add_result(tab, results)
  File
"/usr/local/google/code/chromium/src/tools/chrome_proxy/integration_tests/chrome_proxy_metrics.py",
line 70, in AddResultsForHeaderValidation
    r.url, r.GetHeader('Via'), r.GetHeader('Referer'), r.status))
ChromeProxyMetricException: http://check.googlezip.net/static/: Via header
(None) is not valid (refer=None, status=200)

[  FAILED  ] compression: image (1796 ms)
[ RUN      ] bypass
[       OK ] bypass (2556 ms)
[ RUN      ] compression: javascript
Traceback (most recent call last):
  File "tools/chrome_proxy/../telemetry/telemetry/internal/story_runner.py",
line 84, in _RunStoryAndProcessErrorIfNeeded
    state.RunStory(results)
  File "tools/chrome_proxy/../telemetry/telemetry/page/shared_page_state.py",
line 287, in RunStory
    self._current_page, self._current_tab, results)
  File
"/usr/local/google/code/chromium/src/tools/chrome_proxy/common/chrome_proxy_measurements.py",
line 41, in ValidateAndMeasurePage
    self.AddResults(tab, results)
  File
"/usr/local/google/code/chromium/src/tools/chrome_proxy/integration_tests/chrome_proxy_measurements.py",
line 375, in AddResults
    add_result(tab, results)
  File
"/usr/local/google/code/chromium/src/tools/chrome_proxy/integration_tests/chrome_proxy_metrics.py",
line 70, in AddResultsForHeaderValidation
    r.url, r.GetHeader('Via'), r.GetHeader('Referer'), r.status))
ChromeProxyMetricException: http://check.googlezip.net/static/: Via header
(None) is not valid (refer=None, status=200)

[  FAILED  ] compression: javascript (1699 ms)
[ RUN      ] compression: css
Traceback (most recent call last):
  File "tools/chrome_proxy/../telemetry/telemetry/internal/story_runner.py",
line 84, in _RunStoryAndProcessErrorIfNeeded
    state.RunStory(results)
  File "tools/chrome_proxy/../telemetry/telemetry/page/shared_page_state.py",
line 287, in RunStory
    self._current_page, self._current_tab, results)
  File
"/usr/local/google/code/chromium/src/tools/chrome_proxy/common/chrome_proxy_measurements.py",
line 41, in ValidateAndMeasurePage
    self.AddResults(tab, results)
  File
"/usr/local/google/code/chromium/src/tools/chrome_proxy/integration_tests/chrome_proxy_measurements.py",
line 375, in AddResults
    add_result(tab, results)
  File
"/usr/local/google/code/chromium/src/tools/chrome_proxy/integration_tests/chrome_proxy_metrics.py",
line 70, in AddResultsForHeaderValidation
    r.url, r.GetHeader('Via'), r.GetHeader('Referer'), r.status))
ChromeProxyMetricException: http://check.googlezip.net/static/: Via header
(None) is not valid (refer=None, status=200)

[sclittle, 2015-07-07 20:04:01]

Hmm, if you're building vanilla open source Chromium, the problem might be that
the Data Reduction Proxy auth key isn't being set correctly, so every request is
getting bypassed. This auth key is baked into the closed source part of Chrome
in an effort to prevent abuse of the Data Reduction Proxy.

This looks like it will probably work fine, but if you want, you can either try
building Chromium including the closed source google API keys, or you can get
the auth key value from
https://cs.corp.google.com/#clankium/src/google_apis/internal/google_chrome_a...
and use the Chrome flag --spdy-proxy-auth-value=... to set it.

Please do *not* post that auth key value publicly here or elsewhere if you do
use it.

[sullivan, 2015-07-07 20:09:16]

https://codereview.chromium.org/1218073014/diff/20001/tools/chrome_proxy/inte...
File tools/chrome_proxy/integration_tests/chrome_proxy_pagesets/smoke.py
(right):

https://codereview.chromium.org/1218073014/diff/20001/tools/chrome_proxy/inte...
tools/chrome_proxy/integration_tests/chrome_proxy_pagesets/smoke.py:6: from
telemetry.story import story_set as story_set_module
On 2015/07/07 18:34:45, nednguyen wrote:
> this should be from "telemetry import story"

Done.

https://codereview.chromium.org/1218073014/diff/20001/tools/chrome_proxy/inte...
tools/chrome_proxy/integration_tests/chrome_proxy_pagesets/smoke.py:81: class
SmokeStorySet(story_set_module.StorySet):
On 2015/07/07 18:34:45, nednguyen wrote:
> story.StorySet

Done.

[sullivan, 2015-07-07 20:14:52]

On 2015/07/07 20:04:01, sclittle wrote:
> Hmm, if you're building vanilla open source Chromium, the problem might be
that
> the Data Reduction Proxy auth key isn't being set correctly, so every request
is
> getting bypassed. This auth key is baked into the closed source part of Chrome
> in an effort to prevent abuse of the Data Reduction Proxy.
> 
> This looks like it will probably work fine, but if you want, you can either
try
> building Chromium including the closed source google API keys, or you can get
> the auth key value from
>
https://cs.corp.google.com/#clankium/src/google_apis/internal/google_chrome_a...
> and use the Chrome flag --spdy-proxy-auth-value=... to set it.
> 
> Please do *not* post that auth key value publicly here or elsewhere if you do
> use it.

Thanks! Tests are passing when I follow these instructions.

[sullivan, 2015-07-07 20:15:10]

On 2015/07/07 20:14:52, sullivan wrote:
> On 2015/07/07 20:04:01, sclittle wrote:
> > Hmm, if you're building vanilla open source Chromium, the problem might be
> that
> > the Data Reduction Proxy auth key isn't being set correctly, so every
request
> is
> > getting bypassed. This auth key is baked into the closed source part of
Chrome
> > in an effort to prevent abuse of the Data Reduction Proxy.
> > 
> > This looks like it will probably work fine, but if you want, you can either
> try
> > building Chromium including the closed source google API keys, or you can
get
> > the auth key value from
> >
>
https://cs.corp.google.com/#clankium/src/google_apis/internal/google_chrome_a...
> > and use the Chrome flag --spdy-proxy-auth-value=... to set it.
> > 
> > Please do *not* post that auth key value publicly here or elsewhere if you
do
> > use it.
> 
> Thanks! Tests are passing when I follow these instructions.

I need an OWNER lgtm to submit.

[sclittle, 2015-07-07 20:16:36]

lgtm

[sullivan, 2015-07-07 20:22:40]

The CQ bit was checked by sullivan@chromium.org

[commit-bot: I haz the power, 2015-07-07 20:23:05]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1218073014/40001

[commit-bot: I haz the power, 2015-07-07 22:05:52]

Committed patchset #3 (id:40001)

[commit-bot: I haz the power, 2015-07-07 22:06:54]

Patchset 3 (id:??) landed as
https://crrev.com/db17bf4c4d4bae6268eb2c73e9e3f908d85f1a2d
Cr-Commit-Position: refs/heads/master@{#337685}