Added GPU stress test with multiple tabs in one single window.

Added GPU stress test with multiple tabs in one single window.

BUG=157495
TEST=./gpu_validation/run_gpu_validation_tests

[Danh Nguyen, 2012-12-21 22:21:34]

Hi Nat, Dave,

I'm adding a simple GPU stress test case:
- Opens multiple tabs in a single window.
- Randomly cycles to the tabs and closes them.
Please take a look whenever you have a chance.

Thanks,
Danh

[nduca, 2012-12-21 23:04:31]

I you should call the main script run_gpu_validation_tests. Or run_<whatever
your directory name is>

Your tests themselves should not have options. Unit tests dont have options and
they shouldn't. They test a specific case. And if you want a few scenarios, you
make a few different test instances and have some shared code between them.

Moreover, I think in the gpu validation case, it doesn't make any sense to have
real URLs. Rather, you should use synthetic data that you put in a unittest_data
folder. CCameron has a number of synthetic assets that can be used.

If at some point later we want to use real pages as part of the memory test,
that is okay, but then we need to use the page_set/page_runner system. I think
we should put that off for a while because thats a whole new part of telemetry
for you to learn, and its always better to use fixed synthetic assets for
verification instead of real world sites.

[nduca, 2013-03-13 06:08:48]

Did we give up on this?

[danhn, 2013-03-13 13:30:57]

Hi Nat,

It's been a long time.  I'm going to get this done this week.

Thanks,
Danh


On Tue, Mar 12, 2013 at 11:08 PM, <nduca@chromium.org> wrote:

> Did we give up on this?
>
>
https://codereview.chromium.**org/11668013/<https://codereview.chromium.org/1...
>

[Danh Nguyen, 2013-03-22 23:20:51]

Hi Nat,

I hope I've addressed all of your comments.  Could you take another look at this
CL whenever you have a chance?

Thanks and have a good weekend,

Danh

[nduca, 2013-03-26 06:32:07]

I think there is now a content/test/gpu folder that has telemetry stuff. Maybe
you should build in there.

[danhn, 2013-03-26 15:03:27]

Thanks, Nat.  I'll move them there.  I've copied the test assets from
content/test/data/gpu.  Maybe when I move the test there, I don't need to
copy the test assets.

Danh


On Mon, Mar 25, 2013 at 11:32 PM, <nduca@chromium.org> wrote:

> I think there is now a content/test/gpu folder that has telemetry stuff.
> Maybe
> you should build in there.
>
>
https://codereview.chromium.**org/11668013/<https://codereview.chromium.org/1...
>

[dtu, 2013-03-28 02:11:42]

This doesn't seem like a unit test. Would it be better as a standalone
executable? I guess the determining question for that is, are you expecting to
run a suite of these tests together with no command-line flags?

https://codereview.chromium.org/11668013/diff/9001/tools/gpu_validation/tests...
File tools/gpu_validation/tests/__init__.py (right):

https://codereview.chromium.org/11668013/diff/9001/tools/gpu_validation/tests...
tools/gpu_validation/tests/__init__.py:10: '..', '..', 'telemetry')
Indent.

[nduca, 2013-03-28 18:57:23]

I'm confused, why wouldn't this be a unit test, ultimately?

input: chrome
test: did chrome correctly render each tab after loading many heavy tabs
output: pass fail

Note i've not read the test being performed...

[dtu, 2013-03-28 19:55:32]

On 2013/03/28 18:57:23, nduca wrote:
> I'm confused, why wouldn't this be a unit test, ultimately?
> 
> input: chrome
> test: did chrome correctly render each tab after loading many heavy tabs
> output: pass fail
> 
> Note i've not read the test being performed...

Hrm, okay, makes sense.

[Danh Nguyen, 2013-03-29 14:08:34]

Thanks Dave & Nat for reviewing.

I've moved the test to content/test/gpu as Nat suggested.  Since it doesn't use
page runner for now, I'd like to keep it as a separate suite.

Could you take another look whenever you have a chance?

Thanks,
Danh

https://codereview.chromium.org/11668013/diff/9001/tools/gpu_validation/tests...
File tools/gpu_validation/tests/__init__.py (right):

https://codereview.chromium.org/11668013/diff/9001/tools/gpu_validation/tests...
tools/gpu_validation/tests/__init__.py:10: '..', '..', 'telemetry')
On 2013/03/28 02:11:42, Dave Tu wrote:
> Indent.

Done.  Thanks.

[nduca, 2013-04-02 23:44:25]

@dtu can you review? I'm swamped.

[dtu, 2013-04-03 00:32:49]

just nits. lgtm

https://codereview.chromium.org/11668013/diff/19001/content/test/gpu/gpu_stre...
File content/test/gpu/gpu_stress_tests/gpu_stress_unittest.py (right):

https://codereview.chromium.org/11668013/diff/19001/content/test/gpu/gpu_stre...
content/test/gpu/gpu_stress_tests/gpu_stress_unittest.py:54: tab_nums = range(1,
NUM_TABS + 1)
xrange

https://codereview.chromium.org/11668013/diff/19001/content/test/gpu/gpu_stre...
content/test/gpu/gpu_stress_tests/gpu_stress_unittest.py:61: while num_tabs  >
1:
Why not just always len(self._tabs)?

[Danh Nguyen, 2013-04-03 15:21:57]

Thank you Nat & Dave for your help on this CL.

Danh

https://codereview.chromium.org/11668013/diff/19001/content/test/gpu/gpu_stre...
File content/test/gpu/gpu_stress_tests/gpu_stress_unittest.py (right):

https://codereview.chromium.org/11668013/diff/19001/content/test/gpu/gpu_stre...
content/test/gpu/gpu_stress_tests/gpu_stress_unittest.py:54: tab_nums = range(1,
NUM_TABS + 1)
On 2013/04/03 00:32:49, Dave Tu wrote:
> xrange

xrange doesn't work with random.shuffle() on the next line, so I need to use
range() here.

https://codereview.chromium.org/11668013/diff/19001/content/test/gpu/gpu_stre...
content/test/gpu/gpu_stress_tests/gpu_stress_unittest.py:61: while num_tabs  >
1:
On 2013/04/03 00:32:49, Dave Tu wrote:
> Why not just always len(self._tabs)?

Done.  Thanks, Dave. The loop looks much better now.

[Danh Nguyen, 2013-04-03 15:24:41]

Hi Ken,

I'm adding a GPU stress test which Vangelis asked me to work on a while back. 
Could you help to take a quick look and give me your OWNERS blessing, if that's
possible?

Thanks,
Danh

[nduca, 2013-04-03 17:46:59]

I'm not sure I agree with the previous lgt from @dtu. Why is this not a page
test?

[nduca, 2013-04-03 18:12:48]

There are many parts of page_runner that we bypass with this test. E.g. setting
up the clean browser.

This shouldn't have its own runner. ./gpu/run_Tests should suffice. Figure out
how to hook them together. If it was a page test it would just work.

[Ken Russell (switch to Gerrit), 2013-04-03 22:02:07]

I don't have experience with telemetry so nduca and dtu should review this. They
should also be in the OWNERS file.

[dtu, 2013-04-10 23:52:32]

On 2013/04/03 17:46:59, nduca wrote:
> I'm not sure I agree with the previous lgt from @dtu. Why is this not a page
> test?

Because this test doesn't fit that well into the PageTest framework. Yes, there
are some nice things that you get with PageTest, but there are also some things
that wouldn't work here, like navigating to the pages in a page set. shatch's
similar test, which uses a PageBenchmark, creates a dummy page set and navigates
to a dummy page before doing all the useful test stuff.
https://chromiumcodereview.appspot.com/13328004/

Would the ability to override Navigate() help here?

[nduca, 2013-04-11 03:50:54]

I think we're flailing. I"d like to propose neither this, nor simons change go
in until we get a design doc written on the topic of multi-tab tests.