[LayoutTests] Split up inspector/sources/debugger-async/async-await/async-callstack-async-await.html

[LayoutTests] Split up inspector/sources/debugger-async/async-await/async-callstack-async-await.html

Split up test in order to minimize Timeout flakes

R=hbos@chromium.org, dgozman@chromium.org, littledan@chromium.org
BUG=622696, 621515
Committed: https://crrev.com/459aa22b3344f3f8097e05cf57cc2631ec0ab4b1
Cr-Commit-Position: refs/heads/master@{#402015}

[caitp (gmail), 2016-06-24 14:56:51]

Description was changed from

==========
Split up
inspector/sources/debugger-async/async-await/async-callstack-async-await.html

Split up test in order to minimize Timeout flakes

R=hbos@chromium.org, dgozman@chromium.org, littledan@chromium.org
BUG=622696, 621515
==========

to

==========
[LayoutTests] Split up
inspector/sources/debugger-async/async-await/async-callstack-async-await.html

Split up test in order to minimize Timeout flakes

R=hbos@chromium.org, dgozman@chromium.org, littledan@chromium.org
BUG=622696, 621515
==========

[caitp (gmail), 2016-06-24 19:33:50]

PTAL --- this splits up a test which flakily times out on some buildbots.

[caitp (gmail), 2016-06-24 19:34:46]

On 2016/06/24 19:33:50, caitp wrote:
> PTAL --- this splits up a test which flakily times out on some buildbots.

I think I'm missing a stacktrace, will have to try to fix this

[caitp (gmail), 2016-06-24 19:48:16]

On 2016/06/24 19:34:46, caitp wrote:
> On 2016/06/24 19:33:50, caitp wrote:
> > PTAL --- this splits up a test which flakily times out on some buildbots.
> 
> I think I'm missing a stacktrace, will have to try to fix this

and fixed

[Dan Ehrenberg, 2016-06-24 21:27:38]

Do you have any idea what causes the timeout? Or what differentiates the two
things you're splitting the test into? I'm surprised we time out instead of just
failing with a syntax error on any platform.

https://codereview.chromium.org/2099723002/diff/60001/third_party/WebKit/Layo...
File third_party/WebKit/LayoutTests/TestExpectations (right):

https://codereview.chromium.org/2099723002/diff/60001/third_party/WebKit/Layo...
third_party/WebKit/LayoutTests/TestExpectations:697: crbug.com/621515
inspector/sources/debugger-async/async-await/async-callstack-async-await2.html [
Failure Timeout ]
I would skip these tests until the feature ships, and just run the virtual ones.

[caitp (gmail), 2016-06-24 21:39:08]

On 2016/06/24 21:27:38, Dan Ehrenberg wrote:
> Do you have any idea what causes the timeout? Or what differentiates the two
> things you're splitting the test into? I'm surprised we time out instead of
just
> failing with a syntax error on any platform.

For the Windows failures, it's because LayoutTests are only given so long to run
before being considered a timeout (by default, 6 seconds, for most ports). This
stuff takes some time, so it can exceed the allotted time.

For the [ Failure Timeout ] items in TestExpectations, it's because not all of
the debugger statements are reached, and inspector tests hang until they're all
hit.

[caitp (gmail), 2016-06-24 21:43:03]

The CQ bit was checked by caitpotter88@gmail.com to run a CQ dry run

[commit-bot: I haz the power, 2016-06-24 21:43:26]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[Dan Ehrenberg, 2016-06-24 21:46:12]

On 2016/06/24 at 21:39:08, caitpotter88 wrote:
> On 2016/06/24 21:27:38, Dan Ehrenberg wrote:
> > Do you have any idea what causes the timeout? Or what differentiates the two
> > things you're splitting the test into? I'm surprised we time out instead of
just
> > failing with a syntax error on any platform.
> 
> For the Windows failures, it's because LayoutTests are only given so long to
run before being considered a timeout (by default, 6 seconds, for most ports).
This stuff takes some time, so it can exceed the allotted time.
> 
> For the [ Failure Timeout ] items in TestExpectations, it's because not all of
the debugger statements are reached, and inspector tests hang until they're all
hit.

I wonder if decreasing the time values for setTimeout may result in these tests
timing out less. However, that would result in test expectations for "Timeout"
on the virtual tests that have async/await turned on--but I don't actually see
those expectations put in. Maybe that's all fine.

For test environments where async/await is not supported (i.e., the non-virtual
tests, since async/await is only staged), I'd expect that the whole script with
'async function' does not parse, and runTest finds a missing test. Is this what
is happening? If it's an uninteresting way, it might be best just to skip them.
But what's important is to first verify that that's what's happening.

[caitp (gmail), 2016-06-24 21:53:06]

On 2016/06/24 21:46:12, Dan Ehrenberg wrote:
> On 2016/06/24 at 21:39:08, caitpotter88 wrote:
> > On 2016/06/24 21:27:38, Dan Ehrenberg wrote:
> > > Do you have any idea what causes the timeout? Or what differentiates the
two
> > > things you're splitting the test into? I'm surprised we time out instead
of
> just
> > > failing with a syntax error on any platform.
> > 
> > For the Windows failures, it's because LayoutTests are only given so long to
> run before being considered a timeout (by default, 6 seconds, for most ports).
> This stuff takes some time, so it can exceed the allotted time.
> > 
> > For the [ Failure Timeout ] items in TestExpectations, it's because not all
of
> the debugger statements are reached, and inspector tests hang until they're
all
> hit.
> 
> I wonder if decreasing the time values for setTimeout may result in these
tests
> timing out less. However, that would result in test expectations for "Timeout"
> on the virtual tests that have async/await turned on--but I don't actually see
> those expectations put in. Maybe that's all fine.
> 
> For test environments where async/await is not supported (i.e., the
non-virtual
> tests, since async/await is only staged), I'd expect that the whole script
with
> 'async function' does not parse, and runTest finds a missing test. Is this
what
> is happening? If it's an uninteresting way, it might be best just to skip
them.
> But what's important is to first verify that that's what's happening.

well, the last patchset skips the tests outside of the virtual suite, so I think
that's taken care of. The Promise Chain tests could probably do with a slightly
smaller chain, that might reduce flakes a bit.

[dgozman, 2016-06-24 22:06:15]

> I wonder if decreasing the time values for setTimeout may result in these
tests
> timing out less. However, that would result in test expectations for "Timeout"
> on the virtual tests that have async/await turned on--but I don't actually see
> those expectations put in. Maybe that's all fine.

Inspector tests are just too complex for web platform and take a long time. Even
more when doing breakpoints. Splitting them up is the typical solution.

[dgozman, 2016-06-24 22:07:38]

lgtm

[commit-bot: I haz the power, 2016-06-24 23:12:32]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-06-24 23:12:34]

Dry run: This issue passed the CQ dry run.

[caitp (gmail), 2016-06-24 23:41:45]

The CQ bit was checked by caitpotter88@gmail.com

[commit-bot: I haz the power, 2016-06-24 23:41:58]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-06-24 23:47:01]

Description was changed from

==========
[LayoutTests] Split up
inspector/sources/debugger-async/async-await/async-callstack-async-await.html

Split up test in order to minimize Timeout flakes

R=hbos@chromium.org, dgozman@chromium.org, littledan@chromium.org
BUG=622696, 621515
==========

to

==========
[LayoutTests] Split up
inspector/sources/debugger-async/async-await/async-callstack-async-await.html

Split up test in order to minimize Timeout flakes

R=hbos@chromium.org, dgozman@chromium.org, littledan@chromium.org
BUG=622696, 621515
==========

[commit-bot: I haz the power, 2016-06-24 23:47:03]

Committed patchset #5 (id:80001)

[commit-bot: I haz the power, 2016-06-24 23:48:27]

Description was changed from

==========
[LayoutTests] Split up
inspector/sources/debugger-async/async-await/async-callstack-async-await.html

Split up test in order to minimize Timeout flakes

R=hbos@chromium.org, dgozman@chromium.org, littledan@chromium.org
BUG=622696, 621515
==========

to

==========
[LayoutTests] Split up
inspector/sources/debugger-async/async-await/async-callstack-async-await.html

Split up test in order to minimize Timeout flakes

R=hbos@chromium.org, dgozman@chromium.org, littledan@chromium.org
BUG=622696, 621515

Committed: https://crrev.com/459aa22b3344f3f8097e05cf57cc2631ec0ab4b1
Cr-Commit-Position: refs/heads/master@{#402015}
==========

[commit-bot: I haz the power, 2016-06-24 23:48:28]

Patchset 5 (id:??) landed as
https://crrev.com/459aa22b3344f3f8097e05cf57cc2631ec0ab4b1
Cr-Commit-Position: refs/heads/master@{#402015}