Increase timeout on flaky encrypted-media content tests

Increase timeout on slow encrypted-media layout tests

The default timeout for tests using the w3c test harness is 2 seconds.
For several of these tests that play a little bit of video, this is not
long enough, especially when running using slower builds (like ASAN).
Setting the timeouts to a large value (60s) for both per-test time and
entire test time so that the infrastructure timeouts will be used.

BUG=353770, 353792
TEST=layout tests pass repeatedly using ASAN build

Committed: https://src.chromium.org/viewvc/blink?view=rev&revision=169712

[jrummell, 2014-03-19 20:23:40]

PTAL.

[scherkus (not reviewing), 2014-03-19 20:40:24]

dpranke: do we have some way of tweaking timeout values based on the build
configuration?

for example, with C++ tests we have base/test/test_timeouts.h that will bump up
the values when running under debug, valgrind, etc...

[xhwang, 2014-03-19 20:41:41]

Can you s/flaky/slow and s/content/layout in the title? Otherwise LGTM! Thanks.

[Dirk Pranke, 2014-03-19 20:51:02]

On 2014/03/19 20:40:24, scherkus wrote:
> dpranke: do we have some way of tweaking timeout values based on the build
> configuration?
> 
> for example, with C++ tests we have base/test/test_timeouts.h that will bump
up
> the values when running under debug, valgrind, etc...

You can add these tests to LayoutTests/SlowTests,. which effectively marks the
tests as "Slow" and gives you a 5x longer timeout. We also adjust the timeouts
automatically based on Release vs. Debug and the ASAN bot uses a longer timeout
even than that (which is kinda like the base/test/test_timeouts.h mechanism). 

You can also mark tests as Slow in just the Debug configuration or the ASAN
configuration by adding the entries in the appropriate files, but I wouldn't
recommend that (and it's not usually needed, a test is either Slow across the
board, or not).

You cannot adjust timeout values per-test.

Put differently, the stock timeouts for tests are :

6 seconds in Release
18 seconds in Debug
48 seconds in ASAN Release

A slow test is then expected to be 5x slower still (30 / 90 / 240s).

Does that answer your question?

Does that answer your question?

[jrummell, 2014-03-19 21:52:30]

Updated description (but not subject so emails stay together).

Looks like tests using the w3c harness have their own timeout which is not
dependent on build/OS/whatever.
https://code.google.com/p/chromium/codesearch#chromium/src/third_party/WebKit...

[jrummell, 2014-03-19 22:42:31]

After a discussion with scherkus@ and acolwell@ on how MSE w3c tests work, I've
bumped up the timeouts to 60 seconds for all EME w3c tests so that the
infrastructure timeouts are used.

[xhwang, 2014-03-19 22:45:48]

lgtm++, thanks for investigating this!

[jrummell, 2014-03-20 17:00:15]

The CQ bit was checked by jrummell@chromium.org

[commit-bot: I haz the power, 2014-03-20 17:00:19]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/jrummell@chromium.org/205143002/20001

[commit-bot: I haz the power, 2014-03-20 17:23:08]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2014-03-20 17:23:08]

Try jobs failed on following builders:
  tryserver.blink on win_blink_compile_dbg

[jrummell, 2014-03-20 17:47:17]

The CQ bit was checked by jrummell@chromium.org

[commit-bot: I haz the power, 2014-03-20 17:48:10]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/jrummell@chromium.org/205143002/40001

[commit-bot: I haz the power, 2014-03-20 17:49:03]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2014-03-20 17:49:04]

Failed to apply patch for LayoutTests/TestExpectations:
While running patch -p1 --forward --force --no-backup-if-mismatch;
  patching file LayoutTests/TestExpectations
  Hunk #1 FAILED at 960.
  1 out of 1 hunk FAILED -- saving rejects to file
LayoutTests/TestExpectations.rej

Patch:       LayoutTests/TestExpectations
Index: LayoutTests/TestExpectations
diff --git a/LayoutTests/TestExpectations b/LayoutTests/TestExpectations
index
8009a6f0ff7fc984f5cc66e7b42ede47ec80e6fe..8d968fa4afbf7e82530944ee5052a6120db41494
100644
--- a/LayoutTests/TestExpectations
+++ b/LayoutTests/TestExpectations
@@ -960,6 +960,4 @@ crbug.com/353736 [ Linux ]
fast/forms/select/listbox-appearance-basic.html [ Pas
 crbug.com/353751
http/tests/security/contentSecurityPolicy/1.1/frame-ancestors/frame-ancestors-nested-same-in-same-none-block.html
[ Pass Failure ]
 crbug.com/353763
http/tests/security/contentSecurityPolicy/1.1/frame-ancestors/frame-ancestors-nested-same-in-same-url-block.html
[ Pass Failure ]
 crbug.com/353754 fast/scrolling/hover-during-scroll.html [ Pass Failure ]
-crbug.com/353770 [ Linux ]
media/encrypted-media/encrypted-media-playback-setmediakeys-after-src.html [
Pass Failure Timeout ]
 crbug.com/353790 [ Linux Win ] virtual/threaded/animations/import.html [ Pass
Timeout ]
-crbug.com/353792 media/encrypted-media/encrypted-media-waiting-for-a-key.html [
Pass Failure Crash ]

[jrummell, 2014-03-20 17:58:15]

The CQ bit was checked by jrummell@chromium.org

[commit-bot: I haz the power, 2014-03-20 17:58:23]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/jrummell@chromium.org/205143002/40001

[commit-bot: I haz the power, 2014-03-20 17:58:52]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2014-03-20 17:58:54]

Failed to apply patch for LayoutTests/TestExpectations:
While running patch -p1 --forward --force --no-backup-if-mismatch;
  patching file LayoutTests/TestExpectations
  Hunk #1 FAILED at 960.
  1 out of 1 hunk FAILED -- saving rejects to file
LayoutTests/TestExpectations.rej

Patch:       LayoutTests/TestExpectations
Index: LayoutTests/TestExpectations
diff --git a/LayoutTests/TestExpectations b/LayoutTests/TestExpectations
index
8009a6f0ff7fc984f5cc66e7b42ede47ec80e6fe..8d968fa4afbf7e82530944ee5052a6120db41494
100644
--- a/LayoutTests/TestExpectations
+++ b/LayoutTests/TestExpectations
@@ -960,6 +960,4 @@ crbug.com/353736 [ Linux ]
fast/forms/select/listbox-appearance-basic.html [ Pas
 crbug.com/353751
http/tests/security/contentSecurityPolicy/1.1/frame-ancestors/frame-ancestors-nested-same-in-same-none-block.html
[ Pass Failure ]
 crbug.com/353763
http/tests/security/contentSecurityPolicy/1.1/frame-ancestors/frame-ancestors-nested-same-in-same-url-block.html
[ Pass Failure ]
 crbug.com/353754 fast/scrolling/hover-during-scroll.html [ Pass Failure ]
-crbug.com/353770 [ Linux ]
media/encrypted-media/encrypted-media-playback-setmediakeys-after-src.html [
Pass Failure Timeout ]
 crbug.com/353790 [ Linux Win ] virtual/threaded/animations/import.html [ Pass
Timeout ]
-crbug.com/353792 media/encrypted-media/encrypted-media-waiting-for-a-key.html [
Pass Failure Crash ]

[jrummell, 2014-03-21 00:06:21]

The CQ bit was checked by jrummell@chromium.org

[commit-bot: I haz the power, 2014-03-21 00:06:30]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/jrummell@chromium.org/205143002/50001

[commit-bot: I haz the power, 2014-03-21 00:06:50]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2014-03-21 00:06:50]

Failed to apply patch for LayoutTests/TestExpectations:
While running patch -p1 --forward --force --no-backup-if-mismatch;
  patching file LayoutTests/TestExpectations
  Hunk #1 FAILED at 960.
  1 out of 1 hunk FAILED -- saving rejects to file
LayoutTests/TestExpectations.rej

Patch:       LayoutTests/TestExpectations
Index: LayoutTests/TestExpectations
diff --git a/LayoutTests/TestExpectations b/LayoutTests/TestExpectations
index
8009a6f0ff7fc984f5cc66e7b42ede47ec80e6fe..8d968fa4afbf7e82530944ee5052a6120db41494
100644
--- a/LayoutTests/TestExpectations
+++ b/LayoutTests/TestExpectations
@@ -960,6 +960,4 @@ crbug.com/353736 [ Linux ]
fast/forms/select/listbox-appearance-basic.html [ Pas
 crbug.com/353751
http/tests/security/contentSecurityPolicy/1.1/frame-ancestors/frame-ancestors-nested-same-in-same-none-block.html
[ Pass Failure ]
 crbug.com/353763
http/tests/security/contentSecurityPolicy/1.1/frame-ancestors/frame-ancestors-nested-same-in-same-url-block.html
[ Pass Failure ]
 crbug.com/353754 fast/scrolling/hover-during-scroll.html [ Pass Failure ]
-crbug.com/353770 [ Linux ]
media/encrypted-media/encrypted-media-playback-setmediakeys-after-src.html [
Pass Failure Timeout ]
 crbug.com/353790 [ Linux Win ] virtual/threaded/animations/import.html [ Pass
Timeout ]
-crbug.com/353792 media/encrypted-media/encrypted-media-waiting-for-a-key.html [
Pass Failure Crash ]

[jrummell, 2014-03-21 00:15:25]

The CQ bit was checked by jrummell@chromium.org

[commit-bot: I haz the power, 2014-03-21 00:15:33]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/jrummell@chromium.org/205143002/70001

[commit-bot: I haz the power, 2014-03-21 01:20:32]

Change committed as 169712