Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(256)

Side by Side Diff: webkit/tools/layout_tests/test_expectations.txt

Issue 67198: Add concept of slow tests. (Closed)
Patch Set: do git pull, fix diff Created 11 years, 8 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
OLDNEW
1 // This file consists of lines with specifications of what 1 // This file consists of lines with specifications of what
2 // to expect from layout test cases. The test cases can be directories 2 // to expect from layout test cases. The test cases can be directories
3 // in which case the expectations apply to all test cases in that 3 // in which case the expectations apply to all test cases in that
4 // directory and any subdirectory. The format of the file is along the 4 // directory and any subdirectory. The format of the file is along the
5 // lines of: 5 // lines of:
6 // 6 //
7 // LayoutTests/fast/js/fixme.js = FAIL 7 // LayoutTests/fast/js/fixme.js = FAIL
8 // LayoutTests/fast/js/flaky.js = FAIL PASS 8 // LayoutTests/fast/js/flaky.js = FAIL PASS
9 // LayoutTests/fast/js/crash.js = CRASH TIMEOUT FAIL PASS 9 // LayoutTests/fast/js/crash.js = CRASH TIMEOUT FAIL PASS
10 // 10 //
11 // To add other options: 11 // To add other options:
12 // SKIP : LayoutTests/fast/js/no-good.js = TIMEOUT PASS 12 // SKIP : LayoutTests/fast/js/no-good.js = TIMEOUT PASS
13 // DEBUG : LayoutTests/fast/js/no-good.js = TIMEOUT PASS 13 // DEBUG : LayoutTests/fast/js/no-good.js = TIMEOUT PASS
14 // DEBUG SKIP : LayoutTests/fast/js/no-good.js = TIMEOUT PASS 14 // DEBUG SKIP : LayoutTests/fast/js/no-good.js = TIMEOUT PASS
15 // LINUX DEBUG SKIP : LayoutTests/fast/js/no-good.js = TIMEOUT PASS 15 // LINUX DEBUG SKIP : LayoutTests/fast/js/no-good.js = TIMEOUT PASS
16 // DEFER LINUX WIN : LayoutTests/fast/js/no-good.js = TIMEOUT PASS 16 // DEFER LINUX WIN : LayoutTests/fast/js/no-good.js = TIMEOUT PASS
17 // BUG1234 DEBUG MAC : LayoutTests/fast/js/no-good.js = TIMEOUT PASS 17 // BUG1234 DEBUG MAC : LayoutTests/fast/js/no-good.js = TIMEOUT PASS
18 // 18 //
19 // BUG[0-9]+: See this bug for more information. Every test that isn't marked 19 // BUG[0-9]+: See this bug for more information. Every test that isn't marked
20 // WONTFIX should have a BUG annotation. 20 // WONTFIX should have a BUG annotation.
21 // SKIP: Doesn't run the test. 21 // SKIP: Doesn't run the test.
22 // SLOW: The test takes a long time to run, but does not timeout indefinitely.
22 // WONTFIX: For tests that we never intend to pass on a given platform. 23 // WONTFIX: For tests that we never intend to pass on a given platform.
23 // DEFER: Test does not count in our statistics for the current release. 24 // DEFER: Test does not count in our statistics for the current release.
24 // DEBUG: Expectations apply only to the debug build. 25 // DEBUG: Expectations apply only to the debug build.
25 // RELEASE: Expectations apply only to release build. 26 // RELEASE: Expectations apply only to release build.
26 // LINUX/WIN/MAC: Expectations apply only to these platforms. 27 // LINUX/WIN/MAC: Expectations apply only to these platforms.
27 // 28 //
28 // The format of a test line is as follows: 29 // The format of a test line is as follows:
29 // METADATA : relative/path/to/test.html = EXPECTATIONS 30 // METADATA : relative/path/to/test.html = EXPECTATIONS
30 // 31 //
31 // For example, 32 // For example,
32 // LINUX : LayoutTests/media = PASS FAIL 33 // LINUX : LayoutTests/media = PASS FAIL
33 // 34 //
34 // The above means that all the media tests are flaky, but only on Linux. 35 // The above means that all the media tests are flaky, but only on Linux.
35 // 36 //
36 // Notes: 37 // Notes:
38 // -A test cannot be both SLOW and TIMEOUT
37 // -A test cannot be both DEFER and WONTFIX 39 // -A test cannot be both DEFER and WONTFIX
38 // -A test can be included twice, but not via the same path. 40 // -A test can be included twice, but not via the same path.
39 // -If a test is included twice, then the more precise path wins. 41 // -If a test is included twice, then the more precise path wins.
40 // -CRASH tests cannot be DEFER or WONTFIX 42 // -CRASH tests cannot be DEFER or WONTFIX
41 43
42 // ----------------------------------------------------------------- 44 // -----------------------------------------------------------------
43 // SKIPPED TESTS 45 // SKIPPED TESTS
44 // ----------------------------------------------------------------- 46 // -----------------------------------------------------------------
45 47
46 // The action taken when trying to load the resource 48 // The action taken when trying to load the resource
(...skipping 988 matching lines...) Expand 10 before | Expand all | Expand 10 after
1035 // for the current release. 1037 // for the current release.
1036 BUG10322 DEFER : LayoutTests/accessibility = FAIL 1038 BUG10322 DEFER : LayoutTests/accessibility = FAIL
1037 // Border radius not anti-aliased. 1039 // Border radius not anti-aliased.
1038 // New test. We should fix it, but it doesn't need to block the current release 1040 // New test. We should fix it, but it doesn't need to block the current release
1039 BUG10322 DEFER WIN : LayoutTests/fast/borders/fieldsetBorderRadius.html = FAIL 1041 BUG10322 DEFER WIN : LayoutTests/fast/borders/fieldsetBorderRadius.html = FAIL
1040 // Linux pixeltest failure: radial gradients don't have offset focal point 1042 // Linux pixeltest failure: radial gradients don't have offset focal point
1041 BUG10322 DEFER WIN LINUX : LayoutTests/fast/gradients/generated-gradients.html = FAIL 1043 BUG10322 DEFER WIN LINUX : LayoutTests/fast/gradients/generated-gradients.html = FAIL
1042 // Linux pixeltest failure: radial gradients don't have offset focal point 1044 // Linux pixeltest failure: radial gradients don't have offset focal point
1043 // New test. We should fix it, but it doesn't need to block the current release 1045 // New test. We should fix it, but it doesn't need to block the current release
1044 BUG10322 DEFER WIN LINUX : LayoutTests/fast/gradients/simple-gradients.html = FA IL 1046 BUG10322 DEFER WIN LINUX : LayoutTests/fast/gradients/simple-gradients.html = FA IL
1045 // This test isn't hanging, it just takes 12-13 seconds to run, which is more 1047 // This test isn't hanging, it just takes 12-13 seconds to run.
1046 // than test_shell allows. However, we shouldn't look into the speed of the test 1048 BUG10322 SLOW : LayoutTests/http/tests/misc/acid3.html = FAIL
1047 // until we have reached PASS status.
1048 BUG10322 : LayoutTests/http/tests/misc/acid3.html = TIMEOUT FAIL
1049 1049
1050 // Expectations for this test changed upstream. We should fix this test, but 1050 // Expectations for this test changed upstream. We should fix this test, but
1051 // it doesn't need to block the current release 1051 // it doesn't need to block the current release
1052 BUG10323 DEFER : LayoutTests/http/tests/xmlhttprequest/xmlhttprequest-no-content -length-onProgress.html = FAIL 1052 BUG10323 DEFER : LayoutTests/http/tests/xmlhttprequest/xmlhttprequest-no-content -length-onProgress.html = FAIL
1053 1053
1054 // Times out because this uses dom storage which we don't implement. 1054 // Times out because this uses dom storage which we don't implement.
1055 BUG4360 DEFER SKIP : LayoutTests/security/autocomplete-cleared-on-back.html = TI MEOUT 1055 BUG4360 DEFER SKIP : LayoutTests/security/autocomplete-cleared-on-back.html = TI MEOUT
1056 1056
1057 // Post-MERGE failures: these will all need to be fixed one day 1057 // Post-MERGE failures: these will all need to be fixed one day
1058 1058
(...skipping 188 matching lines...) Expand 10 before | Expand all | Expand 10 after
1247 BUG10353 DEFER : LayoutTests/fast/forms/textarea-rows-cols.html = FAIL 1247 BUG10353 DEFER : LayoutTests/fast/forms/textarea-rows-cols.html = FAIL
1248 // Linux pixeltest failure: We, with Windows, are failing to repaint correctly 1248 // Linux pixeltest failure: We, with Windows, are failing to repaint correctly
1249 BUG10353 WIN LINUX : LayoutTests/fast/repaint/overflow-outline-repaint.html = FA IL 1249 BUG10353 WIN LINUX : LayoutTests/fast/repaint/overflow-outline-repaint.html = FA IL
1250 BUG10353 WIN : LayoutTests/fast/repaint/overflow-scroll-delete.html = FAIL 1250 BUG10353 WIN : LayoutTests/fast/repaint/overflow-scroll-delete.html = FAIL
1251 // Likely just needs to be baselined. 1251 // Likely just needs to be baselined.
1252 BUG10353 MAC LINUX : LayoutTests/fast/forms/form-element-geometry.html = FAIL 1252 BUG10353 MAC LINUX : LayoutTests/fast/forms/form-element-geometry.html = FAIL
1253 BUG10353 MAC LINUX : LayoutTests/fast/replaced/width100percent-textarea.html = F AIL 1253 BUG10353 MAC LINUX : LayoutTests/fast/replaced/width100percent-textarea.html = F AIL
1254 BUG10353 LINUX : LayoutTests/fast/text/drawBidiText.html = FAIL 1254 BUG10353 LINUX : LayoutTests/fast/text/drawBidiText.html = FAIL
1255 BUG10353 LINUX : LayoutTests/http/tests/security/dataURL/xss-DENIED-from-data-ur l-in-foreign-domain-subframe.html = TIMEOUT PASS 1255 BUG10353 LINUX : LayoutTests/http/tests/security/dataURL/xss-DENIED-from-data-ur l-in-foreign-domain-subframe.html = TIMEOUT PASS
1256 chrome/fast/dom/xss-DENIED-javascript-variations.html = FAIL 1256 chrome/fast/dom/xss-DENIED-javascript-variations.html = FAIL
1257 // This test can end up taking longer than the amount of time we give 1257
1258 // each test. DEFERd until we find a better way to deal with it. 1258 BUG_OJAN WIN LINUX SLOW : LayoutTests/http/tests/misc/dns-prefetch-control.html = PASS
1259 BUG10353 DEFER WIN LINUX : LayoutTests/http/tests/misc/dns-prefetch-control.html = TIMEOUT PASS 1259
1260 // This is failing on the mac debug only, not sure why, but it's pretty repeatab le on the bots, so we 1260 // This is failing on the mac debug only, not sure why, but it's pretty repeatab le on the bots, so we
1261 // go to the complexity to split out that one case to track any other changes fo r the other platforms. 1261 // go to the complexity to split out that one case to track any other changes fo r the other platforms.
1262 BUG10353 DEBUG MAC : LayoutTests/http/tests/misc/dns-prefetch-control.html = TIM EOUT FAIL PASS 1262 BUG10353 DEBUG MAC : LayoutTests/http/tests/misc/dns-prefetch-control.html = TIM EOUT FAIL PASS
1263 1263
1264 // NEW FOR MERGE 37604:38097 1264 // NEW FOR MERGE 37604:38097
1265 // This is a Safari specific test used to document the set of global 1265 // This is a Safari specific test used to document the set of global
1266 // constructors they expose and the exact way they are printed. Not 1266 // constructors they expose and the exact way they are printed. Not
1267 // important - we might even want to ignore it. 1267 // important - we might even want to ignore it.
1268 // New tests. We should fix these, but they doesn't need to block the current 1268 // New tests. We should fix these, but they doesn't need to block the current
1269 // release 1269 // release
(...skipping 1186 matching lines...) Expand 10 before | Expand all | Expand 10 after
2456 // due to http://trac.webkit.org/changeset/42428. 2456 // due to http://trac.webkit.org/changeset/42428.
2457 BUG10489 LINUX MAC : chrome/fast/dom/messageport-gc.html = TIMEOUT 2457 BUG10489 LINUX MAC : chrome/fast/dom/messageport-gc.html = TIMEOUT
2458 2458
2459 // New test added by http://trac.webkit.org/changeset/42501. 2459 // New test added by http://trac.webkit.org/changeset/42501.
2460 // Has already been rebaselined for chromium-win. 2460 // Has already been rebaselined for chromium-win.
2461 BUG10529 LINUX : LayoutTests/editing/inserting/6703873.html = FAIL 2461 BUG10529 LINUX : LayoutTests/editing/inserting/6703873.html = FAIL
2462 2462
2463 // This test was changed upstream in http://trac.webkit.org/changeset/42447 2463 // This test was changed upstream in http://trac.webkit.org/changeset/42447
2464 // to include window.showModalDialog, which we give as undefined. 2464 // to include window.showModalDialog, which we give as undefined.
2465 BUG10530 : LayoutTests/fast/dom/Window/window-function-frame-getter-precedence.h tml = FAIL 2465 BUG10530 : LayoutTests/fast/dom/Window/window-function-frame-getter-precedence.h tml = FAIL
OLDNEW
« no previous file with comments | « webkit/tools/layout_tests/run_webkit_tests.py ('k') | webkit/tools/test_shell/test_shell_main.cc » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698