This is to try and get the mac official builder to go green. BUG=none TEST=n...

This is to try and get the mac official builder to go green.  BUG=none TEST=none


Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=71411

[Mihai Parparita -not on Chrome, 2011-01-14 00:07:15]

http://codereview.chromium.org/6301004/diff/1/webkit/tools/layout_tests/test_...
File webkit/tools/layout_tests/test_expectations_chrome.txt (right):

http://codereview.chromium.org/6301004/diff/1/webkit/tools/layout_tests/test_...
webkit/tools/layout_tests/test_expectations_chrome.txt:8: OFFICALGOGREEN MAC
SKIP : media/video-timeupdate-during-playback.html = TIMEOUT PASS
I believe that new-run-webkit-tests is going to get stricter soon about which
modifiers it allows on tests, and OFFICIALGOGREEN will be rejected. BUG_GREEN
may be better.

Also, I don't think you should skip these tests. They're going to start passing
at some point, and we want to be notified of unexpected passes so that they can
be removed from this file.

[kerz_chromium, 2011-01-14 00:10:38]

On 2011/01/14 00:07:15, Mihai Parparita wrote:
>
http://codereview.chromium.org/6301004/diff/1/webkit/tools/layout_tests/test_...
> File webkit/tools/layout_tests/test_expectations_chrome.txt (right):
> 
>
http://codereview.chromium.org/6301004/diff/1/webkit/tools/layout_tests/test_...
> webkit/tools/layout_tests/test_expectations_chrome.txt:8: OFFICALGOGREEN MAC
> SKIP : media/video-timeupdate-during-playback.html = TIMEOUT PASS
> I believe that new-run-webkit-tests is going to get stricter soon about which
> modifiers it allows on tests, and OFFICIALGOGREEN will be rejected. BUG_GREEN
> may be better.
> 
> Also, I don't think you should skip these tests. They're going to start
passing
> at some point, and we want to be notified of unexpected passes so that they
can
> be removed from this file.

Is there something better than SKIP to use?  I want to see our tree go green. 
I'll change the modifier to include BUG_.

[Mihai Parparita -not on Chrome, 2011-01-14 00:15:26]

On Thu, Jan 13, 2011 at 4:10 PM,  <kerz@chromium.org> wrote:
> Is there something better than SKIP to use?  I want to see our tree go
> green.

If you leave out SKIP, then the tests will be run, and if it fails with the
failure that's at the end of the line (e.g. image diffs for IMAGE, image
_and_ text diffs for IMAGE+TEXT) then the run will still be green. This is
how the the build.chromium.org webkit builders are green even though they
fail bunch of tests.

For example,
http://build.chromium.org/p/chromium/builders/Webkit%20Linux%20Tests%20(dbg)(...
at the end:

=> Results: 4679/7629 tests passed (61.3%)

=> Tests to be fixed (2137):
     1 DumpRenderTree crash     ( 0.0%)
     1 test timed out           ( 0.0%)
    32 image and text mismatch  ( 1.5%)
   125 text diff mismatch       ( 5.8%)
    37 image mismatch           ( 1.7%)
  1941 skipped                  (90.8%)

=> Tests that will only be fixed if they crash (WONTFIX) (813):
     6 text diff mismatch       ( 0.7%)
   807 skipped                  (99.3%)

2011-01-13 16:06:53,733 10315 printing.py:543 INFO 7628 tests ran as
expected, 1 didn't:
2011-01-13 16:06:53,733 10315 printing.py:543 INFO

Expected to fail, but passed: (1)
 fast/canvas/canvas-large-dimensions.html

Mihai

> I'll change the modifier to include BUG_.
>
> Description:
> This is to try and get the mac official builder to go green.  BUG=none
> TEST=none
>
>
> Please review this at http://codereview.chromium.org/6301004/
>
> SVN Base: svn://svn.chromium.org/chrome/trunk/src/
>
> Affected files:
>  M     webkit/tools/layout_tests/test_expectations_chrome.txt
>
>
>

[tony, 2011-01-14 00:17:12]

You can run:
  webkit/tools/layout_tests/run_webkit_tests.sh --platform=google-chrome-win
--lint-test-files

to verify that you have the right syntax.  Mihai is correct and the modifier
needs to be BUG_ (or better yet, BUGCR######).

I think if you drop the SKIP, the bots will still go green (assuming you just
copied the results from the bots).

[tony, 2011-01-14 00:18:05]

Err, these are mac expectations so I guess you'd want to run:

webkit/tools/layout_tests/run_webkit_tests.sh --platform=google-chrome-mac
--lint-test-files

[kerz_chromium, 2011-01-14 00:25:16]

OK, have updated to remove the SKIP.

On Thu, Jan 13, 2011 at 4:18 PM, <tony@chromium.org> wrote:

> Err, these are mac expectations so I guess you'd want to run:
>
> webkit/tools/layout_tests/run_webkit_tests.sh --platform=google-chrome-mac
> --lint-test-files
>
>
> http://codereview.chromium.org/6301004/
>

[Mihai Parparita -not on Chrome, 2011-01-14 00:34:21]

LGTM