Align the interface of TestHarness defined in gpu_tests/webgl_conformance.py with that in third_par…

Align the interface of TestHarness defined in gpu_tests/webgl_conformance.py with that in third_party/webgl_conformance.

BUG=314997
NOTRY=true

Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=233243

[Sami, 2013-11-05 10:54:21]

Thanks for the fix, lgtm! I'm trying to think of a way to test that we can catch
failures like this, but it's not straigh forward because that kind of
expected-to-fail test would need to live in the conformance suite which has no
concept of test expectations.

[Ken Russell (switch to Gerrit), 2013-11-05 18:42:24]

Brandon: how could this have been working all along?

[Ken Russell (switch to Gerrit), 2013-11-05 18:43:04]

+zmo as FYI

[Ken Russell (switch to Gerrit), 2013-11-05 18:58:41]

After further review: LGTM. Thank you very much for tracking this down.

The mac_gpu try failure should clear up once
https://codereview.chromium.org/60473002/ lands.

[bajones, 2013-11-05 19:23:07]

On 2013/11/05 18:42:24, Ken Russell wrote:
> Brandon: how could this have been working all along?

Confused me for a second too, but if you look back at the 1.0.1 tests the only
passed success and msg into the harness. This must have broken when we rebased
the tests to the 1.0.3 set and we simply didn't notice.

The nature of Javascript means that !!<string> will return True unless it gets
an empty string, and the boolean in msg would have automatically cast to a
string for the log function. As such nothing would have failed outright, and you
wouldn't notice that anything was broken unless a test was failing and you knew
exactly what to look at. Like Sami said, I'm not really sure how we could catch
errors line this.

[junj, 2013-11-06 00:24:46]

On 2013/11/05 19:23:07, bajones wrote:
> On 2013/11/05 18:42:24, Ken Russell wrote:
> > Brandon: how could this have been working all along?
> 
> Confused me for a second too, but if you look back at the 1.0.1 tests the only
> passed success and msg into the harness. This must have broken when we rebased
> the tests to the 1.0.3 set and we simply didn't notice.
> 
> The nature of Javascript means that !!<string> will return True unless it gets
> an empty string, and the boolean in msg would have automatically cast to a
> string for the log function. As such nothing would have failed outright, and
you
> wouldn't notice that anything was broken unless a test was failing and you
knew
> exactly what to look at. Like Sami said, I'm not really sure how we could
catch
> errors line this.

Thanks for reviewing the patch. Commit it now.

[commit-bot: I haz the power, 2013-11-06 00:26:53]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/jun.a.jiang@intel.com/58833003/1

[commit-bot: I haz the power, 2013-11-06 04:44:40]

Retried try job too often on linux_aura for step(s) browser_tests
http://build.chromium.org/p/tryserver.chromium/buildstatus?builder=linux_aura...

[commit-bot: I haz the power, 2013-11-06 08:15:47]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/jun.a.jiang@intel.com/58833003/1

[commit-bot: I haz the power, 2013-11-06 08:19:32]

Change committed as 233243