Ensure previous test preference overrides are reset between tests

Ensure previous test preference overrides are reset between tests

This patch fixes a bug introduced by [1] where explicit test preference
overrides (e.g., testRunner.overridePreference(...)) would get re-used
in later tests.

BlinkTestController::PrepareForLayoutTest should completely reset
preferences before every test. On the first test run, default_prefs_
is initialized with default preferences plus compositor-test-specific
preferences. To ensure the compositor-test-specific preferences are
reset for subsequent runs, we call OverrideWebkitPrefs(&default_prefs_).
The bug in [1] was that default_prefs_ was read from the previous test
and would include explicitly overridden preferences. This patch removes
this line so default_prefs_ is no longer set to the previous test's
preferences.

Test that compositor-specific preferences are still reset:
run-webkit-tests compositing/animation/state-at-end-event-transform-layer.html fast/canvas/canvas-render-layer.html --additional-driver-flag=--enable-slimming-paint-v2 --num-retries=0 --child-processes=1

Test that explicit preference overrides are reset:
run-webkit-tests third_party/WebKit/LayoutTests/fast/harness/override-preferences.html third_party/WebKit/LayoutTests/fast/harness/results.html --num-retries=0 --child-processes=1

[1] https://crrev.com/a5191dca2b309396a7743816917f19d037ebc39a

R=carlosk@chromium.org,wangxianzhu@chromium.org
BUG=620126
Committed: https://crrev.com/ab2d1cd2ef3c8c12da17db01410d5749f1f4cb08
Cr-Commit-Position: refs/heads/master@{#413470}

[pdr., 2016-08-21 00:11:05]

The CQ bit was checked by pdr@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-08-21 00:11:15]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-08-21 02:40:03]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-08-21 02:40:04]

Dry run: This issue passed the CQ dry run.

[Xianzhu, 2016-08-21 17:15:38]

LGTM. Thanks!

[pdr., 2016-08-22 00:58:07]

The CQ bit was checked by pdr@chromium.org

[commit-bot: I haz the power, 2016-08-22 00:58:16]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-08-22 01:04:15]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-08-22 01:04:16]

Try jobs failed on following builders:
  chromium_presubmit on master.tryserver.chromium.linux (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presub...)

[pdr., 2016-08-22 07:03:34]

pdr@chromium.org changed reviewers:
+ mkwst@chromium.org

[pdr., 2016-08-22 07:04:41]

pdr@chromium.org changed reviewers:
+ peter@chromium.org

[pdr., 2016-08-22 07:04:59]

+reviewer peter for OWNERS, mkwst is out until the 29th

[Peter Beverloo, 2016-08-22 11:23:30]

lgtm

[pdr., 2016-08-22 16:38:17]

The CQ bit was checked by pdr@chromium.org

[commit-bot: I haz the power, 2016-08-22 16:39:05]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[carlosk, 2016-08-22 17:33:01]

Late LGTM. And thanks for the very detailed description.

For the record the reason why I think these tests passed the 1st time is because
we run many content_shell processes in parallel and then the "leaking" happens
less often. But when one runs just a few tests -- like in your repro examples --
the harness launches just a single process and the errors is then evident.

[commit-bot: I haz the power, 2016-08-22 17:49:09]

Committed patchset #1 (id:1)

[commit-bot: I haz the power, 2016-08-22 17:50:38]

Description was changed from

==========
Ensure previous test preference overrides are reset between tests

This patch fixes a bug introduced by [1] where explicit test preference
overrides (e.g., testRunner.overridePreference(...)) would get re-used
in later tests.

BlinkTestController::PrepareForLayoutTest should completely reset
preferences before every test. On the first test run, default_prefs_
is initialized with default preferences plus compositor-test-specific
preferences. To ensure the compositor-test-specific preferences are
reset for subsequent runs, we call OverrideWebkitPrefs(&default_prefs_).
The bug in [1] was that default_prefs_ was read from the previous test
and would include explicitly overridden preferences. This patch removes
this line so default_prefs_ is no longer set to the previous test's
preferences.

Test that compositor-specific preferences are still reset:
run-webkit-tests compositing/animation/state-at-end-event-transform-layer.html
fast/canvas/canvas-render-layer.html
--additional-driver-flag=--enable-slimming-paint-v2 --num-retries=0
--child-processes=1

Test that explicit preference overrides are reset:
run-webkit-tests
third_party/WebKit/LayoutTests/fast/harness/override-preferences.html
third_party/WebKit/LayoutTests/fast/harness/results.html --num-retries=0
--child-processes=1

[1] https://crrev.com/a5191dca2b309396a7743816917f19d037ebc39a

R=carlosk@chromium.org,wangxianzhu@chromium.org
BUG=620126
==========

to

==========
Ensure previous test preference overrides are reset between tests

This patch fixes a bug introduced by [1] where explicit test preference
overrides (e.g., testRunner.overridePreference(...)) would get re-used
in later tests.

BlinkTestController::PrepareForLayoutTest should completely reset
preferences before every test. On the first test run, default_prefs_
is initialized with default preferences plus compositor-test-specific
preferences. To ensure the compositor-test-specific preferences are
reset for subsequent runs, we call OverrideWebkitPrefs(&default_prefs_).
The bug in [1] was that default_prefs_ was read from the previous test
and would include explicitly overridden preferences. This patch removes
this line so default_prefs_ is no longer set to the previous test's
preferences.

Test that compositor-specific preferences are still reset:
run-webkit-tests compositing/animation/state-at-end-event-transform-layer.html
fast/canvas/canvas-render-layer.html
--additional-driver-flag=--enable-slimming-paint-v2 --num-retries=0
--child-processes=1

Test that explicit preference overrides are reset:
run-webkit-tests
third_party/WebKit/LayoutTests/fast/harness/override-preferences.html
third_party/WebKit/LayoutTests/fast/harness/results.html --num-retries=0
--child-processes=1

[1] https://crrev.com/a5191dca2b309396a7743816917f19d037ebc39a

R=carlosk@chromium.org,wangxianzhu@chromium.org
BUG=620126

Committed: https://crrev.com/ab2d1cd2ef3c8c12da17db01410d5749f1f4cb08
Cr-Commit-Position: refs/heads/master@{#413470}
==========

[commit-bot: I haz the power, 2016-08-22 17:50:39]

Patchset 1 (id:??) landed as
https://crrev.com/ab2d1cd2ef3c8c12da17db01410d5749f1f4cb08
Cr-Commit-Position: refs/heads/master@{#413470}