Import web-platform-tests manual tests by default

Import web-platform-tests manual tests by default

A few manual tests already pass or fail in a useful way, like
stable-state-dialogs-manual.html. More importantly, it is difficult to
automate manual tests using wpt_automation if one must first add the
-automation.js script to cause the test to be imported.

The Fullscreen tests are imported and skipped in TestExpectations so
that they can be automated in a coming CL. These tests will all use the
same automation script, so the current import rules wouldn't work.

The wpt import is done with the same revision as the last import:
https://codereview.chromium.org/2448433002

Import wpt@bd99724e428dae78082983eab4675480c43f6234

Using update-w3c-deps in Chromium 55a661e4ed87e54b702de11e69ecc3babb2a96d0.

BUG=660846
Committed: https://crrev.com/96df472c108ec278c919bf704ad5c0eaa2621d6f
Cr-Commit-Position: refs/heads/master@{#428786}

[foolip, 2016-10-31 14:58:19]

The CQ bit was checked by foolip@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-10-31 14:58:36]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[foolip, 2016-10-31 15:02:47]

Description was changed from

==========
Import web-platform-tests manual tests by default

A few manual tests already pass or fail in a useful way, like
stable-state-dialogs-manual.html. More importantly, it is difficult to
automate manual tests using wpt_automation if one must first add the
-automation.js script to cause the test to be imported.

The Fullscreen tests are imported and skipped in TestExpectations so
that they can be automated in a coming CL. These tests will all use the
same automation script, so the current import rules wouldn't work.


The wpt import is done with the same revision as the last import:
https://codereview.chromium.org/2448433002

Import wpt@bd99724e428dae78082983eab4675480c43f6234

Using update-w3c-deps in Chromium 55a661e4ed87e54b702de11e69ecc3babb2a96d0.

BUG=660846
==========

to

==========
Import web-platform-tests manual tests by default

A few manual tests already pass or fail in a useful way, like
stable-state-dialogs-manual.html. More importantly, it is difficult to
automate manual tests using wpt_automation if one must first add the
-automation.js script to cause the test to be imported.

The Fullscreen tests are imported and skipped in TestExpectations so
that they can be automated in a coming CL. These tests will all use the
same automation script, so the current import rules wouldn't work.

The wpt import is done with the same revision as the last import:
https://codereview.chromium.org/2448433002

Import wpt@bd99724e428dae78082983eab4675480c43f6234

Using update-w3c-deps in Chromium 55a661e4ed87e54b702de11e69ecc3babb2a96d0.

BUG=660846
==========

[foolip, 2016-10-31 15:19:35]

foolip@chromium.org changed reviewers:
+ qyearsley@chromium.org, rbyers@chromium.org

[foolip, 2016-10-31 15:19:36]

PTAL

[Rick Byers, 2016-10-31 15:27:10]

LGTM

[foolip, 2016-10-31 15:33:33]

qyearsley@, this will mean that some new manual tests will trickle in with
imports. I don't know if it's worth doing anything about this, treating them
like other timeouts is probably fine, but adding them to skipped imports would
also work. I think that over time we want to either automate or remove these
tests, so it's not a bad thing that they are noticed as long as it's not a
burden.

[qyearsley, 2016-10-31 16:15:26]

On 2016/10/31 at 15:33:33, foolip wrote:
> qyearsley@, this will mean that some new manual tests will trickle in with
imports. I don't know if it's worth doing anything about this, treating them
like other timeouts is probably fine, but adding them to skipped imports would
also work. I think that over time we want to either automate or remove these
tests, so it's not a bad thing that they are noticed as long as it's not a
burden.

Yep, I think that makes sense, LGTM

[commit-bot: I haz the power, 2016-10-31 16:27:50]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-10-31 16:27:51]

Dry run: Try jobs failed on following builders:
  mac_chromium_rel_ng on master.tryserver.chromium.mac (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.mac/builders/mac_chromium_rel_...)

[Dirk Pranke, 2016-10-31 16:33:09]

dpranke@chromium.org changed reviewers:
+ dpranke@chromium.org

[Dirk Pranke, 2016-10-31 16:33:10]

I'm confused by this ... does this mean that a bunch of tests ending in
-manual.html aren't actually manual tests? If so, shouldn't they be renamed?

[foolip, 2016-10-31 16:49:33]

On 2016/10/31 16:33:10, Dirk Pranke wrote:
> I'm confused by this ... does this mean that a bunch of tests ending in
> -manual.html aren't actually manual tests? If so, shouldn't they be renamed?

There are the imported tests and why they work:

imported/wpt/html/browsers/the-window-object/the-windowproxy-object/test-window-proxy-locationbar-manual.html
imported/wpt/html/browsers/the-window-object/the-windowproxy-object/test-window-proxy-menubar-manual.html
imported/wpt/html/browsers/the-window-object/the-windowproxy-object/test-window-proxy-personalbar-manual.html
imported/wpt/html/browsers/the-window-object/the-windowproxy-object/test-window-proxy-scrollbars-manual.html
imported/wpt/html/browsers/the-window-object/the-windowproxy-object/test-window-proxy-statusbar-manual.html
imported/wpt/html/browsers/the-window-object/the-windowproxy-object/test-window-proxy-toolbar-manual.html

These are spec'd to depend on the actual state of the UI, but always initially
return true as implemented in Chromium. Even if they failed, the coverage seems
nice to have.

imported/wpt/html/semantics/embedded-content/media-elements/loading-the-media-resource/stable-state-dialogs-manual.html

I wrote this one. It really is manual, but the alert and other dialogs are
automatically dismissed when running as layout tests.

imported/wpt/html/webappapis/system-state-and-capabilities/the-navigator-object/get-navigatorlanguage-manual.html

This has "Precondition: The user agent's preferred language is set as English
(en)" and fails. Since it runs fast, just letting it fail rather than excluding
for import seems OK.

[Dirk Pranke, 2016-10-31 16:55:59]

ok.

[foolip, 2016-10-31 16:58:16]

The CQ bit was checked by foolip@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-10-31 16:58:31]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[foolip, 2016-10-31 16:58:43]

On 2016/10/31 16:15:26, qyearsley wrote:
> On 2016/10/31 at 15:33:33, foolip wrote:
> > qyearsley@, this will mean that some new manual tests will trickle in with
> imports. I don't know if it's worth doing anything about this, treating them
> like other timeouts is probably fine, but adding them to skipped imports would
> also work. I think that over time we want to either automate or remove these
> tests, so it's not a bad thing that they are noticed as long as it's not a
> burden.
> 
> Yep, I think that makes sense, LGTM

I forgot to update deps_updater_unittest.py, can you check that as well before I
land this?

[qyearsley, 2016-10-31 17:03:19]

On 2016/10/31 at 16:58:43, foolip wrote:
> On 2016/10/31 16:15:26, qyearsley wrote:
> > On 2016/10/31 at 15:33:33, foolip wrote:
> > > qyearsley@, this will mean that some new manual tests will trickle in with
> > imports. I don't know if it's worth doing anything about this, treating them
> > like other timeouts is probably fine, but adding them to skipped imports
would
> > also work. I think that over time we want to either automate or remove these
> > tests, so it's not a bad thing that they are noticed as long as it's not a
> > burden.
> > 
> > Yep, I think that makes sense, LGTM
> 
> I forgot to update deps_updater_unittest.py, can you check that as well before
I land this?

Checked -- deleting those tests LGTM since the corresponding function that they
test is now deleted.

[commit-bot: I haz the power, 2016-10-31 17:49:20]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-10-31 17:49:21]

Dry run: This issue passed the CQ dry run.

[foolip, 2016-10-31 20:18:27]

The CQ bit was checked by foolip@chromium.org

[foolip, 2016-10-31 20:18:28]

The patchset sent to the CQ was uploaded after l-g-t-m from rbyers@chromium.org
Link to the patchset: https://codereview.chromium.org/2463743003/#ps20001
(title: "update deps_updater_unittest.py")

[commit-bot: I haz the power, 2016-10-31 20:18:53]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-10-31 20:26:22]

Description was changed from

==========
Import web-platform-tests manual tests by default

A few manual tests already pass or fail in a useful way, like
stable-state-dialogs-manual.html. More importantly, it is difficult to
automate manual tests using wpt_automation if one must first add the
-automation.js script to cause the test to be imported.

The Fullscreen tests are imported and skipped in TestExpectations so
that they can be automated in a coming CL. These tests will all use the
same automation script, so the current import rules wouldn't work.

The wpt import is done with the same revision as the last import:
https://codereview.chromium.org/2448433002

Import wpt@bd99724e428dae78082983eab4675480c43f6234

Using update-w3c-deps in Chromium 55a661e4ed87e54b702de11e69ecc3babb2a96d0.

BUG=660846
==========

to

==========
Import web-platform-tests manual tests by default

A few manual tests already pass or fail in a useful way, like
stable-state-dialogs-manual.html. More importantly, it is difficult to
automate manual tests using wpt_automation if one must first add the
-automation.js script to cause the test to be imported.

The Fullscreen tests are imported and skipped in TestExpectations so
that they can be automated in a coming CL. These tests will all use the
same automation script, so the current import rules wouldn't work.

The wpt import is done with the same revision as the last import:
https://codereview.chromium.org/2448433002

Import wpt@bd99724e428dae78082983eab4675480c43f6234

Using update-w3c-deps in Chromium 55a661e4ed87e54b702de11e69ecc3babb2a96d0.

BUG=660846
==========

[commit-bot: I haz the power, 2016-10-31 20:26:23]

Committed patchset #2 (id:20001)

[commit-bot: I haz the power, 2016-10-31 20:29:43]

Description was changed from

==========
Import web-platform-tests manual tests by default

A few manual tests already pass or fail in a useful way, like
stable-state-dialogs-manual.html. More importantly, it is difficult to
automate manual tests using wpt_automation if one must first add the
-automation.js script to cause the test to be imported.

The Fullscreen tests are imported and skipped in TestExpectations so
that they can be automated in a coming CL. These tests will all use the
same automation script, so the current import rules wouldn't work.

The wpt import is done with the same revision as the last import:
https://codereview.chromium.org/2448433002

Import wpt@bd99724e428dae78082983eab4675480c43f6234

Using update-w3c-deps in Chromium 55a661e4ed87e54b702de11e69ecc3babb2a96d0.

BUG=660846
==========

to

==========
Import web-platform-tests manual tests by default

A few manual tests already pass or fail in a useful way, like
stable-state-dialogs-manual.html. More importantly, it is difficult to
automate manual tests using wpt_automation if one must first add the
-automation.js script to cause the test to be imported.

The Fullscreen tests are imported and skipped in TestExpectations so
that they can be automated in a coming CL. These tests will all use the
same automation script, so the current import rules wouldn't work.

The wpt import is done with the same revision as the last import:
https://codereview.chromium.org/2448433002

Import wpt@bd99724e428dae78082983eab4675480c43f6234

Using update-w3c-deps in Chromium 55a661e4ed87e54b702de11e69ecc3babb2a96d0.

BUG=660846

Committed: https://crrev.com/96df472c108ec278c919bf704ad5c0eaa2621d6f
Cr-Commit-Position: refs/heads/master@{#428786}
==========

[commit-bot: I haz the power, 2016-10-31 20:29:44]

Patchset 2 (id:??) landed as
https://crrev.com/96df472c108ec278c919bf704ad5c0eaa2621d6f
Cr-Commit-Position: refs/heads/master@{#428786}