service worker: Remove some LayoutTests that are redundant with WPT tests.

service worker: Remove some LayoutTests that are redundant with WPT tests.

This removes some LayoutTests for which there is a version in both
LayoutTests and WPT and Chrome is passing both. Also, the differences
between the versions are trivial, or WPT has more enhanced tests.

Some history: the current WPT tests were largely adapted from our
LayoutTests.  We could not contribute to WPT directly because Blink
infra was not ready (especially wptserve didn't work). And now
everything is working and we have both WPT tests and LayoutTests but
they are slightly different.  This is a step toward unifying them.

This shows the diff between the tests being removed:
http://pastebin.com/QabWh41g

BUG=688116
Review-Url: https://codereview.chromium.org/2695803007
Cr-Commit-Position: refs/heads/master@{#450893}
Committed: https://chromium.googlesource.com/chromium/src/+/992433e26b7b5abd3ff5d584501e6169bbcab16c

[falken, 2017-02-15 07:56:07]

falken@chromium.org changed reviewers:
+ mek@chromium.org, shimazu@chromium.org

[falken, 2017-02-15 07:56:08]

shimazu: ptal
mek: FYI

[falken, 2017-02-15 08:56:01]

The CQ bit was checked by falken@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2017-02-15 08:56:14]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-02-15 09:56:51]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-02-15 09:56:52]

Dry run: This issue passed the CQ dry run.

[falken, 2017-02-15 16:44:10]

On 2017/02/15 07:56:08, falken wrote:
> shimazu: ptal
> mek: FYI

on second thought, mek: ptal

[Marijn Kruisselbrink, 2017-02-15 18:48:46]

I think this lgtm

One comment though, the diff doesn't tell the entire story, there are also
changes in the shared utility method in resources/test-helpers[.sub].js. For
example the upstream version doesn't remove iframes at the end of tests, while
our version does. Probably doesn't matter much, as it shouldn't effect the test
results. (https://github.com/w3c/web-platform-tests/pull/4610/files includes
some changes to the upstream with_iframe that makes it closer to ours, although
at least currently there auto remove is opt-in rather than opt-out).

[falken, 2017-02-16 01:41:31]

Ah thanks that's a good point. Also the worker files js must be checked.

[falken, 2017-02-16 02:05:44]

On 2017/02/16 01:41:31, falken wrote:
> Ah thanks that's a good point. Also the worker files js must be checked.

I've discovered some non-trivial differences. Hold on, will update this CL a
bit.

[shimazu, 2017-02-16 02:11:59]

On 2017/02/16 02:05:44, falken wrote:
> On 2017/02/16 01:41:31, falken wrote:
> > Ah thanks that's a good point. Also the worker files js must be checked.
> 
> I've discovered some non-trivial differences. Hold on, will update this CL a
> bit.

I'd like to do this (mostly) automatically if possible. Do you think it's
possible to do this by tracking the history of repo?
Most of tests exist in wpt has been imported by one commit:
https://github.com/w3c/web-platform-tests/commit/b2e3ab765e26436850208dbdcf74...
so I think it might be possible if we can trace the mozilla history before the
import commit.

[falken, 2017-02-16 02:22:15]

On 2017/02/16 02:11:59, shimazu wrote:
> On 2017/02/16 02:05:44, falken wrote:
> > On 2017/02/16 01:41:31, falken wrote:
> > > Ah thanks that's a good point. Also the worker files js must be checked.
> > 
> > I've discovered some non-trivial differences. Hold on, will update this CL a
> > bit.
> 
> I'd like to do this (mostly) automatically if possible. Do you think it's
> possible to do this by tracking the history of repo?
> Most of tests exist in wpt has been imported by one commit:
>
https://github.com/w3c/web-platform-tests/commit/b2e3ab765e26436850208dbdcf74...
> so I think it might be possible if we can trace the mozilla history before the
> import commit.

I would love if there's an automatic solution. Can you expand on your idea?

Some challenges I'm seeing are:
- how to tell automatically what changes between a php -> py change.
- There are also files where both chromium and WPT made modifications after the
import like fetch-event-network-error-worker.js. It gained a new test in WPT and
chromium tweaked some tests to use a new file called progressive.php.

[falken, 2017-02-16 04:19:06]

Description was changed from

==========
service worker: Remove some LayoutTests that are redundant with WPT tests.

This removes some LayoutTests for which there is a version in both
LayoutTests and WPT and Chrome is passing both. Also, the differences
between the versions are trivial, or WPT has more enhanced tests.

Some history: the current WPT tests were largely adapted from our
LayoutTests.  We could not contribute to WPT directly because Blink
infra was not ready (especially wptserve didn't work). And now
everything is working and we have both WPT tests and LayoutTests but
they are slightly different.  This is a step toward unifying them.

This shows the diff between the tests being removed:
http://pastebin.com/NecmigC4

BUG=688116
==========

to

==========
service worker: Remove some LayoutTests that are redundant with WPT tests.

This removes some LayoutTests for which there is a version in both
LayoutTests and WPT and Chrome is passing both. Also, the differences
between the versions are trivial, or WPT has more enhanced tests.

Some history: the current WPT tests were largely adapted from our
LayoutTests.  We could not contribute to WPT directly because Blink
infra was not ready (especially wptserve didn't work). And now
everything is working and we have both WPT tests and LayoutTests but
they are slightly different.  This is a step toward unifying them.

This shows the diff between the tests being removed:
http://pastebin.com/QabWh41g

BUG=688116
==========

[falken, 2017-02-16 04:22:48]

OK here is a more conservative removal. I updated the pastebin link to show the
diff of subresources (iframe html and worker js) of the tests files as well
(excluding test-helpers.js which I acknowledge is different).

[shimazu, 2017-02-16 04:33:39]

This lgtm.

On 2017/02/16 02:22:15, falken wrote:
> On 2017/02/16 02:11:59, shimazu wrote:
> > On 2017/02/16 02:05:44, falken wrote:
> > > On 2017/02/16 01:41:31, falken wrote:
> > > > Ah thanks that's a good point. Also the worker files js must be checked.
> > > 
> > > I've discovered some non-trivial differences. Hold on, will update this CL
a
> > > bit.
> > 
> > I'd like to do this (mostly) automatically if possible. Do you think it's
> > possible to do this by tracking the history of repo?
> > Most of tests exist in wpt has been imported by one commit:
> >
>
https://github.com/w3c/web-platform-tests/commit/b2e3ab765e26436850208dbdcf74...
> > so I think it might be possible if we can trace the mozilla history before
the
> > import commit.
> 
> I would love if there's an automatic solution. Can you expand on your idea?
> 
> Some challenges I'm seeing are:
> - how to tell automatically what changes between a php -> py change.

Yes, it's one of most difficult part.
Currently we have 22 python scripts vs 32 php scripts, but at my first glance
many of them seems trivial scripts, so that's might be relatively easy to review
manually comparing to HTMLs.

Python:
https://cs.chromium.org/search/?q=file:.py$+file:%5Esrc/third_party/WebKit/La...
PHP:
https://cs.chromium.org/search/?q=file:.php$+file:%5Esrc/third_party/WebKit/L...

> - There are also files where both chromium and WPT made modifications after
the
> import like fetch-event-network-error-worker.js. It gained a new test in WPT
and
> chromium tweaked some tests to use a new file called progressive.php.

I'm still not sure how many tests are exactly the same. 
If we can assume lots of tests are unchanged including the iframe and worker,
automatic removal of only completely unchanged tests might be helpful.

Then I've now tried to listing tests having just two changes (possibly "import
from blink" and "export to wpt") in firefox repository.
These tests would be potentially the same with our layout tests. It seems
relatively large than I expected.

$ for f in $(ls .); do [[ $(hg log -f "$f" --template x | wc -c) == "2" ]] &&
echo "$f"; done

activate-event-after-install-state-change.https.html
activation-after-registration.https.html
active.https.html
claim-not-using-registration.https.html
claim-using-registration.https.html
client-navigate.https.html
fetch-event-after-navigation-within-page.https.html
fetch-event-network-error.https.html
fetch-request-no-freshness-headers.https.html
install-event-type.https.html
installing.https.html
interfaces.https.html
multiple-update.https.html
postmessage.https.html
ready.https.html
register-default-scope.https.html
registration-end-to-end.https.html
registration-events.https.html
rejections.https.html
service-worker-csp-connect.https.html
service-worker-csp-default.https.html
service-worker-csp-script.https.html
shared-worker-controlled.https.html
skip-waiting-without-client.https.html
state.https.html
unregister.https.html

Files under resources/ and ServiceWorkerGlobalScope/ is excluded for this, but
they can be also scanned by the same way.

[falken, 2017-02-16 05:52:12]

On 2017/02/16 04:33:39, shimazu wrote:
> This lgtm.
> 
> On 2017/02/16 02:22:15, falken wrote:
> > On 2017/02/16 02:11:59, shimazu wrote:
> > > On 2017/02/16 02:05:44, falken wrote:
> > > > On 2017/02/16 01:41:31, falken wrote:
> > > > > Ah thanks that's a good point. Also the worker files js must be
checked.
> > > > 
> > > > I've discovered some non-trivial differences. Hold on, will update this
CL
> a
> > > > bit.
> > > 
> > > I'd like to do this (mostly) automatically if possible. Do you think it's
> > > possible to do this by tracking the history of repo?
> > > Most of tests exist in wpt has been imported by one commit:
> > >
> >
>
https://github.com/w3c/web-platform-tests/commit/b2e3ab765e26436850208dbdcf74...
> > > so I think it might be possible if we can trace the mozilla history before
> the
> > > import commit.
> > 
> > I would love if there's an automatic solution. Can you expand on your idea?
> > 
> > Some challenges I'm seeing are:
> > - how to tell automatically what changes between a php -> py change.
> 
> Yes, it's one of most difficult part.
> Currently we have 22 python scripts vs 32 php scripts, but at my first glance
> many of them seems trivial scripts, so that's might be relatively easy to
review
> manually comparing to HTMLs.
> 
> Python:
>
https://cs.chromium.org/search/?q=file:.py$+file:%5Esrc/third_party/WebKit/La...
> PHP:
>
https://cs.chromium.org/search/?q=file:.php$+file:%5Esrc/third_party/WebKit/L...
> 
> > - There are also files where both chromium and WPT made modifications after
> the
> > import like fetch-event-network-error-worker.js. It gained a new test in WPT
> and
> > chromium tweaked some tests to use a new file called progressive.php.
> 
> I'm still not sure how many tests are exactly the same. 
> If we can assume lots of tests are unchanged including the iframe and worker,
> automatic removal of only completely unchanged tests might be helpful.
> 
> Then I've now tried to listing tests having just two changes (possibly "import
> from blink" and "export to wpt") in firefox repository.
> These tests would be potentially the same with our layout tests. It seems
> relatively large than I expected.
> 
> $ for f in $(ls .); do [[ $(hg log -f "$f" --template x | wc -c) == "2" ]] &&
> echo "$f"; done
> 
> activate-event-after-install-state-change.https.html
> activation-after-registration.https.html
> active.https.html
> claim-not-using-registration.https.html
> claim-using-registration.https.html
> client-navigate.https.html
> fetch-event-after-navigation-within-page.https.html
> fetch-event-network-error.https.html
> fetch-request-no-freshness-headers.https.html
> install-event-type.https.html
> installing.https.html
> interfaces.https.html
> multiple-update.https.html
> postmessage.https.html
> ready.https.html
> register-default-scope.https.html
> registration-end-to-end.https.html
> registration-events.https.html
> rejections.https.html
> service-worker-csp-connect.https.html
> service-worker-csp-default.https.html
> service-worker-csp-script.https.html
> shared-worker-controlled.https.html
> skip-waiting-without-client.https.html
> state.https.html
> unregister.https.html
> 
> Files under resources/ and ServiceWorkerGlobalScope/ is excluded for this, but
> they can be also scanned by the same way.

Thanks for investigating! At a glance the list looks somewhat similar what the
diffing is showing as similar tests. Let's keep proceeding and refining our
process.

[falken, 2017-02-16 05:52:47]

The CQ bit was checked by falken@chromium.org

[falken, 2017-02-16 05:52:47]

The patchset sent to the CQ was uploaded after l-g-t-m from mek@chromium.org
Link to the patchset: https://codereview.chromium.org/2695803007/#ps40001
(title: "changes")

[commit-bot: I haz the power, 2017-02-16 05:54:01]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-02-16 08:04:53]

CQ is committing da patch.

Bot data: {"patchset_id": 40001, "attempt_start_ts": 1487224367402020,
"parent_rev": "ebc7290a4f2d9e94b3f85cf57bf251971397cafd", "commit_rev":
"992433e26b7b5abd3ff5d584501e6169bbcab16c"}

[commit-bot: I haz the power, 2017-02-16 08:05:35]

Description was changed from

==========
service worker: Remove some LayoutTests that are redundant with WPT tests.

This removes some LayoutTests for which there is a version in both
LayoutTests and WPT and Chrome is passing both. Also, the differences
between the versions are trivial, or WPT has more enhanced tests.

Some history: the current WPT tests were largely adapted from our
LayoutTests.  We could not contribute to WPT directly because Blink
infra was not ready (especially wptserve didn't work). And now
everything is working and we have both WPT tests and LayoutTests but
they are slightly different.  This is a step toward unifying them.

This shows the diff between the tests being removed:
http://pastebin.com/QabWh41g

BUG=688116
==========

to

==========
service worker: Remove some LayoutTests that are redundant with WPT tests.

This removes some LayoutTests for which there is a version in both
LayoutTests and WPT and Chrome is passing both. Also, the differences
between the versions are trivial, or WPT has more enhanced tests.

Some history: the current WPT tests were largely adapted from our
LayoutTests.  We could not contribute to WPT directly because Blink
infra was not ready (especially wptserve didn't work). And now
everything is working and we have both WPT tests and LayoutTests but
they are slightly different.  This is a step toward unifying them.

This shows the diff between the tests being removed:
http://pastebin.com/QabWh41g

BUG=688116

Review-Url: https://codereview.chromium.org/2695803007
Cr-Commit-Position: refs/heads/master@{#450893}
Committed:
https://chromium.googlesource.com/chromium/src/+/992433e26b7b5abd3ff5d584501e...
==========

[commit-bot: I haz the power, 2017-02-16 08:05:39]

Committed patchset #3 (id:40001) as
https://chromium.googlesource.com/chromium/src/+/992433e26b7b5abd3ff5d584501e...