Import wpt@1b61dad2be6c5100beb565e91c58b8c1084b3c7d

Import wpt@1b61dad2be6c5100beb565e91c58b8c1084b3c7d

Whitelist pointer event manual tests in the import process

Using update-w3c-deps in Chromium c517903e4714fac5e85e9e38fb7316d5df36cc46.

BUG=612924
Committed: https://crrev.com/62794230cc5e644b2fd3a4bde072225724283270
Cr-Commit-Position: refs/heads/master@{#397754}

[Navid Zolghadr, 2016-06-02 20:27:32]

Description was changed from

==========
Import wpt@d3451d24e3f7a9c2c9d3e39c9efeea6e768a70c5

Whitelist pointer event manual tests in the import process

Using update-w3c-deps in Chromium 2139f567e16c062168c9bf9e762c30b59ce7b805.


BUG=612924
==========

to

==========
Import wpt@1b61dad2be6c5100beb565e91c58b8c1084b3c7d

Whitelist pointer event manual tests in the import process

Using update-w3c-deps in Chromium c517903e4714fac5e85e9e38fb7316d5df36cc46.


BUG=612924
==========

[Navid Zolghadr, 2016-06-02 20:29:38]

nzolghadr@chromium.org changed reviewers:
+ dpranke@chromium.org, dtapuska@chromium.org

[Navid Zolghadr, 2016-06-02 20:29:39]

Dirk, I wonder if you are okay with this change. Basically this change
whitelists the manual tests in wpt/pointerevents folder and in the later changes
I'm going to add some scripts to inject the input to these tests.

[dtapuska, 2016-06-02 20:34:30]

On 2016/06/02 20:29:39, Navid Zolghadr wrote:
> Dirk, I wonder if you are okay with this change. Basically this change
> whitelists the manual tests in wpt/pointerevents folder and in the later
changes
> I'm going to add some scripts to inject the input to these tests.

lgtm % this passing on the bots :-)

[Dirk Pranke, 2016-06-02 20:57:24]

"manual tests" means that these end up being pixel tests that need actual
baselines, right? If so, is there some other way to write these tests such that
they don't require baselines?

[Navid Zolghadr, 2016-06-02 21:13:10]

On 2016/06/02 20:57:24, Dirk Pranke wrote:
> "manual tests" means that these end up being pixel tests that need actual
> baselines, right? If so, is there some other way to write these tests such
that
> they don't require baselines?

I don't quite understand what you mean here. What is a pixel test or a baseline?
Do you mean like an -expected output file or something?
Not all these tests need an output file but some of them do. However, there are
already some non-manual tests as well that do have that output file beside them.
Right?

[Dirk Pranke, 2016-06-02 21:23:37]

On 2016/06/02 21:13:10, Navid Zolghadr wrote:
> On 2016/06/02 20:57:24, Dirk Pranke wrote:
> > "manual tests" means that these end up being pixel tests that need actual
> > baselines, right? If so, is there some other way to write these tests such
> that
> > they don't require baselines?
> 
> I don't quite understand what you mean here. What is a pixel test or a
baseline?
> Do you mean like an -expected output file or something?
> Not all these tests need an output file but some of them do. However, there
are
> already some non-manual tests as well that do have that output file beside
them.
> Right?

My understanding is that what the W3C calls a "manual test" is a test where
someone
has to actually look at the web page to determine if it ran correctly. In Blink,
for
non-w3c tests, we do that by capturing the contents of the page as a PNG and
comparing pixels against the expected baselines (-expected.png), hence "pixel
test".

We have thus far not imported any manual tests from the w3c, because we (I) do
not
want us to have to generate baselines and keep them up to date. We only import
testharness-based tests (where you can programmatically determine if a test
passes or
fails) or reference tests, where you can compare the output of one page against
another.

The only non-manual tests that have output next to them are ones that currently
fail,
are expected to do so, and have bugs filed against them (I believe), and these
are only
acceptable because we want to track the coverage we are getting to see if things
start
to fail worse.

Does that help?

[Navid Zolghadr, 2016-06-02 21:31:26]

On 2016/06/02 21:23:37, Dirk Pranke wrote:
> On 2016/06/02 21:13:10, Navid Zolghadr wrote:
> > On 2016/06/02 20:57:24, Dirk Pranke wrote:
> > > "manual tests" means that these end up being pixel tests that need actual
> > > baselines, right? If so, is there some other way to write these tests such
> > that
> > > they don't require baselines?
> > 
> > I don't quite understand what you mean here. What is a pixel test or a
> baseline?
> > Do you mean like an -expected output file or something?
> > Not all these tests need an output file but some of them do. However, there
> are
> > already some non-manual tests as well that do have that output file beside
> them.
> > Right?
> 
> My understanding is that what the W3C calls a "manual test" is a test where
> someone
> has to actually look at the web page to determine if it ran correctly. In
Blink,
> for
> non-w3c tests, we do that by capturing the contents of the page as a PNG and
> comparing pixels against the expected baselines (-expected.png), hence "pixel
> test".
> 
> We have thus far not imported any manual tests from the w3c, because we (I) do
> not
> want us to have to generate baselines and keep them up to date. We only import
> testharness-based tests (where you can programmatically determine if a test
> passes or
> fails) or reference tests, where you can compare the output of one page
against
> another.
> 
> The only non-manual tests that have output next to them are ones that
currently
> fail,
> are expected to do so, and have bugs filed against them (I believe), and these
> are only
> acceptable because we want to track the coverage we are getting to see if
things
> start
> to fail worse.
> 
> Does that help?

I see what you were saying now. That is correct in general about the manual
tests. But non of the pointer event manual tests need a pixel verification. The
manual tests in pointer events are only manual because they need the user input
not user verification. For example it asks user to touch somewhere or click with
the mouse or something like that and still testharness.js will verify whether
the test has passed not. I'm going to add you to all the future reviews that I'm
automating these tests and you can double check if there is anything there that
you might not like. Okay?

[Dirk Pranke, 2016-06-02 21:34:49]

On 2016/06/02 21:31:26, Navid Zolghadr wrote:
> On 2016/06/02 21:23:37, Dirk Pranke wrote:
> > On 2016/06/02 21:13:10, Navid Zolghadr wrote:
> > > On 2016/06/02 20:57:24, Dirk Pranke wrote:
> > > > "manual tests" means that these end up being pixel tests that need
actual
> > > > baselines, right? If so, is there some other way to write these tests
such
> > > that
> > > > they don't require baselines?
> > > 
> > > I don't quite understand what you mean here. What is a pixel test or a
> > baseline?
> > > Do you mean like an -expected output file or something?
> > > Not all these tests need an output file but some of them do. However,
there
> > are
> > > already some non-manual tests as well that do have that output file beside
> > them.
> > > Right?
> > 
> > My understanding is that what the W3C calls a "manual test" is a test where
> > someone
> > has to actually look at the web page to determine if it ran correctly. In
> Blink,
> > for
> > non-w3c tests, we do that by capturing the contents of the page as a PNG and
> > comparing pixels against the expected baselines (-expected.png), hence
"pixel
> > test".
> > 
> > We have thus far not imported any manual tests from the w3c, because we (I)
do
> > not
> > want us to have to generate baselines and keep them up to date. We only
import
> > testharness-based tests (where you can programmatically determine if a test
> > passes or
> > fails) or reference tests, where you can compare the output of one page
> against
> > another.
> > 
> > The only non-manual tests that have output next to them are ones that
> currently
> > fail,
> > are expected to do so, and have bugs filed against them (I believe), and
these
> > are only
> > acceptable because we want to track the coverage we are getting to see if
> things
> > start
> > to fail worse.
> > 
> > Does that help?
> 
> I see what you were saying now. That is correct in general about the manual
> tests. But non of the pointer event manual tests need a pixel verification.
The
> manual tests in pointer events are only manual because they need the user
input
> not user verification. For example it asks user to touch somewhere or click
with
> the mouse or something like that and still testharness.js will verify whether
> the test has passed not. I'm going to add you to all the future reviews that
I'm
> automating these tests and you can double check if there is anything there
that
> you might not like. Okay?

Okay, if things will actually be using testharness.js and not require manual
review,
that seems fine. And, the tests are Skipped now so we don't need baselines.

lgtm . You don't need to cc' me on following CLs unless you really want to.

[Navid Zolghadr, 2016-06-03 16:53:53]

The CQ bit was checked by nzolghadr@chromium.org

[Navid Zolghadr, 2016-06-03 16:53:54]

The patchset sent to the CQ was uploaded after l-g-t-m from
dpranke@chromium.org, dtapuska@chromium.org
Link to the patchset: https://codereview.chromium.org/2020083002/#ps60001
(title: "Fixing tests")

[commit-bot: I haz the power, 2016-06-03 16:54:17]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/2020083002/60001

[commit-bot: I haz the power, 2016-06-03 18:05:00]

Description was changed from

==========
Import wpt@1b61dad2be6c5100beb565e91c58b8c1084b3c7d

Whitelist pointer event manual tests in the import process

Using update-w3c-deps in Chromium c517903e4714fac5e85e9e38fb7316d5df36cc46.


BUG=612924
==========

to

==========
Import wpt@1b61dad2be6c5100beb565e91c58b8c1084b3c7d

Whitelist pointer event manual tests in the import process

Using update-w3c-deps in Chromium c517903e4714fac5e85e9e38fb7316d5df36cc46.


BUG=612924
==========

[commit-bot: I haz the power, 2016-06-03 18:05:02]

Committed patchset #4 (id:60001)

[commit-bot: I haz the power, 2016-06-03 18:07:10]

Description was changed from

==========
Import wpt@1b61dad2be6c5100beb565e91c58b8c1084b3c7d

Whitelist pointer event manual tests in the import process

Using update-w3c-deps in Chromium c517903e4714fac5e85e9e38fb7316d5df36cc46.


BUG=612924
==========

to

==========
Import wpt@1b61dad2be6c5100beb565e91c58b8c1084b3c7d

Whitelist pointer event manual tests in the import process

Using update-w3c-deps in Chromium c517903e4714fac5e85e9e38fb7316d5df36cc46.

BUG=612924

Committed: https://crrev.com/62794230cc5e644b2fd3a4bde072225724283270
Cr-Commit-Position: refs/heads/master@{#397754}
==========

[commit-bot: I haz the power, 2016-06-03 18:07:11]

Patchset 4 (id:??) landed as
https://crrev.com/62794230cc5e644b2fd3a4bde072225724283270
Cr-Commit-Position: refs/heads/master@{#397754}