Add support for khronos gl-cts using its drawElements apis

Implements a windowless chrome gpu command buffer platform for
the gles2 conformance tests in the khronos GL-CTS test
suite using its new drawElements APIs.

Requires the khronos GL-CTS source to be available in
src/third_party/khronos_glcts.  This support targets the
Khronos 3.X branch of ogles/conform at rev 26950.

Define the build option internal_khronos_glcts_tests=1
to build the tests.

Initial support is for Chromium on ChromiumOS.  Other targets
may need to be ported for build support as well.

v2: Fix AUTHORS.  Change naming convention from khronos_conform
    to khronos_glcts. (piman, kbr)
v3: Ran git cl format on egl_native_windowless.cc so it matches
    the chromium style. (piman)
v4: Use AppendArg instead of AppendSwitch to hopefully
    circumvent a repeat of chromium:408251. (kbr)
v5: Handle FilePath's correctly for Windows compatibility.
v6: Rebase; Remove -fno-exceptions from cflags_cc

BUG=chromium:412865
R=piman@chromium.org, kbr@chromium.org, darin, brettw, cpu
TEST=Build and run the khronos_glcts_test binary

Review URL: https://codereview.chromium.org/556333003

Signed-off-by: U. Artie Eoff <ullysses.a.eoff@intel.com>;

Committed: https://crrev.com/ffbb37ebec861db6e93471a5837727ea8d974f91
Cr-Commit-Position: refs/heads/master@{#297567}

[no sievers, 2014-09-10 18:46:49]

sievers@chromium.org changed reviewers:
+ kbr@chromium.org

[no sievers, 2014-09-10 18:47:47]

sievers@chromium.org changed reviewers:
+ skyostil@chromium.org

[U. Artie Eoff, 2014-09-10 18:47:51]

ullysses.a.eoff@intel.com changed reviewers:
+ chad.versace@intel.com, josh.triplett@intel.com
- kbr@chromium.org, skyostil@chromium.org

[U. Artie Eoff, 2014-09-10 18:49:13]

Hello, I'd like you to review my code.

[piman, 2014-09-10 20:12:49]

piman@chromium.org changed reviewers:
+ kbr@chromium.org, zmo@chromium.org

[piman, 2014-09-10 20:12:51]

+zmo/kbr

I would prefer if we didn't duplicate all the support infrastructure between the
2 versions of the conformance tests.

I would also be somewhat reluctant to add infrastructure unless we're committed
to run these tests internally

https://codereview.chromium.org/556333003/diff/1/AUTHORS
File AUTHORS (right):

https://codereview.chromium.org/556333003/diff/1/AUTHORS#newcode485
AUTHORS:485: Intel Corp. <*@intel.com>
We can't add this line, Intel has restricted the CLA to explicitly named people.

https://codereview.chromium.org/556333003/diff/1/build/common.gypi
File build/common.gypi (right):

https://codereview.chromium.org/556333003/diff/1/build/common.gypi#newcode1382
build/common.gypi:1382: 'internal_khronos_conform_tests%': 0,
Does this and internal_gles2_conform_tests need to be separate? I'd assume
whoever has access to one has access to the other one. It'd be simpler to only
have one of these.

https://codereview.chromium.org/556333003/diff/1/gpu/khronos_conform_support/...
File gpu/khronos_conform_support/DEPS (right):

https://codereview.chromium.org/556333003/diff/1/gpu/khronos_conform_support/...
gpu/khronos_conform_support/DEPS:2: "+third_party/khronos_conform",
I don't see this directory in either my local checkout nor any of the DEPS files
(either internal or external)

https://codereview.chromium.org/556333003/diff/1/gpu/khronos_conform_support/...
File gpu/khronos_conform_support/generate_khronos_conform_tests.py (right):

https://codereview.chromium.org/556333003/diff/1/gpu/khronos_conform_support/...
gpu/khronos_conform_support/generate_khronos_conform_tests.py:1: #!/usr/bin/env
python
This file looks very similar to 
gpu/gles2_conform_support/generate_gles2_conform_tests.py. Can we share / avoid
duplication?

[U. Artie Eoff, 2014-09-10 22:12:59]

I do agree that there might be a few similarities between the two versions. 
However, the new Khronos drawElements APIs and how it needs to be constructed,
integrated, compiled and executed are dramatically different.  And things will
get messy if we try to merge support for both into one.

Also, I strongly feel this new port should be done separately so that we can
eventually transition away from the older gles2_conform_support.  The "gles2" in
the name is also restricting and implicative.  We want to add es3+ test support
later and the naming conflicts with that.

Not sure you noticed, but this port does reuse the egl_native stuff from
gles2_conform_support.  That code could be moved out into a shared location if
keeping both versions for a long time is important.

https://codereview.chromium.org/556333003/diff/1/AUTHORS
File AUTHORS (right):

https://codereview.chromium.org/556333003/diff/1/AUTHORS#newcode485
AUTHORS:485: Intel Corp. <*@intel.com>
On 2014/09/10 20:12:50, piman (Very slow to review) wrote:
> We can't add this line, Intel has restricted the CLA to explicitly named
people.

Acknowledged.

https://codereview.chromium.org/556333003/diff/1/build/common.gypi
File build/common.gypi (right):

https://codereview.chromium.org/556333003/diff/1/build/common.gypi#newcode1382
build/common.gypi:1382: 'internal_khronos_conform_tests%': 0,
On 2014/09/10 20:12:50, piman (Very slow to review) wrote:
> Does this and internal_gles2_conform_tests need to be separate? I'd assume
> whoever has access to one has access to the other one. It'd be simpler to only
> have one of these.

I think the "gles2" in the name "internal_gles2_conform_tests" is too narrow and
implies only es2 tests.  We want to add es3 tests and future versions later.  We
could merge these, I suppose, into one form eventually; one which doesn't
constrict meaning so much (like internal_khronos_conform here).  But I'd rather
be able to compile one and not the other.  I think the idea behind this is to
incrementally ramp up with the new drawElements support and phase out the old. 
This requires having both for the transition phase.

https://codereview.chromium.org/556333003/diff/1/gpu/khronos_conform_support/...
File gpu/khronos_conform_support/DEPS (right):

https://codereview.chromium.org/556333003/diff/1/gpu/khronos_conform_support/...
gpu/khronos_conform_support/DEPS:2: "+third_party/khronos_conform",
On 2014/09/10 20:12:50, piman (Very slow to review) wrote:
> I don't see this directory in either my local checkout nor any of the DEPS
files
> (either internal or external)

I guess I'm not certain what this file really does.  But essentially you'll have
to clone your licensed version of the khronos conformance sourcecode into that
path on your end... using whatever process you've put in place internally to get
them. Right?

https://codereview.chromium.org/556333003/diff/1/gpu/khronos_conform_support/...
File gpu/khronos_conform_support/generate_khronos_conform_tests.py (right):

https://codereview.chromium.org/556333003/diff/1/gpu/khronos_conform_support/...
gpu/khronos_conform_support/generate_khronos_conform_tests.py:1: #!/usr/bin/env
python
On 2014/09/10 20:12:50, piman (Very slow to review) wrote:
> This file looks very similar to 
> gpu/gles2_conform_support/generate_gles2_conform_tests.py. Can we share /
avoid
> duplication?

There are quite a few differences between the two test generators.  The new
drawElements framework uses a different test naming scheme and this accounts for
that.  Also, this generator recurses through the .run files and only produces
tests for the .test files (the algorithm is similar to what can be found in the
drawElements framework when it traverses the .run files to produce test names). 
The generate_gles2_conform_tests.py does not recurse.

I suppose the generate_gles2_conform_tests.py could be refactored to support
both schemes... but I think it would likely get a little messy.

[Ken Russell (switch to Gerrit), 2014-09-11 00:03:26]

What portions of the GLES3 test suite are actually run in this new target? Since
Chromium's command buffer and GL bindings still target GLES2, are you running a
subset of the new test suite? Does the test suite run within the command buffer
framework?

CC'ing the ANGLE team, who are particularly interested in the GLES2 conformance
test suite and who would likely be interested in this upgrade.

https://codereview.chromium.org/556333003/diff/1/build/common.gypi
File build/common.gypi (right):

https://codereview.chromium.org/556333003/diff/1/build/common.gypi#newcode1382
build/common.gypi:1382: 'internal_khronos_conform_tests%': 0,
On 2014/09/10 22:12:59, uartie wrote:
> On 2014/09/10 20:12:50, piman (Very slow to review) wrote:
> > Does this and internal_gles2_conform_tests need to be separate? I'd assume
> > whoever has access to one has access to the other one. It'd be simpler to
only
> > have one of these.
> 
> I think the "gles2" in the name "internal_gles2_conform_tests" is too narrow
and
> implies only es2 tests.  We want to add es3 tests and future versions later. 
We
> could merge these, I suppose, into one form eventually; one which doesn't
> constrict meaning so much (like internal_khronos_conform here).  But I'd
rather
> be able to compile one and not the other.  I think the idea behind this is to
> incrementally ramp up with the new drawElements support and phase out the old.

> This requires having both for the transition phase.

Understood that the duplication may be necessary for the time being due to major
differences between the test harnesses. However the naming convention here is
confusing. Since you're explicitly targeting the ES 3.x tests can you please
replace 'khronos' with 'gles3' throughout this CL?

https://codereview.chromium.org/556333003/diff/1/gpu/khronos_conform_support/...
File gpu/khronos_conform_support/DEPS (right):

https://codereview.chromium.org/556333003/diff/1/gpu/khronos_conform_support/...
gpu/khronos_conform_support/DEPS:2: "+third_party/khronos_conform",
On 2014/09/10 22:12:59, uartie wrote:
> On 2014/09/10 20:12:50, piman (Very slow to review) wrote:
> > I don't see this directory in either my local checkout nor any of the DEPS
> files
> > (either internal or external)
> 
> I guess I'm not certain what this file really does.  But essentially you'll
have
> to clone your licensed version of the khronos conformance sourcecode into that
> path on your end... using whatever process you've put in place internally to
get
> them. Right?

It sounds like someone will have to clone Khronos' GLES3 conformance tests into
Chromium's src-internal repository. Frankly, it's quite a pain that the GLES3
conformance tests are not open source. Can you please work within Khronos and
try to get these tests open-sourced? I think the value from the tests comes from
certification, not simply access to the tests' code.

https://codereview.chromium.org/556333003/diff/1/gpu/khronos_conform_support/...
File gpu/khronos_conform_support/generate_khronos_conform_tests.py (right):

https://codereview.chromium.org/556333003/diff/1/gpu/khronos_conform_support/...
gpu/khronos_conform_support/generate_khronos_conform_tests.py:1: #!/usr/bin/env
python
On 2014/09/10 22:12:59, uartie wrote:
> On 2014/09/10 20:12:50, piman (Very slow to review) wrote:
> > This file looks very similar to 
> > gpu/gles2_conform_support/generate_gles2_conform_tests.py. Can we share /
> avoid
> > duplication?
> 
> There are quite a few differences between the two test generators.  The new
> drawElements framework uses a different test naming scheme and this accounts
for
> that.  Also, this generator recurses through the .run files and only produces
> tests for the .test files (the algorithm is similar to what can be found in
the
> drawElements framework when it traverses the .run files to produce test
names). 
> The generate_gles2_conform_tests.py does not recurse.
> 
> I suppose the generate_gles2_conform_tests.py could be refactored to support
> both schemes... but I think it would likely get a little messy.

As unfortunate as code duplication in this area is, I can understand the need to
do it because Khronos' test harness has evolved dramatically between the GLES2
and GLES3 conformance suites.

Can this new test suite subsume the GLES2 conformance tests? Are all of the
areas covered by the GLES2 conformance tests also covered by these new tests?

[U. Artie Eoff, 2014-09-11 01:09:20]

No portions of the GLES3 test cases are run in this new target because the
command buffer doesn't have any ES3 bindings, as you pointed out... at least not
yet.  Only the GLES2 test are run in this target right now (same as
gles2_conform_support).  The primary goal of this patch is to port the ES2 tests
to the *new* khronos framework/APIs so we can *include* the ES3+ tests, later
when ES3+ is supported in the command buffer.

https://codereview.chromium.org/556333003/diff/1/build/common.gypi
File build/common.gypi (right):

https://codereview.chromium.org/556333003/diff/1/build/common.gypi#newcode1382
build/common.gypi:1382: 'internal_khronos_conform_tests%': 0,
On 2014/09/11 00:03:26, Ken Russell wrote:
> On 2014/09/10 22:12:59, uartie wrote:
> > On 2014/09/10 20:12:50, piman (Very slow to review) wrote:
> > > Does this and internal_gles2_conform_tests need to be separate? I'd assume
> > > whoever has access to one has access to the other one. It'd be simpler to
> only
> > > have one of these.
> > 
> > I think the "gles2" in the name "internal_gles2_conform_tests" is too narrow
> and
> > implies only es2 tests.  We want to add es3 tests and future versions later.

> We
> > could merge these, I suppose, into one form eventually; one which doesn't
> > constrict meaning so much (like internal_khronos_conform here).  But I'd
> rather
> > be able to compile one and not the other.  I think the idea behind this is
to
> > incrementally ramp up with the new drawElements support and phase out the
old.
> 
> > This requires having both for the transition phase.
> 
> Understood that the duplication may be necessary for the time being due to
major
> differences between the test harnesses. However the naming convention here is
> confusing. Since you're explicitly targeting the ES 3.x tests can you please
> replace 'khronos' with 'gles3' throughout this CL?

Well... the idea is to target all the tests actually.  That is, to target ES2,
ES30, ES31, and any future ES versions (and who knows, maybe GL21 thru GL44 and
beyond).  All versions can be targeted in one [binary] instance using this new
API.  That is why I chose the more generic naming convention.  In fact, this
particular patch only targets the ES2 tests at the moment, the same as the
gles2_conform_support.  The main difference is it's using the new APIs so that
we can trivially add the other ESx tests later without being constrained by
naming conventions.

https://codereview.chromium.org/556333003/diff/1/gpu/khronos_conform_support/...
File gpu/khronos_conform_support/DEPS (right):

https://codereview.chromium.org/556333003/diff/1/gpu/khronos_conform_support/...
gpu/khronos_conform_support/DEPS:2: "+third_party/khronos_conform",
On 2014/09/11 00:03:26, Ken Russell wrote:
> On 2014/09/10 22:12:59, uartie wrote:
> > On 2014/09/10 20:12:50, piman (Very slow to review) wrote:
> > > I don't see this directory in either my local checkout nor any of the DEPS
> > files
> > > (either internal or external)
> > 
> > I guess I'm not certain what this file really does.  But essentially you'll
> have
> > to clone your licensed version of the khronos conformance sourcecode into
that
> > path on your end... using whatever process you've put in place internally to
> get
> > them. Right?
> 
> It sounds like someone will have to clone Khronos' GLES3 conformance tests
into
> Chromium's src-internal repository. Frankly, it's quite a pain that the GLES3
> conformance tests are not open source. Can you please work within Khronos and
> try to get these tests open-sourced? I think the value from the tests comes
from
> certification, not simply access to the tests' code.
That's opening up a can-o-worms ;).  A license to the Khronos conformance isn't
cheap.  I'm not trained in the art of lobbying.  But I'm sure between the "all
of us", someone could persuade them on this proposal.

https://codereview.chromium.org/556333003/diff/1/gpu/khronos_conform_support/...
File gpu/khronos_conform_support/generate_khronos_conform_tests.py (right):

https://codereview.chromium.org/556333003/diff/1/gpu/khronos_conform_support/...
gpu/khronos_conform_support/generate_khronos_conform_tests.py:1: #!/usr/bin/env
python
On 2014/09/11 00:03:26, Ken Russell wrote:
> On 2014/09/10 22:12:59, uartie wrote:
> > On 2014/09/10 20:12:50, piman (Very slow to review) wrote:
> > > This file looks very similar to 
> > > gpu/gles2_conform_support/generate_gles2_conform_tests.py. Can we share /
> > avoid
> > > duplication?
> > 
> > There are quite a few differences between the two test generators.  The new
> > drawElements framework uses a different test naming scheme and this accounts
> for
> > that.  Also, this generator recurses through the .run files and only
produces
> > tests for the .test files (the algorithm is similar to what can be found in
> the
> > drawElements framework when it traverses the .run files to produce test
> names). 
> > The generate_gles2_conform_tests.py does not recurse.
> > 
> > I suppose the generate_gles2_conform_tests.py could be refactored to support
> > both schemes... but I think it would likely get a little messy.
> 
> As unfortunate as code duplication in this area is, I can understand the need
to
> do it because Khronos' test harness has evolved dramatically between the GLES2
> and GLES3 conformance suites.
> 
> Can this new test suite subsume the GLES2 conformance tests? Are all of the
> areas covered by the GLES2 conformance tests also covered by these new tests?

Yes, this new suite *does* subsume the ES2 conformance tests.  In fact, the ES2
tests are the only tests this patch "subsumes" at the moment.  The pudding in
the cake is (using the newer framework/APIs) we can extend this integration
later to include the ES3+ tests, trivially.  So, yes, all of the areas covered
by the GLES2 conformance tests *are* covered here, and then some (i.e. presuming
there have been more es2 test additions and fixes since the "legacy" khronos
suite).

[Ken Russell (switch to Gerrit), 2014-09-11 01:25:17]

https://codereview.chromium.org/556333003/diff/1/build/common.gypi
File build/common.gypi (right):

https://codereview.chromium.org/556333003/diff/1/build/common.gypi#newcode1382
build/common.gypi:1382: 'internal_khronos_conform_tests%': 0,
On 2014/09/11 01:09:19, uartie wrote:
> On 2014/09/11 00:03:26, Ken Russell wrote:
> > On 2014/09/10 22:12:59, uartie wrote:
> > > On 2014/09/10 20:12:50, piman (Very slow to review) wrote:
> > > > Does this and internal_gles2_conform_tests need to be separate? I'd
assume
> > > > whoever has access to one has access to the other one. It'd be simpler
to
> > only
> > > > have one of these.
> > > 
> > > I think the "gles2" in the name "internal_gles2_conform_tests" is too
narrow
> > and
> > > implies only es2 tests.  We want to add es3 tests and future versions
later.
> 
> > We
> > > could merge these, I suppose, into one form eventually; one which doesn't
> > > constrict meaning so much (like internal_khronos_conform here).  But I'd
> > rather
> > > be able to compile one and not the other.  I think the idea behind this is
> to
> > > incrementally ramp up with the new drawElements support and phase out the
> old.
> > 
> > > This requires having both for the transition phase.
> > 
> > Understood that the duplication may be necessary for the time being due to
> major
> > differences between the test harnesses. However the naming convention here
is
> > confusing. Since you're explicitly targeting the ES 3.x tests can you please
> > replace 'khronos' with 'gles3' throughout this CL?
> 
> Well... the idea is to target all the tests actually.  That is, to target ES2,
> ES30, ES31, and any future ES versions (and who knows, maybe GL21 thru GL44
and
> beyond).  All versions can be targeted in one [binary] instance using this new
> API.  That is why I chose the more generic naming convention.  In fact, this
> particular patch only targets the ES2 tests at the moment, the same as the
> gles2_conform_support.  The main difference is it's using the new APIs so that
> we can trivially add the other ESx tests later without being constrained by
> naming conventions.

I still think a more targeted name is appropriate. If not gles3, then gles, or
khronos_gles, or something. If some future API comes around under Khronos'
umbrella that isn't tested in this particular harness, a naming conflict with
this would be unfortunate. What do you think?

https://codereview.chromium.org/556333003/diff/1/gpu/khronos_conform_support/...
File gpu/khronos_conform_support/DEPS (right):

https://codereview.chromium.org/556333003/diff/1/gpu/khronos_conform_support/...
gpu/khronos_conform_support/DEPS:2: "+third_party/khronos_conform",
On 2014/09/11 01:09:19, uartie wrote:
> On 2014/09/11 00:03:26, Ken Russell wrote:
> > On 2014/09/10 22:12:59, uartie wrote:
> > > On 2014/09/10 20:12:50, piman (Very slow to review) wrote:
> > > > I don't see this directory in either my local checkout nor any of the
DEPS
> > > files
> > > > (either internal or external)
> > > 
> > > I guess I'm not certain what this file really does.  But essentially
you'll
> > have
> > > to clone your licensed version of the khronos conformance sourcecode into
> that
> > > path on your end... using whatever process you've put in place internally
to
> > get
> > > them. Right?
> > 
> > It sounds like someone will have to clone Khronos' GLES3 conformance tests
> into
> > Chromium's src-internal repository. Frankly, it's quite a pain that the
GLES3
> > conformance tests are not open source. Can you please work within Khronos
and
> > try to get these tests open-sourced? I think the value from the tests comes
> from
> > certification, not simply access to the tests' code.
> That's opening up a can-o-worms ;).  A license to the Khronos conformance
isn't
> cheap.  I'm not trained in the art of lobbying.  But I'm sure between the "all
> of us", someone could persuade them on this proposal.

I'll discuss this with some Khronos members and would request that you do the
same.

https://codereview.chromium.org/556333003/diff/1/gpu/khronos_conform_support/...
File gpu/khronos_conform_support/generate_khronos_conform_tests.py (right):

https://codereview.chromium.org/556333003/diff/1/gpu/khronos_conform_support/...
gpu/khronos_conform_support/generate_khronos_conform_tests.py:1: #!/usr/bin/env
python
On 2014/09/11 01:09:20, uartie wrote:
> On 2014/09/11 00:03:26, Ken Russell wrote:
> > On 2014/09/10 22:12:59, uartie wrote:
> > > On 2014/09/10 20:12:50, piman (Very slow to review) wrote:
> > > > This file looks very similar to 
> > > > gpu/gles2_conform_support/generate_gles2_conform_tests.py. Can we share
/
> > > avoid
> > > > duplication?
> > > 
> > > There are quite a few differences between the two test generators.  The
new
> > > drawElements framework uses a different test naming scheme and this
accounts
> > for
> > > that.  Also, this generator recurses through the .run files and only
> produces
> > > tests for the .test files (the algorithm is similar to what can be found
in
> > the
> > > drawElements framework when it traverses the .run files to produce test
> > names). 
> > > The generate_gles2_conform_tests.py does not recurse.
> > > 
> > > I suppose the generate_gles2_conform_tests.py could be refactored to
support
> > > both schemes... but I think it would likely get a little messy.
> > 
> > As unfortunate as code duplication in this area is, I can understand the
need
> to
> > do it because Khronos' test harness has evolved dramatically between the
GLES2
> > and GLES3 conformance suites.
> > 
> > Can this new test suite subsume the GLES2 conformance tests? Are all of the
> > areas covered by the GLES2 conformance tests also covered by these new
tests?
> 
> Yes, this new suite *does* subsume the ES2 conformance tests.  In fact, the
ES2
> tests are the only tests this patch "subsumes" at the moment.  The pudding in
> the cake is (using the newer framework/APIs) we can extend this integration
> later to include the ES3+ tests, trivially.  So, yes, all of the areas covered
> by the GLES2 conformance tests *are* covered here, and then some (i.e.
presuming
> there have been more es2 test additions and fixes since the "legacy" khronos
> suite).

That's very good. We would be highly interested in getting this new target
running on the http://build.chromium.org/p/chromium.gpu.fyi/console waterfall
and removing the gles2_conform_test target in that case. Thank you for
confirming this.

What platforms have you tested this on? Have you at least tested it on desktop
Linux as well as Chrome OS? If we integrate the GLES3 conformance test snapshot
in src-internal, do you expect this will work fairly easily on the platforms
that Chromium's port of the GLES2 conformance test runs on?

[U. Artie Eoff, 2014-09-11 16:50:53]

https://codereview.chromium.org/556333003/diff/1/build/common.gypi
File build/common.gypi (right):

https://codereview.chromium.org/556333003/diff/1/build/common.gypi#newcode1382
build/common.gypi:1382: 'internal_khronos_conform_tests%': 0,
On 2014/09/11 01:25:17, Ken Russell wrote:
> On 2014/09/11 01:09:19, uartie wrote:
> > On 2014/09/11 00:03:26, Ken Russell wrote:
> > > On 2014/09/10 22:12:59, uartie wrote:
> > > > On 2014/09/10 20:12:50, piman (Very slow to review) wrote:
> > > > > Does this and internal_gles2_conform_tests need to be separate? I'd
> assume
> > > > > whoever has access to one has access to the other one. It'd be simpler
> to
> > > only
> > > > > have one of these.
> > > > 
> > > > I think the "gles2" in the name "internal_gles2_conform_tests" is too
> narrow
> > > and
> > > > implies only es2 tests.  We want to add es3 tests and future versions
> later.
> > 
> > > We
> > > > could merge these, I suppose, into one form eventually; one which
doesn't
> > > > constrict meaning so much (like internal_khronos_conform here).  But I'd
> > > rather
> > > > be able to compile one and not the other.  I think the idea behind this
is
> > to
> > > > incrementally ramp up with the new drawElements support and phase out
the
> > old.
> > > 
> > > > This requires having both for the transition phase.
> > > 
> > > Understood that the duplication may be necessary for the time being due to
> > major
> > > differences between the test harnesses. However the naming convention here
> is
> > > confusing. Since you're explicitly targeting the ES 3.x tests can you
please
> > > replace 'khronos' with 'gles3' throughout this CL?
> > 
> > Well... the idea is to target all the tests actually.  That is, to target
ES2,
> > ES30, ES31, and any future ES versions (and who knows, maybe GL21 thru GL44
> and
> > beyond).  All versions can be targeted in one [binary] instance using this
new
> > API.  That is why I chose the more generic naming convention.  In fact, this
> > particular patch only targets the ES2 tests at the moment, the same as the
> > gles2_conform_support.  The main difference is it's using the new APIs so
that
> > we can trivially add the other ESx tests later without being constrained by
> > naming conventions.
> 
> I still think a more targeted name is appropriate. If not gles3, then gles, or
> khronos_gles, or something. If some future API comes around under Khronos'
> umbrella that isn't tested in this particular harness, a naming conflict with
> this would be unfortunate. What do you think?

Yeah, I'd be fine using "khronos_gles_conform".  Or, what about "khronos_glcts"?
 GL-CTS is the true project name of their entire conformance suite afterall (see
their cmake file).  It does make a shorter prefix and also is a bit more of a
targeted name than my original.  If no one likes that, I'll go with the former.

https://codereview.chromium.org/556333003/diff/1/gpu/khronos_conform_support/...
File gpu/khronos_conform_support/DEPS (right):

https://codereview.chromium.org/556333003/diff/1/gpu/khronos_conform_support/...
gpu/khronos_conform_support/DEPS:2: "+third_party/khronos_conform",
On 2014/09/11 01:25:17, Ken Russell wrote:
> On 2014/09/11 01:09:19, uartie wrote:
> > On 2014/09/11 00:03:26, Ken Russell wrote:
> > > On 2014/09/10 22:12:59, uartie wrote:
> > > > On 2014/09/10 20:12:50, piman (Very slow to review) wrote:
> > > > > I don't see this directory in either my local checkout nor any of the
> DEPS
> > > > files
> > > > > (either internal or external)
> > > > 
> > > > I guess I'm not certain what this file really does.  But essentially
> you'll
> > > have
> > > > to clone your licensed version of the khronos conformance sourcecode
into
> > that
> > > > path on your end... using whatever process you've put in place
internally
> to
> > > get
> > > > them. Right?
> > > 
> > > It sounds like someone will have to clone Khronos' GLES3 conformance tests
> > into
> > > Chromium's src-internal repository. Frankly, it's quite a pain that the
> GLES3
> > > conformance tests are not open source. Can you please work within Khronos
> and
> > > try to get these tests open-sourced? I think the value from the tests
comes
> > from
> > > certification, not simply access to the tests' code.
> > That's opening up a can-o-worms ;).  A license to the Khronos conformance
> isn't
> > cheap.  I'm not trained in the art of lobbying.  But I'm sure between the
"all
> > of us", someone could persuade them on this proposal.
> 
> I'll discuss this with some Khronos members and would request that you do the
> same.

I appreciate that.  I'll defer to @chadv on my team to bring it up to Khronos.

https://codereview.chromium.org/556333003/diff/1/gpu/khronos_conform_support/...
File gpu/khronos_conform_support/generate_khronos_conform_tests.py (right):

https://codereview.chromium.org/556333003/diff/1/gpu/khronos_conform_support/...
gpu/khronos_conform_support/generate_khronos_conform_tests.py:1: #!/usr/bin/env
python
On 2014/09/11 01:25:17, Ken Russell wrote:
> On 2014/09/11 01:09:20, uartie wrote:
> > On 2014/09/11 00:03:26, Ken Russell wrote:
> > > On 2014/09/10 22:12:59, uartie wrote:
> > > > On 2014/09/10 20:12:50, piman (Very slow to review) wrote:
> > > > > This file looks very similar to 
> > > > > gpu/gles2_conform_support/generate_gles2_conform_tests.py. Can we
share
> /
> > > > avoid
> > > > > duplication?
> > > > 
> > > > There are quite a few differences between the two test generators.  The
> new
> > > > drawElements framework uses a different test naming scheme and this
> accounts
> > > for
> > > > that.  Also, this generator recurses through the .run files and only
> > produces
> > > > tests for the .test files (the algorithm is similar to what can be found
> in
> > > the
> > > > drawElements framework when it traverses the .run files to produce test
> > > names). 
> > > > The generate_gles2_conform_tests.py does not recurse.
> > > > 
> > > > I suppose the generate_gles2_conform_tests.py could be refactored to
> support
> > > > both schemes... but I think it would likely get a little messy.
> > > 
> > > As unfortunate as code duplication in this area is, I can understand the
> need
> > to
> > > do it because Khronos' test harness has evolved dramatically between the
> GLES2
> > > and GLES3 conformance suites.
> > > 
> > > Can this new test suite subsume the GLES2 conformance tests? Are all of
the
> > > areas covered by the GLES2 conformance tests also covered by these new
> tests?
> > 
> > Yes, this new suite *does* subsume the ES2 conformance tests.  In fact, the
> ES2
> > tests are the only tests this patch "subsumes" at the moment.  The pudding
in
> > the cake is (using the newer framework/APIs) we can extend this integration
> > later to include the ES3+ tests, trivially.  So, yes, all of the areas
covered
> > by the GLES2 conformance tests *are* covered here, and then some (i.e.
> presuming
> > there have been more es2 test additions and fixes since the "legacy" khronos
> > suite).
> 
> That's very good. We would be highly interested in getting this new target
> running on the http://build.chromium.org/p/chromium.gpu.fyi/console waterfall
> and removing the gles2_conform_test target in that case. Thank you for
> confirming this.
> 
> What platforms have you tested this on? Have you at least tested it on desktop
> Linux as well as Chrome OS? If we integrate the GLES3 conformance test
snapshot
> in src-internal, do you expect this will work fairly easily on the platforms
> that Chromium's port of the GLES2 conformance test runs on?

I haven't been able to get *pristine* Chromium to compile successfully on
desktop Linux, yet.  Granted, I haven't spent much time trying to figure out
why.

The only platform I've tested this CL on is ChromiumOS.

I expect there will be a little enabling work from your end to ensure this ran
on other platforms.  I don't have access to all those platforms, nor what that
list of platforms is.  I assume MAC and Windows are on that list.  I could
extrapolate details from the GLES2 conformance support, but I would be
uncomfortable doing that without being able to test it properly.

Nonetheless, I imagine the enabling of other platforms should be fairly easy. 
The nice thing is it can be done incrementally without affecting the legacy
GLES2 conformance support.

Mainly, I'm asking for your support in this realm.

[Ken Russell (switch to Gerrit), 2014-09-11 17:40:04]

https://codereview.chromium.org/556333003/diff/1/build/common.gypi
File build/common.gypi (right):

https://codereview.chromium.org/556333003/diff/1/build/common.gypi#newcode1382
build/common.gypi:1382: 'internal_khronos_conform_tests%': 0,
On 2014/09/11 16:50:52, uartie wrote:
> On 2014/09/11 01:25:17, Ken Russell wrote:
> > On 2014/09/11 01:09:19, uartie wrote:
> > > On 2014/09/11 00:03:26, Ken Russell wrote:
> > > > On 2014/09/10 22:12:59, uartie wrote:
> > > > > On 2014/09/10 20:12:50, piman (Very slow to review) wrote:
> > > > > > Does this and internal_gles2_conform_tests need to be separate? I'd
> > assume
> > > > > > whoever has access to one has access to the other one. It'd be
simpler
> > to
> > > > only
> > > > > > have one of these.
> > > > > 
> > > > > I think the "gles2" in the name "internal_gles2_conform_tests" is too
> > narrow
> > > > and
> > > > > implies only es2 tests.  We want to add es3 tests and future versions
> > later.
> > > 
> > > > We
> > > > > could merge these, I suppose, into one form eventually; one which
> doesn't
> > > > > constrict meaning so much (like internal_khronos_conform here).  But
I'd
> > > > rather
> > > > > be able to compile one and not the other.  I think the idea behind
this
> is
> > > to
> > > > > incrementally ramp up with the new drawElements support and phase out
> the
> > > old.
> > > > 
> > > > > This requires having both for the transition phase.
> > > > 
> > > > Understood that the duplication may be necessary for the time being due
to
> > > major
> > > > differences between the test harnesses. However the naming convention
here
> > is
> > > > confusing. Since you're explicitly targeting the ES 3.x tests can you
> please
> > > > replace 'khronos' with 'gles3' throughout this CL?
> > > 
> > > Well... the idea is to target all the tests actually.  That is, to target
> ES2,
> > > ES30, ES31, and any future ES versions (and who knows, maybe GL21 thru
GL44
> > and
> > > beyond).  All versions can be targeted in one [binary] instance using this
> new
> > > API.  That is why I chose the more generic naming convention.  In fact,
this
> > > particular patch only targets the ES2 tests at the moment, the same as the
> > > gles2_conform_support.  The main difference is it's using the new APIs so
> that
> > > we can trivially add the other ESx tests later without being constrained
by
> > > naming conventions.
> > 
> > I still think a more targeted name is appropriate. If not gles3, then gles,
or
> > khronos_gles, or something. If some future API comes around under Khronos'
> > umbrella that isn't tested in this particular harness, a naming conflict
with
> > this would be unfortunate. What do you think?
> 
> Yeah, I'd be fine using "khronos_gles_conform".  Or, what about
"khronos_glcts"?
>  GL-CTS is the true project name of their entire conformance suite afterall
(see
> their cmake file).  It does make a shorter prefix and also is a bit more of a
> targeted name than my original.  If no one likes that, I'll go with the
former.

Both sound fine. If khronos_glcts is closer to the real name of the project,
let's go with that.

https://codereview.chromium.org/556333003/diff/1/gpu/khronos_conform_support/...
File gpu/khronos_conform_support/generate_khronos_conform_tests.py (right):

https://codereview.chromium.org/556333003/diff/1/gpu/khronos_conform_support/...
gpu/khronos_conform_support/generate_khronos_conform_tests.py:1: #!/usr/bin/env
python
On 2014/09/11 16:50:53, uartie wrote:
> On 2014/09/11 01:25:17, Ken Russell wrote:
> > On 2014/09/11 01:09:20, uartie wrote:
> > > On 2014/09/11 00:03:26, Ken Russell wrote:
> > > > On 2014/09/10 22:12:59, uartie wrote:
> > > > > On 2014/09/10 20:12:50, piman (Very slow to review) wrote:
> > > > > > This file looks very similar to 
> > > > > > gpu/gles2_conform_support/generate_gles2_conform_tests.py. Can we
> share
> > /
> > > > > avoid
> > > > > > duplication?
> > > > > 
> > > > > There are quite a few differences between the two test generators. 
The
> > new
> > > > > drawElements framework uses a different test naming scheme and this
> > accounts
> > > > for
> > > > > that.  Also, this generator recurses through the .run files and only
> > > produces
> > > > > tests for the .test files (the algorithm is similar to what can be
found
> > in
> > > > the
> > > > > drawElements framework when it traverses the .run files to produce
test
> > > > names). 
> > > > > The generate_gles2_conform_tests.py does not recurse.
> > > > > 
> > > > > I suppose the generate_gles2_conform_tests.py could be refactored to
> > support
> > > > > both schemes... but I think it would likely get a little messy.
> > > > 
> > > > As unfortunate as code duplication in this area is, I can understand the
> > need
> > > to
> > > > do it because Khronos' test harness has evolved dramatically between the
> > GLES2
> > > > and GLES3 conformance suites.
> > > > 
> > > > Can this new test suite subsume the GLES2 conformance tests? Are all of
> the
> > > > areas covered by the GLES2 conformance tests also covered by these new
> > tests?
> > > 
> > > Yes, this new suite *does* subsume the ES2 conformance tests.  In fact,
the
> > ES2
> > > tests are the only tests this patch "subsumes" at the moment.  The pudding
> in
> > > the cake is (using the newer framework/APIs) we can extend this
integration
> > > later to include the ES3+ tests, trivially.  So, yes, all of the areas
> covered
> > > by the GLES2 conformance tests *are* covered here, and then some (i.e.
> > presuming
> > > there have been more es2 test additions and fixes since the "legacy"
khronos
> > > suite).
> > 
> > That's very good. We would be highly interested in getting this new target
> > running on the http://build.chromium.org/p/chromium.gpu.fyi/console
waterfall
> > and removing the gles2_conform_test target in that case. Thank you for
> > confirming this.
> > 
> > What platforms have you tested this on? Have you at least tested it on
desktop
> > Linux as well as Chrome OS? If we integrate the GLES3 conformance test
> snapshot
> > in src-internal, do you expect this will work fairly easily on the platforms
> > that Chromium's port of the GLES2 conformance test runs on?
> 
> I haven't been able to get *pristine* Chromium to compile successfully on
> desktop Linux, yet.  Granted, I haven't spent much time trying to figure out
> why.
> 
> The only platform I've tested this CL on is ChromiumOS.
> 
> I expect there will be a little enabling work from your end to ensure this ran
> on other platforms.  I don't have access to all those platforms, nor what that
> list of platforms is.  I assume MAC and Windows are on that list.  I could
> extrapolate details from the GLES2 conformance support, but I would be
> uncomfortable doing that without being able to test it properly.
> 
> Nonetheless, I imagine the enabling of other platforms should be fairly easy. 
> The nice thing is it can be done incrementally without affecting the legacy
> GLES2 conformance support.
> 
> Mainly, I'm asking for your support in this realm.

You certainly have my support. Could you please update this CL with the AUTHORS
change piman requested and the other naming convention changes? I'll help you
get the Khronos files checked in to the appropriate place. May we rely on you to
help advise on issues we may encounter porting this to the other platforms?

[U. Artie Eoff, 2014-09-11 19:45:26]

https://codereview.chromium.org/556333003/diff/1/gpu/khronos_conform_support/...
File gpu/khronos_conform_support/generate_khronos_conform_tests.py (right):

https://codereview.chromium.org/556333003/diff/1/gpu/khronos_conform_support/...
gpu/khronos_conform_support/generate_khronos_conform_tests.py:1: #!/usr/bin/env
python
On 2014/09/11 17:40:04, Ken Russell wrote:
> On 2014/09/11 16:50:53, uartie wrote:
> > On 2014/09/11 01:25:17, Ken Russell wrote:
> > > On 2014/09/11 01:09:20, uartie wrote:
> > > > On 2014/09/11 00:03:26, Ken Russell wrote:
> > > > > On 2014/09/10 22:12:59, uartie wrote:
> > > > > > On 2014/09/10 20:12:50, piman (Very slow to review) wrote:
> > > > > > > This file looks very similar to 
> > > > > > > gpu/gles2_conform_support/generate_gles2_conform_tests.py. Can we
> > share
> > > /
> > > > > > avoid
> > > > > > > duplication?
> > > > > > 
> > > > > > There are quite a few differences between the two test generators. 
> The
> > > new
> > > > > > drawElements framework uses a different test naming scheme and this
> > > accounts
> > > > > for
> > > > > > that.  Also, this generator recurses through the .run files and only
> > > > produces
> > > > > > tests for the .test files (the algorithm is similar to what can be
> found
> > > in
> > > > > the
> > > > > > drawElements framework when it traverses the .run files to produce
> test
> > > > > names). 
> > > > > > The generate_gles2_conform_tests.py does not recurse.
> > > > > > 
> > > > > > I suppose the generate_gles2_conform_tests.py could be refactored to
> > > support
> > > > > > both schemes... but I think it would likely get a little messy.
> > > > > 
> > > > > As unfortunate as code duplication in this area is, I can understand
the
> > > need
> > > > to
> > > > > do it because Khronos' test harness has evolved dramatically between
the
> > > GLES2
> > > > > and GLES3 conformance suites.
> > > > > 
> > > > > Can this new test suite subsume the GLES2 conformance tests? Are all
of
> > the
> > > > > areas covered by the GLES2 conformance tests also covered by these new
> > > tests?
> > > > 
> > > > Yes, this new suite *does* subsume the ES2 conformance tests.  In fact,
> the
> > > ES2
> > > > tests are the only tests this patch "subsumes" at the moment.  The
pudding
> > in
> > > > the cake is (using the newer framework/APIs) we can extend this
> integration
> > > > later to include the ES3+ tests, trivially.  So, yes, all of the areas
> > covered
> > > > by the GLES2 conformance tests *are* covered here, and then some (i.e.
> > > presuming
> > > > there have been more es2 test additions and fixes since the "legacy"
> khronos
> > > > suite).
> > > 
> > > That's very good. We would be highly interested in getting this new target
> > > running on the http://build.chromium.org/p/chromium.gpu.fyi/console
> waterfall
> > > and removing the gles2_conform_test target in that case. Thank you for
> > > confirming this.
> > > 
> > > What platforms have you tested this on? Have you at least tested it on
> desktop
> > > Linux as well as Chrome OS? If we integrate the GLES3 conformance test
> > snapshot
> > > in src-internal, do you expect this will work fairly easily on the
platforms
> > > that Chromium's port of the GLES2 conformance test runs on?
> > 
> > I haven't been able to get *pristine* Chromium to compile successfully on
> > desktop Linux, yet.  Granted, I haven't spent much time trying to figure out
> > why.
> > 
> > The only platform I've tested this CL on is ChromiumOS.
> > 
> > I expect there will be a little enabling work from your end to ensure this
ran
> > on other platforms.  I don't have access to all those platforms, nor what
that
> > list of platforms is.  I assume MAC and Windows are on that list.  I could
> > extrapolate details from the GLES2 conformance support, but I would be
> > uncomfortable doing that without being able to test it properly.
> > 
> > Nonetheless, I imagine the enabling of other platforms should be fairly
easy. 
> > The nice thing is it can be done incrementally without affecting the legacy
> > GLES2 conformance support.
> > 
> > Mainly, I'm asking for your support in this realm.
> 
> You certainly have my support. Could you please update this CL with the
AUTHORS
> change piman requested and the other naming convention changes? I'll help you
> get the Khronos files checked in to the appropriate place. May we rely on you
to
> help advise on issues we may encounter porting this to the other platforms?

Excellent! The CL updates are on the way ;) Yes, I will help advise on any
issues encountered.

[U. Artie Eoff, 2014-09-11 21:17:29]

Updated CL.  I changed the commit message but the description did not get
updated here on the UI.  Why?

[U. Artie Eoff, 2014-09-11 21:20:25]

On 2014/09/11 21:17:29, uartie wrote:
> Updated CL.  I changed the commit message but the description did not get
> updated here on the UI.  Why?

Ok, looks like I had to edit it manually here.

[piman, 2014-09-11 21:39:19]

Ok, if the goal is to remove the existing gles2 CTS support, and replace it with
this, then let's proceed.

https://codereview.chromium.org/556333003/diff/20001/gpu/khronos_glcts_suppor...
File gpu/khronos_glcts_support/native/egl_native_windowless.cc (right):

https://codereview.chromium.org/556333003/diff/20001/gpu/khronos_glcts_suppor...
gpu/khronos_glcts_support/native/egl_native_windowless.cc:18: public:
Can we update this file to the chromium style - e.g. using git cl format?

[U. Artie Eoff, 2014-09-11 22:11:57]

Uploaded patch set 3: Ran git cl format on egl_native_windowless.cc so it
matches the chromium style.

https://codereview.chromium.org/556333003/diff/20001/gpu/khronos_glcts_suppor...
File gpu/khronos_glcts_support/native/egl_native_windowless.cc (right):

https://codereview.chromium.org/556333003/diff/20001/gpu/khronos_glcts_suppor...
gpu/khronos_glcts_support/native/egl_native_windowless.cc:18: public:
On 2014/09/11 21:39:19, piman (Very slow to review) wrote:
> Can we update this file to the chromium style - e.g. using git cl format?

Done.

[piman, 2014-09-11 22:25:41]

The CQ bit was checked by piman@chromium.org

[piman, 2014-09-11 22:25:41]

lgtm

[commit-bot: I haz the power, 2014-09-11 22:27:47]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patchset/556333003/40001

[Ken Russell (switch to Gerrit), 2014-09-11 22:59:22]

Sorry: why did you specify NOTRY=true? That seems like a bad idea. If for some
reason the build breaks with this CL it should be caught. I'm un-CQ'ing this
patch for the moment.

[Ken Russell (switch to Gerrit), 2014-09-11 22:59:27]

The CQ bit was unchecked by kbr@chromium.org

[U. Artie Eoff, 2014-09-11 23:04:11]

On 2014/09/11 22:59:22, Ken Russell wrote:
> Sorry: why did you specify NOTRY=true? That seems like a bad idea. If for some
> reason the build breaks with this CL it should be caught. I'm un-CQ'ing this
> patch for the moment.

Apologies, it wasn't clear to me whether this was standard or not based on the
contributing code documentation that I read.

[Ken Russell (switch to Gerrit), 2014-09-11 23:06:00]

LGTM overall. One question. Feel free to check the "commit" box again once
answered either way.

https://codereview.chromium.org/556333003/diff/40001/gpu/khronos_glcts_suppor...
File gpu/khronos_glcts_support/khronos_glcts_test.cc (right):

https://codereview.chromium.org/556333003/diff/40001/gpu/khronos_glcts_suppor...
gpu/khronos_glcts_support/khronos_glcts_test.cc:71:
cmdline.AppendSwitch("--deqp-archive-dir=" + archive.value());
Are you sure these are correct? See the switch from AppendSwitch to AppendArg in
https://codereview.chromium.org/500133005 because of case sensitivity problems
with paths.

[U. Artie Eoff, 2014-09-11 23:57:12]

The gles2_conform_test changes must be pretty recent since I essentially started
from that for khronos_glcts_test.

Anyhow, I can fix these lines of code up and resubmit the CL.  Curious though,
was it only casting "directories" into lower-case?

Interesting that my khronos_glcts_test.cc is getting diff'd with the
gles2_conform_test.cc... must be an automatic thing in this UI because locally
'git show' doesn't indicate any relationship.

https://codereview.chromium.org/556333003/diff/40001/gpu/khronos_glcts_suppor...
File gpu/khronos_glcts_support/khronos_glcts_test.cc (left):

https://codereview.chromium.org/556333003/diff/40001/gpu/khronos_glcts_suppor...
gpu/khronos_glcts_support/khronos_glcts_test.cc:73: bool success =
base::GetAppOutput(cmdline, &output);
...I assume this return value is true iif the command exit status is 0.

https://codereview.chromium.org/556333003/diff/40001/gpu/khronos_glcts_suppor...
gpu/khronos_glcts_support/khronos_glcts_test.cc:78: (failed_index ==
std::string::npos);
...and, if there is an early-out success (i.e. success==true) I would expect
this result to be false.  Hence causing the test to report FAIL.

https://codereview.chromium.org/556333003/diff/40001/gpu/khronos_glcts_suppor...
File gpu/khronos_glcts_support/khronos_glcts_test.cc (right):

https://codereview.chromium.org/556333003/diff/40001/gpu/khronos_glcts_suppor...
gpu/khronos_glcts_support/khronos_glcts_test.cc:71:
cmdline.AppendSwitch("--deqp-archive-dir=" + archive.value());
On 2014/09/11 23:06:00, Ken Russell wrote:
> Are you sure these are correct? See the switch from AppendSwitch to AppendArg
in
> https://codereview.chromium.org/500133005 because of case sensitivity problems
> with paths.

Hmm... yes this could be problematic if the values are being cast to lower-case
and end up different.  However, these won't cause the
khronos_glcts_test_windowless to early-out with success code.  The
khronos_glcts_test_windowless will exit with ERROR and non-zero status if it
can't access log-filename or archive-dir.

https://codereview.chromium.org/556333003/diff/40001/gpu/khronos_glcts_suppor...
gpu/khronos_glcts_support/khronos_glcts_test.cc:73:
cmdline.AppendSwitch(std::string("--deqp-case=") + test_name);
This --deqp-case concerns me, however, which is what specifies the test to run
(by name)... in the gles2_conform_test_windowless suite '--run' specifies the
test to run (by directory).  But the problem ends up the same.  The
khronos_glcts_test_windowless *does* early-out with 0 exit status even if the
specified test is not found...

[U. Artie Eoff, 2014-09-12 00:03:46]

Ken, was that only an issue on a specific platform?  On ChromiumOS, the
constructed command arguments don't appear to be getting "lower-cased" for me.

[U. Artie Eoff, 2014-09-12 00:23:22]

Uploaded Patch Set 4 : Use AppendArg instead of AppendSwitch to hopefully
circumvent a repeat of chromium:408251

[U. Artie Eoff, 2014-09-12 00:24:45]

The CQ bit was checked by ullysses.a.eoff@intel.com

[commit-bot: I haz the power, 2014-09-12 00:45:24]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patchset/556333003/60001

[Ken Russell (switch to Gerrit), 2014-09-12 00:55:43]

On 2014/09/12 00:03:46, uartie wrote:
> Ken, was that only an issue on a specific platform?  On ChromiumOS, the
> constructed command arguments don't appear to be getting "lower-cased" for me.

I'm guessing it affected Windows due to the comments in
https://codereview.chromium.org/500133005 , though it might have affected Linux
and Mac as well. jmadill@ found the problem and would remember for sure.

Anyway, thanks for updating your new harness.

[Ken Russell (switch to Gerrit), 2014-09-12 01:00:12]

I should have mentioned that the inclusion of your new target in the
chromium_gpu_builder and chromium_gpu_debug_builder targets will necessitate
getting the harness to compile on all of Chromium's platforms, even though it
won't be run yet. Hope you can devote the time to fixing the build problems.

[U. Artie Eoff, 2014-09-12 01:10:46]

On 2014/09/12 01:00:12, Ken Russell wrote:
> I should have mentioned that the inclusion of your new target in the
> chromium_gpu_builder and chromium_gpu_debug_builder targets will necessitate
> getting the harness to compile on all of Chromium's platforms, even though it
> won't be run yet. Hope you can devote the time to fixing the build problems.

I can work on these tomorrow.  The build problems don't block others do they? 
Is there an easy way that I can submit test builds as I work through the issues?

[commit-bot: I haz the power, 2014-09-12 01:11:57]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2014-09-12 01:11:59]

Try jobs failed on following builders:
  chromium_presubmit on tryserver.chromium.linux
(http://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presub...)

[U. Artie Eoff, 2014-09-12 01:22:42]

The CQ bit was unchecked by ullysses.a.eoff@intel.com

[Ken Russell (switch to Gerrit), 2014-09-12 01:28:12]

On 2014/09/12 01:10:46, uartie wrote:
> On 2014/09/12 01:00:12, Ken Russell wrote:
> > I should have mentioned that the inclusion of your new target in the
> > chromium_gpu_builder and chromium_gpu_debug_builder targets will necessitate
> > getting the harness to compile on all of Chromium's platforms, even though
it
> > won't be run yet. Hope you can devote the time to fixing the build problems.
> 
> I can work on these tomorrow.  The build problems don't block others do they? 
> Is there an easy way that I can submit test builds as I work through the
issues?

They don't block anyone else. All of the try jobs you submit have to pass before
your patch will be committed.

You can either upload a new version and re-check the commit box, or use "git cl
try" while on your branch on your local machine, after re-uploading. However, it
might be necessary to get try job access explicitly enabled for your account. If
you try "git cl try" and it doesn't work, let me know.

[Ken Russell (switch to Gerrit), 2014-09-12 01:29:43]

On 2014/09/12 01:11:59, I haz the power (commit-bot) wrote:
> Try jobs failed on following builders:
>   chromium_presubmit on tryserver.chromium.linux
>
(http://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presub...)

Hmm. That's a bit of a problem because the third_party directory hasn't been
added to Chromium's internal sources yet. I'll CC: a couple more people.

[Ken Russell (switch to Gerrit), 2014-09-12 01:32:16]

kbr@chromium.org changed reviewers:
+ brettw@chromium.org, cpu@chromium.org, darin@chromium.org

[Ken Russell (switch to Gerrit), 2014-09-12 01:32:17]

cpu or brettw or darin: looking for your approval for this CL. Ullysses is
working on replacing the harness for the OpenGL ES 2 conformance tests which are
checked in to src-internal with the new harness supporting OpenGL ES 3 (as well
as all the old tests). I haven't yet checked in the new closed sources into
src-internal but will do so shortly. Could you approve this so the harness can
land?

[U. Artie Eoff, 2014-09-12 01:49:57]

On 2014/09/12 01:28:12, Ken Russell wrote:
> On 2014/09/12 01:10:46, uartie wrote:
> > On 2014/09/12 01:00:12, Ken Russell wrote:
> > > I should have mentioned that the inclusion of your new target in the
> > > chromium_gpu_builder and chromium_gpu_debug_builder targets will
necessitate
> > > getting the harness to compile on all of Chromium's platforms, even though
> it
> > > won't be run yet. Hope you can devote the time to fixing the build
problems.
> > 
> > I can work on these tomorrow.  The build problems don't block others do
they? 
> > Is there an easy way that I can submit test builds as I work through the
> issues?
> 
> They don't block anyone else. All of the try jobs you submit have to pass
before
> your patch will be committed.
> 
> You can either upload a new version and re-check the commit box, or use "git
cl
> try" while on your branch on your local machine, after re-uploading. However,
it
> might be necessary to get try job access explicitly enabled for your account.
If
> you try "git cl try" and it doesn't work, let me know.

Hmm... so I have to upload any changes I make to this cl every time I want to
test it out?  That could be dozens of times depending on what platform I'm
trying to get to work.  That sure makes for a slow, painful "modify, try
compile, modify, ..." workflow :-/

[U. Artie Eoff, 2014-09-12 03:23:56]

The CQ bit was checked by ullysses.a.eoff@intel.com

[commit-bot: I haz the power, 2014-09-12 03:25:40]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patchset/556333003/80001

[commit-bot: I haz the power, 2014-09-12 03:40:54]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2014-09-12 03:40:56]

Try jobs failed on following builders:
  chromium_presubmit on tryserver.chromium.linux
(http://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presub...)

[U. Artie Eoff, 2014-09-12 04:08:48]

Looks like FileType::StringType is beating me up on Windows (i.e. there's no
implicit conversion from char* to std::wstring).

https://codereview.chromium.org/556333003/diff/80001/gpu/khronos_glcts_suppor...
File gpu/khronos_glcts_support/khronos_glcts_test.cc (right):

https://codereview.chromium.org/556333003/diff/80001/gpu/khronos_glcts_suppor...
gpu/khronos_glcts_support/khronos_glcts_test.cc:67:
g_deqp_log_dir.Append(FILE_PATH_LITERAL(test_info->name())).
Yep, I used FILE_PATH_LITERAL wrong here :-/  Will fix, but need to figure out
how to make test_info->name() a valid type for the call to Append() on Windows. 
Any thoughts?

https://codereview.chromium.org/556333003/diff/80001/gpu/khronos_glcts_suppor...
gpu/khronos_glcts_support/khronos_glcts_test.cc:96: g_deqp_log_dir =
base::FilePath(argv[1]);
This fails to compile on Windows.  Any hints on how to instantiate a FilePath
from a char* on Windows would be helpful.  Or is there a better way to do this?

[Jamie Madill, 2014-09-12 13:58:25]

On 2014/09/12 00:55:43, Ken Russell wrote:
> I'm guessing it affected Windows due to the comments in
> https://codereview.chromium.org/500133005 , though it might have affected
Linux
> and Mac as well. jmadill@ found the problem and would remember for sure.

I can confirm I saw the problem on Windows. I not sure if it affected any other
platform, as I didn't check them at the time, except after the patch was in.

[U. Artie Eoff, 2014-09-12 14:49:19]

On 2014/09/12 13:58:25, Jamie Madill wrote:
> On 2014/09/12 00:55:43, Ken Russell wrote:
> > I'm guessing it affected Windows due to the comments in
> > https://codereview.chromium.org/500133005 , though it might have affected
> Linux
> > and Mac as well. jmadill@ found the problem and would remember for sure.
> 
> I can confirm I saw the problem on Windows. I not sure if it affected any
other
> platform, as I didn't check them at the time, except after the patch was in.

Yeah, after looking at the CommandLine::AppendSwitch definition, it's clear that
it only happens on Windows due to this call
"LowerASCIIOnWindows(switch_string)".

[U. Artie Eoff, 2014-09-12 16:21:26]

Patchset #5 (id:80001) has been deleted

[U. Artie Eoff, 2014-09-12 16:24:42]

The CQ bit was checked by ullysses.a.eoff@intel.com

[U. Artie Eoff, 2014-09-12 16:24:58]

The CQ bit was unchecked by ullysses.a.eoff@intel.com

[U. Artie Eoff, 2014-09-12 16:26:34]

The CQ bit was checked by ullysses.a.eoff@intel.com

[commit-bot: I haz the power, 2014-09-12 16:27:47]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patchset/556333003/100001

[commit-bot: I haz the power, 2014-09-12 16:39:18]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2014-09-12 16:39:19]

Try jobs failed on following builders:
  chromium_presubmit on tryserver.chromium.linux
(http://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presub...)

[U. Artie Eoff, 2014-09-12 17:02:22]

On 2014/09/12 16:39:19, I haz the power (commit-bot) wrote:
> Try jobs failed on following builders:
>   chromium_presubmit on tryserver.chromium.linux
>
(http://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presub...)

The build problems should be fixed now.  All that's left now should be getting
the new khronos conformance closed sources checked into src-internal.

[U. Artie Eoff, 2014-09-12 20:02:26]

On 2014/09/12 01:28:12, Ken Russell wrote:
> On 2014/09/12 01:10:46, uartie wrote:
> > On 2014/09/12 01:00:12, Ken Russell wrote:
> > > I should have mentioned that the inclusion of your new target in the
> > > chromium_gpu_builder and chromium_gpu_debug_builder targets will
necessitate
> > > getting the harness to compile on all of Chromium's platforms, even though
> it
> > > won't be run yet. Hope you can devote the time to fixing the build
problems.
> > 
> > I can work on these tomorrow.  The build problems don't block others do
they? 
> > Is there an easy way that I can submit test builds as I work through the
> issues?
> 
> They don't block anyone else. All of the try jobs you submit have to pass
before
> your patch will be committed.
> 
> You can either upload a new version and re-check the commit box, or use "git
cl
> try" while on your branch on your local machine, after re-uploading. However,
it
> might be necessary to get try job access explicitly enabled for your account.
If
> you try "git cl try" and it doesn't work, let me know.

Ken, "git cl try" does not work for me... the try jobs just turn purple (try
server exception).  Can you enable this for my account?

[Ken Russell (switch to Gerrit), 2014-09-13 06:31:05]

On 2014/09/12 20:02:26, uartie wrote:
> On 2014/09/12 01:28:12, Ken Russell wrote:
> > On 2014/09/12 01:10:46, uartie wrote:
> > > On 2014/09/12 01:00:12, Ken Russell wrote:
> > > > I should have mentioned that the inclusion of your new target in the
> > > > chromium_gpu_builder and chromium_gpu_debug_builder targets will
> necessitate
> > > > getting the harness to compile on all of Chromium's platforms, even
though
> > it
> > > > won't be run yet. Hope you can devote the time to fixing the build
> problems.
> > > 
> > > I can work on these tomorrow.  The build problems don't block others do
> they? 
> > > Is there an easy way that I can submit test builds as I work through the
> > issues?
> > 
> > They don't block anyone else. All of the try jobs you submit have to pass
> before
> > your patch will be committed.
> > 
> > You can either upload a new version and re-check the commit box, or use "git
> cl
> > try" while on your branch on your local machine, after re-uploading.
However,
> it
> > might be necessary to get try job access explicitly enabled for your
account.
> If
> > you try "git cl try" and it doesn't work, let me know.
> 
> Ken, "git cl try" does not work for me... the try jobs just turn purple (try
> server exception).  Can you enable this for my account?

I've emailed the project administrators. Will let you know once it's enabled.

Does the "Try more bots" UI here in Rietveld work for you to trigger jobs
manually?

Also, please ping the OWNERS in src/third_party about reviewing this change. I
think it should be fine to go in without the CTS being checked in yet to the
internal source tree. I've filed Issue 414003 about that and will take care of
it the week of September 22 (at a Khronos F2F next week).

[U. Artie Eoff, 2014-09-13 15:30:41]

On 2014/09/13 06:31:05, Ken Russell wrote:
> On 2014/09/12 20:02:26, uartie wrote:
> > On 2014/09/12 01:28:12, Ken Russell wrote:
> > > On 2014/09/12 01:10:46, uartie wrote:
> > > > On 2014/09/12 01:00:12, Ken Russell wrote:
> > > > > I should have mentioned that the inclusion of your new target in the
> > > > > chromium_gpu_builder and chromium_gpu_debug_builder targets will
> > necessitate
> > > > > getting the harness to compile on all of Chromium's platforms, even
> though
> > > it
> > > > > won't be run yet. Hope you can devote the time to fixing the build
> > problems.
> > > > 
> > > > I can work on these tomorrow.  The build problems don't block others do
> > they? 
> > > > Is there an easy way that I can submit test builds as I work through the
> > > issues?
> > > 
> > > They don't block anyone else. All of the try jobs you submit have to pass
> > before
> > > your patch will be committed.
> > > 
> > > You can either upload a new version and re-check the commit box, or use
"git
> > cl
> > > try" while on your branch on your local machine, after re-uploading.
> However,
> > it
> > > might be necessary to get try job access explicitly enabled for your
> account.
> > If
> > > you try "git cl try" and it doesn't work, let me know.
> > 
> > Ken, "git cl try" does not work for me... the try jobs just turn purple (try
> > server exception).  Can you enable this for my account?
> 
> I've emailed the project administrators. Will let you know once it's enabled.
> 
> Does the "Try more bots" UI here in Rietveld work for you to trigger jobs
> manually?
> 
> Also, please ping the OWNERS in src/third_party about reviewing this change. I
> think it should be fine to go in without the CTS being checked in yet to the
> internal source tree. I've filed Issue 414003 about that and will take care of
> it the week of September 22 (at a Khronos F2F next week).

No, the "Try more bots" UI does not work for me here either; same outcome.  I'm
only able to trigger try bots by checking the "Commit" option.

I'll ping the OWNERS on Monday. Thanks.

[U. Artie Eoff, 2014-09-15 16:27:50]

cpu, brettw, or darin, could one of you provide a review for this CL?

[cpu_(ooo_6.6-7.5), 2014-09-15 23:14:13]

I don't see code in third party... missing?

[Ken Russell (switch to Gerrit), 2014-09-16 00:33:45]

On 2014/09/15 23:14:13, cpu wrote:
> I don't see code in third party... missing?

I'm away on business travel this week and can't add it until I get back. It
would still be useful to get this checked in in the interim period.

[piman, 2014-09-16 00:40:29]

On Mon, Sep 15, 2014 at 4:14 PM, <cpu@chromium.org> wrote:

> I don't see code in third party... missing?
>

For context (which you can follow upthread), the third-party code is
restricted, and would have to be in src-internal the same way that
gles2_conform is today (with the intent to replace it eventually).
I'm happy to provide more color offline.


> https://codereview.chromium.org/556333003/
>

To unsubscribe from this group and stop receiving emails from it, send an email
to chromium-reviews+unsubscribe@chromium.org.

[U. Artie Eoff, 2014-09-24 12:37:24]

On 2014/09/16 00:33:45, Ken Russell wrote:
> On 2014/09/15 23:14:13, cpu wrote:
> > I don't see code in third party... missing?
> 
> I'm away on business travel this week and can't add it until I get back. It
> would still be useful to get this checked in in the interim period.

Hi Ken, hope you had a good trip.  Are you able to get the third party code
checked in this week?  Either way, who's able to commit this CL in the interim? 
I don't believe I have committer rights.  Would a new rebase be appropriate
first?

[Ken Russell (switch to Gerrit), 2014-09-26 22:22:48]

On 2014/09/24 12:37:24, uartie wrote:
> On 2014/09/16 00:33:45, Ken Russell wrote:
> > On 2014/09/15 23:14:13, cpu wrote:
> > > I don't see code in third party... missing?
> > 
> > I'm away on business travel this week and can't add it until I get back. It
> > would still be useful to get this checked in in the interim period.
> 
> Hi Ken, hope you had a good trip.  Are you able to get the third party code
> checked in this week?  Either way, who's able to commit this CL in the
interim? 
> I don't believe I have committer rights.  Would a new rebase be appropriate
> first?

Sorry, I was not able to this week. I will do it at the beginning of next week.
Please do rebase this CL so that it's ready to go when src-internal is
populated. Thanks.

[cpu_(ooo_6.6-7.5), 2014-09-29 18:13:15]

then just have my rubber-stamp lgtm.

Make sure if that code is ever linked into chrome, you get open source people to
look at the license etc.

[U. Artie Eoff, 2014-09-29 23:21:13]

rebased

[Ken Russell (switch to Gerrit), 2014-10-01 00:05:27]

khronos_glcts has been committed to the internal repo at r61890. It was too
large for gcl to handle so it was committed manually.

I've verified Ullysses' CL locally; it builds most of the test suite, although
there are a few compile errors on desktop Linux. CQ'ing this now; we will work
through those build errors, and then roll src-internal DEPS and get these tests
running on the FYI bots.

[Ken Russell (switch to Gerrit), 2014-10-01 00:05:36]

The CQ bit was checked by kbr@chromium.org

[commit-bot: I haz the power, 2014-10-01 00:06:13]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/556333003/140001

[commit-bot: I haz the power, 2014-10-01 01:08:08]

Committed patchset #7 (id:140001) as 30f42a82c0333000f6880bbcffd5795b79ca6997

[commit-bot: I haz the power, 2014-10-01 01:08:49]

Patchset 7 (id:??) landed as
https://crrev.com/ffbb37ebec861db6e93471a5837727ea8d974f91
Cr-Commit-Position: refs/heads/master@{#297567}