WebCrypto: Add LayoutTests for ECDSA.

WebCrypto: Add LayoutTests for ECDSA.

BUG=399094
Committed: https://src.chromium.org/viewvc/blink?view=rev&revision=185382

[eroman, 2014-11-06 04:43:15]

eroman@chromium.org changed reviewers:
+ abarth@chromium.org

[eroman, 2014-11-06 04:43:15]

Depends on: https://codereview.chromium.org/707743002/ and
https://codereview.chromium.org/698363002/

[eroman, 2014-11-13 19:56:10]

eroman@chromium.org changed reviewers:
+ jww@chromium.org
- abarth@chromium.org

[jww, 2014-11-14 08:18:32]

This lgtm, with the usual Blink grumbles that a lot of this should really be
unit tests, not LayoutTests (and a lot more of them, at that), and that I'd love
to see a wider range of parsing tests (although this really applies across the
board to the WebCrypto tests, and is also related to the unit test issue).

https://codereview.chromium.org/707753002/diff/40001/LayoutTests/crypto/ecdsa...
File LayoutTests/crypto/ecdsa-generateKey-parseAlgorithm-failures.html (right):

https://codereview.chromium.org/707753002/diff/40001/LayoutTests/crypto/ecdsa...
LayoutTests/crypto/ecdsa-generateKey-parseAlgorithm-failures.html:35: return
crypto.subtle.generateKey({name: 'ECDSA', namedCurve: 'P-256aa'}, extractable ,
keyUsages);
Perhaps getting nit picky, but I'd love to see a lot more bad-parsing checks.
For example, the base cases, bad prefixes, etc. This really applies more
generally to the crypto tests, though.

[eroman, 2014-11-14 17:30:14]

Thanks!

I have more tests, especially of bad data, on the Chromium-side unit-tests:
https://chromium.googlesource.com/chromium/src/+/master/content/test/data/web...
https://chromium.googlesource.com/chromium/src/+/master/content/test/data/web...
And the test runner for those files:
https://chromium.googlesource.com/chromium/src/+/master/content/child/webcryp...

I agree that the breakdown for testing is fragmented and inconsistent at the
moment. My general idea has been:

----------------
  LayoutTests
----------------
  * End to end tests going through WebCrypto API
  * Test structured cloning
  * Test access from WebWorkers
  * Test some algorithm parsing failures
  * Tests imported from WebKit

----------------
  Chromium-side unit tests
----------------
  * Fuller algorithm-specific testing with known inputs/outputs (much more
carefully tests boundary cases)
  * Error testing (doing error tests on the Blink side is frustrating because
then when you want to change the errors it requires  complicated 3-side patch to
change the expectations and implementation)
  * Invalid data (especially badly formatted keys, wrong sized or corrupted
inputs)
  * Things hard or impossible to express in LayoutTests -- for instance tests
with buffers that are too large (can build fake buffers in C++ where the length
is close to say 31 bits, however no data was allocated, and make sure an error
happens rather than overflow or reading from the buffer.


One of the difficulties has been a desire to do atomic commits with both code
and tests. This has led to more of the testing being done on the Chromium-side
(as part of content/webcrypto) just by virtue of this being where the final
changelists to enable algorithms lives.

Having written both types of tests, my preference is actually for LayoutTests
over Unit tests. I would like to eventually move the majority of the testing to
being Javascript-based (whether W3C tests or LayoutTests):
  https://code.google.com/p/chromium/issues/detail?id=375569

The reason I prefer the JavaScript tests is:

  * Validates actual WebCrypto usage -- there are a number of layers in getting
from the Javascript through the Blink API, and do the openssl/nss
implementations that unit-tests alone are insufficient.

  * Easier to write. The C++ tests require tons of boiler plate to set up the
calls. This has in fact motivated me to expression more tests as lists of JSON
inputs/outputs (https://code.google.com/p/chromium/issues/detail?id=428941)

  * More readable than the equivalent C++ tests

  * Can be shared between WebCryto implementors. There is a desire to create a
W3C validation suite for WebCrypto, and having a shared set of tests would be
great.


The challenge with LayoutTests however is:
  * Split repositories
  * Harder to adjust for platform differences (notably OpenSSL vs NSS)


Let me know what your thinking is around this!

[eroman, 2014-11-14 17:32:36]

https://codereview.chromium.org/707753002/diff/40001/LayoutTests/crypto/ecdsa...
File LayoutTests/crypto/ecdsa-generateKey-parseAlgorithm-failures.html (right):

https://codereview.chromium.org/707753002/diff/40001/LayoutTests/crypto/ecdsa...
LayoutTests/crypto/ecdsa-generateKey-parseAlgorithm-failures.html:35: return
crypto.subtle.generateKey({name: 'ECDSA', namedCurve: 'P-256aa'}, extractable ,
keyUsages);
On 2014/11/14 08:18:32, jww wrote:
> Perhaps getting nit picky, but I'd love to see a lot more bad-parsing checks.
> For example, the base cases, bad prefixes, etc. This really applies more
> generally to the crypto tests, though.

Let me get back to you on this as a follow-up.

[eroman, 2014-11-14 17:32:39]

The CQ bit was checked by eroman@chromium.org

[commit-bot: I haz the power, 2014-11-14 17:33:43]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/707753002/40001

[commit-bot: I haz the power, 2014-11-14 17:54:31]

Committed patchset #3 (id:40001) as 185382