Chromium Code Reviews| Index: docs/testing/web_platform_tests.md |
| diff --git a/docs/testing/web_platform_tests.md b/docs/testing/web_platform_tests.md |
| new file mode 100644 |
| index 0000000000000000000000000000000000000000..c390fba512a06d8518a3db64c0e864af94ba8400 |
| --- /dev/null |
| +++ b/docs/testing/web_platform_tests.md |
| @@ -0,0 +1,103 @@ |
| +# Web Platform Tests |
| + |
| +The W3C provides a large number of conformance tests for different aspects of |
|
foolip
2017/01/29 10:21:47
Nit: s/W3C/Web Platform Tests project/
qyearsley
2017/01/29 19:44:41
Done
|
| +the Web Platform. Currently the tests are hosted on GitHub. There are two main |
| +repositories, one for the CSS Working Group (aka csswg-test), and one for pretty |
| +much everything else (aka web-platform-tests). There is a plan to merge |
| +csswg-test into web-platform-tests, so later, there will be only one repository. |
| + |
| +[TOC] |
| + |
| +## Import |
| + |
| +Chromium has mirrors (web-platform-tests, csswg-test) of the GitHub repos, and |
| +periodically imports a subset of the tests so that they are run as part of the |
| +regular Blink layout test testing process. |
| + |
| +Note that currently the main reason we do not run more of the W3C tests is |
| +because they are (probably) mostly redundant with Blink's existing tests, and so |
|
foolip
2017/01/29 10:21:47
I don't think this is the reason really. Although
qyearsley
2017/01/29 19:44:42
This was just copied over from the earlier sites p
|
| +we would double the running time of the layout tests for little benefit. Ideally |
| +we would identify the tests that are redundant and remove Blink's copies, so |
| +that we run just the W3C tests where possible. |
| + |
| +The end goals for this whole process are to: Be able to run the W3C tests |
| +unmodified locally just as easily as we can run the Blink tests Ensure that we |
|
foolip
2017/01/29 10:21:47
Missing punctuation, or was this a bullet list?
qyearsley
2017/01/29 19:44:41
This was a numbered list, but I think it can be a
|
| +are tracking tip-of-tree in the W3C repositories as closely as possible Run as |
| +many of the W3C tests as possible |
| + |
| +### Automatic Import Process |
| + |
| +There is an automatic [w3c-test-autoroller |
| +builder](https://build.chromium.org/p/chromium.infra.cron/builders/w3c-test-autoroller) |
| +for regularly updating our local copies of web-platform-tests. |
| + |
| +```bash |
| +Tools/Scripts/wpt-import --auto-import wpt |
| +Tools/Scripts/wpt-import --auto-import css |
| +``` |
| + |
| +### Importing New W3C Tests |
| + |
| +We control which tests are imported via `LayoutTests/W3CImportExpectations`, |
| +which has a list of directories to skip during import. This means that any new |
| +tests and directories that show up in the W3C repos are automatically imported. |
| + |
| +To pull the latest versions of the tests that are currently being imported |
| +(i.e., you don't need to change the blacklist), all you have to do is run: |
|
foolip
2017/01/29 10:21:47
Suggest blocklist, or the filename itself.
qyearsley
2017/01/29 19:44:41
Now changed to just have the filename of the scrip
|
| + |
| +```bash |
| +Tools/Scripts/wpt-import wpt |
| +Tools/Scripts/wpt-import css |
| +``` |
| + |
| +That script will pull the latest version of the tests from our mirrors of the |
| +W3C repos. If any new versions of tests are found, they will be committed |
| +locally to your local repository. You may then upload the changes. |
| + |
| +If you wish to add more tests (by un-skipping some of the directories currently |
| +skipped in `W3CImportExpectations`), you can modify that file locally and commit |
| +it, and on the next auto-import, the new tests should be imported. If you want |
| +to import immediately, you can also run `wpt-import --allow-local-commits`. |
| + |
| +## Contributing Blink tests back to the W3C |
| + |
| +If you need to make changes to Web Platform Tests, just commit your changes |
| +directly to our version in `LayoutTests/external/wpt` and the changes will be |
| +automatically upstreamed within 24 hours. |
| + |
| +Note: if you're adding a new test in `external/wpt`, you'll need to re-generate |
| +MANIFEST.json manually until [CL 2644783003](https://crrev.com/2644783003) is |
| +landed. The command to do so is: |
| + |
| +```bash |
| +Tools/Scripts/webkitpy/thirdparty/wpt/wpt/manifest --work \ |
| + --tests-root=LayoutTests/external/wpt |
| +``` |
| + |
| +### Where can I find the code for the WPT import and export tools? |
| + |
| +- Exporter: |
| + [Tools/Scripts/wpt-export](../../third_party/WebKit/Tools/Scripts/wpt-export) |
| +- Importer: |
| + [Tools/Scripts/wpt-import](../../third_party/WebKit/Tools/Scripts/wpt-import) |
| +- Libraries: |
| + [Tools/Scripts/webkitpy/w3c](../../third_party/WebKit/Tools/Scripts/webkitpy/w3c) |
| + |
| +### Will the exported commits be linked to my GitHub profile? |
| + |
| +The email you commit with (e.g. user@chromium.org) will be the author of the |
| +commit on GitHub. You can add it as a secondary address on your GitHub account |
| +to link your exported commits to your GitHub profile. What if there are |
| +conflicts? |
|
foolip
2017/01/29 10:21:47
A link to GitHub went missing here, and missing se
qyearsley
2017/01/29 19:44:42
Fixed.
|
| + |
| +This cannot be avoided entirely as the two repositories are independent, but |
| +should be rare with frequent imports and exports. When it does happen, manual |
| +intervention will be needed and in non-trivial cases you may be asked to help |
| +resolve the conflict. |
| + |
| +### Direct pull requests |
| + |
| +It's still possible to make direct pull requests to web-platform-tests. The |
| +processes for getting new tests committed the W3C repos are at |
| +[http://testthewebforward.org/docs/]. Some specifics are at |
| +[http://testthewebforward.org/docs/github-101.html]. |