Chromium Code Reviews| OLD | NEW |
|---|---|
| (Empty) | |
| 1 # Web Platform Tests | |
| 2 | |
| 3 The W3C provides a large number of conformance tests for different aspects of | |
|
foolip
2017/01/29 10:21:47
Nit: s/W3C/Web Platform Tests project/
qyearsley
2017/01/29 19:44:41
Done
| |
| 4 the Web Platform. Currently the tests are hosted on GitHub. There are two main | |
| 5 repositories, one for the CSS Working Group (aka csswg-test), and one for pretty | |
| 6 much everything else (aka web-platform-tests). There is a plan to merge | |
| 7 csswg-test into web-platform-tests, so later, there will be only one repository. | |
| 8 | |
| 9 [TOC] | |
| 10 | |
| 11 ## Import | |
| 12 | |
| 13 Chromium has mirrors (web-platform-tests, csswg-test) of the GitHub repos, and | |
| 14 periodically imports a subset of the tests so that they are run as part of the | |
| 15 regular Blink layout test testing process. | |
| 16 | |
| 17 Note that currently the main reason we do not run more of the W3C tests is | |
| 18 because they are (probably) mostly redundant with Blink's existing tests, and so | |
|
foolip
2017/01/29 10:21:47
I don't think this is the reason really. Although
qyearsley
2017/01/29 19:44:42
This was just copied over from the earlier sites p
| |
| 19 we would double the running time of the layout tests for little benefit. Ideally | |
| 20 we would identify the tests that are redundant and remove Blink's copies, so | |
| 21 that we run just the W3C tests where possible. | |
| 22 | |
| 23 The end goals for this whole process are to: Be able to run the W3C tests | |
| 24 unmodified locally just as easily as we can run the Blink tests Ensure that we | |
|
foolip
2017/01/29 10:21:47
Missing punctuation, or was this a bullet list?
qyearsley
2017/01/29 19:44:41
This was a numbered list, but I think it can be a
| |
| 25 are tracking tip-of-tree in the W3C repositories as closely as possible Run as | |
| 26 many of the W3C tests as possible | |
| 27 | |
| 28 ### Automatic Import Process | |
| 29 | |
| 30 There is an automatic [w3c-test-autoroller | |
| 31 builder](https://build.chromium.org/p/chromium.infra.cron/builders/w3c-test-auto roller) | |
| 32 for regularly updating our local copies of web-platform-tests. | |
| 33 | |
| 34 ```bash | |
| 35 Tools/Scripts/wpt-import --auto-import wpt | |
| 36 Tools/Scripts/wpt-import --auto-import css | |
| 37 ``` | |
| 38 | |
| 39 ### Importing New W3C Tests | |
| 40 | |
| 41 We control which tests are imported via `LayoutTests/W3CImportExpectations`, | |
| 42 which has a list of directories to skip during import. This means that any new | |
| 43 tests and directories that show up in the W3C repos are automatically imported. | |
| 44 | |
| 45 To pull the latest versions of the tests that are currently being imported | |
| 46 (i.e., you don't need to change the blacklist), all you have to do is run: | |
|
foolip
2017/01/29 10:21:47
Suggest blocklist, or the filename itself.
qyearsley
2017/01/29 19:44:41
Now changed to just have the filename of the scrip
| |
| 47 | |
| 48 ```bash | |
| 49 Tools/Scripts/wpt-import wpt | |
| 50 Tools/Scripts/wpt-import css | |
| 51 ``` | |
| 52 | |
| 53 That script will pull the latest version of the tests from our mirrors of the | |
| 54 W3C repos. If any new versions of tests are found, they will be committed | |
| 55 locally to your local repository. You may then upload the changes. | |
| 56 | |
| 57 If you wish to add more tests (by un-skipping some of the directories currently | |
| 58 skipped in `W3CImportExpectations`), you can modify that file locally and commit | |
| 59 it, and on the next auto-import, the new tests should be imported. If you want | |
| 60 to import immediately, you can also run `wpt-import --allow-local-commits`. | |
| 61 | |
| 62 ## Contributing Blink tests back to the W3C | |
| 63 | |
| 64 If you need to make changes to Web Platform Tests, just commit your changes | |
| 65 directly to our version in `LayoutTests/external/wpt` and the changes will be | |
| 66 automatically upstreamed within 24 hours. | |
| 67 | |
| 68 Note: if you're adding a new test in `external/wpt`, you'll need to re-generate | |
| 69 MANIFEST.json manually until [CL 2644783003](https://crrev.com/2644783003) is | |
| 70 landed. The command to do so is: | |
| 71 | |
| 72 ```bash | |
| 73 Tools/Scripts/webkitpy/thirdparty/wpt/wpt/manifest --work \ | |
| 74 --tests-root=LayoutTests/external/wpt | |
| 75 ``` | |
| 76 | |
| 77 ### Where can I find the code for the WPT import and export tools? | |
| 78 | |
| 79 - Exporter: | |
| 80 [Tools/Scripts/wpt-export](../../third_party/WebKit/Tools/Scripts/wpt-export ) | |
| 81 - Importer: | |
| 82 [Tools/Scripts/wpt-import](../../third_party/WebKit/Tools/Scripts/wpt-import ) | |
| 83 - Libraries: | |
| 84 [Tools/Scripts/webkitpy/w3c](../../third_party/WebKit/Tools/Scripts/webkitpy /w3c) | |
| 85 | |
| 86 ### Will the exported commits be linked to my GitHub profile? | |
| 87 | |
| 88 The email you commit with (e.g. user@chromium.org) will be the author of the | |
| 89 commit on GitHub. You can add it as a secondary address on your GitHub account | |
| 90 to link your exported commits to your GitHub profile. What if there are | |
| 91 conflicts? | |
|
foolip
2017/01/29 10:21:47
A link to GitHub went missing here, and missing se
qyearsley
2017/01/29 19:44:42
Fixed.
| |
| 92 | |
| 93 This cannot be avoided entirely as the two repositories are independent, but | |
| 94 should be rare with frequent imports and exports. When it does happen, manual | |
| 95 intervention will be needed and in non-trivial cases you may be asked to help | |
| 96 resolve the conflict. | |
| 97 | |
| 98 ### Direct pull requests | |
| 99 | |
| 100 It's still possible to make direct pull requests to web-platform-tests. The | |
| 101 processes for getting new tests committed the W3C repos are at | |
| 102 [http://testthewebforward.org/docs/]. Some specifics are at | |
| 103 [http://testthewebforward.org/docs/github-101.html]. | |
| OLD | NEW |