docs/testing/web_platform_tests.md - Issue 2717643002: Call out the importance of the --work argument for wpt manifest

Side by Side Diff: docs/testing/web_platform_tests.md

Issue 2717643002: Call out the importance of the --work argument for wpt manifest (Closed)

Patch Set: Created 3 years, 9 months ago

Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.

Jump to:

« no previous file with comments | « no previous file | no next file » | no next file with comments »

OLD	NEW
1 # web-platform-tests	1 # web-platform-tests

2	2

3 Interoperability between browsers is	3 Interoperability between browsers is

4 [critical](https://www.chromium.org/blink/platform-predictability) to Chromium's	4 [critical](https://www.chromium.org/blink/platform-predictability) to Chromium's

5 mission of improving the web. We believe that leveraging and contributing to a	5 mission of improving the web. We believe that leveraging and contributing to a

6 shared test suite is one of the most important tools in achieving	6 shared test suite is one of the most important tools in achieving

7 interoperability between browsers. The [web-platform-tests	7 interoperability between browsers. The [web-platform-tests

8 repository](https://github.com/w3c/web-platform-tests) is the primary shared	8 repository](https://github.com/w3c/web-platform-tests) is the primary shared

9 test suite where all browser engines are collaborating. There's also a	9 test suite where all browser engines are collaborating. There's also a

10 [csswg-test repository](https://github.com/w3c/csswg-test) for CSS tests, but	10 [csswg-test repository](https://github.com/w3c/csswg-test) for CSS tests, but

(...skipping 83 matching lines...) Expand 10 before \| Expand all \| Expand 10 after Loading...
94	94

95 Note: if you're adding a new test in `external/wpt`, you'll need to re-generate	95 Note: if you're adding a new test in `external/wpt`, you'll need to re-generate

96 MANIFEST.json manually until [CL 2644783003](https://crrev.com/2644783003) is	96 MANIFEST.json manually until [CL 2644783003](https://crrev.com/2644783003) is

97 landed. The command to do so is:	97 landed. The command to do so is:

98	98

99 ```bash	99 ```bash

100 Tools/Scripts/webkitpy/thirdparty/wpt/wpt/manifest --work \	100 Tools/Scripts/webkitpy/thirdparty/wpt/wpt/manifest --work \

101 --tests-root=LayoutTests/external/wpt	101 --tests-root=LayoutTests/external/wpt

102 ```	102 ```

103	103

	104 Note: the `--work` argument is important even if you have no local changes.

	105 Without it the `manifest` script will try to generate a manifest for all files

	106 in the Chromium tree and take a very long time.

	107 See [wpt-tools issue #171](https://github.com/w3c/wpt-tools/issues/171).

	108

104 Most tests are written using testharness.js, see	109 Most tests are written using testharness.js, see

105 [Writing Layout Tests](./writing_layout_tests.md) and	110 [Writing Layout Tests](./writing_layout_tests.md) and

106 [Layout Tests Tips](./layout_tests_tips.md) for general guidelines.	111 [Layout Tests Tips](./layout_tests_tips.md) for general guidelines.

107	112

108 ### Write tests against specifications	113 ### Write tests against specifications

109	114

110 Tests in web-platform-tests are expected to match behavior defined by the	115 Tests in web-platform-tests are expected to match behavior defined by the

111 relevant specification. In other words, all assertions that a test makes	116 relevant specification. In other words, all assertions that a test makes

112 should be derived from a specification's normative requirements, and not go	117 should be derived from a specification's normative requirements, and not go

113 beyond them. It is often necessary to change the specification to clarify what	118 beyond them. It is often necessary to change the specification to clarify what

(...skipping 54 matching lines...) Expand 10 before \| Expand all \| Expand 10 after Loading...
168 that will be automatically upstreamed. There will be no additional review in	173 that will be automatically upstreamed. There will be no additional review in

169 web-platform-tests as part of the export process.	174 web-platform-tests as part of the export process.

170	175

171 If upstream reviewers have feedback on the changes, discuss on the pull request	176 If upstream reviewers have feedback on the changes, discuss on the pull request

172 created during export, and if necessary work on a new pull request to iterate	177 created during export, and if necessary work on a new pull request to iterate

173 until everyone is satisfied.	178 until everyone is satisfied.

174	179

175 When reviewing tests, check that they match the relevant specification, which	180 When reviewing tests, check that they match the relevant specification, which

176 may not fully match the implementation. See also	181 may not fully match the implementation. See also

177 [Write tests against specifications](#Write-tests-against-specifications).	182 [Write tests against specifications](#Write-tests-against-specifications).

OLD	NEW