| OLD | NEW |
| 1 JSON Reporter Protocol | 1 JSON Reporter Protocol |
| 2 ====================== | 2 ====================== |
| 3 | 3 |
| 4 The test runner supports a JSON reporter which provides a machine-readable | 4 The test runner supports a JSON reporter which provides a machine-readable |
| 5 representation of the test runner's progress. This reporter is intended for use | 5 representation of the test runner's progress. This reporter is intended for use |
| 6 by IDEs and other tools to present a custom view of the test runner's operation | 6 by IDEs and other tools to present a custom view of the test runner's operation |
| 7 without needing to parse output intended for humans. | 7 without needing to parse output intended for humans. |
| 8 | 8 |
| 9 Note that the test runner is highly asynchronous, and users of this protocol | 9 Note that the test runner is highly asynchronous, and users of this protocol |
| 10 shouldn't make assumptions about the ordering of events beyond what's explicitly | 10 shouldn't make assumptions about the ordering of events beyond what's explicitly |
| (...skipping 133 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
| 144 ``` | 144 ``` |
| 145 | 145 |
| 146 A group event is emitted before any `TestStartEvent`s for tests in a given | 146 A group event is emitted before any `TestStartEvent`s for tests in a given |
| 147 group. This is the only event that contains the full metadata about a group; | 147 group. This is the only event that contains the full metadata about a group; |
| 148 future events will refer to the group by its opaque ID. | 148 future events will refer to the group by its opaque ID. |
| 149 | 149 |
| 150 This includes the implicit group at the root of each suite, which has a `null` | 150 This includes the implicit group at the root of each suite, which has a `null` |
| 151 name. However, it does *not* include implicit groups for the virtual suites | 151 name. However, it does *not* include implicit groups for the virtual suites |
| 152 generated to represent loading test files. | 152 generated to represent loading test files. |
| 153 | 153 |
| 154 The group should be considered skipped if `group.metadata.skip` is `true`. When | 154 If the group is skipped, a single `TestStartEvent` will be emitted for a test |
| 155 a group is skipped, a single `TestStartEvent` will be emitted for a test within | 155 within the group, followed by a `TestDoneEvent` marked as skipped. The |
| 156 that group that will also be skipped. | 156 `group.metadata.skip` field should *not* be considered authoritative for |
| 157 determining whether a group is skipped. |
| 157 | 158 |
| 158 ### TestStartEvent | 159 ### TestStartEvent |
| 159 | 160 |
| 160 ``` | 161 ``` |
| 161 class TestStartEvent extends Event { | 162 class TestStartEvent extends Event { |
| 162 String type = "testStart"; | 163 String type = "testStart"; |
| 163 | 164 |
| 164 // Metadata about the test that started. | 165 // Metadata about the test that started. |
| 165 Test test; | 166 Test test; |
| 166 } | 167 } |
| 167 ``` | 168 ``` |
| 168 | 169 |
| 169 An event emitted when a test begins running. This is the only event that | 170 An event emitted when a test begins running. This is the only event that |
| 170 contains the full metadata about a test; future events will refer to the test by | 171 contains the full metadata about a test; future events will refer to the test by |
| 171 its opaque ID. | 172 its opaque ID. |
| 172 | 173 |
| 173 The test should be considered skipped if `test.metadata.skip` is `true`. | 174 If the test is skipped, its `TestDoneEvent` will have `skipped` set to `true`. |
| 175 The `test.metadata.skip` field should *not* be considered authoritative for |
| 176 determining whether a test is skipped. |
| 174 | 177 |
| 175 ### PrintEvent | 178 ### PrintEvent |
| 176 | 179 |
| 177 ``` | 180 ``` |
| 178 class PrintEvent extends Event { | 181 class PrintEvent extends Event { |
| 179 String type = "print"; | 182 String type = "print"; |
| 180 | 183 |
| 181 // The ID of the test that printed a message. | 184 // The ID of the test that printed a message. |
| 182 int testID; | 185 int testID; |
| 183 | 186 |
| (...skipping 39 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
| 223 String type = "testDone"; | 226 String type = "testDone"; |
| 224 | 227 |
| 225 // The ID of the test that completed. | 228 // The ID of the test that completed. |
| 226 int testID; | 229 int testID; |
| 227 | 230 |
| 228 // The result of the test. | 231 // The result of the test. |
| 229 String result; | 232 String result; |
| 230 | 233 |
| 231 // Whether the test's result should be hidden. | 234 // Whether the test's result should be hidden. |
| 232 bool hidden; | 235 bool hidden; |
| 236 |
| 237 // Whether the test (or some part of it) was skipped. |
| 238 bool skipped; |
| 233 } | 239 } |
| 234 ``` | 240 ``` |
| 235 | 241 |
| 236 An event emitted when a test completes. The `result` attribute indicates the | 242 An event emitted when a test completes. The `result` attribute indicates the |
| 237 result of the test: | 243 result of the test: |
| 238 | 244 |
| 239 * `"success"` if the test had no errors. | 245 * `"success"` if the test had no errors. |
| 240 | 246 |
| 241 * `"failure"` if the test had a `TestFailure` but no other errors. | 247 * `"failure"` if the test had a `TestFailure` but no other errors. |
| 242 | 248 |
| (...skipping 150 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
| 393 class Metadata { | 399 class Metadata { |
| 394 // Whether the test case will be skipped by the test runner. | 400 // Whether the test case will be skipped by the test runner. |
| 395 bool skip; | 401 bool skip; |
| 396 | 402 |
| 397 // The reason the test case is skipped, if the user provided it. | 403 // The reason the test case is skipped, if the user provided it. |
| 398 String? skipReason; | 404 String? skipReason; |
| 399 } | 405 } |
| 400 ``` | 406 ``` |
| 401 | 407 |
| 402 The metadata attached to a test by a user. | 408 The metadata attached to a test by a user. |
| 409 |
| 410 Note that the `skip` field should not be considered authoritative. A test may be |
| 411 skipped even if `skip` is set to `false`. |
| OLD | NEW |