OLD | NEW |
1 JSON Reporter Protocol | 1 JSON Reporter Protocol |
2 ====================== | 2 ====================== |
3 | 3 |
4 The test runner supports a JSON reporter which provides a machine-readable | 4 The test runner supports a JSON reporter which provides a machine-readable |
5 representation of the test runner's progress. This reporter is intended for use | 5 representation of the test runner's progress. This reporter is intended for use |
6 by IDEs and other tools to present a custom view of the test runner's operation | 6 by IDEs and other tools to present a custom view of the test runner's operation |
7 without needing to parse output intended for humans. | 7 without needing to parse output intended for humans. |
8 | 8 |
9 ## Usage | 9 ## Usage |
10 | 10 |
(...skipping 34 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
45 ## Reading this Document | 45 ## Reading this Document |
46 | 46 |
47 Each major type of JSON object used by the protocol is described by a class. | 47 Each major type of JSON object used by the protocol is described by a class. |
48 Classes have names which are referred to in this document, but are not used as | 48 Classes have names which are referred to in this document, but are not used as |
49 part of the protocol. Classes have typed attributes, which refer to the types | 49 part of the protocol. Classes have typed attributes, which refer to the types |
50 and names of attributes in the JSON objects. If an attribute's type is another | 50 and names of attributes in the JSON objects. If an attribute's type is another |
51 class, that refers to a nested object. The special type `List<...>` indicates a | 51 class, that refers to a nested object. The special type `List<...>` indicates a |
52 JSON list of the given type. | 52 JSON list of the given type. |
53 | 53 |
54 Classes can "extend" one another, meaning that the subclass has all the | 54 Classes can "extend" one another, meaning that the subclass has all the |
55 properties of the superclass. Concrete subclasses can be distinguished by the | 55 attributes of the superclass. Concrete subclasses can be distinguished by the |
56 specific value of their `type` attribute. Classes may be abstract, indicating | 56 specific value of their `type` attribute. Classes may be abstract, indicating |
57 that only their subclasses will ever be used. | 57 that only their subclasses will ever be used. |
58 | 58 |
59 ## Events | 59 ## Events |
60 | 60 |
61 ### Event | 61 ### Event |
62 | 62 |
63 ``` | 63 ``` |
64 abstract class Event { | 64 abstract class Event { |
65 // The type of the event. | 65 // The type of the event. |
(...skipping 22 matching lines...) Expand all Loading... |
88 String protocolVersion; | 88 String protocolVersion; |
89 | 89 |
90 // The version of the test runner being used. | 90 // The version of the test runner being used. |
91 String runnerVersion; | 91 String runnerVersion; |
92 } | 92 } |
93 ``` | 93 ``` |
94 | 94 |
95 A single start event is emitted before any other events. It indicates that the | 95 A single start event is emitted before any other events. It indicates that the |
96 test runner has started running. | 96 test runner has started running. |
97 | 97 |
| 98 ### GroupEvent |
| 99 |
| 100 ``` |
| 101 class GroupEvent extends Event { |
| 102 String type = "group"; |
| 103 |
| 104 /// Metadata about the group. |
| 105 Group group; |
| 106 } |
| 107 ``` |
| 108 |
| 109 A group event is emitted before any `TestStartEvent`s for tests in a given |
| 110 group. This is the only event that contains the full metadata about a group; |
| 111 future events will refer to the group by its opaque ID. |
| 112 |
| 113 This includes the implicit group at the root of each suite, which has a `null` |
| 114 name. However, it does *not* include implicit groups for the virtual suites |
| 115 generated to represent loading test files. |
| 116 |
| 117 The group should be considered skipped if `group.metadata.skip` is `true`. When |
| 118 a group is skipped, a single `TestStartEvent` will be emitted for a test within |
| 119 that group that will also be skipped. |
| 120 |
98 ### TestStartEvent | 121 ### TestStartEvent |
99 | 122 |
100 ``` | 123 ``` |
101 class TestStartEvent extends Event { | 124 class TestStartEvent extends Event { |
102 String type = "testStart"; | 125 String type = "testStart"; |
103 | 126 |
104 // Metadata about the test that started. | 127 // Metadata about the test that started. |
105 Test test; | 128 Test test; |
106 } | 129 } |
107 ``` | 130 ``` |
(...skipping 94 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
202 String type = "done"; | 225 String type = "done"; |
203 | 226 |
204 // Whether all tests succeeded (or were skipped). | 227 // Whether all tests succeeded (or were skipped). |
205 bool success; | 228 bool success; |
206 } | 229 } |
207 ``` | 230 ``` |
208 | 231 |
209 An event indicating the result of the entire test run. This will be the final | 232 An event indicating the result of the entire test run. This will be the final |
210 event emitted by the reporter. | 233 event emitted by the reporter. |
211 | 234 |
212 ## Test Information | 235 ## Other Classes |
213 | 236 |
214 ### Test | 237 ### Test |
215 | 238 |
216 ``` | 239 ``` |
217 class Test { | 240 class Test { |
218 // An opaque ID for the test. | 241 // An opaque ID for the test. |
219 int id; | 242 int id; |
220 | 243 |
221 // The name of the test, including prefixes from any containing groups. | 244 // The name of the test, including prefixes from any containing groups. |
222 String name; | 245 String name; |
223 | 246 |
224 // The test's metadata, including metadata from any containing groups and the | 247 // The IDs of groups containing this test, in order from outermost to |
225 // test suite itself. | 248 // innermost. |
| 249 List<int> groupIDs; |
| 250 |
| 251 // The test's metadata, including metadata from any containing groups. |
226 Metadata metadata; | 252 Metadata metadata; |
227 } | 253 } |
228 ``` | 254 ``` |
229 | 255 |
230 A single test case. The test's ID is unique in the context of this test run. | 256 A single test case. The test's ID is unique in the context of this test run. |
231 It's used elsewhere in the protocol to refer to this test without including its | 257 It's used elsewhere in the protocol to refer to this test without including its |
232 full representation. | 258 full representation. |
233 | 259 |
| 260 Most tests will have at least one group ID, representing the implicit root |
| 261 group. However, some may not; these should be treated as having no group |
| 262 metadata. |
| 263 |
| 264 ### Group |
| 265 |
| 266 ``` |
| 267 class Group { |
| 268 // An opaque ID for the group. |
| 269 int id; |
| 270 |
| 271 // The name of the group, including prefixes from any containing groups. |
| 272 String? name; |
| 273 |
| 274 // The ID of the group's parent group, unless it's the root group. |
| 275 int? parentID; |
| 276 |
| 277 // The group's metadata, including metadata from any containing groups. |
| 278 Metadata metadata; |
| 279 } |
| 280 ``` |
| 281 |
| 282 A group containing test cases. The group's ID is unique in the context of this |
| 283 test run. It's used elsewhere in the protocol to refer to this test without |
| 284 including its full representation. |
| 285 |
| 286 The implicit group at the root of each test suite has `null` `name` and |
| 287 `parentID` attributes. |
| 288 |
234 ### Metadata | 289 ### Metadata |
235 | 290 |
236 ``` | 291 ``` |
237 class Metadata { | 292 class Metadata { |
238 // Whether the test case will be skipped by the test runner. | 293 // Whether the test case will be skipped by the test runner. |
239 bool skip; | 294 bool skip; |
240 | 295 |
241 // The reason the test case is skipped, if the user provided it. | 296 // The reason the test case is skipped, if the user provided it. |
242 String? skipReason; | 297 String? skipReason; |
243 } | 298 } |
244 ``` | 299 ``` |
245 | 300 |
246 The metadata attached to a test by a user. | 301 The metadata attached to a test by a user. |
OLD | NEW |