Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(135)

Side by Side Diff: json_reporter.md

Issue 1469863005: Add JSON protocol support for groups. (Closed) Base URL: git@github.com:dart-lang/test@master
Patch Set: Code review changes Created 5 years, 1 month ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
« no previous file with comments | « no previous file | json_reporter.schema.json » ('j') | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 JSON Reporter Protocol 1 JSON Reporter Protocol
2 ====================== 2 ======================
3 3
4 The test runner supports a JSON reporter which provides a machine-readable 4 The test runner supports a JSON reporter which provides a machine-readable
5 representation of the test runner's progress. This reporter is intended for use 5 representation of the test runner's progress. This reporter is intended for use
6 by IDEs and other tools to present a custom view of the test runner's operation 6 by IDEs and other tools to present a custom view of the test runner's operation
7 without needing to parse output intended for humans. 7 without needing to parse output intended for humans.
8 8
9 ## Usage 9 ## Usage
10 10
(...skipping 34 matching lines...) Expand 10 before | Expand all | Expand 10 after
45 ## Reading this Document 45 ## Reading this Document
46 46
47 Each major type of JSON object used by the protocol is described by a class. 47 Each major type of JSON object used by the protocol is described by a class.
48 Classes have names which are referred to in this document, but are not used as 48 Classes have names which are referred to in this document, but are not used as
49 part of the protocol. Classes have typed attributes, which refer to the types 49 part of the protocol. Classes have typed attributes, which refer to the types
50 and names of attributes in the JSON objects. If an attribute's type is another 50 and names of attributes in the JSON objects. If an attribute's type is another
51 class, that refers to a nested object. The special type `List<...>` indicates a 51 class, that refers to a nested object. The special type `List<...>` indicates a
52 JSON list of the given type. 52 JSON list of the given type.
53 53
54 Classes can "extend" one another, meaning that the subclass has all the 54 Classes can "extend" one another, meaning that the subclass has all the
55 properties of the superclass. Concrete subclasses can be distinguished by the 55 attributes of the superclass. Concrete subclasses can be distinguished by the
56 specific value of their `type` attribute. Classes may be abstract, indicating 56 specific value of their `type` attribute. Classes may be abstract, indicating
57 that only their subclasses will ever be used. 57 that only their subclasses will ever be used.
58 58
59 ## Events 59 ## Events
60 60
61 ### Event 61 ### Event
62 62
63 ``` 63 ```
64 abstract class Event { 64 abstract class Event {
65 // The type of the event. 65 // The type of the event.
(...skipping 22 matching lines...) Expand all
88 String protocolVersion; 88 String protocolVersion;
89 89
90 // The version of the test runner being used. 90 // The version of the test runner being used.
91 String runnerVersion; 91 String runnerVersion;
92 } 92 }
93 ``` 93 ```
94 94
95 A single start event is emitted before any other events. It indicates that the 95 A single start event is emitted before any other events. It indicates that the
96 test runner has started running. 96 test runner has started running.
97 97
98 ### GroupEvent
99
100 ```
101 class GroupEvent extends Event {
102 String type = "group";
103
104 /// Metadata about the group.
105 Group group;
106 }
107 ```
108
109 A group event is emitted before any `TestStartEvent`s for tests in a given
110 group. This is the only event that contains the full metadata about a group;
111 future events will refer to the group by its opaque ID.
112
113 This includes the implicit group at the root of each suite, which has a `null`
114 name. However, it does *not* include implicit groups for the virtual suites
115 generated to represent loading test files.
116
117 The group should be considered skipped if `group.metadata.skip` is `true`. When
118 a group is skipped, a single `TestStartEvent` will be emitted for a test within
119 that group that will also be skipped.
120
98 ### TestStartEvent 121 ### TestStartEvent
99 122
100 ``` 123 ```
101 class TestStartEvent extends Event { 124 class TestStartEvent extends Event {
102 String type = "testStart"; 125 String type = "testStart";
103 126
104 // Metadata about the test that started. 127 // Metadata about the test that started.
105 Test test; 128 Test test;
106 } 129 }
107 ``` 130 ```
(...skipping 94 matching lines...) Expand 10 before | Expand all | Expand 10 after
202 String type = "done"; 225 String type = "done";
203 226
204 // Whether all tests succeeded (or were skipped). 227 // Whether all tests succeeded (or were skipped).
205 bool success; 228 bool success;
206 } 229 }
207 ``` 230 ```
208 231
209 An event indicating the result of the entire test run. This will be the final 232 An event indicating the result of the entire test run. This will be the final
210 event emitted by the reporter. 233 event emitted by the reporter.
211 234
212 ## Test Information 235 ## Other Classes
213 236
214 ### Test 237 ### Test
215 238
216 ``` 239 ```
217 class Test { 240 class Test {
218 // An opaque ID for the test. 241 // An opaque ID for the test.
219 int id; 242 int id;
220 243
221 // The name of the test, including prefixes from any containing groups. 244 // The name of the test, including prefixes from any containing groups.
222 String name; 245 String name;
223 246
224 // The test's metadata, including metadata from any containing groups and the 247 // The IDs of groups containing this test, in order from outermost to
225 // test suite itself. 248 // innermost.
249 List<int> groupIDs;
250
251 // The test's metadata, including metadata from any containing groups.
226 Metadata metadata; 252 Metadata metadata;
227 } 253 }
228 ``` 254 ```
229 255
230 A single test case. The test's ID is unique in the context of this test run. 256 A single test case. The test's ID is unique in the context of this test run.
231 It's used elsewhere in the protocol to refer to this test without including its 257 It's used elsewhere in the protocol to refer to this test without including its
232 full representation. 258 full representation.
233 259
260 Most tests will have at least one group ID, representing the implicit root
261 group. However, some may not; these should be treated as having no group
262 metadata.
263
264 ### Group
265
266 ```
267 class Group {
268 // An opaque ID for the group.
269 int id;
270
271 // The name of the group, including prefixes from any containing groups.
272 String? name;
273
274 // The ID of the group's parent group, unless it's the root group.
275 int? parentID;
276
277 // The group's metadata, including metadata from any containing groups.
278 Metadata metadata;
279 }
280 ```
281
282 A group containing test cases. The group's ID is unique in the context of this
283 test run. It's used elsewhere in the protocol to refer to this test without
284 including its full representation.
285
286 The implicit group at the root of each test suite has `null` `name` and
287 `parentID` attributes.
288
234 ### Metadata 289 ### Metadata
235 290
236 ``` 291 ```
237 class Metadata { 292 class Metadata {
238 // Whether the test case will be skipped by the test runner. 293 // Whether the test case will be skipped by the test runner.
239 bool skip; 294 bool skip;
240 295
241 // The reason the test case is skipped, if the user provided it. 296 // The reason the test case is skipped, if the user provided it.
242 String? skipReason; 297 String? skipReason;
243 } 298 }
244 ``` 299 ```
245 300
246 The metadata attached to a test by a user. 301 The metadata attached to a test by a user.
OLDNEW
« no previous file with comments | « no previous file | json_reporter.schema.json » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698