Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(251)

Side by Side Diff: json_reporter.md

Issue 1637193003: Add suite information to the JSON reporter. (Closed) Base URL: git@github.com:dart-lang/test@master
Patch Set: fix changelog Created 4 years, 11 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
« no previous file with comments | « CHANGELOG.md ('k') | json_reporter.schema.json » ('j') | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 JSON Reporter Protocol 1 JSON Reporter Protocol
2 ====================== 2 ======================
3 3
4 The test runner supports a JSON reporter which provides a machine-readable 4 The test runner supports a JSON reporter which provides a machine-readable
5 representation of the test runner's progress. This reporter is intended for use 5 representation of the test runner's progress. This reporter is intended for use
6 by IDEs and other tools to present a custom view of the test runner's operation 6 by IDEs and other tools to present a custom view of the test runner's operation
7 without needing to parse output intended for humans. 7 without needing to parse output intended for humans.
8 8
9 Note that the test runner is highly asynchronous, and users of this protocol 9 Note that the test runner is highly asynchronous, and users of this protocol
10 shouldn't make assumptions about the ordering of events beyond what's explicitly 10 shouldn't make assumptions about the ordering of events beyond what's explicitly
(...skipping 83 matching lines...) Expand 10 before | Expand all | Expand 10 after
94 String protocolVersion; 94 String protocolVersion;
95 95
96 // The version of the test runner being used. 96 // The version of the test runner being used.
97 String runnerVersion; 97 String runnerVersion;
98 } 98 }
99 ``` 99 ```
100 100
101 A single start event is emitted before any other events. It indicates that the 101 A single start event is emitted before any other events. It indicates that the
102 test runner has started running. 102 test runner has started running.
103 103
104 ### SuiteEvent
105
106 ```
107 class SuiteEvent extends Event {
108 String type = "suite";
109
110 /// Metadata about the suite.
111 Suite suite;
112 }
113 ```
114
115 A suite event is emitted before any `GroupEvent`s for groups in a given test
116 suite. This is the only event that contains the full metadata about a suite;
117 future events will refer to the suite by its opaque ID.
118
104 ### GroupEvent 119 ### GroupEvent
105 120
106 ``` 121 ```
107 class GroupEvent extends Event { 122 class GroupEvent extends Event {
108 String type = "group"; 123 String type = "group";
109 124
110 /// Metadata about the group. 125 /// Metadata about the group.
111 Group group; 126 Group group;
112 } 127 }
113 ``` 128 ```
(...skipping 129 matching lines...) Expand 10 before | Expand all | Expand 10 after
243 ### Test 258 ### Test
244 259
245 ``` 260 ```
246 class Test { 261 class Test {
247 // An opaque ID for the test. 262 // An opaque ID for the test.
248 int id; 263 int id;
249 264
250 // The name of the test, including prefixes from any containing groups. 265 // The name of the test, including prefixes from any containing groups.
251 String name; 266 String name;
252 267
268 // The ID of the suite containing this test.
269 int suiteID;
270
253 // The IDs of groups containing this test, in order from outermost to 271 // The IDs of groups containing this test, in order from outermost to
254 // innermost. 272 // innermost.
255 List<int> groupIDs; 273 List<int> groupIDs;
256 274
257 // The test's metadata, including metadata from any containing groups. 275 // The test's metadata, including metadata from any containing groups.
258 Metadata metadata; 276 Metadata metadata;
259 } 277 }
260 ``` 278 ```
261 279
262 A single test case. The test's ID is unique in the context of this test run. 280 A single test case. The test's ID is unique in the context of this test run.
263 It's used elsewhere in the protocol to refer to this test without including its 281 It's used elsewhere in the protocol to refer to this test without including its
264 full representation. 282 full representation.
265 283
266 Most tests will have at least one group ID, representing the implicit root 284 Most tests will have at least one group ID, representing the implicit root
267 group. However, some may not; these should be treated as having no group 285 group. However, some may not; these should be treated as having no group
268 metadata. 286 metadata.
269 287
288 ### Suite
289
290 ```
291 class Suite {
292 // An opaque ID for the group.
293 int id;
294
295 // The platform on which the suite is running.
296 String? platform;
297
298 // The path to the suite's file.
299 String path;
300 }
301 ```
302
303 A test suite corresponding to a loaded test file. The suite's ID is unique in
304 the context of this test run. It's used elsewhere in the protocol to refer to
305 this suite without including its full representation.
306
307 A suite's platform is one of the platforms that can be passed to the
308 `--platform` option, or `null` if there is no platform (for example if the file
309 doesn't exist at all). Its path is relative to the root of the current package.
alexander.doroshko 2016/02/11 14:26:43 Spec says that Suite.path is relative to the curre
nweiz 2016/02/12 00:58:25 Good to know. I put out https://codereview.chromiu
310
311 Suites don't include their own metadata. Instead, that metadata is present on
312 the root-level group.
313
270 ### Group 314 ### Group
271 315
272 ``` 316 ```
273 class Group { 317 class Group {
274 // An opaque ID for the group. 318 // An opaque ID for the group.
275 int id; 319 int id;
276 320
277 // The name of the group, including prefixes from any containing groups. 321 // The name of the group, including prefixes from any containing groups.
278 String? name; 322 String? name;
279 323
324 // The ID of the suite containing this group.
325 int suiteID;
326
280 // The ID of the group's parent group, unless it's the root group. 327 // The ID of the group's parent group, unless it's the root group.
281 int? parentID; 328 int? parentID;
282 329
283 // The group's metadata, including metadata from any containing groups. 330 // The group's metadata, including metadata from any containing groups.
284 Metadata metadata; 331 Metadata metadata;
285 } 332 }
286 ``` 333 ```
287 334
288 A group containing test cases. The group's ID is unique in the context of this 335 A group containing test cases. The group's ID is unique in the context of this
289 test run. It's used elsewhere in the protocol to refer to this test without 336 test run. It's used elsewhere in the protocol to refer to this group without
290 including its full representation. 337 including its full representation.
291 338
292 The implicit group at the root of each test suite has `null` `name` and 339 The implicit group at the root of each test suite has `null` `name` and
293 `parentID` attributes. 340 `parentID` attributes.
294 341
295 ### Metadata 342 ### Metadata
296 343
297 ``` 344 ```
298 class Metadata { 345 class Metadata {
299 // Whether the test case will be skipped by the test runner. 346 // Whether the test case will be skipped by the test runner.
300 bool skip; 347 bool skip;
301 348
302 // The reason the test case is skipped, if the user provided it. 349 // The reason the test case is skipped, if the user provided it.
303 String? skipReason; 350 String? skipReason;
304 } 351 }
305 ``` 352 ```
306 353
307 The metadata attached to a test by a user. 354 The metadata attached to a test by a user.
OLDNEW
« no previous file with comments | « CHANGELOG.md ('k') | json_reporter.schema.json » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698