OLD | NEW |
---|---|
(Empty) | |
1 # The Design of Reflectable Capabilities | |
sigurdm
2015/08/05 08:31:56
It would be nice to refer to http://www.dartdocs.o
eernst
2015/08/05 14:22:19
Done; it actually works to simply replace '0.1.0'
| |
2 | |
3 This document is intended to document the design choices that we have | |
4 made regarding the class `ReflectCapability` and its subtypes, which is a | |
5 set of classes declared in the library | |
6 `package:reflectable/capability.dart` in | |
7 [package reflectable][package_reflectable]. We use the word | |
8 **capability** to designate instances of subtypes of class | |
9 `ReflectCapability`. This class and its subtypes are used when specifying | |
10 the level of support that a client of the package reflectable will get | |
11 for reflective operations in a given context, e.g., for instances of a | |
12 specific class. We use the word **client** when referring to any software | |
13 artifact which is importing and using the package reflectable, generally | |
sigurdm
2015/08/05 08:31:56
"software artifact" reads like we're writing a leg
eernst
2015/08/05 14:22:19
Obviously, it's illegal. ;-)
The point was that i
| |
14 assuming that it is a package. Using one or another capability as | |
15 metadata on a class `C` in client code may determine whether or not it is | |
16 possible to reflectively `invoke` a method on an instance of `C` via an | |
17 `InstanceMirror`. Given that the motivation for having the package | |
18 reflectable in the first place is to save space consumed by less frugal | |
19 kinds of reflection, the ability to restrict reflection support to the | |
20 actual needs is a core point in the design of the package. | |
21 | |
22 [package_reflectable]: https://github.com/dart-lang/reflectable | |
23 | |
24 # Context and Design Ideas | |
25 | |
26 To understand the topics covered in this document, we need to briefly | |
27 outline how to understand the package reflectable as a whole. Then we | |
28 proceed to explain how we partition the universe of possible kinds of | |
29 support for reflection, such that we have a set of kinds of reflection to | |
30 choose from. Finally we explain how capabilities are used to make a | |
31 selection among these choices, and how they can be applied to specific | |
32 parts of the client program. | |
33 | |
34 ## The Package Reflectable | |
35 | |
36 The package reflectable is an example of support for mirror-based | |
37 introspective reflection in object-oriented languages in general, and it | |
38 should be understandable as such [1]. More specifically, the | |
39 reflection API offered by the package reflectable has been copied | |
40 verbatim from the API offered by the package `dart:mirrors`, and then | |
41 modified in a few ways. As a result, code using `dart:mirrors` should be | |
42 very similar to corresponding code using package reflectable. The | |
43 differences that do exist were introduced for two reasons: | |
44 | |
45 * By design, some operations which are declared as top-level functions in | |
46 `dart:mirrors` are declared as methods on the class `Reflectable` in | |
47 the package reflectable, because instances of subclasses thereof, known | |
48 as **reflectors**, are intended to play the role as mirror systems | |
49 [1, or search 'mirror systems' below], and these operations are | |
50 mirror system specific. For instance, the top-level function `reflect` | |
51 in `dart:mirrors` corresponds to two different methods (with different | |
52 semantics, so they cannot be merged) for two different mirror systems. | |
53 | |
54 * Some proposals have been made for changes to the `dart:mirrors` API. We | |
55 took the opportunity to try out an **updated API** by making | |
56 modifications in the signatures of certain methods. For instance, | |
57 `InstanceMirror.invoke` will return the result from the method | |
58 invocation, not an `InstanceMirror` wrapping it. In general, mirror | |
59 operations **return base level values** rather than mirrors thereof in | |
60 the cases where the mirrors are frequently discarded immediately, and | |
61 where it is easy to create the mirror if needed. Mirror class method | |
62 signatures have also been modified in one more way: Where | |
63 `dart:mirrors` methods accept arguments or return results involving | |
64 `Symbol`, package reflectable uses **`String`**. This helps avoiding | |
65 difficulties associated with minification (which is an automated, | |
66 pervasive renaming procedure that is applied to programs mainly in | |
67 order to save space), because `String` values remain unchanged | |
68 throughout compilation. | |
69 | |
70 In summary, the vast majority of the API offered by the package | |
71 reflectable is identical to the API offered by `dart:mirrors`, and design | |
72 documents about that API or about reflection in general [2,3] | |
73 will serve to document the underlying ideas and design choices. | |
74 | |
75 ## Reflection Capability Design | |
76 | |
77 The obvious novel element in package reflectable is that it allows | |
78 clients to specify the level of support for reflection in a new way, by | |
79 using capabilities in metadata. This section outlines the semantics of | |
80 reflection capabilities, i.e., which kinds of criteria they should be | |
81 able to express. | |
82 | |
83 In general, we maintain the property that the specifications of | |
84 reflection support with one reflector (that is, inside one mirror-system) | |
85 are **monotone**, in the sense that any program having a certain amount | |
86 of reflection support specifications will support at least as many | |
87 reflective operations if additional specifications are added to that | |
88 reflector. In other words, reflection support specifications can request | |
89 additional features, they can never prevent any reflection features from | |
90 being supported. As a result, we obtain a modularity law: a programmer | |
91 who browses source code and encounters a reflection support specification | |
92 `S` somewhere can always trust that the corresponding kind of reflection | |
93 support will be present in the program. Other parts of the program may | |
94 still add even more reflection support, but they cannot withdraw the | |
95 features requested by `S`. Similarly, the specifications are | |
96 **idempotent**, that is, multiple specifications requesting the same | |
97 feature or overlapping feature sets are harmless, it makes no difference | |
98 whether a particular thing has been requested once or several times. | |
99 | |
100 ### Mirror API Based Capabilities | |
101 | |
102 The level of support for reflection may in principle be specified in many | |
103 ways: there is a plethora of ways to use reflection, and ideally the | |
104 client should be able to request support for exactly that which is | |
105 needed. In order to drastically simplify this universe of possibilities | |
106 and still maintain a useful level of expressive power, we have decided to | |
107 use the following stratification as an overall framework for the design: | |
108 | |
109 * The most basic kind of reflection support specification simply | |
110 addresses the API of the mirror classes directly, that is, it is | |
111 concerned with "turning on" support for the use of individual methods | |
112 or small groups of methods in the mirror classes. For instance, it is | |
113 possible to turn on support for `InstanceMirror.invoke` using one | |
114 capability, and another capability will turn on | |
115 `ClassMirror.invoke`. In case a supported method is called it behaves | |
116 like the corresponding method in a corresponding mirror class from | |
117 `dart:mirrors`; in case an unsupported method is called, an exception | |
118 is thrown. | |
119 | |
120 * As a refinement of the API based specification, we have chosen to focus | |
121 on the specification of allowable argument values given to specific | |
122 methods in the API. For instance, it is possible to specify a predicate | |
123 which is used to filter existing method names such that | |
124 `InstanceMirror.invoke` is supported for methods whose name satisfies | |
125 that predicate. An example usage could be testing, where reflective | |
126 invocation of all methods whose name ends in `...Test` might be a | |
127 convenient feature, as opposed to the purely static approach where | |
128 someone would have to write a centralized listing of all such methods, | |
129 which could then be used to call them. | |
130 | |
131 With these mechanisms, it is possible to specify support for reflection | |
132 in terms of mirrors and the features that they offer, independently of | |
133 the actual source code in the client program. | |
134 | |
135 ### Reflectee Based Capabilities | |
136 | |
137 Another dimension in the support for reflection is the selection of which | |
138 parts of the client program the mirrors will be able to reflect upon, | |
139 both when a `ClassMirror` reflects upon one of those classes, and when an | |
140 `InstanceMirror` reflects upon one of its instances. In short, this | |
141 dimension is concerned with the available selection of reflectees. | |
142 | |
143 The general feature covering this type of specification is | |
144 **quantification** over source code elements—in particular over | |
145 classes (future extensions will deal with other entities). In this area | |
146 we have focused on the mechanisms listed below. Note that `MyReflectable` | |
147 is assumed to be the name of a subclass of `Reflectable` and | |
148 `myReflectable` is assumed to be a `const` instance of `MyReflectable`, | |
149 by canonicalization *the* unique `const` instance thereof. This allows us | |
150 to refer to the general concept of a reflector in terms of the example, | |
151 `myReflectable`, along with its class and similar associated | |
152 declarations. | |
153 | |
154 * Reflection support is initiated by invoking one of the methods | |
155 `reflect` or `reflectType` on `myReflectable`. We have chosen to omit | |
156 the capability to do `reflect` (in the sense that this is always | |
157 possible) because there is little reason for having reflection at all | |
158 without support for instance mirrors. In contrast, we have chosen to | |
159 have a capability for obtaining class mirrors and similar source code | |
160 oriented mirrors, which also controls the ability to perform | |
161 `reflectType`; this is because having these mirrors may have | |
162 substantial cost in terms of program size. Finally, we have chosen to | |
163 omit the method `reflectClass`, because it may be replaced by | |
164 `reflectType`, followed by `originalDeclaration` when | |
165 `isOriginalDeclaration` is `false`. | |
166 | |
167 * The basic mechanism to get reflection support for a class `C` is to | |
168 attach metadata to it, and this metadata must be a reflector such as | |
169 `myReflectable`. The class `Reflectable` has a constructor which is | |
170 `const` and takes a single argument of type `List<ReflectCapability>` | |
171 and another constructor which takes up to ten arguments of type | |
172 `ReflectCapability` (thus avoiding the boilerplate that explicitly | |
173 makes it a list). `MyReflectable` must have a single constructor which | |
174 is `const` and takes zero arguments. It is thus enforced that | |
175 `MyReflectable` passes the `List<ReflectCapability>` in its constructor | |
176 via a superinitializer, such that every instance of `MyReflectable` has | |
177 the same state, "the same capabilities". In summary, this basic | |
178 mechanism will request reflection support for one class, at the level | |
179 specified by the capabilities stored in the metadata. | |
180 | |
181 * The reflection support specification can be non-local, that is, it | |
182 could be placed in a different location in the program than on the | |
183 target class itself. This is needed when there is a need to request | |
184 reflection support for a class in a library that cannot be edited (it | |
185 could be predefined, it could be provided by a third party such that | |
186 modifications incur repeated maintenance after updates, etc.). This | |
187 feature has been known as **side tags** since the beginnings of the | |
188 package reflectable. Currently they are attached as metadata to an | |
189 import directive for the library | |
190 `package:reflectable/reflectable.dart`, but they could in principle be | |
191 attached to any program element that admits metadata, or they could | |
192 occur in other `const` contexts, as long as there is a well-defined | |
193 convention for finding them such that they can have an effect. | |
194 | |
195 * Quantification generalizes the single-class specifications by allowing | |
196 a single specification to specify that the capabilities given as its | |
197 arguments should apply to a set of classes or other program | |
198 elements. It is easy to provide quantification mechanisms, but we do | |
199 not want to pollute the package with a bewildering richness of | |
200 quantification mechanisms, so each of the ones we have should be | |
201 comprehensible and reasonably powerful, and they should not overlap. So | |
202 far, we have focused on the following variants: | |
203 * It should be possible to specify that one or more specific classes | |
204 get a specific level of reflection support; this is a simple | |
205 generalization of side tags where the target is a list of classes | |
206 rather than a single class. | |
207 * It should be possible to request reflection support for a set of | |
208 classes chosen in a more abstract manner than by | |
209 enumeration. Obvious candidate quantification mechanisms quantify | |
210 over all superclasses of a given class; over all supertypes of a | |
211 given class; over all subclasses of a given class; over all | |
212 subtypes of a given class; and over all classes whose name matches | |
213 a given pattern. | |
214 * Quantification as in the previous bullet is centralized because it | |
215 is based on one specification which is then used to 'query' the | |
216 whole program (or certain large parts of it) for matching | |
217 entities. It is common and useful to supplement this with a | |
218 decentralized mechanism, where programmers manually enumerate the | |
219 members of a set, for instance by attaching a certain marker as | |
220 metadata to those members. This makes it possible to maintain the | |
221 set precisely and explicitly, even in the cases where the members | |
222 do not share obvious common traits that makes the centralized | |
223 approach convenient. A good example is that a set of methods can be | |
224 given reflective support by annotating them with metadata; for | |
225 instance, we may wish to be able to reflectively invoke all methods | |
226 marked with @businessRule. | |
227 | |
228 We subscribe to a point of view where reflective operations are divided | |
229 into (a) operations concerned with the dynamic behavior of class | |
230 instances, and (b) operations concerned with the structure of the | |
231 program; let us call the former **behavioral operations** and the latter | |
232 **introspective operations**. As an example, using | |
233 `InstanceMirror.invoke` in order to execute a method on the reflectee | |
234 is a behavioral operation, whereas it is an introspective operation to | |
235 use `ClassMirror.declarations` in order to investigate the set of members | |
236 that an instance of the reflected class would have. | |
237 | |
238 An important consequence of this distinction is that behavioral | |
239 operations are concerned with the actual behaviors of objects, which | |
240 means that inherited method implementations have the same status as | |
241 method implementations declared in the class which is the runtime type of | |
242 the reflectee. Conversely, introspective operations are concerned with | |
243 source code entities such as declarations, and hence the `declarations` | |
244 reported for a given class does *not* include inherited declarations, | |
245 they must be found by explicitly iterating over the superclass chain. | |
246 | |
247 Finally, we need to mention the notion of a **mirror system**, that is, a | |
248 set of features which provides support for mirror based reflection. This | |
249 is because we may have several of them: With a choice of a level of | |
250 reflection support (based on the mirror APIs), and a choice of classes | |
251 for which this level of support should be provided (based on reflectee | |
252 selection), it is reasonable to say that we have specified a mirror | |
253 system. Using multiple mirror systems is relevant in cases where some | |
254 classes (and/or their instances) require very different levels of | |
255 support. For example, when a few classes require extensive reflection | |
256 support and a large number of other classes require just a little bit, | |
257 using a powerful mirror system with the former and a minimalist one with | |
258 the latter may be worth the effort, due to the globally improved resource | |
259 economy. Some extra complexity must be expected; e.g., if we can obtain | |
260 both a "cheap" and a "powerful" mirror for the same object it will happen | |
261 via something like `myCheapReflectable.reflect(o)` and | |
262 `myPowerfulReflectable.reflect(o)`, and it is then up to the programmer | |
263 to avoid asking the cheap one to do powerful things. | |
264 | |
265 # Specifying Reflection Capabilities | |
266 | |
267 As mentioned on page 1, reflection capabilities are specified using the | |
268 subtype hierarchy rooted in the class **`ReflectCapability`**, in | |
269 `package:reflectable/capability.dart`. Instances of these classes are | |
270 used to build something that may well be considered as abstract syntax | |
271 trees for a domain specific language. This section describes how this | |
272 setup can be used to specify reflection support. | |
273 | |
274 The subtype hierarchy under `ReflectCapability` is sealed, in the sense | |
275 that there is a set of subtypes of `ReflectCapability` in that library, | |
276 and there should never be any other subtypes of that class; the language | |
277 does not enforce this concept, but it is a convention that should be | |
278 followed. | |
279 | |
280 Being used as `const` values, instances of these classes obviously cannot | |
281 have mutable state, but some of them do contain `const` values such as | |
282 `String`s or other capabilities. They do not have methods, except the | |
283 ones that they inherit from `Object`. Altogether, this means that | |
284 instances of these classes cannot "do anything", but they can be used to | |
285 build immutable trees, and the universe of possible trees is fixed | |
286 because the set of classes is fixed. This makes the trees similar to | |
287 abstract syntax trees, and we can ascribe a semantics to these syntax | |
288 trees from the outside. That semantics may be implemented by an | |
289 interpreter or a translator. The sealedness of the set of classes | |
290 involved is required because an unknown subtype of `ReflectCapability` | |
291 would not have a semantics, and interpreters and translators would not be | |
292 able to handle them (and we haven't been convinced that a suitable level | |
293 of extensibility in those interpreters and translators is worth the | |
294 effort). | |
295 | |
296 In other words, we specify reflection capabilities by building an | |
297 abstract syntax tree for an expression in a domain specific language; let | |
298 us call that language the **reflectable capability language**. | |
299 | |
300 It is obviously possible to have multiple representations of expressions | |
301 in this language, and we have considered introducing a traditional, | |
302 textual syntax for it. We could have a parser that accepts a `String`, | |
303 parses it, and yields an abstract syntax tree consisting of instances of | |
304 subtypes of `ReflectCapability`, or reports a syntax error. A | |
305 `Reflectable` constructor taking a `String` argument could be provided, | |
306 and the `String` could be parsed when needed. This would be a convenient | |
307 (but less safe) way for programmers to specify reflection support, | |
308 possibly as an alternative to the current approach where the abstract | |
309 syntax trees must be specified directly. | |
310 | |
311 However, the textual syntax is used in this document only because it is | |
312 concise and easy to read, it has not been (and might never be) | |
313 implemented. Hence, actual code using the reflectable capability language | |
314 will have to use the more verbose form that directly builds an object | |
315 structure representing an abstract syntax tree for that | |
316 expression. Example code showing how this is done can be found in the | |
317 package test_reflectable. | |
318 | |
319 In this document, we will discuss this language in terms of its | |
320 grammatical structure, along with an informal semantics of each | |
321 construct. | |
322 | |
323 ## Specifying Mirror API Based Capabilities | |
324 | |
325 Figure 1 shows the raw material for the elements in one part of the | |
326 reflectable capability language grammar. The left side of the figure | |
327 contains tokens representing abstract concepts for clustering, and the | |
328 right side contains tokens representing each of the methods in the entire | |
329 mirror API. A few tokens represent more than one method (for instance, | |
330 all of `VariableMirror`, `MethodMirror`, and `TypeVariableMirror` have an | |
331 `isStatic` getter, and `metadata` is also defined in two classes), but | |
332 they have been merged into one token because those methods play the same | |
333 role semantically in all contexts where they occur. In other cases where | |
334 the semantics differ (`invoke`, `invokeGetter`, `invokeSetter`, and | |
335 `declarations`) there are multiple tokens for each method name, | |
336 indicating the enclosing mirror class with a prefix ending in `_`. | |
337 | |
338 | |
sigurdm
2015/08/05 08:31:56
I think github will render the table properly with
floitsch
2015/08/05 11:02:49
I would prefer not to have HTML in the document, u
eernst
2015/08/05 14:22:19
Indeed. Removed.
eernst
2015/08/05 14:22:19
The HTML-free (markdown-style) tables are not pret
| |
339 | |
340 <table> | |
341 <tr> | |
342 <td><strong>Concept</strong></td> | |
343 <td><strong>Specialization<strong></td> | |
344 </tr> | |
345 <tr> | |
346 <td><i>invocation</i></td> | |
347 <td> | |
348 instance_invoke | class_invoke | | |
sigurdm
2015/08/05 08:31:56
Github-flavored markdown does not parse underscore
eernst
2015/08/05 14:22:19
Using that now.
| |
349 library_invoke | instance_invokeGetter | | |
350 class_invokeGetter | library_invokeGetter | | |
351 instance_invokeSetter | class_invokeSetter | | |
352 library_invokeSetter | delegate | | |
353 apply | newInstance | |
354 </td> | |
355 </tr> | |
356 <tr> | |
357 <td><i>naming</i></td> | |
358 <td>simpleName | qualifiedName | constructorName</td> | |
359 </tr> | |
360 <tr> | |
361 <td><i>classification</i></td> | |
362 <td>isPrivate | isTopLevel | isImport | isExport | | |
363 isDeferred | isShow | isHide | isOriginalDeclaration | | |
364 isAbstract | isStatic | isSynthetic | isRegularMethod | | |
365 isOperator | isGetter | isSetter | isConstructor | | |
366 isConstConstructor | isGenerativeConstructor | | |
367 isRedirectingConstructor | isFactoryConstructor | isFinal | | |
368 isConst | isOptional | isNamed | hasDefaultValue | | |
369 hasReflectee | hasReflectedType | |
370 </td> | |
371 </tr> | |
372 <tr> | |
373 <td><i>annotation</i></td> | |
374 <td>metadata</td> | |
375 </tr> | |
376 <tr> | |
377 <td><i>typing</i></td> | |
378 <td>instance_type | variable_type | parameter_type | | |
379 typeVariables | typeArguments | originalDeclaration | | |
380 isSubtypeOf | isAssignableTo | superclass | superinterfaces | | |
381 mixin | isSubclassOf | returnType | upperBound | referent | |
382 </td> | |
383 </tr> | |
384 <tr> | |
385 <td><i>concretization</i></td> | |
386 <td>reflectee | reflectedType</td> | |
387 </tr> | |
388 <tr> | |
389 <td><i>introspection</i></td> | |
390 <td>owner | function | uri | library_declarations | | |
391 class_declarations | libraryDependencies | sourceLibrary | | |
392 targetLibrary | prefix | combinators | instanceMembers | | |
393 staticMembers | parameters | callMethod | defaultValue | |
394 </td> | |
395 </tr> | |
396 <tr> | |
397 <td><i>text</i></td> | |
398 <td>location | source</td> | |
399 </tr> | |
400 </table> | |
401 | |
402 **Figure 1.** Reflectable capability language API raw material. | |
403 | |
404 Figure 2 shows a reduction of this raw material to a set of capabilities | |
405 that we consider reasonable. It does not allow programmers to select | |
406 their capabilities with the same degree of detail, but we expect that the | |
407 complexity reduction is sufficiently valuable to justify the less | |
408 fine-grained control. | |
409 | |
410 We have added *`RegExp`* arguments, specifying that each of these | |
411 capabilities can meaningfully apply a pattern matching constraint to | |
412 select the methods, getters, etc. which are included. With the empty | |
413 *`RegExp`* as the default value, all entities in the relevant category | |
414 are included when no *`RegExp`* is given. Similarly, we have created | |
415 variants taking a *`MetadataClass`* argument which expresses that an | |
416 entity in the relevant category is included iff it has been annotated | |
417 with metadata whose type is a subtype of the given | |
418 *`MetadataClass`*. This provides support for centralized and slightly | |
419 abstract selection of entities using regular expressions, and it provides | |
420 support for decentralized selection of entities using metadata to | |
421 explicitly mark the entities. | |
422 | |
423 It is important to note that the *`MetadataClass`* is potentially | |
424 unrelated to the package reflectable: We expect the use case where some | |
425 class `C` from a package `P` unrelated to reflectable happens to fit | |
426 well, because instances of `C` are already attached as metadata to the | |
427 relevant set of members, which would in turn be the case because the need | |
428 for reflection arises as a consequence of the semantics of `C` as | |
429 metadata relative to `P`. | |
430 | |
431 | |
432 | |
433 <table> | |
434 <tr> | |
435 <td><strong>Non-terminal</strong></td> | |
436 <td><strong>Expansion</strong></td> | |
437 </tr> | |
438 <tr> | |
439 <td><i>apiSelection</i></td> | |
440 <td><i>invocation</i> | <i>naming</i> | <i>classification</i> | | |
441 <i>annotation</i> | <i>typing</i> | <i>introspection</i> | |
442 </td> | |
443 </tr> | |
444 <tr> | |
445 <td><i>invocation</i></td> | |
446 <td>instanceInvoke([<i>RegExp</i>]) | | |
447 instanceInvokeMeta(<i>MetadataClass</i>) | | |
448 staticInvoke([<i>RegExp</i>]) | | |
449 staticInvokeMeta(<i>MetadataClass</i>) | | |
450 newInstance([<i>RegExp</i>]) | | |
451 newInstanceMeta(<i>MetadataClass</i>) | |
452 </td> | |
453 </tr> | |
454 <tr> | |
455 <td><i>naming</i></td> | |
456 <td>name</td> | |
457 </tr> | |
458 <tr> | |
459 <td><i>classification</i></td> | |
460 <td>classify</td> | |
461 </tr> | |
462 <tr> | |
463 <td><i>annotation</i></td> | |
464 <td>metadata</td> | |
465 </tr> | |
466 <tr> | |
467 <td><i>typing</i></td> | |
468 <td>type([<i>UpperBound</i>]) | typeRelations</td> | |
469 </tr> | |
470 <tr> | |
471 <td><i>introspection</i></td> | |
472 <td>owner | declarations | uri | libraryDependencies</td> | |
473 </tr> | |
474 </table> | |
475 | |
476 **Figure 2.** Reflectable capability language API grammar tokens. | |
477 | |
478 The category *text* was removed because we do not plan to support | |
479 reflective access to the source code as a whole at this point; *naming* | |
480 has been expressed atomically as name because we do not want to | |
481 distinguish among the different kinds of names, and similarly for all the | |
482 *classification* predicates. The category *concretization* was removed | |
483 because it is trivial to support these features, so they are always | |
484 enabled. | |
485 | |
486 We have omitted `apply` and `function` because we do not have support for | |
487 `ClosureMirror` and we do not expect to get it anytime soon. | |
488 | |
489 Moreover, `delegate` was made implicit such that the ability to invoke a | |
490 method implies the ability to delegate to it. | |
491 | |
492 The category *typing* was simplified in several ways: `instance_type` was | |
493 renamed into `type` because of its prominence. It optionally receives an | |
494 *`UpperBound`* argument which puts a limit on the available class mirrors | |
495 (class mirrors will only be supported for classes which are subclasses of | |
496 that *`UpperBound`*). The method `reflectType` on reflectors is only | |
497 supported when this capability is present, and only on class mirrors | |
498 passing the *`UpperBound`*, if any. The capabilities `variable_type`, | |
499 `parameter_type`, and `returnType` were unified into `type` because they | |
500 are concerned with lookups for the same kind of mirrors. To give some | |
501 control over the level of detail in the type related mirrors, | |
502 `typeVariables`, `typeArguments`, `originalDeclaration`, `isSubtypeOf`, | |
503 `isAssignableTo`, `superclass`, `superinterfaces`, `mixin`, | |
504 `isSubclassOf`, `upperBound`, and `referent` were unified into | |
505 `typeRelations`; they all address relations among types, type variables, | |
506 and `typedef`s, and it may be a substantial extra cost to preserve | |
507 information about these topics if it is not used. | |
508 | |
509 The category *introspection* was also simplified: We unified | |
510 `class_declarations`, `library_declarations`, `instanceMembers`, | |
511 `staticMembers`, `callMethod`, `parameters`, and `defaultValue` into | |
512 `declarations`. Finally we unified the import and export properties into | |
513 `libraryDependencies` such that it subsumes `sourceLibrary`, | |
514 `targetLibrary`, `prefix`, and `combinators`. We have retained the | |
515 `owner` capability separately, because we expect the ability to look up | |
516 the enclosing declaration for a given declaration to be too costly to | |
517 include implicitly as part of another capability; and we have retained | |
518 the `uri` capability separately because the preservation of information | |
519 about URIs in JavaScript translated code (which is needed in order to | |
520 implement the method uri on a library mirror) has been characterized as a | |
521 security problem in some contexts. | |
522 | |
523 Note that certain reflective methods are **non-elementary** in the sense | |
524 that they can be implemented entirely based on other reflective methods, | |
525 the **elementary** ones. This affects the following capabilities: | |
526 `isSubtypeOf`, `isAssignableTo`, `isSubclassOf`, `instanceMembers`, and | |
527 `staticMembers`. These methods can be implemented in a general manner even | |
528 for transformed programs, so they are provided as part of the package | |
529 reflectable rather than being generated. Hence, they are supported if and | |
530 only if the methods they rely on are supported. This is what it means | |
531 when we say that `instanceMembers` is 'unified into' `declarations`. | |
532 | |
533 ### Covering Multiple API Based Capabilities Concisely | |
534 | |
535 In order to avoid overly verbose syntax in the cases where relatively | |
536 broad reflection support is requested, we have chosen to introduce some | |
537 grouping tokens. They do not contribute anything new, they just offer a | |
538 more concise notation for certain selections of capabilities that are | |
539 expected to occur together frequently. Figure 3 shows these grouping | |
540 tokens. As an aid to remember what this additional syntax means, we have | |
541 used words ending in 'ing' to give a hint about the tiny amount of | |
542 abstraction involved in grouping several capabilities into a single | |
543 construct. | |
544 | |
545 | |
546 | |
547 <table> | |
sigurdm
2015/08/05 08:31:56
The tables could also be formatted as
|header1|h
eernst
2015/08/05 14:22:19
Did that; works nicely, except that I can't see a
| |
548 <tr> | |
549 <td><strong>Group</strong></td> | |
550 <td><strong>Meaning</strong></td> | |
551 </tr> | |
552 <tr> | |
553 <td>invoking([<i>RegExp</i>])</td> | |
554 <td>instanceInvoke([<i>RegExp</i>]), staticInvoke([<i>RegExp</i>]), | |
555 newInstance([<i>RegExp</i>]) | |
556 </td> | |
557 </tr> | |
558 <tr> | |
559 <td>invokingMeta(<i>MetadataClass</i>)</td> | |
sigurdm
2015/08/05 08:31:56
Italics should be possible with _word_
eernst
2015/08/05 14:22:19
Not in nested html, which is the reason why I coul
| |
560 <td>instanceInvokeMeta(<i>MetadataClass</i>), | |
561 staticInvokeMeta(<i>MetadataClass</i>), | |
562 newInstanceMeta(<i>MetadataClass</i>) | |
563 </td> | |
564 </tr> | |
565 <tr> | |
566 <td>typing([<i>UpperBound</i>])</td> | |
567 <td>type([<i>UpperBound</i>]), name, classify, metadata, | |
568 typeRelations, owner, declarations, uri, libraryDependencies | |
569 </td> | |
570 </tr> | |
571 </table> | |
572 | |
573 **Figure 3.** Grouping tokens for the reflectable capability language. | |
574 | |
575 <!-- Glitch: It seems to be impossible to specify italics inside a code --> | |
576 <!-- block, which means that we cannot indicate that 'RegExp' is a --> | |
577 <!-- meta-variable whereas 'invoking' is concrete syntax. No solution --> | |
578 <!-- seems to exist for this problem, or at least none known to --> | |
579 <!-- stackoverflow. This affects several expressions in the next --> | |
580 <!-- paragraph. --> | |
581 | |
582 The semantics of including the capability `invoking(RegExp)` where | |
583 *`RegExp`* stands for a given argument is identical to the semantics of | |
584 including all three capabilities in the same row on the right hand side | |
585 of the figure, giving all of them the same *`RegExp`* as | |
586 argument. Similarly, `invoking()` without an argument requests support | |
587 for reflective invocation of all instance methods, all static methods, | |
588 and all constructors. The semantics of including the capability | |
589 `invokingMeta(MetadataClass)` is the same as the semantics of including | |
590 all three capabilities to the right in the same row, with the same | |
591 argument. Finally, the semantics of including `typing(UpperBound)` is to | |
592 request support for all the capabilities on the right, passing | |
593 *`UpperBound`* to the `type` capability; that is, requesting support for | |
594 every feature associated with information about the program structure, | |
595 bounded by the given *`UpperBound`*; and `typing()` without an argument | |
596 works the same, only with `type()`. | |
597 | |
598 ## Specifying Reflectee Based Capabilities | |
599 | |
600 In the previous section we found a way to specify mirror API based | |
601 capabilities as a grammar. It is very simple, because it consists of | |
602 terminals only, apart from the fact that some of these terminals take an | |
603 argument that is used to restrict the supported arguments to the matching | |
604 names. As shown in Fig. 2, the non-terminal *`apiSelection`* covers them | |
605 all. We shall use them several at a time, so the typical usage is a list, | |
606 written as *`apiSelection*`*. | |
607 | |
608 In this section we discuss how the reflection support specified by a | |
609 given *`apiSelection*`* can be requested for a specific set of | |
610 program elements. Currently the only supported kind of program element is | |
611 classes, but this will be generalized later. The program elements that | |
612 receive reflection support are called the **targets** of the | |
613 specification, and the specification itself is given as a | |
614 superinitializer in a subclass (call it `MyReflectable`) of class | |
615 `Reflectable`, with a unique instance (call it `myReflectable`). Now, | |
616 `myReflectable` is used as metadata somewhere in the program, and each | |
617 kind of capability is only applicable as an annotation in certain | |
618 locations, which is discussed below. | |
619 | |
620 Figure 4 shows how capabilities and annotations can be constructed, | |
621 generally starting from an *`apiSelection*`*. The non-terminals in | |
622 this part of the grammar have been named after the intended location of | |
623 the metadata which is or contains a capability of the corresponding kind. | |
624 | |
625 | |
626 | |
627 <table> | |
628 <tr> | |
629 <td><strong>Non-terminals</strong></td> | |
630 <td><strong>Expansions</strong></td> | |
631 </tr> | |
632 <tr> | |
633 <td>reflector</td> | |
634 <td>Reflectable(targetMetadata)</td> | |
635 </tr> | |
636 <tr> | |
637 <td>targetMetadata</td> | |
638 <td>apiSelection* | subtypeQuantify(apiSelection*) | | |
639 admitSubtype(apiSelection*) | |
640 </td> | |
641 </tr> | |
642 <tr> | |
643 <td>globalMetadata</td> | |
644 <td>globalQuantify(RegExp, reflector) | | |
645 globalQuantifyMeta(MetadataClass, reflector) | |
646 </td> | |
647 </tr> | |
648 </table> | |
649 | |
650 **Figure 4.** Reflectable capability language target selection. | |
651 | |
652 In practice, a *`reflector`* is an instance of a subclass of class | |
653 `Reflectable` that is directly attached to a class as metadata, or passed | |
654 to a global quantifier; in the running example it is the object | |
655 `myReflectable`. The reflector has one piece of state that we model with | |
656 *`targetMetadata`*. In the grammar in Fig. 4 we use the identifier | |
657 `Reflectable` to stand for all the subclasses, and we model the state by | |
658 letting it take the corresponding *`targetMetadata`* as an argument. The | |
659 semantics of annotating a class with a given *`reflector`* depends on the | |
660 *`targetMetadata`*, as described below. | |
661 | |
662 A *`targetMetadata`* capability can be a base level set of capabilities, | |
663 that is, an *`apiSelection*`*, and it can also be a quantifier taking | |
664 such an *`apiSelection*`* as an argument. The semantics of attaching | |
665 a *`reflector`* containing a plain *`apiSelection*`* to a target | |
666 class `C` is that reflection support at the level specified by the given | |
667 *`apiSelection*`* is provided for the class `C` and instances | |
668 thereof. The semantics of attaching a *`reflector`* containing | |
669 `subtypeQuantify(apiSelection*)` to a class `C` is that the reflection | |
670 support specified by the given *`apiSelection*`* is provided for all | |
671 classes which are subtypes of the class `C`, including `C` itself, and | |
672 their instances. The semantics of attaching a *`reflector`* containing | |
673 `admitSubtype(apiSelection*)` to a class `C` is a pragmatic mix of the | |
674 former two which is subtle enough to warrant a slightly more detailed | |
675 discussion, given in the next section. The basic idea is that it allows | |
676 instances of subtypes of the target class to be treated as if they were | |
677 instances of the target class. | |
678 | |
679 Finally, we support side tags using global quantifiers, | |
680 `globalQuantify(RegExp, reflector)` and | |
681 `globalQuantifyMeta(MetadataClass, reflector)`. Currently, we have | |
682 decided that they must be attached as metadata to an import statement | |
683 importing `package:reflectable/reflectable.dart`, but we may relax this | |
684 restriction if other placements turn out to be helpful in practice. Due | |
685 to the monotone semantics of capabilities it is not a problem if a given | |
686 program contains more than one such *`globalMetadata`*, the provided | |
687 reflection support will simply be the least one that satisfies all | |
688 requests. | |
689 | |
690 The semantics of having `globalQuantify(RegExp, reflector)` in a program | |
691 is ideally identical to the semantics of having the given *`reflector`* | |
692 attached directly to each of those classes in the program whose qualified | |
693 name matches the given *`RegExp`*. Similarly, the semantics of having | |
694 `globalQuantifyMeta(MetadataClass, reflector)` in a program is ideally | |
695 identical to the semantics of having the given *`reflector`* attached | |
696 directly to each of those classes whose metadata includes an instance of | |
697 type *`MetadataClass`*. At this point, however, we must add an adjustment | |
698 to the ideal goal that the semantics is identical: Access to private | |
699 declarations may not be fully supported with a *`globalMetadata`*. This | |
700 is discussed in the next section. | |
701 | |
702 ### Completely or Partially Mirrored Instances? | |
703 | |
704 Traditionally, it is assumed that reflective access to an instance, a | |
705 class, or some other entity will provide a complete and faithful view of | |
706 that entity. For instance, it should be possible for reflective code to | |
707 access features declared as private even when that reflective code is | |
708 located in a context where non-reflective access to the same features | |
709 would not be allowed. Moreover, when a reflective lookup is used to learn | |
710 which class a given object is an instance of, it is expected that the | |
711 response describes the actual runtime type of the object, and not some | |
712 superclass such as the statically known type of that object in some | |
713 context. | |
714 | |
715 In the package reflectable there are reasons for violating this | |
716 completeness assumption, and some of them are built-in consequences of | |
717 the reasons for having this package in the first place. In other words, | |
718 these restrictions will not go away entirely. Other restrictions may be | |
719 lifted in the future, because they were introduced based on certain | |
720 trade-offs made in the implementation of the package. | |
721 | |
722 The main motivation for providing the package reflectable is that the | |
723 more general support for reflection provided by the `dart:mirrors` | |
724 package tends to be too costly at runtime in terms of program | |
725 size. Hence, it is a core point for package reflectable to specify a | |
726 restricted version of reflection that fits the purposes of a given | |
727 program, such that it can be done using a significantly smaller amount of | |
728 space. Consequently, it will be perfectly normal for such a program to | |
729 have reflective support for an object without reflective access to, say, | |
730 some of its methods. There are several other kinds of coverage which is | |
731 incomplete by design, and they are not a problem: they are part of the | |
732 reason for using package reflectable in the first place. | |
733 | |
734 The following subsections discuss two different situations where some | |
735 restrictions apply that are not there by design. We first discuss cases | |
736 where access to private features is incomplete, and then we discuss the | |
737 consequences of admitting subtypes as specified with | |
738 `admitSubtype(apiSelection*)`. | |
739 | |
740 #### Privacy Related Restrictions | |
741 | |
742 The restrictions discussed in this subsection are motivated by trade-offs | |
743 in the implementation in package reflectable, so we need to mention some | |
744 implementation details. The package reflectable has been designed for | |
745 program transformation, i.e., it is intended that a source to source | |
746 transformer shall be able to receive a given program (which is using | |
747 package reflectable, and indirectly `dart:mirrors`) as input, and | |
748 transform it to an equivalent program that does not use `dart:mirrors`, | |
749 generally by generating mirror implementation classes containing | |
750 ordinary, non-reflective code. | |
751 | |
752 Ordinary code cannot violate privacy restrictions. Hence, reflective | |
753 operations cannot, say, read or write a private field in a different | |
754 library. The implication is that private access can only be supported for | |
755 classes declared in a library which can be transformed, because only then | |
756 can the generated mirror implementation class coexist with the target | |
757 class. Using a transformer as we currently do (and plan to do in the | |
758 future), all libraries in the client package can be transformed. However, | |
759 libraries outside the client package cannot be transformed, which in | |
760 particular matters for libraries from other packages, and for pre-defined | |
761 libraries. | |
762 | |
763 For some libraries which cannot be transformed, it would be possible to | |
764 create a local copy of the library in the client package and modify the | |
765 program globally such that it uses that local copy, and such that the | |
766 semantics of the copied library is identical to the semantics of the | |
767 original. This cannot be done for libraries that contain language | |
768 primitives which cannot be implemented in the language; for instance, the | |
769 pre-defined class int cannot be replaced by a user-defined | |
770 class. Moreover, the need to copy and adjust one library could propagate | |
771 to another library, e.g., if the latter imports the former. Hence, not | |
772 even this workaround would enable transformation of all libraries. We do | |
773 not currently have any plans to use this kind of techniques, and hence | |
774 only libraries in the current package can be transformed. | |
775 | |
776 Given that the main transformation technique for package reflectable is | |
777 to generate a number of mirror classes for each target class, this means | |
778 that access to private declarations cannot be supported for classes in | |
779 libraries that cannot be transformed. This applies to private classes as | |
780 a whole, and it applies to private declarations in public classes. | |
781 | |
782 It should be noted that transformation of libraries imported from other | |
783 packages might be manageable to implement, but it requires changes to the | |
784 basic tools used to process Dart programs, e.g., pub. Alternatively, it | |
785 is possible that this restriction can be lifted in general in the future, | |
786 if the tools (compilers, analyzers, etc.) are modified to support a | |
787 privacy overriding mechanism. | |
788 | |
789 #### Considerations around Admitting Subtypes | |
790 | |
791 When a *`targetMetadata`* on the form *`apiSelection*`* is attached to a | |
792 given class `C`, the effect is that reflection support is provided for | |
793 the class `C` and for instances of `C`. However, that support can be | |
794 extended to give partial reflection support for instances of subtypes of | |
795 `C` in a way that does not incur further costs in terms of program size: | |
796 A mirror generated for instances of class `C` can have a `reflectee` (the | |
797 object being mirrored by that mirror) whose type is a proper subtype of | |
798 `C`. A *`targetMetadata`* on the form `admitSubtype(apiSelection*)` is | |
799 used to specify exactly this: It enables an instance mirror to hold a | |
800 reflectee which is an instance of a proper subtype of the type that the | |
801 mirror was generated for. | |
802 | |
803 The question arises which instance mirror to use for a given object *O* | |
804 with runtime type `D` which is given as an argument to the operation | |
805 `reflect` on a reflector, when there is no mirror class which was created | |
806 for exactly `D`. This is the situation where a subtype reflectee is | |
807 actually admitted. In general, there may be multiple candidate mirror | |
808 classes corresponding to classes `C1, C2, .. Ck` which are "least | |
809 supertypes of `D`" in the sense that no type `E` is a proper supertype of | |
810 `D` and a proper subtype of `Ci` for any `i` (this also implies that no | |
811 two classes `Ci` and `Cj` are subtypes of each other). The language | |
812 specification includes an algorithm which will find a uniquely determined | |
813 supertype of `C1 .. Ck` which is called their **least upper bound**. We | |
814 cannot use this algorithm directly because we have an arbitrary subset of | |
815 the types in a type hierarchy rather than all types, and then we need to | |
816 make a similar decision for this "sparse" subtype hierarchy that only | |
817 includes classes with reflection support from the given | |
818 reflector. Nevertheless, we expect that it is possible to create a | |
819 variant of the least upper bound algorithm which will work for these | |
820 sparse subtype hierarchies. | |
821 | |
822 It should be noted that a very basic invariant which is commonly assumed | |
823 for reflection support in various languages is violated: An instance | |
824 mirror constructed for type `C` can have a reflectee which is an instance | |
825 of a proper subtype `D`. Of course, not all mirror systems have anything | |
826 like the notion of a mirror that is constructed for a given type, but the | |
827 corresponding problem is relevant everywhere: The mirror will not report | |
828 on the properties of the object as-is, it will report on the properties | |
829 of instances of a supertype. This is a kind of incompleteness, and it | |
830 even causes the mirror to give plain *incorrect* descriptions of the | |
831 object in some cases. | |
832 | |
833 In particular, assume that an object *O* with runtime type `D` is given, | |
834 and that we have an instance mirror *IM* whose reflectee is *O*. Assume | |
835 that the class of *IM* was generated for instances of a class `C`, which | |
836 is a proper supertype of `D`. It is only because of `admitSubtype` that | |
837 it is even possible for *IM* to have a `reflectee` whose `runtimeType` is | |
838 not `C`. In many situations this discrepancy makes no difference and *IM* | |
839 works fine with *O*, but it is informative to focus on a case where it | |
840 really matters: | |
841 | |
842 <!-- Cannot make 'IM' italic in 'IM.type', hence using a rather messy --> | |
843 <!-- mixed notation. --> | |
844 | |
845 Let us use a reflective operation on *IM* to get a class mirror for the | |
846 class of *O*. *IM*.`type` will return an instance *CM* of the class mirror | |
847 for `C`, not a class mirror for *O*'s actual runtime type `D`. If a | |
848 programmer uses this approach to look up the name of the class of an | |
849 object like *O*, the answer will simply be wrong, it says `"C"` and it | |
850 should have said `"D"`. Similarly, if we traverse the superclasses we | |
851 will never see the class `D`, nor the intermediate classes between `D` | |
852 and `C`. A real-world example is serialization: if we look up the | |
853 declarations of fields in order to serialize the reflectee then we will | |
854 silently fail to include the fields declared in the ignored subclasses | |
855 down to `D`. In general, there are many unpleasant surprises waiting for | |
856 the naive user of this feature, so it should be considered to be an | |
857 expert-only option. | |
858 | |
859 Why not just do the "right thing" and return a class mirror for `D`? It is | |
860 not possible to simply check the `runtimeType` of `reflectee` in the | |
861 implementation of the method type, and then deliver a class mirror of `D` | |
862 because, typically, there *is* no class mirror for `D`. In fact, the whole | |
863 point of having the `admitSubtype` quantifier is that it saves space | |
864 because a potentially large number of subtypes of a given type can be | |
865 given partial reflection support without the need to generate a | |
866 correspondingly large number of mirror classes. | |
867 | |
868 To further clarify what it means to get 'partial' reflective support, | |
869 consider some cases: | |
870 | |
871 Reflectively calling instance methods on *O* which are declared in `C` or | |
872 inherited into `C` will work as expected, and standard object-oriented | |
873 method invocation will ensure that it is the correct method | |
874 implementation for *O* which is called, not just the most specific | |
875 implementation which is available in `C`. | |
876 | |
877 Calling instance methods on *O* which are declared in a proper subtype of | |
878 `C`, including methods from `D` itself, will not work. This is because | |
879 the class of *IM* has been generated under the assumption that no such | |
880 methods exist, it only knows about `C` methods. As mentioned, if we fetch | |
881 the class of *O* we may get a proper supertype of the actual class of | |
882 *O*, and hence all the derived operations will be similarly affected. For | |
883 instance, the declarations from *CM* will be the declarations in `C`, and | |
884 they have nothing to do with the declarations in `D`. Similarly, if we | |
885 traverse the superclasses then we will only see a strict suffix of the | |
886 actual list of superclasses of the class of *O*. | |
887 | |
888 Based on these serious issues, we have decided that when an instance | |
889 mirror is associated with the `admitSubtype` quantifier, it shall be an | |
890 error to execute the `type` method in order to obtain a mirror of a | |
891 class, because it is very unlikely to work as intended when that class is | |
892 in fact not the class of the reflectee. It would be possible to allow it | |
893 in the cases where the match happens to be perfect, but this would be | |
894 difficult for programmers to use, and they may as well use `reflectType` | |
895 directly if they want to reflect upon a class which is not taken directly | |
896 from an instance. | |
897 | |
898 In summary, there is a delicate trade-off to make in the case where an | |
899 entire subtype hierarchy should be equipped with reflection support. The | |
900 trade off is to either pay the price in terms of program size and get | |
901 full support (using `subtypeQuantify`); or to save space aggressively and | |
902 in return tolerate the partial support for reflection (using | |
903 `admitSubtype`). | |
904 | |
905 # Summary | |
906 | |
907 We have described the design of the capabilities used in the package | |
908 reflectable to specify the desired level of support for reflection. The | |
909 underlying idea is that the capabilities at the base level specify a | |
910 selection of operations from the API of the mirror classes, along with | |
911 some simple restrictions on the allowable arguments to those | |
912 operations. On top of that, the API based capabilities can be associated | |
913 with specific parts of the target program (though at this point only | |
914 classes) such that exactly those classes will have the reflection support | |
915 specified with the API based capabilities. The target classes can be | |
916 selected individually, by adding a reflector as metadata on each target | |
917 class. Alternatively, target classes can be selected by quantification: | |
918 It is possible to quantify over all subtypes, in which case not only the | |
919 class `C` that holds the metadata receives reflection support, but also all | |
920 subtypes of `C`. Finally, it is possible to admit instances of subtypes as | |
921 reflectees of a small set of mirrors, such that partial reflection | |
922 support is achieved for many classes, without the cost of having many | |
923 mirror classes. | |
924 | |
925 # References | |
926 | |
927 1. Gilad Bracha and David Ungar. "Mirrors: design principles for | |
928 meta-level facilities of object-oriented programming languages". ACM | |
929 SIGPLAN Notices. 24 Oct. 2004: 331-344. | |
930 2. Brian Cantwell Smith. "Procedural reflection in programming | |
931 languages." 1982. | |
932 3. Jonathan M. Sobel and Daniel P. Friedman. "An introduction to | |
933 reflection-oriented programming." Proceedings of Reflection. | |
934 Apr. 1996. | |
OLD | NEW |