OLD | NEW |
(Empty) | |
| 1 # The Design of Reflectable Capabilities |
| 2 |
| 3 This document is intended to give a conceptually based presentation of the |
| 4 design choices that we have made regarding the class `ReflectCapability` |
| 5 and its subtypes, which is a set of classes declared in the library |
| 6 `package:reflectable/capability.dart` in |
| 7 [package reflectable][package_reflectable]. For a more programming oriented |
| 8 point of view, please consult the |
| 9 [library documentation][dartdoc_for_capability]. |
| 10 We use the word **capability** to designate instances of subtypes of class |
| 11 `ReflectCapability`. This class and its subtypes are used when specifying |
| 12 the level of support that a client of the package reflectable will get |
| 13 for reflective operations in a given context, e.g., for instances of a |
| 14 specific class. We use the word **client** when referring to a library |
| 15 which is importing and using the package reflectable, or a package |
| 16 containing such a library. Using one or another capability as |
| 17 metadata on a class `C` in client code may determine whether or not it is |
| 18 possible to reflectively `invoke` a method on an instance of `C` via an |
| 19 `InstanceMirror`. Given that the motivation for having the package |
| 20 reflectable in the first place is to save space consumed by less frugal |
| 21 kinds of reflection, the ability to restrict reflection support to the |
| 22 actual needs is a core point in the design of the package. |
| 23 |
| 24 [package_reflectable]: https://github.com/dart-lang/reflectable |
| 25 [dartdoc_for_capability]: http://www.dartdocs.org/documentation/reflectable/0.1.
0/index.html#reflectable/reflectable-capability |
| 26 |
| 27 # Context and Design Ideas |
| 28 |
| 29 To understand the topics covered in this document, we need to briefly |
| 30 outline how to understand the package reflectable as a whole. Then we |
| 31 proceed to explain how we partition the universe of possible kinds of |
| 32 support for reflection, such that we have a set of kinds of reflection to |
| 33 choose from. Finally we explain how capabilities are used to make a |
| 34 selection among these choices, and how they can be applied to specific |
| 35 parts of the client program. |
| 36 |
| 37 ## The Package Reflectable |
| 38 |
| 39 The package reflectable is an example of support for mirror-based |
| 40 introspective reflection in object-oriented languages in general, and it |
| 41 should be understandable as such [1]. More specifically, the |
| 42 reflection API offered by the package reflectable has been copied |
| 43 verbatim from the API offered by the package `dart:mirrors`, and then |
| 44 modified in a few ways. As a result, code using `dart:mirrors` should be |
| 45 very similar to corresponding code using package reflectable. The |
| 46 differences that do exist were introduced for two reasons: |
| 47 |
| 48 * By design, some operations which are declared as top-level functions in |
| 49 `dart:mirrors` are declared as methods on the class `Reflectable` in |
| 50 the package reflectable, because instances of subclasses thereof, known |
| 51 as **reflectors**, are intended to play the role as mirror systems |
| 52 [1, or search 'mirror systems' below], and these operations are |
| 53 mirror system specific. For instance, the top-level function `reflect` |
| 54 in `dart:mirrors` corresponds to two different methods (with different |
| 55 semantics, so they cannot be merged) for two different mirror systems. |
| 56 |
| 57 * Some proposals have been made for changes to the `dart:mirrors` API. We |
| 58 took the opportunity to try out an **updated API** by making |
| 59 modifications in the signatures of certain methods. For instance, |
| 60 `InstanceMirror.invoke` will return the result from the method |
| 61 invocation, not an `InstanceMirror` wrapping it. In general, mirror |
| 62 operations **return base level values** rather than mirrors thereof in |
| 63 the cases where the mirrors are frequently discarded immediately, and |
| 64 where it is easy to create the mirror if needed. Mirror class method |
| 65 signatures have also been modified in one more way: Where |
| 66 `dart:mirrors` methods accept arguments or return results involving |
| 67 `Symbol`, package reflectable uses **`String`**. This helps avoiding |
| 68 difficulties associated with minification (which is an automated, |
| 69 pervasive renaming procedure that is applied to programs mainly in |
| 70 order to save space), because `String` values remain unchanged |
| 71 throughout compilation. |
| 72 |
| 73 In summary, the vast majority of the API offered by the package |
| 74 reflectable is identical to the API offered by `dart:mirrors`, and design |
| 75 documents about that API or about reflection in general [2,3] |
| 76 will serve to document the underlying ideas and design choices. |
| 77 |
| 78 ## Reflection Capability Design |
| 79 |
| 80 The obvious novel element in package reflectable is that it allows |
| 81 clients to specify the level of support for reflection in a new way, by |
| 82 using capabilities in metadata. This section outlines the semantics of |
| 83 reflection capabilities, i.e., which kinds of criteria they should be |
| 84 able to express. |
| 85 |
| 86 In general, we maintain the property that the specifications of |
| 87 reflection support with one reflector (that is, inside one mirror-system) |
| 88 are **monotone**, in the sense that any program having a certain amount |
| 89 of reflection support specifications will support at least as many |
| 90 reflective operations if additional specifications are added to that |
| 91 reflector. In other words, reflection support specifications can request |
| 92 additional features, they can never prevent any reflection features from |
| 93 being supported. As a result, we obtain a modularity law: a programmer |
| 94 who browses source code and encounters a reflection support specification |
| 95 `S` somewhere can always trust that the corresponding kind of reflection |
| 96 support will be present in the program. Other parts of the program may |
| 97 still add even more reflection support, but they cannot withdraw the |
| 98 features requested by `S`. Similarly, the specifications are |
| 99 **idempotent**, that is, multiple specifications requesting the same |
| 100 feature or overlapping feature sets are harmless, it makes no difference |
| 101 whether a particular thing has been requested once or several times. |
| 102 |
| 103 ### Mirror API Based Capabilities |
| 104 |
| 105 The level of support for reflection may in principle be specified in many |
| 106 ways: there is a plethora of ways to use reflection, and ideally the |
| 107 client should be able to request support for exactly that which is |
| 108 needed. In order to drastically simplify this universe of possibilities |
| 109 and still maintain a useful level of expressive power, we have decided to |
| 110 use the following stratification as an overall framework for the design: |
| 111 |
| 112 * The most basic kind of reflection support specification simply |
| 113 addresses the API of the mirror classes directly, that is, it is |
| 114 concerned with "turning on" support for the use of individual methods |
| 115 or small groups of methods in the mirror classes. For instance, it is |
| 116 possible to turn on support for `InstanceMirror.invoke` using one |
| 117 capability, and another capability will turn on |
| 118 `ClassMirror.invoke`. In case a supported method is called it behaves |
| 119 like the corresponding method in a corresponding mirror class from |
| 120 `dart:mirrors`; in case an unsupported method is called, an exception |
| 121 is thrown. |
| 122 |
| 123 * As a refinement of the API based specification, we have chosen to focus |
| 124 on the specification of allowable argument values given to specific |
| 125 methods in the API. For instance, it is possible to specify a predicate |
| 126 which is used to filter existing method names such that |
| 127 `InstanceMirror.invoke` is supported for methods whose name satisfies |
| 128 that predicate. An example usage could be testing, where reflective |
| 129 invocation of all methods whose name ends in `...Test` might be a |
| 130 convenient feature, as opposed to the purely static approach where |
| 131 someone would have to write a centralized listing of all such methods, |
| 132 which could then be used to call them. |
| 133 |
| 134 With these mechanisms, it is possible to specify support for reflection |
| 135 in terms of mirrors and the features that they offer, independently of |
| 136 the actual source code in the client program. |
| 137 |
| 138 ### Reflectee Based Capabilities |
| 139 |
| 140 Another dimension in the support for reflection is the selection of which |
| 141 parts of the client program the mirrors will be able to reflect upon, |
| 142 both when a `ClassMirror` reflects upon one of those classes, and when an |
| 143 `InstanceMirror` reflects upon one of its instances. In short, this |
| 144 dimension is concerned with the available selection of reflectees. |
| 145 |
| 146 The general feature covering this type of specification is |
| 147 **quantification** over source code elements—in particular over |
| 148 classes (future extensions will deal with other entities). In this area |
| 149 we have focused on the mechanisms listed below. Note that `MyReflectable` |
| 150 is assumed to be the name of a subclass of `Reflectable` and |
| 151 `myReflectable` is assumed to be a `const` instance of `MyReflectable`, |
| 152 by canonicalization *the* unique `const` instance thereof. This allows us |
| 153 to refer to the general concept of a reflector in terms of the example, |
| 154 `myReflectable`, along with its class and similar associated |
| 155 declarations. |
| 156 |
| 157 * Reflection support is initiated by invoking one of the methods |
| 158 `reflect` or `reflectType` on `myReflectable`. We have chosen to omit |
| 159 the capability to do `reflect` (in the sense that this is always |
| 160 possible) because there is little reason for having reflection at all |
| 161 without support for instance mirrors. In contrast, we have chosen to |
| 162 have a capability for obtaining class mirrors and similar source code |
| 163 oriented mirrors, which also controls the ability to perform |
| 164 `reflectType`; this is because having these mirrors may have |
| 165 substantial cost in terms of program size. Finally, we have chosen to |
| 166 omit the method `reflectClass`, because it may be replaced by |
| 167 `reflectType`, followed by `originalDeclaration` when |
| 168 `isOriginalDeclaration` is `false`. |
| 169 |
| 170 * The basic mechanism to get reflection support for a class `C` is to |
| 171 attach metadata to it, and this metadata must be a reflector such as |
| 172 `myReflectable`. The class `Reflectable` has a constructor which is |
| 173 `const` and takes a single argument of type `List<ReflectCapability>` |
| 174 and another constructor which takes up to ten arguments of type |
| 175 `ReflectCapability` (thus avoiding the boilerplate that explicitly |
| 176 makes it a list). `MyReflectable` must have a single constructor which |
| 177 is `const` and takes zero arguments. It is thus enforced that |
| 178 `MyReflectable` passes the `List<ReflectCapability>` in its constructor |
| 179 via a superinitializer, such that every instance of `MyReflectable` has |
| 180 the same state, "the same capabilities". In summary, this basic |
| 181 mechanism will request reflection support for one class, at the level |
| 182 specified by the capabilities stored in the metadata. |
| 183 |
| 184 * The reflection support specification can be non-local, that is, it |
| 185 could be placed in a different location in the program than on the |
| 186 target class itself. This is needed when there is a need to request |
| 187 reflection support for a class in a library that cannot be edited (it |
| 188 could be predefined, it could be provided by a third party such that |
| 189 modifications incur repeated maintenance after updates, etc.). This |
| 190 feature has been known as **side tags** since the beginnings of the |
| 191 package reflectable. Currently they are attached as metadata to an |
| 192 import directive for the library |
| 193 `package:reflectable/reflectable.dart`, but they could in principle be |
| 194 attached to any program element that admits metadata, or they could |
| 195 occur in other `const` contexts, as long as there is a well-defined |
| 196 convention for finding them such that they can have an effect. |
| 197 |
| 198 * Quantification generalizes the single-class specifications by allowing |
| 199 a single specification to specify that the capabilities given as its |
| 200 arguments should apply to a set of classes or other program |
| 201 elements. It is easy to provide quantification mechanisms, but we do |
| 202 not want to pollute the package with a bewildering richness of |
| 203 quantification mechanisms, so each of the ones we have should be |
| 204 comprehensible and reasonably powerful, and they should not overlap. So |
| 205 far, we have focused on the following variants: |
| 206 * It should be possible to specify that one or more specific classes |
| 207 get a specific level of reflection support; this is a simple |
| 208 generalization of side tags where the target is a list of classes |
| 209 rather than a single class. |
| 210 * It should be possible to request reflection support for a set of |
| 211 classes chosen in a more abstract manner than by |
| 212 enumeration. Obvious candidate quantification mechanisms quantify |
| 213 over all superclasses of a given class; over all supertypes of a |
| 214 given class; over all subclasses of a given class; over all |
| 215 subtypes of a given class; and over all classes whose name matches |
| 216 a given pattern. |
| 217 * Quantification as in the previous bullet is centralized because it |
| 218 is based on one specification which is then used to 'query' the |
| 219 whole program (or certain large parts of it) for matching |
| 220 entities. It is common and useful to supplement this with a |
| 221 decentralized mechanism, where programmers manually enumerate the |
| 222 members of a set, for instance by attaching a certain marker as |
| 223 metadata to those members. This makes it possible to maintain the |
| 224 set precisely and explicitly, even in the cases where the members |
| 225 do not share obvious common traits that makes the centralized |
| 226 approach convenient. A good example is that a set of methods can be |
| 227 given reflective support by annotating them with metadata; for |
| 228 instance, we may wish to be able to reflectively invoke all methods |
| 229 marked with @businessRule. |
| 230 |
| 231 We subscribe to a point of view where reflective operations are divided |
| 232 into (a) operations concerned with the dynamic behavior of class |
| 233 instances, and (b) operations concerned with the structure of the |
| 234 program; let us call the former **behavioral operations** and the latter |
| 235 **introspective operations**. As an example, using |
| 236 `InstanceMirror.invoke` in order to execute a method on the reflectee |
| 237 is a behavioral operation, whereas it is an introspective operation to |
| 238 use `ClassMirror.declarations` in order to investigate the set of members |
| 239 that an instance of the reflected class would have. |
| 240 |
| 241 An important consequence of this distinction is that behavioral |
| 242 operations are concerned with the actual behaviors of objects, which |
| 243 means that inherited method implementations have the same status as |
| 244 method implementations declared in the class which is the runtime type of |
| 245 the reflectee. Conversely, introspective operations are concerned with |
| 246 source code entities such as declarations, and hence the `declarations` |
| 247 reported for a given class does *not* include inherited declarations, |
| 248 they must be found by explicitly iterating over the superclass chain. |
| 249 |
| 250 Finally, we need to mention the notion of a **mirror system**, that is, a |
| 251 set of features which provides support for mirror based reflection. This |
| 252 is because we may have several of them: With a choice of a level of |
| 253 reflection support (based on the mirror APIs), and a choice of classes |
| 254 for which this level of support should be provided (based on reflectee |
| 255 selection), it is reasonable to say that we have specified a mirror |
| 256 system. Using multiple mirror systems is relevant in cases where some |
| 257 classes (and/or their instances) require very different levels of |
| 258 support. For example, when a few classes require extensive reflection |
| 259 support and a large number of other classes require just a little bit, |
| 260 using a powerful mirror system with the former and a minimalist one with |
| 261 the latter may be worth the effort, due to the globally improved resource |
| 262 economy. Some extra complexity must be expected; e.g., if we can obtain |
| 263 both a "cheap" and a "powerful" mirror for the same object it will happen |
| 264 via something like `myCheapReflectable.reflect(o)` and |
| 265 `myPowerfulReflectable.reflect(o)`, and it is then up to the programmer |
| 266 to avoid asking the cheap one to do powerful things. |
| 267 |
| 268 # Specifying Reflection Capabilities |
| 269 |
| 270 As mentioned on page 1, reflection capabilities are specified using the |
| 271 subtype hierarchy rooted in the class **`ReflectCapability`**, in |
| 272 `package:reflectable/capability.dart`. Instances of these classes are |
| 273 used to build something that may well be considered as abstract syntax |
| 274 trees for a domain specific language. This section describes how this |
| 275 setup can be used to specify reflection support. |
| 276 |
| 277 The subtype hierarchy under `ReflectCapability` is sealed, in the sense |
| 278 that there is a set of subtypes of `ReflectCapability` in that library, |
| 279 and there should never be any other subtypes of that class; the language |
| 280 does not enforce this concept, but it is a convention that should be |
| 281 followed. |
| 282 |
| 283 Being used as `const` values, instances of these classes obviously cannot |
| 284 have mutable state, but some of them do contain `const` values such as |
| 285 `String`s or other capabilities. They do not have methods, except the |
| 286 ones that they inherit from `Object`. Altogether, this means that |
| 287 instances of these classes cannot "do anything", but they can be used to |
| 288 build immutable trees, and the universe of possible trees is fixed |
| 289 because the set of classes is fixed. This makes the trees similar to |
| 290 abstract syntax trees, and we can ascribe a semantics to these syntax |
| 291 trees from the outside. That semantics may be implemented by an |
| 292 interpreter or a translator. The sealedness of the set of classes |
| 293 involved is required because an unknown subtype of `ReflectCapability` |
| 294 would not have a semantics, and interpreters and translators would not be |
| 295 able to handle them (and we haven't been convinced that a suitable level |
| 296 of extensibility in those interpreters and translators is worth the |
| 297 effort). |
| 298 |
| 299 In other words, we specify reflection capabilities by building an |
| 300 abstract syntax tree for an expression in a domain specific language; let |
| 301 us call that language the **reflectable capability language**. |
| 302 |
| 303 It is obviously possible to have multiple representations of expressions |
| 304 in this language, and we have considered introducing a traditional, |
| 305 textual syntax for it. We could have a parser that accepts a `String`, |
| 306 parses it, and yields an abstract syntax tree consisting of instances of |
| 307 subtypes of `ReflectCapability`, or reports a syntax error. A |
| 308 `Reflectable` constructor taking a `String` argument could be provided, |
| 309 and the `String` could be parsed when needed. This would be a convenient |
| 310 (but less safe) way for programmers to specify reflection support, |
| 311 possibly as an alternative to the current approach where the abstract |
| 312 syntax trees must be specified directly. |
| 313 |
| 314 However, the textual syntax is used in this document only because it is |
| 315 concise and easy to read, it has not been (and might never be) |
| 316 implemented. Hence, actual code using the reflectable capability language |
| 317 will have to use the more verbose form that directly builds an object |
| 318 structure representing an abstract syntax tree for that |
| 319 expression. Example code showing how this is done can be found in the |
| 320 package test_reflectable. |
| 321 |
| 322 In this document, we will discuss this language in terms of its |
| 323 grammatical structure, along with an informal semantics of each |
| 324 construct. |
| 325 |
| 326 ## Specifying Mirror API Based Capabilities |
| 327 |
| 328 Figure 1 shows the raw material for the elements in one part of the |
| 329 reflectable capability language grammar. The left side of the figure |
| 330 contains tokens representing abstract concepts for clustering, and the |
| 331 right side contains tokens representing each of the methods in the entire |
| 332 mirror API. A few tokens represent more than one method (for instance, |
| 333 all of `VariableMirror`, `MethodMirror`, and `TypeVariableMirror` have an |
| 334 `isStatic` getter, and `metadata` is also defined in two classes), but |
| 335 they have been merged into one token because those methods play the same |
| 336 role semantically in all contexts where they occur. In other cases where |
| 337 the semantics differ (`invoke`, `invokeGetter`, `invokeSetter`, and |
| 338 `declarations`) there are multiple tokens for each method name, |
| 339 indicating the enclosing mirror class with a prefix ending in `_`. |
| 340 |
| 341 | **Strong** | **Specialization** | |
| 342 | ------------------------------ | ------------------------------ | |
| 343 | *invocation* | `instance_invoke` \| `class_invoke` \| `libra
ry_invoke` \| `instance_invokeGetter` \| `class_invokeGetter` \| `library_invoke
Getter` \| `instance_invokeSetter` \| `class_invokeSetter` \| `library_invokeSet
ter` \| `delegate` \| `apply` \| `newInstance` | |
| 344 | *naming* | `simpleName` \| `qualifiedName` \| `construct
orName` | |
| 345 | *classification* | `isPrivate` \| `isTopLevel` \| `isImport` \|
`isExport` \| `isDeferred` \| `isShow` \| `isHide` \| `isOriginalDeclaration` \|
`isAbstract` \| `isStatic` \| `isSynthetic` \| `isRegularMethod` \| `isOperator
` \| `isGetter` \| `isSetter` \| `isConstructor` \| `isConstConstructor` \| `isG
enerativeConstructor` \| `isRedirectingConstructor` \| `isFactoryConstructor` \|
`isFinal` \| `isConst` \| `isOptional` \| `isNamed` \| `hasDefaultValue` \| `ha
sReflectee` \| `hasReflectedType` | |
| 346 | *annotation* | `metadata` | |
| 347 | *typing* | `instance_type` \| `variable_type` \| `parame
ter_type` \| `typeVariables` \| `typeArguments` \| `originalDeclaration` \| `isS
ubtypeOf` \| `isAssignableTo` \| `superclass` \| `superinterfaces` \| `mixin` \|
`isSubclassOf` \| `returnType` \| `upperBound` \| `referent` | |
| 348 | *concretization* | `reflectee` \| `reflectedType` | |
| 349 | *introspection* | `owner` \| `function` \| `uri` \| `library_de
clarations` \| `class_declarations` \| `libraryDependencies` \| `sourceLibrary`
\| `targetLibrary` \| `prefix` \| `combinators` \| `instanceMembers` \| `staticM
embers` \| `parameters` \| `callMethod` \| `defaultValue` | |
| 350 | *text* | `location` \| `source` | |
| 351 |
| 352 **Figure 1.** Reflectable capability language API raw material. |
| 353 |
| 354 Figure 2 shows a reduction of this raw material to a set of capabilities |
| 355 that we consider reasonable. It does not allow programmers to select |
| 356 their capabilities with the same degree of detail, but we expect that the |
| 357 complexity reduction is sufficiently valuable to justify the less |
| 358 fine-grained control. |
| 359 |
| 360 We have added *`RegExp`* arguments, specifying that each of these |
| 361 capabilities can meaningfully apply a pattern matching constraint to |
| 362 select the methods, getters, etc. which are included. With the empty |
| 363 *`RegExp`* as the default value, all entities in the relevant category |
| 364 are included when no *`RegExp`* is given. Similarly, we have created |
| 365 variants taking a *`MetadataClass`* argument which expresses that an |
| 366 entity in the relevant category is included iff it has been annotated |
| 367 with metadata whose type is a subtype of the given |
| 368 *`MetadataClass`*. This provides support for centralized and slightly |
| 369 abstract selection of entities using regular expressions, and it provides |
| 370 support for decentralized selection of entities using metadata to |
| 371 explicitly mark the entities. |
| 372 |
| 373 It is important to note that the *`MetadataClass`* is potentially |
| 374 unrelated to the package reflectable: We expect the use case where some |
| 375 class `C` from a package `P` unrelated to reflectable happens to fit |
| 376 well, because instances of `C` are already attached as metadata to the |
| 377 relevant set of members, which would in turn be the case because the need |
| 378 for reflection arises as a consequence of the semantics of `C` as |
| 379 metadata relative to `P`. |
| 380 |
| 381 | **Non-terminal** | **Expansion** | |
| 382 | ------------------------------ | ------------------------------ | |
| 383 | *apiSelection* | *invocation* \| *naming* \| *classification*
\| *annotation* \| *typing* \| *introspection* | |
| 384 | *invocation* | `instanceInvoke([`*`RegExp`*`])` \| `instance
InvokeMeta(`*`MetadataClass`*`)` \| `staticInvoke([`*`RegExp`*`])` \| `staticInv
okeMeta(`*`MetadataClass`*`)` \| `newInstance([`*`RegExp`*`])` \| `newInstanceMe
ta(`*`MetadataClass`*`)` | |
| 385 | *naming* | `name` | |
| 386 | *classification* | `classify` | |
| 387 | *annotation* | `metadata` | |
| 388 | *typing* | `type([`*`UpperBound`*`])` \| `typeRelations`
| |
| 389 | *introspection* | `owner` \| `declarations` \| `uri` \| `librar
yDependencies` | |
| 390 |
| 391 **Figure 2.** Reflectable capability language API grammar tokens. |
| 392 |
| 393 The category *text* was removed because we do not plan to support |
| 394 reflective access to the source code as a whole at this point; *naming* |
| 395 has been expressed atomically as name because we do not want to |
| 396 distinguish among the different kinds of names, and similarly for all the |
| 397 *classification* predicates. The category *concretization* was removed |
| 398 because it is trivial to support these features, so they are always |
| 399 enabled. |
| 400 |
| 401 We have omitted `apply` and `function` because we do not have support for |
| 402 `ClosureMirror` and we do not expect to get it anytime soon. |
| 403 |
| 404 Moreover, `delegate` was made implicit such that the ability to invoke a |
| 405 method implies the ability to delegate to it. |
| 406 |
| 407 The category *typing* was simplified in several ways: `instance_type` was |
| 408 renamed into `type` because of its prominence. It optionally receives an |
| 409 *`UpperBound`* argument which puts a limit on the available class mirrors |
| 410 (class mirrors will only be supported for classes which are subclasses of |
| 411 that *`UpperBound`*). The method `reflectType` on reflectors is only |
| 412 supported when this capability is present, and only on class mirrors |
| 413 passing the *`UpperBound`*, if any. The capabilities `variable_type`, |
| 414 `parameter_type`, and `returnType` were unified into `type` because they |
| 415 are concerned with lookups for the same kind of mirrors. To give some |
| 416 control over the level of detail in the type related mirrors, |
| 417 `typeVariables`, `typeArguments`, `originalDeclaration`, `isSubtypeOf`, |
| 418 `isAssignableTo`, `superclass`, `superinterfaces`, `mixin`, |
| 419 `isSubclassOf`, `upperBound`, and `referent` were unified into |
| 420 `typeRelations`; they all address relations among types, type variables, |
| 421 and `typedef`s, and it may be a substantial extra cost to preserve |
| 422 information about these topics if it is not used. |
| 423 |
| 424 The category *introspection* was also simplified: We unified |
| 425 `class_declarations`, `library_declarations`, `instanceMembers`, |
| 426 `staticMembers`, `callMethod`, `parameters`, and `defaultValue` into |
| 427 `declarations`. Finally we unified the import and export properties into |
| 428 `libraryDependencies` such that it subsumes `sourceLibrary`, |
| 429 `targetLibrary`, `prefix`, and `combinators`. We have retained the |
| 430 `owner` capability separately, because we expect the ability to look up |
| 431 the enclosing declaration for a given declaration to be too costly to |
| 432 include implicitly as part of another capability; and we have retained |
| 433 the `uri` capability separately because the preservation of information |
| 434 about URIs in JavaScript translated code (which is needed in order to |
| 435 implement the method uri on a library mirror) has been characterized as a |
| 436 security problem in some contexts. |
| 437 |
| 438 Note that certain reflective methods are **non-elementary** in the sense |
| 439 that they can be implemented entirely based on other reflective methods, |
| 440 the **elementary** ones. This affects the following capabilities: |
| 441 `isSubtypeOf`, `isAssignableTo`, `isSubclassOf`, `instanceMembers`, and |
| 442 `staticMembers`. These methods can be implemented in a general manner even |
| 443 for transformed programs, so they are provided as part of the package |
| 444 reflectable rather than being generated. Hence, they are supported if and |
| 445 only if the methods they rely on are supported. This is what it means |
| 446 when we say that `instanceMembers` is 'unified into' `declarations`. |
| 447 |
| 448 ### Covering Multiple API Based Capabilities Concisely |
| 449 |
| 450 In order to avoid overly verbose syntax in the cases where relatively |
| 451 broad reflection support is requested, we have chosen to introduce some |
| 452 grouping tokens. They do not contribute anything new, they just offer a |
| 453 more concise notation for certain selections of capabilities that are |
| 454 expected to occur together frequently. Figure 3 shows these grouping |
| 455 tokens. As an aid to remember what this additional syntax means, we have |
| 456 used words ending in 'ing' to give a hint about the tiny amount of |
| 457 abstraction involved in grouping several capabilities into a single |
| 458 construct. |
| 459 |
| 460 |
| 461 | **Group** | **Meaning** | |
| 462 | ------------------------------ | -------------------------------- | |
| 463 | `invoking([`*`RegExp`*`])` | `instanceInvoke([`*`RegExp`*`])`, `staticInvo
ke([`*`RegExp`*`])`, `newInstance([`*`RegExp`*`])` | |
| 464 | `invokingMeta(`*`MetadataClass`*`)` | `instanceInvokeMeta(`*`MetadataClass`*`)
`, `staticInvokeMeta(`*`MetadataClass`*`)`, `newInstanceMeta(`*`MetadataClass`*`
>)` | |
| 465 | `typing([`*`UpperBound`*`])` | `type([`*`UpperBound`*`])`, `name`, `classify
`, `metadata`, `typeRelations`, `owner`, `declarations`, `uri`, `libraryDependen
cies` | |
| 466 |
| 467 **Figure 3.** Grouping tokens for the reflectable capability language. |
| 468 |
| 469 The semantics of including the capability `invoking(`*`RegExp`*`)` where |
| 470 *`RegExp`* stands for a given argument is identical to the semantics of |
| 471 including all three capabilities in the same row on the right hand side |
| 472 of the figure, giving all of them the same *`RegExp`* as |
| 473 argument. Similarly, `invoking()` without an argument requests support |
| 474 for reflective invocation of all instance methods, all static methods, |
| 475 and all constructors. The semantics of including the capability |
| 476 `invokingMeta(`*`MetadataClass`*`)` is the same as the semantics of including |
| 477 all three capabilities to the right in the same row, with the same |
| 478 argument. Finally, the semantics of including `typing(`*`UpperBound`*`)` is to |
| 479 request support for all the capabilities on the right, passing |
| 480 *`UpperBound`* to the `type` capability; that is, requesting support for |
| 481 every feature associated with information about the program structure, |
| 482 bounded by the given *`UpperBound`*; and `typing()` without an argument |
| 483 works the same, only with `type()`. |
| 484 |
| 485 ## Specifying Reflectee Based Capabilities |
| 486 |
| 487 In the previous section we found a way to specify mirror API based |
| 488 capabilities as a grammar. It is very simple, because it consists of |
| 489 terminals only, apart from the fact that some of these terminals take an |
| 490 argument that is used to restrict the supported arguments to the matching |
| 491 names. As shown in Fig. 2, the non-terminal *`apiSelection`* covers them |
| 492 all. We shall use them several at a time, so the typical usage is a list, |
| 493 written as *`apiSelection*`*. |
| 494 |
| 495 In this section we discuss how the reflection support specified by a |
| 496 given *`apiSelection*`* can be requested for a specific set of |
| 497 program elements. Currently the only supported kind of program element is |
| 498 classes, but this will be generalized later. The program elements that |
| 499 receive reflection support are called the **targets** of the |
| 500 specification, and the specification itself is given as a |
| 501 superinitializer in a subclass (call it `MyReflectable`) of class |
| 502 `Reflectable`, with a unique instance (call it `myReflectable`). Now, |
| 503 `myReflectable` is used as metadata somewhere in the program, and each |
| 504 kind of capability is only applicable as an annotation in certain |
| 505 locations, which is discussed below. |
| 506 |
| 507 Figure 4 shows how capabilities and annotations can be constructed, |
| 508 generally starting from an *`apiSelection*`*. The non-terminals in |
| 509 this part of the grammar have been named after the intended location of |
| 510 the metadata which is or contains a capability of the corresponding kind. |
| 511 |
| 512 |**Non-terminals** | **Expansions** | |
| 513 | ------------------------------- | ------------------------------ | |
| 514 | *reflector* | `Reflectable(`*`targetMetadata`*`)` | |
| 515 | *targetMetadata* | *`apiSelection*`* \| `subtypeQuantify(`*`api
Selection*`*`)` \| `admitSubtype(`*`apiSelection*`*`)` | |
| 516 | *globalMetadata* | `globalQuantify(`*`RegExp`*`, `*`reflector`*
`)` \| `globalQuantifyMeta(`*`MetadataClass`*`, `*`reflector`*`)` | |
| 517 |
| 518 **Figure 4.** Reflectable capability language target selection. |
| 519 |
| 520 In practice, a *`reflector`* is an instance of a subclass of class |
| 521 `Reflectable` that is directly attached to a class as metadata, or passed |
| 522 to a global quantifier; in the running example it is the object |
| 523 `myReflectable`. The reflector has one piece of state that we model with |
| 524 *`targetMetadata`*. In the grammar in Fig. 4 we use the identifier |
| 525 `Reflectable` to stand for all the subclasses, and we model the state by |
| 526 letting it take the corresponding *`targetMetadata`* as an argument. The |
| 527 semantics of annotating a class with a given *`reflector`* depends on the |
| 528 *`targetMetadata`*, as described below. |
| 529 |
| 530 A *`targetMetadata`* capability can be a base level set of capabilities, |
| 531 that is, an *`apiSelection*`*, and it can also be a quantifier taking |
| 532 such an *`apiSelection*`* as an argument. The semantics of attaching |
| 533 a *`reflector`* containing a plain *`apiSelection*`* to a target |
| 534 class `C` is that reflection support at the level specified by the given |
| 535 *`apiSelection*`* is provided for the class `C` and instances |
| 536 thereof. The semantics of attaching a *`reflector`* containing |
| 537 `subtypeQuantify(`*`apiSelection*`*`)` to a class `C` is that the reflection |
| 538 support specified by the given *`apiSelection*`* is provided for all |
| 539 classes which are subtypes of the class `C`, including `C` itself, and |
| 540 their instances. The semantics of attaching a *`reflector`* containing |
| 541 `admitSubtype(`*`apiSelection*`*`)` to a class `C` is a pragmatic mix of the |
| 542 former two which is subtle enough to warrant a slightly more detailed |
| 543 discussion, given in the next section. The basic idea is that it allows |
| 544 instances of subtypes of the target class to be treated as if they were |
| 545 instances of the target class. |
| 546 |
| 547 Finally, we support side tags using global quantifiers, |
| 548 `globalQuantify(`*`RegExp`*`, `*`reflector`*`)` and |
| 549 `globalQuantifyMeta(`*`MetadataClass`*`, `*`reflector`*`)`. Currently, we have |
| 550 decided that they must be attached as metadata to an import statement |
| 551 importing `package:reflectable/reflectable.dart`, but we may relax this |
| 552 restriction if other placements turn out to be helpful in practice. Due |
| 553 to the monotone semantics of capabilities it is not a problem if a given |
| 554 program contains more than one such *`globalMetadata`*, the provided |
| 555 reflection support will simply be the least one that satisfies all |
| 556 requests. |
| 557 |
| 558 The semantics of having `globalQuantify(`*`RegExp`*`, `*`reflector`*`)` in a pro
gram |
| 559 is ideally identical to the semantics of having the given *`reflector`* |
| 560 attached directly to each of those classes in the program whose qualified |
| 561 name matches the given *`RegExp`*. Similarly, the semantics of having |
| 562 `globalQuantifyMeta(`*`MetadataClass`*`, `*`reflector`*`)` in a program is ideal
ly |
| 563 identical to the semantics of having the given *`reflector`* attached |
| 564 directly to each of those classes whose metadata includes an instance of |
| 565 type *`MetadataClass`*. At this point, however, we must add an adjustment |
| 566 to the ideal goal that the semantics is identical: Access to private |
| 567 declarations may not be fully supported with a *`globalMetadata`*. This |
| 568 is discussed in the next section. |
| 569 |
| 570 ### Completely or Partially Mirrored Instances? |
| 571 |
| 572 Traditionally, it is assumed that reflective access to an instance, a |
| 573 class, or some other entity will provide a complete and faithful view of |
| 574 that entity. For instance, it should be possible for reflective code to |
| 575 access features declared as private even when that reflective code is |
| 576 located in a context where non-reflective access to the same features |
| 577 would not be allowed. Moreover, when a reflective lookup is used to learn |
| 578 which class a given object is an instance of, it is expected that the |
| 579 response describes the actual runtime type of the object, and not some |
| 580 superclass such as the statically known type of that object in some |
| 581 context. |
| 582 |
| 583 In the package reflectable there are reasons for violating this |
| 584 completeness assumption, and some of them are built-in consequences of |
| 585 the reasons for having this package in the first place. In other words, |
| 586 these restrictions will not go away entirely. Other restrictions may be |
| 587 lifted in the future, because they were introduced based on certain |
| 588 trade-offs made in the implementation of the package. |
| 589 |
| 590 The main motivation for providing the package reflectable is that the |
| 591 more general support for reflection provided by the `dart:mirrors` |
| 592 package tends to be too costly at runtime in terms of program |
| 593 size. Hence, it is a core point for package reflectable to specify a |
| 594 restricted version of reflection that fits the purposes of a given |
| 595 program, such that it can be done using a significantly smaller amount of |
| 596 space. Consequently, it will be perfectly normal for such a program to |
| 597 have reflective support for an object without reflective access to, say, |
| 598 some of its methods. There are several other kinds of coverage which is |
| 599 incomplete by design, and they are not a problem: they are part of the |
| 600 reason for using package reflectable in the first place. |
| 601 |
| 602 The following subsections discuss two different situations where some |
| 603 restrictions apply that are not there by design. We first discuss cases |
| 604 where access to private features is incomplete, and then we discuss the |
| 605 consequences of admitting subtypes as specified with |
| 606 `admitSubtype(`*`apiSelection*`*`)`. |
| 607 |
| 608 #### Privacy Related Restrictions |
| 609 |
| 610 The restrictions discussed in this subsection are motivated by trade-offs |
| 611 in the implementation in package reflectable, so we need to mention some |
| 612 implementation details. The package reflectable has been designed for |
| 613 program transformation, i.e., it is intended that a source to source |
| 614 transformer shall be able to receive a given program (which is using |
| 615 package reflectable, and indirectly `dart:mirrors`) as input, and |
| 616 transform it to an equivalent program that does not use `dart:mirrors`, |
| 617 generally by generating mirror implementation classes containing |
| 618 ordinary, non-reflective code. |
| 619 |
| 620 Ordinary code cannot violate privacy restrictions. Hence, reflective |
| 621 operations cannot, say, read or write a private field in a different |
| 622 library. The implication is that private access can only be supported for |
| 623 classes declared in a library which can be transformed, because only then |
| 624 can the generated mirror implementation class coexist with the target |
| 625 class. Using a transformer as we currently do (and plan to do in the |
| 626 future), all libraries in the client package can be transformed. However, |
| 627 libraries outside the client package cannot be transformed, which in |
| 628 particular matters for libraries from other packages, and for pre-defined |
| 629 libraries. |
| 630 |
| 631 For some libraries which cannot be transformed, it would be possible to |
| 632 create a local copy of the library in the client package and modify the |
| 633 program globally such that it uses that local copy, and such that the |
| 634 semantics of the copied library is identical to the semantics of the |
| 635 original. This cannot be done for libraries that contain language |
| 636 primitives which cannot be implemented in the language; for instance, the |
| 637 pre-defined class int cannot be replaced by a user-defined |
| 638 class. Moreover, the need to copy and adjust one library could propagate |
| 639 to another library, e.g., if the latter imports the former. Hence, not |
| 640 even this workaround would enable transformation of all libraries. We do |
| 641 not currently have any plans to use this kind of techniques, and hence |
| 642 only libraries in the current package can be transformed. |
| 643 |
| 644 Given that the main transformation technique for package reflectable is |
| 645 to generate a number of mirror classes for each target class, this means |
| 646 that access to private declarations cannot be supported for classes in |
| 647 libraries that cannot be transformed. This applies to private classes as |
| 648 a whole, and it applies to private declarations in public classes. |
| 649 |
| 650 It should be noted that transformation of libraries imported from other |
| 651 packages might be manageable to implement, but it requires changes to the |
| 652 basic tools used to process Dart programs, e.g., pub. Alternatively, it |
| 653 is possible that this restriction can be lifted in general in the future, |
| 654 if the tools (compilers, analyzers, etc.) are modified to support a |
| 655 privacy overriding mechanism. |
| 656 |
| 657 #### Considerations around Admitting Subtypes |
| 658 |
| 659 When a *`targetMetadata`* on the form *`apiSelection*`* is attached to a |
| 660 given class `C`, the effect is that reflection support is provided for |
| 661 the class `C` and for instances of `C`. However, that support can be |
| 662 extended to give partial reflection support for instances of subtypes of |
| 663 `C` in a way that does not incur further costs in terms of program size: |
| 664 A mirror generated for instances of class `C` can have a `reflectee` (the |
| 665 object being mirrored by that mirror) whose type is a proper subtype of |
| 666 `C`. A *`targetMetadata`* on the form `admitSubtype(`*`apiSelection*`*`)` is |
| 667 used to specify exactly this: It enables an instance mirror to hold a |
| 668 reflectee which is an instance of a proper subtype of the type that the |
| 669 mirror was generated for. |
| 670 |
| 671 The question arises which instance mirror to use for a given object *O* |
| 672 with runtime type `D` which is given as an argument to the operation |
| 673 `reflect` on a reflector, when there is no mirror class which was created |
| 674 for exactly `D`. This is the situation where a subtype reflectee is |
| 675 actually admitted. In general, there may be multiple candidate mirror |
| 676 classes corresponding to classes `C1, C2, .. Ck` which are "least |
| 677 supertypes of `D`" in the sense that no type `E` is a proper supertype of |
| 678 `D` and a proper subtype of `Ci` for any `i` (this also implies that no |
| 679 two classes `Ci` and `Cj` are subtypes of each other). The language |
| 680 specification includes an algorithm which will find a uniquely determined |
| 681 supertype of `C1 .. Ck` which is called their **least upper bound**. We |
| 682 cannot use this algorithm directly because we have an arbitrary subset of |
| 683 the types in a type hierarchy rather than all types, and then we need to |
| 684 make a similar decision for this "sparse" subtype hierarchy that only |
| 685 includes classes with reflection support from the given |
| 686 reflector. Nevertheless, we expect that it is possible to create a |
| 687 variant of the least upper bound algorithm which will work for these |
| 688 sparse subtype hierarchies. |
| 689 |
| 690 It should be noted that a very basic invariant which is commonly assumed |
| 691 for reflection support in various languages is violated: An instance |
| 692 mirror constructed for type `C` can have a reflectee which is an instance |
| 693 of a proper subtype `D`. Of course, not all mirror systems have anything |
| 694 like the notion of a mirror that is constructed for a given type, but the |
| 695 corresponding problem is relevant everywhere: The mirror will not report |
| 696 on the properties of the object as-is, it will report on the properties |
| 697 of instances of a supertype. This is a kind of incompleteness, and it |
| 698 even causes the mirror to give plain *incorrect* descriptions of the |
| 699 object in some cases. |
| 700 |
| 701 In particular, assume that an object *O* with runtime type `D` is given, |
| 702 and that we have an instance mirror *IM* whose reflectee is *O*. Assume |
| 703 that the class of *IM* was generated for instances of a class `C`, which |
| 704 is a proper supertype of `D`. It is only because of `admitSubtype` that |
| 705 it is even possible for *IM* to have a `reflectee` whose `runtimeType` is |
| 706 not `C`. In many situations this discrepancy makes no difference and *IM* |
| 707 works fine with *O*, but it is informative to focus on a case where it |
| 708 really matters: |
| 709 |
| 710 <!-- Cannot make 'IM' italic in 'IM.type', hence using a rather messy --> |
| 711 <!-- mixed notation. --> |
| 712 |
| 713 Let us use a reflective operation on *IM* to get a class mirror for the |
| 714 class of *O*. *IM*.`type` will return an instance *CM* of the class mirror |
| 715 for `C`, not a class mirror for *O*'s actual runtime type `D`. If a |
| 716 programmer uses this approach to look up the name of the class of an |
| 717 object like *O*, the answer will simply be wrong, it says `"C"` and it |
| 718 should have said `"D"`. Similarly, if we traverse the superclasses we |
| 719 will never see the class `D`, nor the intermediate classes between `D` |
| 720 and `C`. A real-world example is serialization: if we look up the |
| 721 declarations of fields in order to serialize the reflectee then we will |
| 722 silently fail to include the fields declared in the ignored subclasses |
| 723 down to `D`. In general, there are many unpleasant surprises waiting for |
| 724 the naive user of this feature, so it should be considered to be an |
| 725 expert-only option. |
| 726 |
| 727 Why not just do the "right thing" and return a class mirror for `D`? It is |
| 728 not possible to simply check the `runtimeType` of `reflectee` in the |
| 729 implementation of the method type, and then deliver a class mirror of `D` |
| 730 because, typically, there *is* no class mirror for `D`. In fact, the whole |
| 731 point of having the `admitSubtype` quantifier is that it saves space |
| 732 because a potentially large number of subtypes of a given type can be |
| 733 given partial reflection support without the need to generate a |
| 734 correspondingly large number of mirror classes. |
| 735 |
| 736 To further clarify what it means to get 'partial' reflective support, |
| 737 consider some cases: |
| 738 |
| 739 Reflectively calling instance methods on *O* which are declared in `C` or |
| 740 inherited into `C` will work as expected, and standard object-oriented |
| 741 method invocation will ensure that it is the correct method |
| 742 implementation for *O* which is called, not just the most specific |
| 743 implementation which is available in `C`. |
| 744 |
| 745 Calling instance methods on *O* which are declared in a proper subtype of |
| 746 `C`, including methods from `D` itself, will not work. This is because |
| 747 the class of *IM* has been generated under the assumption that no such |
| 748 methods exist, it only knows about `C` methods. As mentioned, if we fetch |
| 749 the class of *O* we may get a proper supertype of the actual class of |
| 750 *O*, and hence all the derived operations will be similarly affected. For |
| 751 instance, the declarations from *CM* will be the declarations in `C`, and |
| 752 they have nothing to do with the declarations in `D`. Similarly, if we |
| 753 traverse the superclasses then we will only see a strict suffix of the |
| 754 actual list of superclasses of the class of *O*. |
| 755 |
| 756 Based on these serious issues, we have decided that when an instance |
| 757 mirror is associated with the `admitSubtype` quantifier, it shall be an |
| 758 error to execute the `type` method in order to obtain a mirror of a |
| 759 class, because it is very unlikely to work as intended when that class is |
| 760 in fact not the class of the reflectee. It would be possible to allow it |
| 761 in the cases where the match happens to be perfect, but this would be |
| 762 difficult for programmers to use, and they may as well use `reflectType` |
| 763 directly if they want to reflect upon a class which is not taken directly |
| 764 from an instance. |
| 765 |
| 766 In summary, there is a delicate trade-off to make in the case where an |
| 767 entire subtype hierarchy should be equipped with reflection support. The |
| 768 trade off is to either pay the price in terms of program size and get |
| 769 full support (using `subtypeQuantify`); or to save space aggressively and |
| 770 in return tolerate the partial support for reflection (using |
| 771 `admitSubtype`). |
| 772 |
| 773 # Summary |
| 774 |
| 775 We have described the design of the capabilities used in the package |
| 776 reflectable to specify the desired level of support for reflection. The |
| 777 underlying idea is that the capabilities at the base level specify a |
| 778 selection of operations from the API of the mirror classes, along with |
| 779 some simple restrictions on the allowable arguments to those |
| 780 operations. On top of that, the API based capabilities can be associated |
| 781 with specific parts of the target program (though at this point only |
| 782 classes) such that exactly those classes will have the reflection support |
| 783 specified with the API based capabilities. The target classes can be |
| 784 selected individually, by adding a reflector as metadata on each target |
| 785 class. Alternatively, target classes can be selected by quantification: |
| 786 It is possible to quantify over all subtypes, in which case not only the |
| 787 class `C` that holds the metadata receives reflection support, but also all |
| 788 subtypes of `C`. Finally, it is possible to admit instances of subtypes as |
| 789 reflectees of a small set of mirrors, such that partial reflection |
| 790 support is achieved for many classes, without the cost of having many |
| 791 mirror classes. |
| 792 |
| 793 # References |
| 794 |
| 795 1. Gilad Bracha and David Ungar. "Mirrors: design principles for |
| 796 meta-level facilities of object-oriented programming languages". ACM |
| 797 SIGPLAN Notices. 24 Oct. 2004: 331-344. |
| 798 2. Brian Cantwell Smith. "Procedural reflection in programming |
| 799 languages." 1982. |
| 800 3. Jonathan M. Sobel and Daniel P. Friedman. "An introduction to |
| 801 reflection-oriented programming." Proceedings of Reflection. |
| 802 Apr. 1996. |
OLD | NEW |