Newsletter 2017-09-01.

docs/newsletter/20170901.md

+# Dart Language and Library Newsletter
+2017-09-01
+@floitschG
+
+Welcome to the Dart Language and Library Newsletter.
+
+## The Case Against Call
+Dart 1.x supports callable objects. By adding a `call` method to a class, instances of this class can be invoked as if they were functions:
+
+``` dart
+class Square {
+  int call(int x) => x * x;
+  toString() => "Function that squares its input";
+}
+
+main() {
+  var s = new Square();
+  print(s(4));  // => 16.
+  print(s);  // => Function that squares its input.
+  print(s is int Function(int));  // => true.
+}
+```
+
+Note that `Square` doesn't need to implement any `Function` interface: as soon as there is a `call` method, all instances of the class can be used as if they were closures.
+
+While we generally like the this feature (let's be honest: it's pretty cool), the language team is trying to eventually remove it from the language. In this section, we explain the reasons for this decision.
+
+### Wrong Name
+Despite referring to the feature as the "call operator", it is actually not implemented as an operator. Instead of writing the call operator similarly to other operators (like plus, minus, ...), it's just a special method name.
+
+As an operator we would write the `Square` class from above as follows:
+``` dart
+class Square {
+  int operator() (int x) => x * x;
+}
+```
+
+Some developers actually prefer the "call" name, but the operator syntax wouldn't just be more consistent. It would also remove the weird case where we can tear off `call` methods infinitely:
+
+``` dart
+var s = new Square();
+var f = s.call.call.call.call.call.call;
+print(f(3));  // => 9;
+```
+
+If the `call` operator was an actual operator, there wouldn't be any way to tear off the operator itself.
+
+### Tear-Offs are Too Good
+Tearing off a function is trivial in Dart. Simply referring to the corresponding method member tears off the bound function:
+
+``` dart
+class Square {
+  int square(int x) => x * x;
+}
+
+main() {
+  var s = new Square();
+  var f = s.square;
+  print(f(3));  // => 9.
+}
+```
+
+The most obvious reason for a call-operator is to masquerade an instance as a function. However, with easy tear-offs, one can just tear off the method and pass that one instead. The only pattern where this doesn't work, is if users need to cast a function type back to an object, or if they rely on specific `hashCode`, equality or `toString`.
+
+The following contrived example shows how a program could use these properties.
+
+``` dart
+// The `Element` class and the `disposedElements` getter are provided
+// by some framework.
+
+/// An element that reacts to mouse clicks.
+class Element {
+  /// The element's click handler is a function that takes a `MouseEvent`.
+  void Function(MouseEvent) clickCallback;
+}
+
+/// A stream that informs the user of elements that have been disposed.
+Stream<Element> disposedElements = ...;
+
+// ============= The following code corresponds to user code. =====
+
+// Attaches a click handler to the element of the given name
+// and writes the clicks to a file.
+void logClicks(String name) {
+  var sink = new File("$name.txt").openWrite();
+  var element = screen.getElement(name);
+  element.clickCallback = sink.writeln;
+}
+
+main() {
+  logClicks('demo');
+  logClicks('demo2');
+  disposedElements.listen((element) {
+    // Would like to close the file for the registered handlers.
+    // ------
+  });
+}
+```
+In the beginning of `main` the program registers some callbacks on UI elements. However, when these elements are disposed of, the program currently does not know how to find the `IOSink` that corresponds to the element that is removed.
+
+One easy solution is to add a global map that stores the mapping between the elements and the opened sinks. Alternatively, we can introduce a callable class that stores the open file:
+
+``` dart
+// A class that connects the open output file with the handlers.
+class ClickHandler {
+  final IOSink sink;
+  ClickHandler(this.sink);
+  void call(Object event) {
+    sink.writeln(event);
+  }
+}
+
+// Attaches a click handler to the element of the given name
+// and writes the clicks to a file.
+void logClicks(String name) {
+  var sink = new File("$name.txt").openWrite();
+  var handler = new ClickHandler(sink);
+  var element = screen.getElement(name);
+  // Uses the callable object as handler.
+  element.clickCallback = handler;
+}
+
+main() {
+  logClicks('demo');
+  logClicks('demo2');
+  disposedElements.listen((element) {
+    // ============
+    // Casts the function back to a `ClickHandler` class.
+    var handler = element.clickCallback as ClickHandler;
+    // Now we can close the sink.
+    handler.sink.close();
+  });
+}
+```
+
+By using a callable class, the program can store additional information with the callback. When the framework tells us which element has been disposed, the program can retrieve the handler, cast it back to `ClickHandler` and read the `IOSink` out of it.
+
+Fortunately, these patterns are very rare, and usually there are many other ways to solve the problem. If you know real world programs that require these properties, please let us know.
+
+### Typing
+A class that represents, at the same time, a nominal type and a structural function type tremendously complicates the type system.
+
+As a first example, let's observe a class that uses a generic type as parameter type to its `call` method:
+
+``` dart
+class A<T> {
+  void call(T arg) {};
+}
+
+main() {
+  var a = new A<num>();
+  A<Object> a2 = a;  // OK.
+  void Function(int) f = a;  // OK.
+  // But:
+  A<int> a3 = a;  // Error.
+  void Function(Object) f2 = a;  // Error.
+}
+```
+
+Because Dart's generic types are covariant, we are allowed to assign `a` to `a2`. This is the usual `List<Apple>` is a `List<Fruit>`. (This is not always a safe assignment, but Dart adds checks to ensure that programs still respect heap soundness.)
+
+Similarly, it feels natural to say that `a` which represents a `void Function(T)`, with `T` equal to `num`, can be used as a `void Function(int)`. After all, if the method is only invoked with integers, then the `num` is clearly good enough.
+
+Note that the assignment to `a2` uses a supertype (`Object`) of `num` at the left-hand side, whereas the assignment to `f` uses a subtype (`int`). We say that the assignment to `a2` is *covariant*, whereas the assignment to `f` is *contravariant* on the generic type argument.
+
+Our type system can handle these cases, and correctly inserts the necessary checks to ensure soundness. However, it would be nice, if we didn't have to deal with objects that are, effectively, bivariant.
+
+Things get even more complicated when we look at subtyping rules for `call` methods. Take the following "simple" example:
+
+``` dart
+class C {
+  void call(void Function(C) callback) => callback(this);
+}
+
+main() {
+  C c = new C();
+  c((_) => null);  // <=== ok.
+  c(c);  // <=== ok?
+}
+```
+
+Clearly, `C` has a `call` method and is thus a function. The invocation `c((_) => null)` is equivalent to `c.call((_) => null)`. So far, things are simple. The difficulty arises when `c` is passed an instance of type `C` (in this case `c` itself).
+
+The type system has to decide if an instance of type `C` (here `c`) is assignable to the parameter type. For simplicity, we only focus on subtyping, which corresponds to the intuitive "Apple" can be assigned to "Fruit". Usually, subtyping is written using the "<:" operator: `Apple <: Fruit`. This notation will make this text shorter (and *slightly* more formal).
+
+In our example, the type system thus wants to answer: `C <: void Function(C)`? Since `C` is compared to a function type, we have to look at `C`'s `call` method and use that type instead: `void Function(void Function(C))`. The type system can now compare these types structurally: `void Function(void Function(C)) <: void Function(C)`?
+
+It starts by looking at the return types. In our case these are trivially assignable: both are `void`. Next up are the parameter types: `void Function(C)` on the left, and `C` on the right. Since these types are in parameter position, we have to invert the operands. Formally, this inversion is due to the fact that argument types are in contravariant position. Intuitively, it's easy to see that a function that takes a *fruit function* (`Function(Fruit)`) can always be used in places where an *apple function* (`Function(Apple)`) is required: `Function(Fruit) <: Function(Apple)` because `Apple <: Fruit`.
+
+Getting back to our example, we had just concluded that the return types of `void Function(void Function(C)) <: void Function(C)` matched and were looking at the parameter types. After switching sides we have to check whether `C <: void Function(C)`.
+
+If this looks familiar, you paid attention: this is the question we tried to answer in the first place…
+
+Fundamentally, this means that Dart (with the `call` method) features recursive types. Depending on the resolution algorithm of the type system we can now either conclude that:
+- `C <: void Function(C)`, if we use a co-inductive algorithm that tracks recursion (which is just fancy wording for saying that we assume everything works and try to see if things break), or
+- `C </: void Function(C)`, if we use an inductive algorithm that tracks recursion. (Start with nothing, and build up the truth).
+
+This is just one out of multiple issues that `call` methods bring to Dart's typing system. Fortunately, we are not the first ones to solve these problems. Recursive type systems exist in the wild, and there are known algorithms to deal with them (for example Amadio and Cardelli http://lucacardelli.name/Papers/SRT.pdf), but they add lots of complexity to the type system.
+
+### Conclusion
+Given all the complications the `call` method, the language team intends to eventually remove this feature from the language.
+
+Our plan was to slowly phase `call` methods out over time, but we are now investigating, if we should take the jump with Dart 2.0, so that we can present a simpler type system for our Dart 2.0 specification.
+
+At this stage we are still collecting information, including looking at existing programs, and gathering feedback. If you use this feature and don't see an easy work-around please let us know.
+
+## Limitations on Generic Types
+A common operation in Dart is to look through an iterable, and only keep objects of a specific type.
+
+``` dart
+class A {}
+class B extends A {}
+
+void main() {
+  var itA = new Iterable<A>.generate(5, (i) => i.isEven ? new A() : new B());
+  var itB = itA.where((x) => x is B);
+}
+```
+In this example, `itA` is an `Iterable` that contains both `A`s and `B`s. The `where` method then filters these elements and returns an `Iterable` that just contains `B`s. It would thus be great to be able to use the returned `Iterable` as an `Iterable<B>`. Unfortunately, that's not the case:
+``` dart
+print(itB is Iterable<B>);  // => false.
+print(itB.runtimeType);  // => Iterable<A>.
+```
+The dynamic type of `itB` is still `Iterable<A>`. This becomes obvious, when looking at the signature of `where`: `Iterable<E> where(bool test(E element))` (where `E` is the generic type of the receiver `Iterable`).
+
+It's natural to wonder if we could improve the `where` function and allow the user to provide a generic type when they want to: `itA.where<B>((x) => x is B)`. If the user provides a type, then the returned iterable should have that generic type. Otherwise, the original type should be used:
+
+``` dart
+// We would like the following return types:
+var anotherItA = itA.where(randomBool);  // an Iterable<A>.
+var itB = itA.where<B>((x) => x is B);  // an Iterable<B>.
+```
+
+The signature of `where` would need to look somehow similar to:
+``` dart
+Iterable<T> where<T>(bool test(E element));
+```
+This signature would work for the second case, where the user provided a generic argument to the call, but would fail for the first case. Since there is no way for the type inference to find a type for the generic type, it would fill that type with `dynamic`. So, `anotherItA` would just be an `Iterable<dynamic>` and not `Iterable<A>`.
+
+The only way to provide "default" values for generics is to use the `extends` clause such as:
+``` dart
+Iterable<T> where<T extends E>(bool test(E element));
+```
+This is because Dart's type inference uses the bound of a generic type when no generic argument is provided.
+
+Running our tests, this looks promising:
+``` dart
+var anotherItA = itA.where(randomBool);
+print(anotherItA.runtimeType);  // => Iterable<A>.
+
+var itB = itA.where<B>((x) => x is B);
+print(itB.runtimeType);  // => Iterable<B>.
+```
+
+Clearly, given the title of this section, there must be a catch...
+
+While our simple examples work, adding this generic type breaks down with covariant generics (`List<Apple>` is a `List<Fruit>`). Let's try our new `where` function on a more sophisticated example:
+
+``` dart
+int nonNullLength(Iterable<Object> objects) {
+  return objects.where((x) => x != null).length;
+}
+
+var list = [1, 2];  // a List<int>.
+print(nonNullLength(list));
+```
+
+The `nonNullLength` function just filters out all elements that are `null` and returns the length of the resulting `Iterable`. Without our update to the `where` function this works perfectly. However, with our new function we get an error.
+
+The `where` in `nonNullLength` has no generic argument, and the type inference has to fill it in. Without any provided generic argument and no contextual information, the type inference uses the bound of the generic parameter. For our improved `where` function the generic parameter clause is `T extends E` and the bound is thus `E`. Within `nonNullLength` the provided argument `objects` is of type `Iterable<Object>` and the inference has to assume that `E` equals `Object`. The compiler statically inserts `Object` as generic argument to `where`.
+
+Clearly, `Object` is not a subtype of `int` (the actual generic type `E` of the provided `Iterable`). As such, a dynamic check must stop the execution and report an error. In Dart 2.0 the `nonNullLength` function would therefore throw.
+
+Type inference is only available in strong mode and Dart 2.0, and, so far, only DDC supports the new type system. (Also, this particular check is only implemented in a very recent DDC.) Eventually, all our tools will implement the required checks.
+
+Without actual default values for generic parameters, there isn't any good way to support a type-based `where`. At the moment, the language team has no intentions of adding this feature. However, we are going to add a new method on `Iterable` to filter for specific types. A new function, `of<T>()` or `ofType<T>`, will allow developers to filter an `Iterable` and get a new `Iterable` of the requested type.