Implement stricter rule about self referencing typedefs (fix issue 13675).

runtime/vm/object.cc

Index: runtime/vm/object.cc
===================================================================
--- runtime/vm/object.cc	(revision 29405)
+++ runtime/vm/object.cc	(working copy)
@@ -2416,29 +2416,46 @@
 
 void Class::PatchSignatureFunction(const Function& signature_function) const {
   ASSERT(!signature_function.IsNull());
+  set_signature_function(signature_function);
   const Class& owner_class = Class::Handle(signature_function.Owner());
   ASSERT(!owner_class.IsNull());
-  TypeArguments& type_parameters = TypeArguments::Handle();
-  // A signature class extends class Instance and is parameterized in the same
-  // way as the owner class of its non-static signature function.
-  // It is not type parameterized if its signature function is static.
-  // In case of a function type alias, the function owner is the alias class
-  // instead of the enclosing class.
-  if (!signature_function.is_static() &&
-      (owner_class.NumTypeParameters() > 0) &&
-      !signature_function.HasInstantiatedSignature()) {
-    type_parameters = owner_class.type_parameters();
-  }
-  set_signature_function(signature_function);
-  set_type_parameters(type_parameters);
+  // A signature class extends class Instance and is either not parameterized or
+  // parameterized with exactly the same list of type parameters as the owner
+  // class of its function.
+  // In case of a function type alias, the function owner is the alias class,
+  // which is also the signature class. The signature class is therefore
+  // parameterized according to the alias class declaration, even if the
+  // function type is not generic.
+  // Otherwise, if the function is static or if its signature type is
+  // non-generic, i.e. it does not depend on any type parameter of the owner
+  // class, then the signature class is not parameterized, although the owner
+  // class may be.
   if (owner_class.raw() == raw()) {
     // This signature class is an alias, which cannot be the canonical
     // signature class for this signature function.
     ASSERT(!IsCanonicalSignatureClass());
-  } else if (signature_function.signature_class() == Object::null()) {
-    // Make this signature class the canonical signature class.
-    signature_function.set_signature_class(*this);
-    ASSERT(IsCanonicalSignatureClass());
+    // Do not modify the declared type parameters of the alias, even if unused.
+  } else {
+    // Copy the type parameters only for an instance function type that is not
+    // instantiated, i.e. that depends on the type parameters of the owner
+    // class.
+    // TODO(regis): Verify that it is not a problem for the copied type
+    // parameters to refer to the owner class rather than to the signature

[rmacnak, 2013/10/29 00:44:21]

I don't think there's a way to tell reflectively. We can only get to this type
parameters through the typedef, and assign (mirror) ownership there. Unlike
classes, where one can get at type variables through type arguments of other
classes or return/parameter/field types.

[regis, 2013/10/29 16:59:24]

In case of a typedef, the type parameters are not copied. So no problem there. I
think that instantiation of these type parameters will always occur via the
owner class as instantiator, which is correct. However, if reflection allows you
to get to the signature class of a closurized function, and then to its type
parameters, you may notice that they belong to the function owner class, rather
than to this signature class. This is my concern.

+    // class. In other words, uninstantiated function types should only get
+    // instantiated by the owner class as instantiator and never by the
+    // signature class itself.
+    TypeArguments& type_parameters = TypeArguments::Handle();
+    if (!signature_function.is_static() &&
+        (owner_class.NumTypeParameters() > 0) &&
+        !signature_function.HasInstantiatedSignature()) {
+      type_parameters = owner_class.type_parameters();
+    }
+    set_type_parameters(type_parameters);
+    if (signature_function.signature_class() == Object::null()) {
+      // Make this signature class the canonical signature class.
+      signature_function.set_signature_class(*this);
+      ASSERT(IsCanonicalSignatureClass());
+    }
   }
   set_is_prefinalized();
 }