Added generic method syntax to language specification.

docs/language/informal/generic-method-syntax.md

+# Feature: Generic Method Syntax
+
+## Feature: Generic Method Syntax
+
+**This document** is an informal specification of the support for generic methods and functions which has been implemented in `dart2js` with option `--generic-method-syntax`, starting with commit [acc5f59](https://github.com/dart-lang/sdk/commit/acc5f59a99d5d8747459c935e6360ac325606cc6). In SDK 1.21 this feature is available by default (i.e., also without the option) in the virtual machine and the analyzer, as well as in `dart2js`.

[Lasse Reichstein Nielsen, 2017/03/06 09:45:14]

Consider breaking paragraphs into shorter lines.

If nothing else, it makes it easier to comment on in the code-review tool!

+
+The **motivation for** having this **feature** is that it enables partial support for generic methods and functions, thus providing a bridge between not having generic methods and having full support for generic methods. In particular, code declaring and using generic methods may be type checked and compiled in strong mode, and the same code will now be acceptable in standard (non-strong) mode as well. The semantics is different in certain cases, but standard mode analysis will emit diagnostic messages (e.g., errors) for that.
+
+In this document, the word **routine** will be used when referring to something which can be a method, a top level function, a local function, or a function literal expression.

[Lasse Reichstein Nielsen, 2017/03/06 09:45:15]

The "routine" name is a little odd, and I'm not sure this definition makes me
any wiser.
I'm guessing the point is that it *omits* getters, setters and constructors.
Since I don't know what "method" covers anyway, I wouldn't know if they are
included in that, so be explicit. (For example, the spec says "the implicit
getter method" at some point).

Maybe something like:
"That is, any callable declaration except getters, setters and constructors."

Also, as defined here, it's not clear whether "routine" refers to syntax or to
semantic values - method, top-level function or local function can be either,
and literal expressions is only syntax (which evaluate to function values).

+
+With **this feature** it is possible to compile code where generic methods and functions are declared, implemented, and invoked. The runtime semantics does not include reification of type arguments. Evaluations of the runtime value of a routine type parameter is a runtime error or yields `dynamic`, depending on the context. No type checking takes place at usages of a method or function type parameter in the body, and no type checking regarding explicitly specified or omitted type arguments takes place at call sites.

[Lasse Reichstein Nielsen, 2017/03/06 09:45:14]

"in the body" of what?

[Lasse Reichstein Nielsen, 2017/03/06 09:45:14]

"The runtime semantics does not include reification of arguments"

Very roundabout statement to say that something doesn't happen.
How about just "Type arguments are not available at runtime."

(Less is more!)

[Lasse Reichstein Nielsen, 2017/03/06 09:45:14]

All in all, sentence is very hard to read, and I'm not sure what it's saying.

[Lasse Reichstein Nielsen, 2017/03/06 09:45:14]

"Evaluations of the runtime value of ..."
I wouldn't say that you evaluate a value. You evaluate syntax, not values, so
maybe "Evaluation of a routine type parameter as an expression...". I also
wouldn't use "evaluate" about "type expression", only actual expressions. That
leaves us without a word for what you do to the 'type' "T" to find the type it's
bound to.
Maybe: "The semantics of a routine type parameter is always either the dynamic
type or an error."?

+
+In short, generic methods and functions are supported syntactically, and the runtime semantics prevents dynamic usages of the type argument values, but it allows all usages where that dynamic value is not required. For instance, a generic routine type parameter, `T`, cannot be used in an expression like `x is T`, but it can be used as a type annotation. In a context where other tools may perform type checking, this allows for a similar level of expressive power as do language designs where type arguments are erased at compile time.

[Lasse Reichstein Nielsen, 2017/03/06 09:45:14]

Please break into separate lines, it's impossible to comment on these blocks in
the code review tool :)

(That's a bug, NOT a feature!)

[Lasse Reichstein Nielsen, 2017/03/06 09:45:14]

What does "is not required mean"? Probably not syntactically, so you have an
idea that you are trying to express, but this sentence doesn't do that. It then
gives examples that are not helping - you are not saying why `x is T` is
different form `T x`. 
(The reasoning, as I understand it, is that *validation* is allowed but *tests*
are not - we can continue running by assuming that a validation succeeded (even
if it didn't) but we cannot pick a branch after a test. That rules out `is`
tests (which create a boolean) and `on` tests, but type annotations and `as`
casts. It's unclear what use as a class type-parameter means: `new List<T>`, `is
List<T>`, but I think we treat that as a type annotation use.

So, write that (if that is what you intend).

+
+The **motivation for** this **document** is that it serves as an informal specification for the implementation of support for the generic method syntax feature in all Dart tools.

[Lasse Reichstein Nielsen, 2017/03/06 09:45:14]

That's a long sentence. Far too many null-words to say something that can be
said shorter. Maybe:

Motivation: This document is an informal specification for generic method syntax
in Dart tools.

Allow me a reference:
http://www.kommunikationsforum.dk/log/multimedia/billeder%20til%20artikler/me...

+
+## Syntax
+
+The syntactic elements which are added or modified in order to support this feature are as follows, based on grammar rules given in the Dart Language Specification (Aug 19, 2015).

[Lasse Reichstein Nielsen, 2017/03/06 09:45:14]

Maybe: which -> that  (not sure if the "which" here is restrictive or
descriptive, but if it's descriptive, there should be commas).

Consider avoiding the inserted phrase completely:

The following syntactic elements are added or modified to support this feature,
based on ...

+
+```
+formalParameterPart:
+        typeParameters? formalParameterList
+functionSignature:
+        metadata returnType? identifier formalParameterPart
+typeParameter:
+        metadata identifier (('extends'|'super') type)?
+functionExpression:
+        formalParameterPart functionBody
+fieldFormalParameter:
+        metadata finalConstVarOrType? 'this' '.' identifier
+        formalParameterPart?
+argumentPart:
+        typeArguments? arguments
+selector:
+        assignableSelector | argumentPart
+assignableExpression:
+        primary (argumentPart* assignableSelector)+ |
+        'super' unconditionalAssignableSelector |
+        identifier
+cascadeSection:
+        '..' (cascadeSelector argumentPart*)
+        (assignableSelector argumentPart*)*
+        (assignmentOperator expressionWithoutCascade)?
+```
+
+In a [draft specification](https://codereview.chromium.org/1177073002) of generic methods from June 2015, the number of grammar changes is significantly higher, but that form can be obtained via renaming.

[Lasse Reichstein Nielsen, 2017/03/06 09:45:14]

Renaming of what? Is this sentence important, or just a historical curiosity?

That is, consider removing it - an implementer won't care.

+
+This extension to the grammar gives rise to an **ambiguity** where the same tokens may be angle brackets of a type argument list as well as relational operators. For instance, `foo(a<b,c>(d))`[^1] may be parsed as a `postfixExpression` on the form `primary arguments` where the arguments are two relational expressions (`a<b` and `c>(d)`),  and it may also be parsed such that there is a single argument which is an invocation of a generic function (`a<b,c>(d)`).  The ambiguity is resolved in **favor** of the **generic function invocation** whenever the `primary` is followed by a balanced pair of angle brackets where the next token after the final `>` is a left parenthesis (in short, we are "looking at `< .. >(`").

[Lasse Reichstein Nielsen, 2017/03/06 09:45:14]

looking at `e< ... >(`

That is: add `e` before the `<`. I think it's clearer that way.

(Break into lines!)

+
+This implies that an expression like `foo(a<b,2>(d))` will be rejected because it is parsed such that `foo` gets one argument which must be a generic function invocation, but `2` cannot parse correctly as a `type`. This is a breaking change, because the same expression used to parse correctly as an invocation of `foo` with two arguments.
+
+The **reason** why the generic function invocation is favored over the relational expressions is that it is considered to be a rare exception that this ambiguity arises: It requires a balanced set of angle brackets followed by a left parenthesis, which is already an unusual form. On top of that, the style guide recommendation to use named parameters for boolean arguments helps making this situation even less common.
+
+If it does occur then there is an easy **workaround**: an extra set of parentheses (as in `foo(a<b,(2>(d)))`) will resolve the ambiguity in the direction of relational expressions; or we might simply be able to remove the parentheses around the last expression (as in `foo(a<b,2>d)`), which will also eliminate the ambiguity.
+
+It should be noted that parsing techniques like recursive descent seem to conflict with this approach to disambiguation: Determining whether the remaining input starts with a balanced expression on the form `<` .. `>` seems to imply a need for an unbounded lookahead. However, if some type of "diet" parsing is used and various kinds of bracket tokens are matched up during the lexical analysis then it takes only a simple O(1) check in the parser to perform the required check.

[Lasse Reichstein Nielsen, 2017/03/06 09:45:14]

"diet parsing" isn't a concept that the reader can be assumed to be familiar
with (if it's a known concept at all, and not just a name someone invented
locally).
Maybe "if using a multi-pass parsing strategy that matches brace-pairs before
parsing expressions - perhaps during tokenization."
 

+
+## Scope of the Mechanism
+
+With the syntax in place, it is obvious that certain potential extensions have **not** been **included**. 
+
+For instance, constructors, setters, getters, and operators cannot be declared as generic. Actual type arguments cannot be passed at invocation sites for setters, getters, and operators, and for constructors there is a need to find a way to distinguish between type arguments for the new instance and type arguments for the constructor itself. It is possible that some of these restrictions will be lifted in a full-fledged version of this extension.

[Lasse Reichstein Nielsen, 2017/03/06 09:45:15]

I wouldn't say that getters, setters or operators are potential extensions.
Getters and setters are actually syntactically viable (x.foo<int> = 42), but
they only makes sense if you can have generic fields too, which are not in the
cards. Operators could meaningfully be generic, but the syntax is against it.

So, only constructors is really a *potential* extension, not just a speculative
one. 

Drop the "it's possible" sentence. Maybe it will, maybe it won't, we'll know
when we do it, but until then it's idle speculation that someone might mistake
for an actual plan.

+
+Conversely, the inclusion of lower bounds for type parameters (using the keyword `super`) serves to demonstrate that lower bounds fit well into the syntax. There is no guarantee that a final version of generic methods will support lower bounds, and it is not required that an implementation must support them.

[Lasse Reichstein Nielsen, 2017/03/06 09:45:14]

Then don't include it. Implementations should implement this informal
specification. Don't include anything in it that they don't need to implement
anyway, that's just a recipe for confusion.

+
+In general, the support for generic methods offered by this feature is not intended to be complete, it is **intended** to allow **for** **experiments** such that a final version of the generic method support can be designed well, **and** it is intended to allow for the **subset of usages** where reification is not required.

[Lasse Reichstein Nielsen, 2017/03/06 09:45:14]

This is confusing. That's not what you said above as the motivation for this
document.
Is it an experiment or a specification of the actual generic method syntax?
Should the VM implement this or something else?
Where is the specification for what it should implement? (and why is this not
it?)

So, just the facts.

+
+## Resolution and Type Checking
+
+In order to be useful, the support for generic methods and functions must be sufficiently complete and consistent to **avoid spurious** diagnostic **messages**. In particular, even though no regular type checks take place at usages of routine type parameters in the body where they are in scope, those type parameters should be resolved. If they had been ignored then any usage of a routine type parameter `X` would give rise to a `Cannot resolve type X` error message, or the usage might resolve to other declarations of `X` in enclosing scopes such as a class type parameter, both of which is unacceptable.

[Lasse Reichstein Nielsen, 2017/03/06 09:45:14]

"In particular, even though no regular type checks take place at usages of
routine type parameters in the body where they are in scope"

Can't parse this, or guess what it's trying to say. Try rewriting. 

(What is a "Regular type check"? Why would the usage of a type parameter
introduce a type check? You can't use a variable where it's not in scope - it's
not that variable then, just similar syntax). 

Are you trying to say that:
- Formal type parameters do introduce a type variable in scope,
- but those type variable won't be bound to the actual type arguments.
?
(and as commentary: if they didn't introduce a variable in the lexical scope, it
wouldn't be possible to use the variable without errors, and we want that to be
possible too).

+
+In `dart2js` resolution, the desired behavior has been achieved by adding a new type parameter **scope** and putting the type parameters into that scope, giving each of them the bound `dynamic`. The type parameter scope is the current scope during resolution of the routine signature and the type parameter bounds, it encloses the formal parameter scope of the routine, and the formal parameter scope in turn encloses the body scope.
+
+This implies that every usage of a routine type parameter is treated during **type checking** as if it had been an alias for the type `dynamic`.
+
+Static checks for **invocations** of methods or functions where type arguments are passed are omitted entirely: The type arguments are parsed, but no checks are applied to certify that the given routine accepts type arguments, and no checks are applied for bound violations. Similarly, no checks are performed for invocations where no type arguments are passed, whether or not the given routine is statically known to accept type arguments.
+
+Certain usages of a routine type parameter `X` give rise to **errors**: It is a compile-time error if `X` is used as a type literal (e.g., `foo(X)`), or in an expression on the form `e is X` or `e is! X`.

[Lasse Reichstein Nielsen, 2017/03/06 09:45:14]

I think we should also disallow `on T { ... }` in catch statements.
It's basically the same as an is-test, and it throws for a malformed type.
  

+
+It could be argued that it should be a warning or an error if a routine type parameter `X` is used in an expression on the form `e as X`. The blind success of this test at runtime may introduce bugs into correct programs in situations where the type constraint is violated; in particular, this could cause "wrong" objects to propagate through local variables and parameters and even into data structures (say, when a `List<T>` is actually a `List<dynamic>`, because `T` is not present at runtime when the list is created). However, considering that these type constraint violations are expected to be rare, and considering that it is common to require that programs compile without warnings, we have chosen to omit this warning. A tool is still free to emit a hint, or in some other way indicate that there is an issue.
+
+## Dynamic semantics
+
+If a routine invocation specifies actual type arguments, e.g., `int` in the **invocation** `f<int>(42)`, those type arguments will not be evaluated at runtime, and they will not be passed to the routine in the invocation. Similarly, no type arguments are ever passed to a generic routine due to call-site inference. This corresponds to the fact that the type arguments have no runtime representation.

[Lasse Reichstein Nielsen, 2017/03/06 09:45:14]

This is Dart 1 only, so don't mention inference.

If you have inference, then it's strong mode, and then the parameters *are*
passed and usable.

+
+When the body of a generic **routine** is **executed**, usages of the formal type parameters will either result in a run-time error, or they will yield the value yielded by an evaluation of `dynamic`, following the treatment of malformed types in Dart. There are the following cases:

[Lasse Reichstein Nielsen, 2017/03/06 09:45:14]

Again, I prefer not to use "evaluate" and "value" about types (except as type
literals, which are expressions). Just say that the type parameter *refers* to
the dynamic type, or that it is (as if it is) replaced by the type `dynamic`.

+
+When `X` is a routine type parameter, the evaluation of `e is X`, `e is! X`, and `X` used as a type literal proceeds as if `X` had been a malformed type, producing a dynamic error if the type test itself is reached; the evaluation of `e as X` has the same outcome as the evaluation of `e`.
+
+Note that the forms containing `is` are compile-time errors, which means that implementations are free to reject the program, or to compile the program with a different runtime semantics for these expressions. The rationale for `dart2js` to allow the construct and compile it to a run time error is that (1) this allows more programs using generic methods to be compiled, and (2) an `is` expression that blindly returns `true` every time (or `false` every time) may silently introduce a bug into an otherwise correct program, so the expression must fail if it is ever evaluated.

[Lasse Reichstein Nielsen, 2017/03/06 09:45:14]

You didn't say above that it's a compile-time error, so why "note" it here? And
it probably isn't a compile-time error.

Dart2js is not free to compile a compile-time error into a runtime error. 
Compile-time errors can happen at runtime, if you interleave parsing and
execution like the VM does, but even then, the entire function containing the
compile-time error should be rejected. Only failing if the *is* is executed is
not OK for a compile-time error.

So, is it a compile-time error, or is it a static warning/runtime error? Pick
one.

+
+When `X` is a routine type parameter which is passed as a type argument to a generic class instantiation `G` which is itself used in `e is G`, `e is! G`, `e as G`, and `G` used as a type literal, evaluation again proceeds as if `X` were a malformed type, which in this case means that it is treated like `dynamic`.

[Lasse Reichstein Nielsen, 2017/03/06 09:45:14]

Be more explicit (meaning, actually write "G<X>"):

If `X` is a routine type parameter and `G<X>` is a generic class instantiation
(and similarly if X is one of multiple type arguments), then `e is G<X>` and `e
is! G<X>` and the type literal `G<X>` proceeds as if `X` is malformed type,
which ...

+
+This may be surprising, so let us consider a couple of examples: When `X` is a routine type parameter, `42 is X` raises a dynamic error, `<int>[42] is List<X>` yields the value `true`, and `42 as X` yields `42`, no matter whether the syntax for the invocation of the routine included an actual type argument, and, if so, no matter which value the actual type argument would have had at the invocation.

[Lasse Reichstein Nielsen, 2017/03/06 09:45:14]

Drop the "no matter ..." part. We already established that arguments are
ignored.

+
+Object construction is similar: When `X` is a routine type parameter which is a passed as a type argument to a constructor invocation, the value passed to the constructor will be the value yielded by an evaluation of `dynamic`.

[Lasse Reichstein Nielsen, 2017/03/06 09:45:14]

Explicit example: , e.g., `new Map<int, X>()` or `<X>[]`,

the value yielded by an evaluation of `dynamic` -> the dynamic type.

+
+In **checked mode**, when `X` is a routine type parameter which is used as a type annotation or in a generic type `G` used as a type annotation, no checked mode checks will ever fail for initialization or assignment to a local variable or parameter whose type annotation is `X`, and if the type annotation is a generic type `G` that contains `X`, checked mode checks will succeed or fail as if `X` had been the type `dynamic`.

[Lasse Reichstein Nielsen, 2017/03/06 09:45:14]

Feels like it's repeating itself here. G<X> is G<dynamic> in every way. There is
no advantage in splitting it into cases when all the cases are the same.

A raw X acts like a malformed type in most cases:
- static warning/runtime error when used directly in `is` checks, `on` checks,
and as a type literal, dynamic everywhere else. When used as a type parameter
again (<T>[], new Foo<T>(), x.bar<T>()), it's always treated as dynamic.

This differs from a malformed type only in `as` checks (right?).

+
+## Changes
+
+2017-Jan-04: Changed 'static error' to 'compile-time error', which is the phrase that the language specification uses.
+
+## Notes
+
+[^1]: These expressions violate the common style in Dart with respect to spacing and capitalization. That is because the ambiguity implies conflicting requirements, and we do not want to bias the appearance in one of the two directions.