Assorted fixes to async: yield may suspend in async* methods; yield* skips emptysequences; definiti…

docs/language/dartLangSpec.tex

 \documentclass{article}
 \usepackage{epsfig}
 \usepackage{color}
 \usepackage{dart}
 \usepackage{bnf}
 \usepackage{hyperref}
 \usepackage{lmodern}
 \newcommand{\code}[1]{{\sf #1}}
 \title{Dart Programming Language  Specification \\
 {\large Version 1.9}}
@@ skipping 3074 matching lines @@
 
 $(T_1$ $a_1, \ldots, T_n$ $a_n, [T_{n+1}$ $x_{n+1} = d_1, \ldots,  T_{n+k}$ $x_{n+k} = d_k]) => e$ 
 is 
 
 $(T_1 \ldots, T_n, [T_{n+1}$ $x_{n+1}, \ldots, T_{n+k}$ $x_{n+k}]) \rightarrow T_0$, where $T_0$ is the static type of $e$. 
 
 \LMHash{}
 The static type of a function literal of the form 
 
 $(T_1$ $a_1, \ldots, T_n$ $a_n, [T_{n+1}$ $x_{n+1} = d_1, \ldots,  T_{n+k}$ $x_{n+k} = d_k])$ \ASYNC{} $=> e$
-is $(T_1 \ldots, T_n, [T_{n+1}$ $x_{n+1}, \ldots, T_{n+k}$ $x_{n+k}]) \rightarrow Future<flatten(T_0)>$, where $T_0$ is the static type of $e$ and  $flatten(T) = flatten(S)$ if $T = Future<S>$, and $T$ otherwise. 
+is $(T_1 \ldots, T_n, [T_{n+1}$ $x_{n+1}, \ldots, T_{n+k}$ $x_{n+k}]) \rightarrow Future<flatten(T_0)>$, where $T_0$ is the static type of $e$ and  $flatten(T)$ is defined as follows:
+
+If $T <: Future$ then let $S$ be a type such that $T << Future<S>$ and for all $R$, if $T << Future<R>$ then $S << R$.  
+
+\rationale{
+This ensures that $Future<S>$ is the most specific instantiation of \cd{Future} that is a super type of $T$.
+}
+
+Then $flatten(T) = flatten(S)$ iff $S \ne T$.  
+
+In any other circumstance, $flatten(T) = T$.

[Paul Berry, 2015/03/17 01:51:51]

I don't think this addresses the infinite regress problem in the example I gave
yesterday where class C<T> implements Future<C<C<T>>>.  AFAICT, the evaluation
of flatten(C<dynamic>) goes as follows:

C<dynamic> <: Future, so we must find a type S such that C<dynamic> << Future<S>
and for all R, if C<dynamic> << Future<R> then S << R.  The only S which
satisfies this is C<C<dynamic>>.  Since C<C<dynamic>> != C<dynamic>
flatten(C<dynamic>) = flatten(C<C<dynamic>>).

Now we must evaluate flatten(C<C<dynamic>>).  By parallel logic to the above,
this works out to flatten(C<C<dynamic>>) = flatten(C<C<C<dynamic>>>).  And so on
and so forth.

+
+
 
 \rationale{
 We collapse multiple layers of futures into one. If $e$ evaluates to a future $f$, the future will not invoke its \code{then()} callback until f completes to a non-future value, and so the result of an await is never a future, and the result of an async function will never have type \code{Future$<X>$} where $X$ itself is an invocation of \code{Future}. 
+
+The sole exception to that would be a type $X$ that extended or implemented \code{Future$<X>$}. In that case, the result type is $X$ and no further unwrapping takes place. As an example of why this is necessary, consider
+
+\cd{\CLASS{} C$<$T$>$  \IMPLEMENTS{}  C$<$C$<$T$>>$ \ldots }

[Paul Berry, 2015/03/17 01:51:51]

This is an illegal definition (a class cannot implement itself).

I'm guessing you meant to quote the case I mentioned yesterday, which was:

\cd{\CLASS{} C$<$T$>$  \IMPLEMENTS{}  Future$<$C$<$C$<$T$>>>$ \ldots }

As I mentioned above, I don't think we've successfully avoided an infinite
regress in this case.

+
+Here, a naive definition of $flatten$ diverges; there is not even a fixed point.
+
 }
 
 
 \LMHash{}
 The static type of a function literal of the form 
 
 $(T_1$ $a_1, \ldots, T_n$ $a_n, \{T_{n+1}$ $x_{n+1} : d_1, \ldots,  T_{n+k}$ $x_{n+k} : d_k\}) => e$ 
 is 
 
 $(T_1 \ldots, T_n, \{T_{n+1}$ $x_{n+1}, \ldots, T_{n+k}$ $x_{n+k}\}) \rightarrow T_0$, where $T_0$ is the static type of $e$. 
@@ skipping 412 matching lines @@
 }
 
 \LMHash{}
 The iterable implementation must comply with the contract of \code{Iterable} and should not take any steps identified as exceptionally efficient in that contract.
 
 \commentary {
 The contract explicitly mentions a number of situations where certain iterables could be more efficient than normal. For example, by precomputing their length. Normal iterables must iterate over their elements to determine their length. This is certainly true in the case of a synchronous generator, where each element is computed by a function. It would not be acceptable to pre-compute the results of the generator and cache them, for example.
 }
 
 \LMHash{}
-When iteration over the iterable is started, by getting an iterator $j$ from the iterable and calling \code{moveNext()} on it, execution of the body of $f$ will begin. When $f$ terminates, $j$ is positioned after its last element, so that its current value is \NULL{} and the current call to \code{moveNext()} on $j$ returns false, as will all further calls.
+When iteration over the iterable is started, by getting an iterator $j$ from the iterable and calling \code{moveNext()}, execution of the body of $f$ will begin. When $f$ terminates, $j$ is positioned after its last element, so that its current value is \NULL{} and the current call to \code{moveNext()} on $j$ returns false, as will all further calls.
 
 Each iterator starts a separate computation. If the \SYNC* function is impure, the sequence of values yielded by each iterator may differ. 
 
 \commentary{
 One can derive more than one iterator from a given iterable.   Note that operations on the iterable itself can create distinct iterators. An example would be \code{length}.  It is conceivable that different iterators might yield sequences of different length. The same care needs to be taken when writing \SYNC* functions as when
 writing an \code{Iterator} class. In particular, it should handle multiple
 simultaneous iterators gracefully. If the iterator depends on external state
 that might change, it should check that the state is still valid after every
 yield (and maybe throw a \code{ConcurrentModificationError} if it isn't).
 }
@@ skipping 2364 matching lines @@
 Next, $o$ is added to the iterable or stream associated with the immediately enclosing function. 
 
 \LMHash{}
 If the enclosing function $m$ is marked \ASYNC* and the stream $u$ associated with $m$ has been canceled, then let $c$ be the \FINALLY{} clause (\ref{try}) of the innermost enclosing try-finally statement, if any. If $c$ is defined, let $h$ be the handler induced by $c$. If $h$ is defined, control is transferred to $h$. If $h$ is undefined, the immediately enclosing function terminates.
 
 \rationale{
 The stream associated with an asynchronous generator could be canceled by any code with a reference to that stream at any point where the generator was passivated.  Such a cancellation constitutes an irretrievable error for the generator. At this point, the only plausible action for the generator is to clean up after itself via its \FINALLY{} clauses.
 }
 
 \LMHash{}
+Otherwise, if the enclosing function $m$ is marked \ASYNC* (\ref{functions}) then the enclosing function may suspend.
+
+\rationale {
+If a \YIELD{} occurred inside an infinite loop and the enclosing function never suspended, there might not be an opportunity for consumers of the  enclosing stream to run and access the data in the stream.  The stream might then accumulate an unbounded number of elements. Such a situation is untenable. Therefore, we allow the enclosing function to be suspended when a new value is added to its associated stream. However, it is not essential (and in fact, can be quite costly) to suspend the function on every \YIELD{}. The implementation is free to decide how often to suspend the enclosing function. The only requirement is that consumers are not blocked indefinitely.
+}
+
+
+\LMHash{}
 If the enclosing function $m$ is marked \SYNC* (\ref{functions}) then:
 \begin{itemize}
 \item
-Execution of the function $m$ immediately enclosing $s$ is suspended until the method \code{moveNext()} is invoked upon the iterator used to initiate the current invocation of $m$. 
+Execution of the function $m$ immediately enclosing $s$ is suspended until the nullary method \code{moveNext()} is invoked upon the iterator used to initiate the current invocation of $m$. 
 \item
 The current call to \code{moveNext()} returns \TRUE.
 \end{itemize}
 
 \LMHash{}
 It is a compile-time error if a yield statement appears in a function that is not a generator function.
 
 \LMHash{}
 Let $T$ be the static type of $e$ and let $f$ be the immediately enclosing function.  It is a static type warning if either:
 \begin{itemize}
@@ skipping 10 matching lines @@
 \LMHash{}
  The {\em yield-each statement} adds a series of values to the result of a generator function (\ref{functions}).
  
  \begin{grammar}
 {\bf yieldEachStatement:}
    \YIELD* expression `{\escapegrammar ;}'
       .
 \end{grammar}
 
 \LMHash{}
-Execution of a statement s of the form \code{\YIELD* $e$;}  proceeds as follows:
+Execution of a statement $s$ of the form \code{\YIELD* $e$;}  proceeds as follows:
 
 \LMHash{}
-First, the expression $e$ is evaluated to an object $o$. If the immediately enclosing function $m$ is synchronous, then it is a dynamic error if the class of $o$ does not implement \code{Iterable}.  If $m$ asynchronous, then it is a dynamic error if the class of $o$ does not implement \code{Stream}. Next, for each element $x$ of $o$:
+First, the expression $e$ is evaluated to an object $o$. 
+
+\LMHash{}
+If the immediately enclosing function $m$ is marked \SYNC* (\ref{functions}), then:
+\begin{enumerate}
+\item It is a dynamic error if the class of $o$ does not implement \code{Iterable}.  Otherwise
+\item The method \cd{iterator} is invoked upon $o$ returning an object $i$. 
+\item \label{moveNext} The \cd{moveNext} method of $i$ is invoked on it with no arguments. If \cd{moveNext} returns \FALSE{} execution of $s$ is complete. Otherwise
+\item An element $x$ is extracted from $i$ and $x$ is added to the iterable or stream associated with $m$.
+Execution of the function $m$ immediately enclosing $s$ is suspended until the nullary method \code{moveNext()} is invoked upon the iterator used to initiate the current invocation of $m$, at which point execution of $s$ continues at \ref{moveNext}. 
+\item
+The current call to \code{moveNext()} returns \TRUE.
+\end{enumerate}
+
+\LMHash{}
+If $m$ is marked \ASYNC* (\ref{functions}), then:
+\begin{itemize}
+\item  It is a dynamic error if the class of $o$ does not implement \code{Stream}. Otherwise
+\item For each element $x$ of $o$:
 \begin{itemize}
 \item
-If $m$ is marked \ASYNC* (\ref{functions}) and the stream $u$ associated with $m$ has been paused,  then execution of $m$ is suspended until $u$ is resumed or canceled.
+If the stream $u$ associated with $m$ has been paused,  then execution of $m$ is suspended until $u$ is resumed or canceled.
+ \item
+If the stream $u$ associated with $m$ has been canceled, then let $c$ be the \FINALLY{} clause (\ref{try}) of the innermost enclosing try-finally statement, if any. If $c$ is defined,  let $h$ be the handler induced by $c$. If $h$ is defined, control is transferred to $h$. If $h$ is undefined, the immediately enclosing function terminates.
 \item
- $x$ is added to the iterable or stream associated with $m$ in the order it appears in $o$.  
- \item
-If $m$ is marked \ASYNC* and the stream $u$ associated with $m$ has been canceled, then let $c$ be the \FINALLY{} clause (\ref{try}) of the innermost enclosing try-finally statement, if any. If $c$ is defined,  let $h$ be the handler induced by $c$. If $h$ is defined, control is transferred to $h$. If $h$ is undefined, the immediately enclosing function terminates.
+Otherwise,  $x$ is added to the stream associated with $m$ in the order it appears in $o$.  The function $m$ may suspend.
+\end{itemize}
+\item If the stream $o$ is done, execution of $s$ is complete. 
 \end{itemize}
 
-\LMHash{}
-If the enclosing function is marked \SYNC* (\ref{functions}) then:
-\begin{itemize}
-\item
-Execution of the function $m$ immediately enclosing $s$ is suspended until the method \code{moveNext()} is invoked upon the iterator used to initiate the current invocation of $m$. 
-\item
-The current call to \code{moveNext()} returns \TRUE.
-\end{itemize}
 
 \LMHash{}
 It is a compile-time error if a yield-each statement appears in a function that is not a generator function.
 
 \LMHash{}
 Let $T$ be the static type of $e$ and let $f$ be the immediately enclosing function.  It is a static type warning if $T$ may not be assigned to the declared return type of $f$.  If $f$ is synchronous it is a static  type warning if $T$ may not be assigned to \code{Iterable}.  If $f$ is asynchronous it is a static  type warning if $T$ may not be assigned to \code{Stream}. 
 
 
 \subsection{ Assert}
 \LMLabel{assert}
@@ skipping 1402 matching lines @@
 
 The invariant that each normative paragraph is associated with a line
 containing the text \LMHash{} should be maintained.  Extra occurrences
 of \LMHash{} can be added if needed, e.g., in order to make
 individual \item{}s in itemized lists addressable.  Each \LM.. command
 must occur on a separate line.  \LMHash{} must occur immediately
 before the associated paragraph, and \LMLabel must occur immediately
 after the associated \section{}, \subsection{} etc.
 
 ----------------------------------------------------------------------