Update parser to support <t>, <template>, /> syntax

sky/specs/parsing.md

 Parsing
 =======
 
-Parsing in Sky is a strict pipeline consisting of four stages:
+Parsing in Sky is a strict pipeline consisting of five stages:
 
 - decoding, which converts incoming bytes into Unicode characters
-  using UTF-8
+  using UTF-8.
 
-- normalising, which converts certain sequences of characters
+- normalising, which manipulates the sequence of characters.
 
-- tokenising, which converts these characters into tokens
+- tokenising, which converts these characters into three kinds of
+  tokens: character tokens, start tag tokens, and end tag tokens.
+  Character tokens have a single character value. Tag tokens have a
+  tag name, and a list of name/value pairs known as attributes.
 
-- tree construction, which converts these tokens into a tree of nodes
+- token cleanup, which converts sequences of character tokens into
+  string tokens, and removes duplicate attributes in tag tokens.
+
+- tree construction, which converts these tokens into a tree of nodes.
 
 Later stages cannot affect earlier stages.
 
 When a sequence of bytes is to be parsed, there is always a defined
-_parsing context_, which is either "application" or "module".
+_parsing context_, which is either an Application object or a Module
+object.
 
 
 Decoding stage
 --------------
 
 To decode a sequence of bytes _bytes_ for parsing, the [UTF-8
 decoder](https://encoding.spec.whatwg.org/#utf-8-decoder) must be used
 to transform _bytes_ into a sequence of characters _characters_.
 
 This sequence must then be passed to the normalisation stage.
@@ skipping 18 matching lines @@
 Tokenisation stage
 ------------------
 
 To tokenise a sequence of characters, a state machine is used.
 
 Initially, the state machine must begin in the **signature** state.
 
 Each character in turn must be processed according to the rules of the
 state at the time the character is processed. A character is processed
 once it has been _consumed_. This produces a stream of tokens; the
-tokens must be passed to the tree construction stage.
+tokens must be passed to the token cleanup stage.
 
 When the last character is consumed, the tokeniser ends.
 
 
 ### Expecting a string ###
 
 When the user agent is to _expect a string_, it must run these steps:
 
 1. Let _expectation_ be the string to expect. When this string is
    indexed, the first character has index 0.
@@ skipping 10 matching lines @@
 
 6. Switch to the **expect a string** state.
 
 
 ### Tokeniser states ###
 
 #### **Signature** state ####
 
 If the current character is...
 
-* '```#```': If the _parsing context_ is not "application", switch to
+* '```#```': If the _parsing context_ is not an Application, switch to
   the _failed signature_ state. Otherwise, expect the string
   "```#!mojo mojo:sky```", with _after signature_ as the _success_
   state and _failed signature_ as the _failure_ state.
 
-* '```S```': If the _parsing context_ is not "module", switch to the
+* '```S```': If the _parsing context_ is not a Module, switch to the
   _failed signature_ state. Otherwise, expect the string
   "```SKY MODULE```", with _after signature_ as the _success_ state,
   and _failed signature_ as the _failure_ state.
 
 * Anything else: Jump to the **failed signature** state.
 
 
 #### **Expect a string** state ####
 
 If the current character is not the same as the <i>index</i>th character in
@@ skipping 284 matching lines @@
   tag** state.
 
 * Anything else: Append the current character to the tag name, and
   consume the current character. Stay in this state.
 
 
 ### **Void tag** state ###
 
 If the current character is...
 
-* '```>```': Consume the current character. Switch to the **after
+* '```>```': Consume the current character. Switch to the **after void
   tag** state.
 
 * Anything else: Switch to the **before attribute name** state without
   consuming the current character.
 
 
 ### **Before attribute name** state ###
 
 If the current character is...
 
@@ skipping 137 matching lines @@
 
 If the tag token was a start tag token and the tag name was
 '```script```', then and switch to the **script raw data** state.
 
 If the tag token was a start tag token and the tag name was
 '```style```', then and switch to the **style raw data** state.
 
 Otherwise, switch to the **data** state.
 
 
+### **After void tag** state ###
+
+Emit the tag token.
+
+If the tag token is a start tag token, emit an end tag token with the
+same tag name.
+
+Switch to the **data** state.
+
+
 ### **Comment start 1** state ###
 
 If the current character is...
 
 * '```-```': Consume the character and switch to the **comment start
   2** state.
 
 * '```>```': Emit character tokens for '```<!>```'. Consume the
   current character. Switch to the **data** state.
 
@@ skipping 294 matching lines @@
 
 * Any other character in the range '```0```'..'```9```',
   '```a```'..'```f```', '```A```'..'```F```': Consume the character
   and stay in this state.
 
 * Anything else: Run the _emitting operation_ for all but the last
   character in _raw value_, and switch to the **data state** without
   consuming the current character.
 
 
-Tree construction
------------------
+Token cleanup stage
+-------------------
 
-To construct a node tree from a _sequence of tokens_ and a document _document_:
+Replace each sequence of character tokens with a single string token
+whose value is the concatenation of all the characters in the
+character tokens.
+
+For each start tag token, remove all but the first name/value pair for
+each name (i.e. remove duplicate attributes, keeping only the first
+one).
+
+For each end tag token, remove the attributes entirely.
+
+If the token is a start tag token, notify the JavaScript token stream
+callback of the token.
+
+Then, pass the tokens to the tree construction stage.
+
+
+Tree construction stage
+-----------------------
+
+To construct a node tree from a _sequence of tokens_ and a document
+_document_:
 
 1. Initialize the _stack of open nodes_ to be _document_.
-2. Consider each token _token_ in the _sequence of tokens_ in turn.
-   - If _token_ is a text token,
-     1. Create a text node _node_ with character data _token.data_.
+2. Consider each token _token_ in the _sequence of tokens_ in turn, as
+   follows. If a token is to be skipped, then jump straight to the
+   next token, without doing any more work with the skipped token.
+   - If _token_ is a string token,
+     1. If the value of the token contains only U+0020 and U+000A
+        characters, and there is no ```t``` element on the _stack of
+        open nodes_, then skip the token.
+     2. Create a text node _node_ whose character data is the value of
+        the token.
+     3. Append _node_ to the top node in the _stack of open nodes_.
+   - If _token_ is a start tag token,
+     1. Create an element _node_ with tag name and attributes given by
+        the token.
      2. Append _node_ to the top node in the _stack of open nodes_.
-   - If _token_ is a start tag token,
-     1. Create an element _node_ with tag name _token.tagName_ and attributes
-        _token.attributes_.
-     2. Append _node_ to the top node in the _stack of open nodes_.
-     3. If the _token.selfClosing_ flag is not set, push _node_ onto the
-        _stack of open elements_.
-     4. If _token.tagName_ is _script_, TODO: Execute the script.
-   - If _token_ is an end tag token,
-     1. If the _stack of open nodes_ contains a node whose _tagName_ is
-        _token.tagName_,
-        - Pop nodes from the _stack of open nodes_ until a node with
-          a _tagName_ equal to _token.tagName_ has been popped.
-     2. Otherwise, ignore _token_.
-   - If _token_ is a comment token,
-     1. Ignore _token_.
-   - If _token_ is an EOF token,
-     1. Pop all the nodes from the _stack of open nodes_.
-     2. Signal _document_ that parsing is complete.
-
-TODO(ianh): &lt;template>, &lt;t>
+   - If _token_ is an end tag token:
+     1. Let _node_ be the topmost node in the _stack of open nodes_
+        whose tag name is the same as the token's tag name, if any. If
+        there isn't one, skip this token.
+     2. If there's a ```template``` element in the _stack of open
+        nodes_ above _node_, then skip this token.
+     3. Pop nodes from the _stack of open nodes_ until _node_ has been
+        popped.
+     4. If _node_'s tag name is ```script```, then yield until there
+        are no pending import loads, then execute the script given by
+        the element's contents.
+3. Yield until there are no pending import loads.
+3. Fire a ```load``` event at the _parsing context_ object.