Add data-URI support class to dart:core (next to Uri).

sdk/lib/core/uri.dart

Index: sdk/lib/core/uri.dart
diff --git a/sdk/lib/core/uri.dart b/sdk/lib/core/uri.dart
index 78f42de6bdd52bbf322cd55c9ab71673557bbad9..89c0a78862a44749579a1b7484280494cc0539f0 100644
--- a/sdk/lib/core/uri.dart
+++ b/sdk/lib/core/uri.dart
@@ -2309,23 +2309,40 @@ class Uri {
    */
   static String _uriDecode(String text,
                            {bool plusToSpace: false,
-                            Encoding encoding: UTF8}) {
+                            Encoding encoding: UTF8,
+                            int start: 0,
+                            int end}) {
+    if (end == null) end = text.length;

[floitsch, 2015/11/07 02:56:26]

if we have start and end, shouldn't we at least check that start <= end?

[Lasse Reichstein Nielsen, 2015/11/09 10:27:38]

This is a private function, I usually don't check parameters there because I'm
the only one calling it. If I'm worried, I'll add an assert.

I also usually don't make them have optional parameters.
I'll convert this to non-optional parameters, and add an assert for good
measure.

     // First check whether there is any characters which need special handling.
     bool simple = true;
-    for (int i = 0; i < text.length && simple; i++) {
+    for (int i = start; i < end && simple; i++) {
       var codeUnit = text.codeUnitAt(i);
       simple = codeUnit != _PERCENT && codeUnit != _PLUS;

[floitsch, 2015/11/07 02:56:26]

I prefer:

if (codeUnit == _PERCENT || codeUnit == _PLUS) {
  simple = false;
  break;  (and remove the && simple check in the loop-condition)
}

In the end it's doing the same, but imho it's more obvious of what happens,
since everything is in one place.

[Lasse Reichstein Nielsen, 2015/11/09 10:27:38]

Done.

     }
     List<int> bytes;
     if (simple) {
       if (encoding == UTF8 || encoding == LATIN1) {
-        return text;
-      } else {
+        return text.substring(start, end);
+      } else if (start == 0 && end == text.length) {
         bytes = text.codeUnits;
+      } else {
+        var decoder = encoding.decoder;
+        var result;
+        var conversionSink = decoder.startChunkedConversion(
+          new ChunkedConversionSink((list) {
+            result = list.join();
+          }));
+        if (conversionSink is ByteConversionSink) {
+          conversionSink.addSlice(text.codeUnits, start, end, true);
+        } else {
+          conversionSink.add(text.codeUnits.sublist(start, end));
+          conversionSink.close();
+        }
+        return result;
       }
     } else {
       bytes = new List();
-      for (int i = 0; i < text.length; i++) {
+      for (int i = start; i < end; i++) {
         var codeUnit = text.codeUnitAt(i);
         if (codeUnit > 127) {
           throw new ArgumentError("Illegal percent encoding in URI");
@@ -2597,4 +2614,518 @@ class Uri {
       0xfffe,   // 0x60 - 0x6f  0111111111111111
                 //              pqrstuvwxyz   ~
       0x47ff];  // 0x70 - 0x7f  1111111111100010
+
+}
+
+// --------------------------------------------------------------------
+// Data URI
+// --------------------------------------------------------------------
+
+/**
+ * A representation of a `data:` URI.
+ *
+ * Data URIs are non-hierarchial URIs that contain can contain any data.
+ * They are defined by [RFC 2397](https://tools.ietf.org/html/rfc2397).
+ *
+ * This class allows parsing the URI text and extracting individual parts of the
+ * URI, as well as  building the URI text from structured parts.
+ */
+class DataUri {

[nweiz, 2015/10/15 21:09:03]

It seems strange that this isn't a subclass of Uri. It certainly satisfies the
"is a" criterion, and I'd expect to be able to pass a data URI anywhere I could
pass a normal URI. Having to explicitly convert back and forth sounds awkward.

[Lasse Reichstein Nielsen, 2015/10/28 13:55:47]

The Uri class is designed only for hierarchial URIs and data: URIs are not
hierarchial, so it is not really an "is-a" relation. They both relate to the
generic URI concept, but they are different and have widely different meanings.

If you have a URI string with a "data:" scheme, you can create a DataUri object
to access its parts, just as you can create a Uri object for an "http:" schemed
URI.

[nweiz, 2015/10/29 00:28:36]

That's not communicated anywhere in the documentation, and it's certainly not
communicated by the name. If it only supported hierarchical URLs, then it should
be called "Url", not "Uri". As it is, the name and the documentation clearly say
that it does represent the generic URI concept, even if it has some getters that
are particularly useful for the hierarchical subset.

That also doesn't match the way it's used in practice. APIs like
Isolate.spawnUri take Uris, and work perfectly well when they're
non-hierarchical—they even tend to support Data URIs in particular. It seems
perverse to say that these core methods support data URIs but not the canonical
Dart object used to represent data URIs.

Even this class itself supports fromUri and toUri. How do those methods make
sense if a Uri should only be hierarchical?

[Lasse Reichstein Nielsen, 2015/11/03 18:02:52]

Good points.
So this class is not really a "data URI", it's more like a "data URI helper"
class. It's not a URI, but it gives access to the scheme specific structure of
Data URIs.

It's still not functionality I want to merge into the URI class, or that I would
want to share with the URI functionality, so I guess the best option is to find
a better name for the class.

DataUriHelper is ... bad. ("Helper", "Manager", "Impl" etc. are non-descriptive
words added to distinguish something from something else that would otherwise
have the same name).

DataUriParser makes more sense, but misses that it is a way to create them.

Suggestions?

+  static const int _noScheme = -1;
+  /**
+   * Contains the text content of a `data:` URI, with or without a
+   * leading `data:`.
+   *
+   * If [_separatorIndices] starts with `4` (the index of the `:`), then
+   * there is a leading `data:`, otherwise _separatorIndices starts with
+   * `-1`.
+   */
+  final String _text;
+
+  /**
+   * List of the separators (';', '=' and ',') in the text.
+   *
+   * Starts with the index of the index of the `:` in `data:` of the mimeType.
+   * That is always either -1 or 4, depending on whether `_text` includes the
+   * `data:` scheme or not.
+   *
+   * The first speparator ends the mime type. We don't bother with finding
+   * the '/' inside the mime type.
+   *
+   * Each two separators after that marks a parameter key and value.
+   *
+   * If there is a single separator left, it ends the "base64" marker.
+   *
+   * So the following separators are found for a text:
+   *
+   *     data:text/plain;foo=bar;base64,ARGLEBARGLE=
+   *         ^          ^   ^   ^      ^
+   *
+   */
+  List<int> _separatorIndices;
+
+  DataUri._(this._text,
+            this._separatorIndices);
+
+  /** The entire content of the data URI, including the leading `data:`. */
+  String get text => _separatorIndices[0] == _noScheme ? "data:$_text" : _text;
+
+  /**
+   * Creates a `data:` URI containing the contents as percent-encoded text.
+   */
+  factory DataUri.fromString(String content,

[nweiz, 2015/10/15 21:09:03]

Why can't you base64-encode text, or percent-encode binary data? If my text
contains a lot of non-URL-safe characters (particularly non-ASCII characters)
that would need to be encoded anyway, base64 may well end up being more compact.

[Lasse Reichstein Nielsen, 2015/10/16 14:38:45]

You can't base-64 encode text - base64 encoding only works on bytes and strings
are not bytes.

I could add an implicit UTF-8 encoding of the text, but I prefer to keep this
simple and explicit. If you need to UTF-8 encode, you choose to do that.

The other direction: bytes to percent-escaped is possible, but rarely useful
(unless you happen to know that your bytes contain a lot of ASCII characters
that won't be escaped, and in that case I don't mind asking you to create a
string from them).

[nweiz, 2015/10/19 19:51:20]

I was thinking something parallel to File.writeAsString and similar APIs: it
takes a string as well as an encoding parameter that defaults 
to UTF-8.
Making users convert to a string won't work in general; Dart's UTF-16 strings
can't fully represent binary data in a way that will be correctly translated to
UTF-8 (or any other encoding).

[Lasse Reichstein Nielsen, 2015/10/28 13:55:47]

True. We probably should have an encoding here: The "content" string is
converted to bytes, and there is no reason to fix that to UTF-8.

(Well, except that we have to write the code to do the encoding since the
functions in Uri only works on strings).

The content of a data: URI is always a sequence of bytes, so encoding
information is important - so we should use it and store it.

+                             {mimeType: "text/plain",
+                              Iterable<DataUriParameter> parameters}) {

[nweiz, 2015/10/15 21:09:03]

Right now, the encoding is implicitly always UTF-8, but the corresponding 
charset parameter isn't automatically included.

Ideally, the user would be able to specify an encoding and a charset parameter
would be added based on that. Otherwise, this should add charset=UTF-8, and the
documentation should be explicit about what encoding is used.

[Lasse Reichstein Nielsen, 2015/10/16 14:38:45]

Good point. The default, if nothing is written, is charset=US-ASCII which won't
be correct here.

[Lasse Reichstein Nielsen, 2015/10/28 13:55:47]

I've changed this to add an "Encoding charset" parameter which is used to
convert the string to bytes (and which may throw if the string isn't
compatible). It defaults to US-ASCII if you don't specify anything.

+    StringBuffer buffer = new StringBuffer();
+    List indices = [_noScheme];
+    _writeUri(mimeType, parameters, buffer, indices);
+    indices.add(buffer.length);
+    buffer.write(',');
+    buffer.write(Uri.encodeComponent(content));

[nweiz, 2015/10/15 21:09:04]

URI.encodeComponent doesn't encode a number of reserved characters ("!~*'()"). I
assume most parsers will do the right thing anyway, but this is a place where
the implementation diverges from the spec.

[Lasse Reichstein Nielsen, 2015/10/16 14:38:45]

ACK. The syntax for parameter keys an values are RFC 2045 tokens which do not
contain percent encodings (they may contain percent, but it wouldn't mean it's
encoded).
We should not encode here, just validate, and do it against the _tokenCharTable,
and fail on any invalid character.
If you need to encode something, you should do it explicitly before calling
here.

[nweiz, 2015/10/19 19:51:20]

You should certainly percent-encode *here*, because it's the data. But the data
URI spec says you should also percent-encode parameters: "parameter values
should use the URL Escaped encoding instead of quoted string if the parameter
values contain any 'tspecial'.".

[Lasse Reichstein Nielsen, 2015/10/28 13:55:47]

Ack, rereading the RFC again. And again.
It's almost as if it's incompletely specified :)

The format of a data URI is:
  'data:' (type '/' subtype)? (';' attribute '=' value)*
    (';base64')? ',' uric*

where type/subtype/attribute/value comes from RFC 2045 and uric comes from RFC
2396.
The type, subtype, attribute and value are percent-encoded if they contain
characters that are not token characters (ASCII minus SPACE, CTLs and tspecial)
- or '%' itself, I guess.
If the content (a sequence of bytes) is not base-64 encoded (RFC 4648), then it
percent escapes (RFC 2396) non-uric characters.

It's not clear what happens if a non-ASCII character is included in attribute or
value. It needs to be encoded to bytes somehow because that's all we can
represent with percent escapes. 
There is an RFC(2231) which defines how to add encoding to attributes
(foo*=utf-8'en-us'%A8%94 - the 'foo*' means that the name is foo and the content
is special). We probably don't want to support that.

I think I'll just UTF-8 + percent encode parameter keys and values if they
contain non-ASCII characters, and percent-encode non-token ASCII chars.


For the data part, the allowed character is uric := reserved | unreserved |
escape , so it doesn't need to escape reserved characters. It's still not
exactly the correct characters, I'll make a new table with the correct
characters to not escape (which is annoyingly close to Uri._encodFullTable, but
doesn't contain `#`.

[nweiz, 2015/10/29 00:28:36]

I think that's the right behavior. I believe there's a blanket rule for URIs
that percent-escapes should be interpreted as UTF-8 unless otherwise specified.

+    return new DataUri._(buffer.toString(), indices);
+  }
+
+  /**
+   * Creates a `data:` URI string containing the base-64 encoded content bytes.
+   *
+   * It defaults to having the mime-type `application/octet-stream`.
+   */
+  factory DataUri.fromBytes(List<int> bytes,
+                            {mimeType: "application/octet-stream",
+                             Iterable<DataUriParameter> parameters}) {
+    StringBuffer buffer = new StringBuffer();
+    List indices = [_noScheme];
+    _writeUri(mimeType, parameters, buffer, indices);
+    indices.add(buffer.length);
+    buffer.write(';base64,');
+    indices.add(buffer.length - 1);
+    BASE64.encoder.startChunkedConversion(
+                       new StringConversionSink.fromStringSink(buffer))
+          .addSlice(bytes, 0, bytes.length, true);
+    return new DataUri._(buffer.toString(), indices);
+  }
+
+  /**
+   * Creates a `DataUri` from a [Uri] which must have `data` as [Uri.scheme].
+   *
+   * The [uri] must have scheme `data` and no authority, query or fragment,
+   * and the path must be valid as a data URI.
+   */
+  factory DataUri.fromUri(Uri uri) {

[nweiz, 2015/10/15 21:09:03]

This should document when it will throw FormatExceptions, especially since some
syntax errors are detected eagerly and some are not. Same goes for parse().

[Lasse Reichstein Nielsen, 2015/10/16 14:38:45]

Good point.

[Lasse Reichstein Nielsen, 2015/10/28 13:55:47]

Documented on parse, referenced here.

+    if (uri.scheme != "data") {
+      throw new ArgumentError.value(uri, "uri",
+                                    "Scheme must be 'data'");
+    }
+    if (uri.hasAuthority) {
+      throw new ArgumentError.value(uri, "uri",
+                                    "Data uri must not have authority");
+    }
+    if (uri.hasQuery) {
+      throw new ArgumentError.value(uri, "uri",
+                                    "Data uri must not have a query part");
+    }
+    if (uri.hasFragment) {
+      throw new ArgumentError.value(uri, "uri",
+                                    "Data uri must not have a fragment part");
+    }

[nweiz, 2015/10/15 21:09:03]

According to https://simonsapin.github.io/data-urls/ (which is referenced by
browser vendors and so is at least somewhat authoritative), the query should be
included in the parsing algorithm and the fragment should be ignored.

[Lasse Reichstein Nielsen, 2015/10/16 14:38:45]

True - '?' is a valid uric, so the query should be included.

[Lasse Reichstein Nielsen, 2015/10/28 13:55:47]

True. Fixed.

+    return _parse(uri.path, 0);
+  }
+
+  /**
+   * Writes the initial part of a `data:` uri, from after the "data:"
+   * until just before the ',' before the data, or before a `;base64,`
+   * marker.
+   *
+   * Of an [indices] list is passed, separator indices are stored in that
+   * list.
+   */
+  static void _writeUri(String mimeType,
+                        Iterable<DataUriParameter> parameters,
+                        StringBuffer buffer, List indices) {
+    if (mimeType == null) {
+      mimeType = "text/plain";
+    }
+    if (mimeType.isEmpty ||
+        identical(mimeType, "text/plain") ||

[nweiz, 2015/10/15 21:09:03]

Consider omitting the text/plain mime type, since it's the default anyway.

[Lasse Reichstein Nielsen, 2015/10/16 14:38:45]

Good idea.

+        identical(mimeType, "application/octet-stream")) {
+      buffer.write(mimeType);  // Common cases need no escaping.
+    } else {
+      int slashIndex = _validateMimeType(mimeType);
+      if (slashIndex < 0) {
+        throw new ArgumentError.value(mimeType, "mimeType",
+                                      "Invalid MIME type");
+      }
+      buffer.write(Uri._uriEncode(_tokenCharTable,
+                                  mimeType.substring(0, slashIndex)));
+      buffer.write("/");
+      buffer.write(Uri._uriEncode(_tokenCharTable,
+                                  mimeType.substring(slashIndex + 1)));
+    }
+    if (parameters != null) {
+      for (var parameter in parameters) {
+        if (indices != null) indices.add(buffer.length);
+        buffer.write(';');
+        // Encode any non-RFC2045-token character as well as '%' and '#'.
+        buffer.write(Uri._uriEncode(_tokenCharTable, parameter.key));
+        if (indices != null) indices.add(buffer.length);
+        buffer.write('=');
+        buffer.write(Uri._uriEncode(_tokenCharTable, parameter.value));
+      }
+    }
+  }
+
+  /**
+   * Checks mimeType is valid-ish (`token '/' token`).
+   *
+   * Returns the index of the slash, or -1 if the mime type is not
+   * considered valid.
+   *
+   * Currently only looks for slashes, all other characters will be
+   * percent-encoded as UTF-8 if necessary.
+   */
+  static int _validateMimeType(String mimeType) {
+    int slashIndex = -1;
+    for (int i = 0; i < mimeType.length; i++) {
+      var char = mimeType.codeUnitAt(i);
+      if (char != Uri._SLASH) continue;
+      if (slashIndex < 0) {
+        slashIndex = i;
+        continue;
+      }
+      return -1;
+    }
+    return slashIndex;
+  }
+
+  /**
+   * Creates a [Uri] with the content of [DataUri.fromString].
+   *
+   * The resulting URI will have `data` as scheme and the remainder
+   * of the data URI as path.
+   *
+   * Equivalent to creating a `DataUri` using `new DataUri.fromString` and
+   * calling `toUri` on the result.
+   */
+  static Uri uriFromString(String content,
+                           {mimeType: "text/plain",
+                            Iterable<DataUriParameter> parameters}) {
+    var buffer = new StringBuffer();
+    _writeUri(mimeType, parameters, buffer, null);
+    buffer.write(',');
+    buffer.write(Uri.encodeComponent(content));
+    return new Uri(scheme: "data", path: buffer.toString());
+  }
+
+  /**
+   * Creates a [Uri] with the content of [DataUri.fromBytes].
+   *
+   * The resulting URI will have `data` as scheme and the remainder
+   * of the data URI as path.
+   *
+   * Equivalent to creating a `DataUri` using `new DataUri.fromBytes` and
+   * calling `toUri` on the result.
+   */
+  static Uri uriFromBytes(List<int> bytes,
+                          {mimeType: "text/plain",
+                           Iterable<DataUriParameter> parameters}) {
+    var buffer = new StringBuffer();
+    _writeUri(mimeType, parameters, buffer, null);
+    buffer.write(';base64,');
+    BASE64.encoder.startChunkedConversion(buffer)
+          .addSlice(bytes, 0, bytes.length, true);
+    return new Uri(scheme: "data", path: buffer.toString());
+  }
+
+  /**
+   * Parses a string as a `data` URI.
+   */
+  static DataUri parse(String uri) {
+    if (!uri.startsWith("data:")) {
+      throw new FormatException("Does not start with 'data:'", uri, 0);
+    }
+    return _parse(uri, 5);
+  }
+
+  /**
+   * Converts a `DataUri` to a [Uri].
+   *
+   * Returns a `Uri` with scheme `data` and the remainder of the data URI
+   * as path.
+   */
+  Uri toUri() {
+    String content = _text;
+    int colonIndex = _separatorIndices[0];
+    if (colonIndex >= 0) {
+      content = _text.substring(colonIndex + 1);
+    }
+    return new Uri._internal("data", null, null, null, content, null, null);
+  }
+
+  /**
+   * The MIME type of the data URI.
+   *
+   * A data URI consists of a "media type" followed by data.
+   * The mediatype starts with a MIME type and can be followed by
+   * extra parameters.
+   *
+   * Example:
+   *
+   *     data:text/plain;encoding=utf-8,Hello%20World!
+   *
+   * This data URI has the media type `text/plain;encoding=utf-8`, which is the
+   * MIME type `text/plain` with the parameter `encoding` with value `utf-8`.
+   * See [RFC 2045](https://tools.ietf.org/html/rfc2045) for more detail.
+   *
+   * If the first part of the data URI is empty, it defaults to `text/plain`.
+   */
+  String get mimeType {
+    int start = _separatorIndices[0] + 1;
+    int end = _separatorIndices[1];
+    if (start == end) return "text/plain";
+    return Uri._uriDecode(_text, start: start, end: end);
+  }
+
+  /**
+   * Whether the data is base64 encoded or not.
+   */
+  bool get isBase64 => _separatorIndices.length.isOdd;
+
+  /**
+   * The content part of the data URI, as its actual representation.
+   *
+   * This string may contain percent escapes.
+   */
+  String get contentText => _text.substring(_separatorIndices.last + 1);
+
+  /**
+   * The content part of the data URI as bytes.
+   *
+   * If the data is base64 encoded, it will be decoded to bytes.
+   *
+   * If the data is not base64 encoded, it will be decoded by unescaping
+   * percent-escaped characters and returning byte values of each unescaped
+   * character. The bytes will not be, e.g., UTF-8 decoded.
+   */
+  List<int> contentAsBytes() {
+    String text = _text;
+    int start = _separatorIndices.last + 1;
+    if (isBase64) {
+      if (text.endsWith("%3D")) {
+        return BASE64.decode(Uri._uriDecode(text, start: start,
+                                            encoding: LATIN1));

[nweiz, 2015/10/15 21:09:04]

Why does this assume a LATIN1 encoding? It should be based on the parameters,
and per spec it should default to ASCII.

[Lasse Reichstein Nielsen, 2015/10/16 14:38:45]

This function is not creating text, only bytes, so "%c4" should evaluate to the
byte 0xc4. This is really a hack to treat escapes as the bytes they represent,
which is what LATIN-1 encoding does.

[nweiz, 2015/10/19 19:51:20]

It would be good to document that in a comment.

[Lasse Reichstein Nielsen, 2015/10/28 13:55:47]

No longer necessary using the new BASE64 decoder
(but it does assume that percent-escapes only occurs in the padding).

+      }
+      return BASE64.decode(text.substring(start));
+    }
+
+    // Not base64, do percent-decoding and return the remaining bytes.
+    // Compute result size.
+    const int percent = 0x25;
+    int length = text.length - start;
+    for (int i = start; i < text.length; i++) {
+      var codeUnit = text.codeUnitAt(i);
+      if (codeUnit == percent) {
+        i += 2;
+        length -= 2;
+      }
+    }
+    // Fill result array.
+    Uint8List result = new Uint8List(length);
+    if (length == text.length) {
+      result.setRange(0, length, text.codeUnits, start);
+      return result;
+    }
+    int index = 0;
+    for (int i = start; i < text.length; i++) {
+      var codeUnit = text.codeUnitAt(i);
+      if (codeUnit != percent) {
+        result[index++] = codeUnit;
+      } else {
+        if (i + 2 < text.length) {
+          var digit1 = _hexDigit(text.codeUnitAt(i + 1));
+          var digit2 = _hexDigit(text.codeUnitAt(i + 2));
+          if (digit1 >= 0 && digit2 >= 0) {
+            int byte = digit1 * 16 + digit2;
+            result[index++] = byte;
+            i += 2;
+            continue;
+          }
+        }
+        throw new FormatException("Invalid percent escape", text, i);
+      }
+    }
+    assert(index == result.length);
+    return result;
+  }
+
+  // Converts a UTF-16 code-unit to its value as a hex digit.
+  // Returns -1 for non-hex digits.
+  int _hexDigit(int char) {
+    const int char_0 = 0x30;
+    const int char_a = 0x61;
+
+    int digit = char ^ char_0;
+    if (digit <= 9) return digit;
+    char = ((char | 0x20) - char_a) & 0xFFFF;
+    if (char < 6) return 10 + char;
+    return -1;
+  }
+
+  /**
+   * Returns a string created from the content of the data URI.
+   *
+   * If the content is base64 encoded, it will be decoded to bytes and then
+   * decoded to a string using [encoding].
+   *
+   * If the content is not base64 encoded, it will first have percent-escapes
+   * converted to bytes and then the character codes and byte values are
+   * decoded using [encoding].
+   */
+  String contentAsString({Encoding encoding: UTF8}) {

[nweiz, 2015/10/15 21:09:03]

The encoding should be taken from the URI's parameters, at least by default.

[Lasse Reichstein Nielsen, 2015/10/16 14:38:45]

I really, really don't want to parse the "charset" header.
It can take a lot of values that we won't be able to satisfy anyway.
I prefer to leave the parsing of parameters to the user who can then err
appropriately if the parameters are not understandable.

On the other hand, we could promise to understand any of the names accepted by
Encoding.getByName (basically ASCII, LATIN-1 and UTF-8 by a number of names),
and if we can't understand the charset parameter, we can fall back on LATIN-1
(because LATIN-1 decoding can't fail - all byte sequences are valid - so we just
give you the bytes).

If we do that, we should also have a getter that gives you the encoding that
would be used (maybe "Encoding get charsetEncoding").

[nweiz, 2015/10/19 19:51:20]

I think never failing by default is less useful than generally doing the right
thing by default. I'd say:

* If the user passes in an encoding, use that.
* Otherwise, if the charset declares a recognized encoding, use that.
* Otherwise, fail with a useful message.

If the user wants to be sure that they absolutely never fail to decode, they can
still do so, but the default is to provide a good error message in cases where
we're reasonably confident that a correct decoding isn't possible using
dart:convert.

[Lasse Reichstein Nielsen, 2015/10/28 13:55:47]

Sounds reasonable. If there is no charset parameter, the default is US-ASCII, so
the failure should only happen if we have a present-but-unrecognizable charset
parameter.

+    String text = _text;
+    int start = _separatorIndices.last + 1;
+    if (isBase64) {
+      var converter = BASE64.decoder.fuse(encoding.decoder);
+      if (text.endsWith("%3D")) {
+        return converter.convert(Uri._uriDecode(text, start: start,
+                                                encoding: LATIN1));
+      }
+      return converter.convert(text.substring(start));
+    }
+    return Uri._uriDecode(text, start: start, encoding: encoding);
+  }
+
+  /**
+   * An iterable over the parameters of the data URI.
+   *
+   * A data URI may contain parameters between the the MIMI type and the

[nweiz, 2015/10/15 21:09:03]

Nit: "MIMI" -> "MIME"

[Lasse Reichstein Nielsen, 2015/10/16 14:38:45]

:) 

+   * data. This iterates through those parameters, returning each as a
+   * [DataUriParameter] pair of key and value.
+   */
+  Iterable<DataUriParameter> get parameters sync* {
+    for (int i = 3; i < _separatorIndices.length; i += 2) {
+      var start = _separatorIndices[i - 2] + 1;
+      var equals = _separatorIndices[i - 1];
+      var end = _separatorIndices[i];

[nweiz, 2015/10/15 21:09:04]

It looks like this will incorrectly accept invalid URIs that mix up semicolons,
commas, and maybe equals as well.

[Lasse Reichstein Nielsen, 2015/10/16 14:38:45]

The parser should avoid that. This assumes that the separator indices are
correct and provided by the parser.

[nweiz, 2015/10/19 19:51:20]

That might be true. It's a good thing to write tests for, though.

[Lasse Reichstein Nielsen, 2015/10/28 13:55:47]

It's not even possible to test it, because it's all hidden behind private
functions and constructors. I'll add some assertions to ensure that invariants
hold.

+      String key = Uri._uriDecode(_text, start: start, end: equals);
+      String value = Uri._uriDecode(_text, start: equals + 1, end: end);

[nweiz, 2015/10/15 21:09:03]

What about whitespace? If the spec is interpreted like other similar specs,
there should be implicit whitespace between all tokens which will end up encoded
as percent-escapes, and thus find its way into the slices of text this uses. If
we want to match browser behavior (which we should), we ought to strip this
whitespace.

Also, mime types and parameters that are whitespace-only should be disallowed.

[Lasse Reichstein Nielsen, 2015/10/16 14:38:45]

The URI grammars don't generally allow space characters. There is no implicit
whitespace between the tokens - spaces are simply not allowed. If you include a
space character, it needs to be escaped, but then it will also be treated as
meaningful - escaped characters are included literally.

Are you sure browsers allow whitespace in data URIs?

I don't want to validate the parameters (I have no idea what they mean anyway),
so as long as they are syntactically correct, it should be fine. Space is not
allowed in RFC 2045 token (which parameter key and value must be). Percent is
allowed, but probably shouldn't mean percent-encoding, so I guess we should just
check each string against token chars and fail if they are invalid.

[nweiz, 2015/10/19 19:51:20]

Chrome accepts literal spaces in data URIs, but not "%20". I suppose because we
don't support literal spaces in Uri objects anyway we don't need to worry about
it.
The data URI spec says that percent-encoding can be used to escape non-token
chars in parameter values.

[Lasse Reichstein Nielsen, 2015/10/28 13:55:47]

ACK, so we must escape existing percent characters.

[Lasse Reichstein Nielsen, 2015/10/28 13:55:47]

It seems Chrome distinguishes between "data:text/html;%20charset=UTF-8,<%c2%80>"
and "data:text/html;charset=UTF-8,<%c2%80>", where only the latter is
interpreted as UTF-8, so whitespace is not ignored.
(What is shown on in the address bar is not the plain URI, it does some tweaks
to make it more readable).

+      yield new DataUriParameter(key, value);
+    }
+  }
+
+  static DataUri _parse(String text, int start) {
+    assert(start == 0 || start == 5);
+    assert((start == 5) == text.startsWith("data:"));
+
+    /// Character codes.
+    const int comma     = 0x2c;
+    const int slash     = 0x2f;
+    const int semicolon = 0x3b;
+    const int equals    = 0x3d;
+    List indices = [start - 1];
+    int slashIndex = -1;
+    var char;
+    int i = start;
+    for (; i < text.length; i++) {
+      char = text.codeUnitAt(i);
+      if (char == comma || char == semicolon) break;
+      if (char == slash) {
+        if (slashIndex < 0) {
+          slashIndex = i;
+          continue;
+        }
+        throw new FormatException("Invalid MIME type", text, i);
+      }
+    }
+    if (slashIndex < 0 && i > start) {
+      // An empty MIME type is allowed, but if non-empty it must contain
+      // exactly one slash.
+      throw new FormatException("Invalid MIME type", text, i);
+    }
+    while (char != comma) {
+      // parse parameters and/or "base64".
+      indices.add(i);
+      i++;
+      int equalsIndex = -1;
+      for (; i < text.length; i++) {
+        char = text.codeUnitAt(i);
+        if (char == equals) {
+          if (equalsIndex < 0) equalsIndex = i;
+        } else if (char == semicolon || char == comma) {
+          break;
+        }
+      }
+      if (equalsIndex >= 0) {
+        indices.add(equalsIndex);
+      } else {
+        // Have to be final "base64".
+        var lastSeparator = indices.last;
+        if (char != comma ||
+            i != lastSeparator + 7 /* "base64,".length */ ||
+            !text.startsWith("base64", lastSeparator + 1)) {
+          throw new FormatException("Expecting '='", text, i);
+        }
+        break;
+      }
+    }
+    indices.add(i);
+    return new DataUri._(text, indices);
+  }
+
+  String toString() => text;
+
+  // Table of the `token` characters of RFC 2045 in a URI.
+  //
+  // A token is any US-ASCII character except SPACE, control characters and
+  // `tspecial` characters. The `tspecial` category is:
+  // '(', ')', '<', '>', '@', ',', ';', ':', '\', '"', '/', '[, ']', '?', '='.
+  //
+  // In a data URI, we also need to escape '%' and '#' characters.
+  static const _tokenCharTable = const [
+                //             LSB             MSB
+                //              |               |
+      0x0000,   // 0x00 - 0x0f  00000000 00000000
+      0x0000,   // 0x10 - 0x1f  00000000 00000000
+                //               !  $ &'   *+ -.
+      0x6cd2,   // 0x20 - 0x2f  01001011 00110110
+                //              01234567 89
+      0x03ff,   // 0x30 - 0x3f  11111111 11000000
+                //               ABCDEFG HIJKLMNO
+      0xfffe,   // 0x40 - 0x4f  01111111 11111111
+                //              PQRSTUVW XYZ   ^_
+      0xc7ff,   // 0x50 - 0x5f  11111111 11100011
+                //              `abcdefg hijklmno
+      0xffff,   // 0x60 - 0x6f  11111111 11111111
+                //              pqrstuvw xyz{|}~
+      0x7fff];  // 0x70 - 0x7f  11111111 11111110
+}
+
+/**
+ * A parameter of a data URI.
+ *
+ * A parameter is a key and a value.
+ *
+ * The key and value are the actual values to be encoded into the URI.
+ * They will be escaped if necessary when creating a data URI,
+ * and have been unescaped when extracted from a data URI.
+ */
+class DataUriParameter {

[nweiz, 2015/10/15 21:09:03]

Why isn't this just a map? Maps are much easier to use than custom data types.

[Lasse Reichstein Nielsen, 2015/10/16 14:38:45]

Because the same parameter name may occur more than once.
Maps are not suitable for that (HTTP headers have have that problem).

[nweiz, 2015/10/19 19:51:20]

Can they? The MIME spec isn't explicit about this, but it seems to imply that
it's possible to find a single canonical value of a parameter with a given name.
Do you know of content types that require the use of multiple parameters with
the same name?

[Lasse Reichstein Nielsen, 2015/10/28 13:55:47]

Good point. It doesn't actually look like parameters can be repeated, so I'll
change it to a Map<String, String>.

+  /** Parameter key. */
+  final String key;
+  /** Parameter value. */
+  final String value;
+  DataUriParameter(this.key, this.value);
+
+  /**
+   * Creates an iterable of parameters from a map from key to value.
+   *
+   * Parameter keys are not required to be unique in a data URI, but
+   * when they are, a map can be used to represent the parameters, and
+   * this function provides a way to access the map pairs as parameter
+   * values.
+   */
+  static Iterable<DataUriParameter> fromMap(Map<String, String> headers) sync* {
+    for (String key in headers.keys) {
+      yield new DataUriParameter(key, headers[key]);
+    }
+  }
 }