Make input/output formats pluggable, adapt to new libraries

pkg/serialization/lib/serialization.dart

Index: pkg/serialization/lib/serialization.dart
diff --git a/pkg/serialization/lib/serialization.dart b/pkg/serialization/lib/serialization.dart
index 2b1377280a35a2e7fd74a23f7dc3811e17cdd0e7..bb4c2b877cb979c857d884e4ae94dcf147d0c4cc 100644
--- a/pkg/serialization/lib/serialization.dart
+++ b/pkg/serialization/lib/serialization.dart
@@ -19,10 +19,10 @@
  *      String output = serialization.write(address);
  *
  * This creates a new serialization and adds a rule for address objects. Right
- * now it has to be passed an address instance because we can't write Address
- * as a literal. Then we ask the Serialization to write the address and we get
- * back a String which is a [JSON] representation of the state of it and related
- * objects.
+ * now it has to be passed an address instance because of limitations using
+ * Address as a literal. Then we ask the Serialization to write the address
+ * and we get back a String which is a [JSON] representation of the state of
+ * it and related objects.
  *
  * The version above used reflection to automatically identify the public
  * fields of the address object. We can also specify those fields explicitly.
@@ -44,9 +44,42 @@
  *            constructor: "",
  *            excludeFields: ["other", "stuff"]);
  *
+ * Writing Rules
+ * =============
  * We can also use a completely non-reflective rule to serialize and
- * de-serialize objects. This can be more cumbersome, but it does work in
- * dart2js, where mirrors are not yet implemented.
+ * de-serialize objects. This can be more work, but it does work in
+ * dart2js, where mirrors are not yet implemented. We can specify this in two
+ * ways. First, we can write our own SerializationRule class that has methods
+ * for our Address class.
+ *
+ *      class AddressRule extends CustomRule {
+ *        bool appliesTo(instance, Writer w) => instance.runtimeType == Address;

[Jennifer Messerly, 2013/01/11 02:21:53]

"instance is Address"?

[Alan Knight, 2013/01/11 19:18:11]

I actually did that on purpose, because if we have subclasses of Address then an
"is" test would also pick those up, and I think we need to have separate rules
for each subclass. But if I'm doing that I should explain it.

[Jennifer Messerly, 2013/01/11 21:54:25]

Yeah, comment would be nice. That makes sense.
Do you need to check for "instance != null" here, or is that handled before we
get to appliesTo?

fwiw, this is one of the first times I've seen an exact type test used in Dart
code. It's neat it can be written so easily.

[Alan Knight, 2013/01/11 22:35:00]

null will  get caught by the PrimitiveRule that comes first. Although it appears
that null.runtimeType works, and returns the type Null.

+ *        getState(instance) => [instance.street, instance.city];
+ *        create(state) => new Address();
+ *        setState(Address a, List state) {
+ *          a.street = state[0];
+ *          a.city = state[1];
+ *        }
+ *      }
+ *
+ * The class needs four different methods. The [appliesTo] method tells us if
+ * the rule should be used to write an object. The [getState] method should
+ * return all the state of the object that we want to recreate,
+ * and should be either a Map or a List. If you want to write to human-readable
+ * formats where it's useful to be able to look at the data as a map from
+ * field names to values, then it's better to return it as a map. Otherwise it's
+ * more efficient to return it as a list. You just need to be sure that the
+ * [create] and [setState] methods interpret the same way as [getState] does.
+ *
+ * The [create] method will create the new object and return it. While it's
+ * possible to create the object and set all its state in this one method, that
+ * increases the likelihood of problems with cycles. So it's better to use the
+ * minimum necessary information in [create] and do more of the work in
+ * [setState].
+ *
+ * The other way to do this is not creating a subclass, but by using a
+ * [ClosureRule] and giving it functions for how to create
+ * the address.
  *
  *      addressToMap(a) => {"number" : a.number, "street" : a.street,
  *          "city" : a.city};
@@ -54,30 +87,20 @@
  *      fillInAddress(Map m, Address a) => a.city = m["city"];
  *      var serialization = new Serialization()
  *        ..addRule(
- *            new ClosureToMapRule(anAddress.runtimeType,
+ *            new ClosureRule(anAddress.runtimeType,
  *                addressToMap, createAddress, fillInAddress);
  *
- * Note that there are three different functions provided. The addressToMap
- * function takes the fields we want serialized from the Address and puts them
- * into a map. The createAddress function creates a new address using a map like
- * the one returned by the first function. And the fillInAddress function fills
- * in any  remaining state in the created object.
- *
- * At the moment, however, using this rule increases the probability of problems
- * with cycles. The problem is that before passing values to the user-supplied
- * functions it has to inflate any references to be the real objects. Since it
- * doesn't know which ones the creation function uses it has to inflate all of
- * them. For example, consider a Node class with parent, leftChild, and
- * rightChild, and the parent field was final and set in the constructor. When
- * we inflate all of the values we will end up with a cycle and can't
- * de-serialize. If we know which fields are used by the constructor we can
- * inflate only those, which is what BasicRule does. We expect to make a richer
- * API for rules not using reflection, but there's a tension between providing
- * the serialization process with enough information and making it more work
- * to specify.
+ * In this case we have created standalone functions rather than
+ * methods in a subclass and we pass them to the constructor of
+ * [ClosureRule]. In this case we've also had them use maps rather than
+ * lists for the state, but either would work as long as the rule is
+ * consistent with the representation it uses. We pass it the runtimeType
+ * of the object, and functions equivalent to the methods on [CustomRule]
  *
+ * Constant Values
+ * ===============
  * There are cases where the constructor needs values that we can't easily get
- * the serialized object. For example, we may just want to pass null, or a
+ * from the serialized object. For example, we may just want to pass null, or a
  * constant value. To support this, we can specify as constructor fields
  * values that aren't field names. If any value isn't a String, it will be
  * treated as a constant and passed unaltered to the constructor.
@@ -93,34 +116,34 @@
  *
  * Writing
  * =======
- * To write objects, we use the write() methods. There are two variations.
+ * To write objects, we use the write() method.
+ *
+ *       var output = serialization.write(someObject);
  *
- *       String output = serialization.write(someObject);
- *       List output = serialization.writeFlat(someObject);
+ * By default this uses a representation in which objects are represented as
+ * maps keyed by field name, but in which references between objects have been
+ * converted into Reference objects. This is then encoded as a [json] string.
  *
- * The first uses a representation in which objects are represented as maps
- * keyed by field name, but in which references between objects have been
- * converted into Reference objects. This is then encoded as a [JSON] string.
+ * We can write objects in different formats by passing a [Format] object to
+ * the [write] method or by getting a [Writer] object. The available formats
+ * include the default, a simple "flat" format that doesn't include field names,
+ * and a simple JSON format that produces output more suitable for talking to
+ * services that expect JSON in a predefined format. Examples of these are
  *
- * The second representation holds all the objects as a List of simple types.
- * For practical use you may want to convert that to a [JSON] or other encoded
- * representation as well.
+ *      String output = serialization.write(address, new SimpleMapFormat());
+ *      List output = serialization.write(address, new SimpleFlatFormat());
+ *      Map output = serialization.write(address, new SimpleJSONFormat());
+ * Or, using a [Writer] explicitly
+ *      var writer = serialization.newWriter(new SimpleFlatFormat());
+ *      var output = writer.write(address);
  *
- * Both representations are primarily intended as proofs of concept for
- * different types of representation, and we expect to generalize that to a
- * pluggable mechanism for different representations.
+ * These representations are not yet considered stable.
  *
  * Reading
  * =======
- * To read objects, the corresponding methods are [read] and [readFlat].
+ * To read objects, the corresponding [read] method can be used.
  *
- *       List input = serialization.read(aString);
- *       List input = serialization.readFlat(aList);
- *
- * There is also a convenience method for the case of reading a single object.
- *
- *       Object result = serialization.readOne(aString);
- *       Object result = serialization.readOneFlat(aString);
+ *       Address input = serialization.read(aString);
  *
  * When reading, the serialization instance doing the reading must be configured
  * with compatible rules to the one doing the writing. It's possible for the
@@ -129,21 +152,27 @@
  * same. The simplest way to achieve this is by having the serialization
  * variable [selfDescribing] be true. In that case the rules themselves are also
  * stored along with the serialized data, and can be read back on the receiving
- * end. Note that this does not yet work for [ClosureToMapRule]. The
- * [selfDescribing] variable is true by default.
+ * end. Note that this may not work for all rules or all formats. The
+ * [selfDescribing] variable is true by default, but the SimpleJSONFormat does

[Jennifer Messerly, 2013/01/11 02:21:53]

SimpleJsonFormat?

I think we do camel case (e.g. innerHTML became innerHtml)

[Alan Knight, 2013/01/11 19:18:11]

Done.

+ * not support it, since the point is to provide a representation in a form
+ * other services might expect. Using CustomRule or ClosureRule also does not
+ * yet work with the [selfDescribing] variable.
  *
+ * Named Objects

[Jennifer Messerly, 2013/01/11 02:21:53]

like these new sections!

+ * =============
  * When reading, some object references should not be serialized, but should be
  * connected up to other instances on the receiving side. A notable example of
  * this is when serialization rules have been stored. Instances of BasicRule
  * take a [ClassMirror] in their constructor, and we cannot serialize those. So
  * when we read the rules, we must provide a Map<String, Object> which maps from
  * the simple name of classes we are interested in to a [ClassMirror]. This can
- * be provided either in the [externalObjects] variable of the Serialization,
- * or as an additional parameter to the reading methods.
+ * be provided either in the [namedObjects] variable of the Serialization,
+ * or as an additional parameter to the reading and writing methods on the
+ * [Reader] or [Writer] respectively.
  *
  *     new Serialization()
  *       ..addRuleFor(new Person(), constructorFields: ["name"])
- *       ..externalObjects['Person'] = reflect(new Person()).type;
+ *       ..namedObjects['Person'] = reflect(new Person()).type;
  */
 library serialization;
 
@@ -155,6 +184,7 @@ import 'dart:json' as json;
 part 'src/reader_writer.dart';
 part 'src/serialization_rule.dart';
 part 'src/basic_rule.dart';
+part 'src/format.dart';
 
 /**
  * This class defines a particular serialization scheme, in terms of
@@ -291,13 +321,12 @@ class Serialization {
   }
 
   /**
-   * This is the basic method to write out an object graph rooted at
-   * [object] and return the result. Right now this is hard-coded to return
-   * a String from a custom [JSON] format, but that is likely to change to be
-   * more pluggable in the near future.
+   * This writes out an object graph rooted at [object] and returns the result.
+   * The [format] parameter determines the form of the result. The default
+   * format returns a String in [json] format.
    */
-  String write(Object object) {
-    return newWriter().write(object);
+  write(Object object, [Format format]) {
+    return newWriter(format).write(object);
   }
 
   /**
@@ -305,15 +334,8 @@ class Serialization {
    * want to do something more complex with the writer than just returning
    * the final result.
    */
-  Writer newWriter() => new Writer(this);
-
-  /**
-   * Write out the tree in a custom flat format, returning a list containing
-   * only "simple" types: num, String, and bool.
-   */
-  List writeFlat(Object object) {
-    return newWriter().writeFlat(object);
-  }
+  Writer newWriter([Format format]) =>
+      new Writer(this, format);
 
   /**
    * Read the serialized data from [input] and return the root object
@@ -333,13 +355,13 @@ class Serialization {
    * you want to do something more complex with the reader than just returning
    * the final result.
    */
-  Reader newReader() => new Reader(this);
+  Reader newReader([Format format]) => new Reader(this, format);
 
   /**
    * Return the list of SerializationRule that apply to [object]. For
    * internal use, but public because it's used in testing.
    */
-  List<SerializationRule> rulesFor(object, Writer w) {
+  Iterable<SerializationRule> rulesFor(object, Writer w) {
     // This has a couple of edge cases.
     // 1) The owning object may have indicated we should use a different
     // rule than the default.
@@ -363,17 +385,17 @@ class Serialization {
       target = object;
       candidateRules = _rules;
     }
-    List applicable =
-        candidateRules.where((each) => each.appliesTo(target, w)).toList();
+    Iterable applicable = candidateRules.where(
+        (each) => each.appliesTo(target, w));
 
     if (applicable.isEmpty) {
       return [addRuleFor(target)];
     }
 
     if (applicable.length == 1) return applicable;
-    var first = applicable[0];
+    var first = applicable.first;
     var finalRules = applicable.where(
-        (x) => !x.mustBePrimary || (x == first)).toList();
+        (x) => !x.mustBePrimary || (x == first));
 
     if (finalRules.isEmpty) throw new SerializationException(
         'No valid rule found for object $object');
@@ -386,7 +408,7 @@ class Serialization {
    * If there are new rule classes created, they will need to be described
    * here.
    */
-  Serialization _ruleSerialization() {
+  Serialization ruleSerialization() {
     // TODO(alanknight): There's an extensibility issue here with new rules.
     // TODO(alanknight): How to handle rules with closures? They have to
     // exist on the other side, but we might be able to hook them up by name,