Make input/output formats pluggable, adapt to new libraries

pkg/serialization/lib/serialization.dart

 // Copyright (c) 2012, the Dart project authors.  Please see the AUTHORS file
 // for details. All rights reserved. Use of this source code is governed by a
 // BSD-style license that can be found in the LICENSE file.
 
 /**
  * This provides a general-purpose serialization facility for Dart objects. A
  * [Serialization] is defined in terms of [SerializationRule]s and supports
  * reading and writing to different formats.
  *
  * Setup
  * =====
  * A simple example of usage is
  *
  *      var address = new Address();
  *      address.street = 'N 34th';
  *      address.city = 'Seattle';
  *      var serialization = new Serialization()
  *          ..addRuleFor(address);
  *      String output = serialization.write(address);
  *
  * This creates a new serialization and adds a rule for address objects. Right
- * now it has to be passed an address instance because we can't write Address
- * as a literal. Then we ask the Serialization to write the address and we get
- * back a String which is a [JSON] representation of the state of it and related
- * objects.
+ * now it has to be passed an address instance because of limitations using
+ * Address as a literal. Then we ask the Serialization to write the address
+ * and we get back a String which is a [JSON] representation of the state of
+ * it and related objects.
  *
  * The version above used reflection to automatically identify the public
  * fields of the address object. We can also specify those fields explicitly.
  *
  *      var serialization = new Serialization()
  *        ..addRuleFor(address,
  *            constructor: "create",
  *            constructorFields: ["number", "street"],
  *            fields: ["city"]);
  *
  * This rule still uses reflection to access the fields, but does not try to
  * identify which fields to use, but instead uses only the "number" and "street"
  * fields that we specified. We may also want to tell it to identify the
  * fields, but to specifically omit certain fields that we don't want
  * serialized.
  *
  *      var serialization = new Serialization()
  *        ..addRuleFor(address,
  *            constructor: "",
  *            excludeFields: ["other", "stuff"]);
  *
+ * Writing Rules
+ * =============
  * We can also use a completely non-reflective rule to serialize and
- * de-serialize objects. This can be more cumbersome, but it does work in
- * dart2js, where mirrors are not yet implemented.
+ * de-serialize objects. This can be more work, but it does work in
+ * dart2js, where mirrors are not yet implemented. We can specify this in two
+ * ways. First, we can write our own SerializationRule class that has methods
+ * for our Address class.
+ *
+ *      class AddressRule extends CustomRule {
+ *        bool appliesTo(instance, Writer w) => instance.runtimeType == Address;

[Jennifer Messerly, 2013/01/11 02:21:53]

"instance is Address"?

[Alan Knight, 2013/01/11 19:18:11]

I actually did that on purpose, because if we have subclasses of Address then an
"is" test would also pick those up, and I think we need to have separate rules
for each subclass. But if I'm doing that I should explain it.

[Jennifer Messerly, 2013/01/11 21:54:25]

Yeah, comment would be nice. That makes sense.
Do you need to check for "instance != null" here, or is that handled before we
get to appliesTo?

fwiw, this is one of the first times I've seen an exact type test used in Dart
code. It's neat it can be written so easily.

[Alan Knight, 2013/01/11 22:35:00]

null will  get caught by the PrimitiveRule that comes first. Although it appears
that null.runtimeType works, and returns the type Null.

+ *        getState(instance) => [instance.street, instance.city];
+ *        create(state) => new Address();
+ *        setState(Address a, List state) {
+ *          a.street = state[0];
+ *          a.city = state[1];
+ *        }
+ *      }
+ *
+ * The class needs four different methods. The [appliesTo] method tells us if
+ * the rule should be used to write an object. The [getState] method should
+ * return all the state of the object that we want to recreate,
+ * and should be either a Map or a List. If you want to write to human-readable
+ * formats where it's useful to be able to look at the data as a map from
+ * field names to values, then it's better to return it as a map. Otherwise it's
+ * more efficient to return it as a list. You just need to be sure that the
+ * [create] and [setState] methods interpret the same way as [getState] does.
+ *
+ * The [create] method will create the new object and return it. While it's
+ * possible to create the object and set all its state in this one method, that
+ * increases the likelihood of problems with cycles. So it's better to use the
+ * minimum necessary information in [create] and do more of the work in
+ * [setState].
+ *
+ * The other way to do this is not creating a subclass, but by using a
+ * [ClosureRule] and giving it functions for how to create
+ * the address.
  *
  *      addressToMap(a) => {"number" : a.number, "street" : a.street,
  *          "city" : a.city};
  *      createAddress(Map m) => new Address.create(m["number"], m["street"]);
  *      fillInAddress(Map m, Address a) => a.city = m["city"];
  *      var serialization = new Serialization()
  *        ..addRule(
- *            new ClosureToMapRule(anAddress.runtimeType,
+ *            new ClosureRule(anAddress.runtimeType,
  *                addressToMap, createAddress, fillInAddress);
  *
- * Note that there are three different functions provided. The addressToMap
- * function takes the fields we want serialized from the Address and puts them
- * into a map. The createAddress function creates a new address using a map like
- * the one returned by the first function. And the fillInAddress function fills
- * in any  remaining state in the created object.
+ * In this case we have created standalone functions rather than
+ * methods in a subclass and we pass them to the constructor of
+ * [ClosureRule]. In this case we've also had them use maps rather than
+ * lists for the state, but either would work as long as the rule is
+ * consistent with the representation it uses. We pass it the runtimeType
+ * of the object, and functions equivalent to the methods on [CustomRule]
  *
- * At the moment, however, using this rule increases the probability of problems
- * with cycles. The problem is that before passing values to the user-supplied
- * functions it has to inflate any references to be the real objects. Since it
- * doesn't know which ones the creation function uses it has to inflate all of
- * them. For example, consider a Node class with parent, leftChild, and
- * rightChild, and the parent field was final and set in the constructor. When
- * we inflate all of the values we will end up with a cycle and can't
- * de-serialize. If we know which fields are used by the constructor we can
- * inflate only those, which is what BasicRule does. We expect to make a richer
- * API for rules not using reflection, but there's a tension between providing
- * the serialization process with enough information and making it more work
- * to specify.
- *
+ * Constant Values
+ * ===============
  * There are cases where the constructor needs values that we can't easily get
- * the serialized object. For example, we may just want to pass null, or a
+ * from the serialized object. For example, we may just want to pass null, or a
  * constant value. To support this, we can specify as constructor fields
  * values that aren't field names. If any value isn't a String, it will be
  * treated as a constant and passed unaltered to the constructor.
  *
  * In some cases a non-constructor field should not be set using field
  * access or a setter, but should be done by calling a method. For example, it
  * may not be possible to set a List field "foo", and you need to call an
  * addFoo() method for each entry in the list. In these cases, if you are using
  * a BasicRule for the object you can call the setFieldWith() method.
  *
  *       s..addRuleFor(fooHolderInstance).setFieldWith("foo",
  *           (parent, value) => for (var each in value) parent.addFoo(value));
  *
  * Writing
  * =======
- * To write objects, we use the write() methods. There are two variations.
+ * To write objects, we use the write() method.
  *
- *       String output = serialization.write(someObject);
- *       List output = serialization.writeFlat(someObject);
+ *       var output = serialization.write(someObject);
  *
- * The first uses a representation in which objects are represented as maps
- * keyed by field name, but in which references between objects have been
- * converted into Reference objects. This is then encoded as a [JSON] string.
+ * By default this uses a representation in which objects are represented as
+ * maps keyed by field name, but in which references between objects have been
+ * converted into Reference objects. This is then encoded as a [json] string.
  *
- * The second representation holds all the objects as a List of simple types.
- * For practical use you may want to convert that to a [JSON] or other encoded
- * representation as well.
+ * We can write objects in different formats by passing a [Format] object to
+ * the [write] method or by getting a [Writer] object. The available formats
+ * include the default, a simple "flat" format that doesn't include field names,
+ * and a simple JSON format that produces output more suitable for talking to
+ * services that expect JSON in a predefined format. Examples of these are
  *
- * Both representations are primarily intended as proofs of concept for
- * different types of representation, and we expect to generalize that to a
- * pluggable mechanism for different representations.
+ *      String output = serialization.write(address, new SimpleMapFormat());
+ *      List output = serialization.write(address, new SimpleFlatFormat());
+ *      Map output = serialization.write(address, new SimpleJSONFormat());
+ * Or, using a [Writer] explicitly
+ *      var writer = serialization.newWriter(new SimpleFlatFormat());
+ *      var output = writer.write(address);
+ *
+ * These representations are not yet considered stable.
  *
  * Reading
  * =======
- * To read objects, the corresponding methods are [read] and [readFlat].
+ * To read objects, the corresponding [read] method can be used.
  *
- *       List input = serialization.read(aString);
- *       List input = serialization.readFlat(aList);
- *
- * There is also a convenience method for the case of reading a single object.
- *
- *       Object result = serialization.readOne(aString);
- *       Object result = serialization.readOneFlat(aString);
+ *       Address input = serialization.read(aString);
  *
  * When reading, the serialization instance doing the reading must be configured
  * with compatible rules to the one doing the writing. It's possible for the
  * rules to be different, but they need to be able to read the same
  * representation. For most practical purposes right now they should be the
  * same. The simplest way to achieve this is by having the serialization
  * variable [selfDescribing] be true. In that case the rules themselves are also
  * stored along with the serialized data, and can be read back on the receiving
- * end. Note that this does not yet work for [ClosureToMapRule]. The
- * [selfDescribing] variable is true by default.
+ * end. Note that this may not work for all rules or all formats. The
+ * [selfDescribing] variable is true by default, but the SimpleJSONFormat does

[Jennifer Messerly, 2013/01/11 02:21:53]

SimpleJsonFormat?

I think we do camel case (e.g. innerHTML became innerHtml)

[Alan Knight, 2013/01/11 19:18:11]

Done.

+ * not support it, since the point is to provide a representation in a form
+ * other services might expect. Using CustomRule or ClosureRule also does not
+ * yet work with the [selfDescribing] variable.
  *
+ * Named Objects

[Jennifer Messerly, 2013/01/11 02:21:53]

like these new sections!

+ * =============
  * When reading, some object references should not be serialized, but should be
  * connected up to other instances on the receiving side. A notable example of
  * this is when serialization rules have been stored. Instances of BasicRule
  * take a [ClassMirror] in their constructor, and we cannot serialize those. So
  * when we read the rules, we must provide a Map<String, Object> which maps from
  * the simple name of classes we are interested in to a [ClassMirror]. This can
- * be provided either in the [externalObjects] variable of the Serialization,
- * or as an additional parameter to the reading methods.
+ * be provided either in the [namedObjects] variable of the Serialization,
+ * or as an additional parameter to the reading and writing methods on the
+ * [Reader] or [Writer] respectively.
  *
  *     new Serialization()
  *       ..addRuleFor(new Person(), constructorFields: ["name"])
- *       ..externalObjects['Person'] = reflect(new Person()).type;
+ *       ..namedObjects['Person'] = reflect(new Person()).type;
  */
 library serialization;
 
 import 'src/mirrors_helpers.dart';
 import 'src/serialization_helpers.dart';
 import 'dart:async';
 import 'dart:json' as json;
 
 part 'src/reader_writer.dart';
 part 'src/serialization_rule.dart';
 part 'src/basic_rule.dart';
+part 'src/format.dart';
 
 /**
  * This class defines a particular serialization scheme, in terms of
  * [SerializationRule] instances, and supports reading and writing them.
  * See library comment for examples of usage.
  */
 class Serialization {
 
   /**
    * The serialization is controlled by the list of Serialization rules. These
@@ skipping 116 matching lines @@
    * handle most simple cases, but for adding an arbitrary rule, including
    * a SerializationRule subclass which you have created, you can use this
    * method.
    */
   void addRule(SerializationRule rule) {
     rule.number = _rules.length;
     _rules.add(rule);
   }
 
   /**
-   * This is the basic method to write out an object graph rooted at
-   * [object] and return the result. Right now this is hard-coded to return
-   * a String from a custom [JSON] format, but that is likely to change to be
-   * more pluggable in the near future.
+   * This writes out an object graph rooted at [object] and returns the result.
+   * The [format] parameter determines the form of the result. The default
+   * format returns a String in [json] format.
    */
-  String write(Object object) {
-    return newWriter().write(object);
+  write(Object object, [Format format]) {
+    return newWriter(format).write(object);
   }
 
   /**
    * Return a new [Writer] object for this serialization. This is useful if you
    * want to do something more complex with the writer than just returning
    * the final result.
    */
-  Writer newWriter() => new Writer(this);
-
-  /**
-   * Write out the tree in a custom flat format, returning a list containing
-   * only "simple" types: num, String, and bool.
-   */
-  List writeFlat(Object object) {
-    return newWriter().writeFlat(object);
-  }
+  Writer newWriter([Format format]) =>
+      new Writer(this, format);
 
   /**
    * Read the serialized data from [input] and return the root object
    * from the result. If there are objects that need to be resolved
    * in the current context, they should be provided in [externals] as a
    * Map from names to values. In particular, in the current implementation
    * any class mirrors needed should be provided in [externals] using the
    * class name as a key. In addition to the [externals] map provided here,
    * values will be looked up in the [externalObjects] map.
    */
   read(String input, [Map externals = const {}]) {
     return newReader().read(input, externals);
   }
 
   /**
    * Return a new [Reader] object for this serialization. This is useful if
    * you want to do something more complex with the reader than just returning
    * the final result.
    */
-  Reader newReader() => new Reader(this);
+  Reader newReader([Format format]) => new Reader(this, format);
 
   /**
    * Return the list of SerializationRule that apply to [object]. For
    * internal use, but public because it's used in testing.
    */
-  List<SerializationRule> rulesFor(object, Writer w) {
+  Iterable<SerializationRule> rulesFor(object, Writer w) {
     // This has a couple of edge cases.
     // 1) The owning object may have indicated we should use a different
     // rule than the default.
     // 2) We may not have a rule, in which case we lazily create a BasicRule.
     // 3) Rules are allowed to say mustBePrimary, meaning that they can be used
     // iff no other rule was chosen first.
     // TODO(alanknight): Can the mustBePrimary mechanism be removed or changed.
     // It adds an order dependency to the rules, and is messy. Reconsider in the
     // light of a more general mechanism for multiple rules per object.
     // TODO(alanknight): Finding which rules apply seems likely to be a
     // bottleneck, particularly with the current reflective implementation.
     // Consider how to improve it. e.g. cache the list of rules by class. But
     // be careful of issues like rules which have arbitrary predicates. Or
     // consider having the arbitrary predicates be secondary to an initial
     // class-based lookup mechanism.
     var target, candidateRules;
     if (object is DesignatedRuleForObject) {
       target = object.target;
       candidateRules = object.possibleRules(_rules);
     } else {
       target = object;
       candidateRules = _rules;
     }
-    List applicable =
-        candidateRules.where((each) => each.appliesTo(target, w)).toList();
+    Iterable applicable = candidateRules.where(
+        (each) => each.appliesTo(target, w));
 
     if (applicable.isEmpty) {
       return [addRuleFor(target)];
     }
 
     if (applicable.length == 1) return applicable;
-    var first = applicable[0];
+    var first = applicable.first;
     var finalRules = applicable.where(
-        (x) => !x.mustBePrimary || (x == first)).toList();
+        (x) => !x.mustBePrimary || (x == first));
 
     if (finalRules.isEmpty) throw new SerializationException(
         'No valid rule found for object $object');
     return finalRules;
   }
 
   /**
    * Create a Serialization for serializing SerializationRules. This is used
    * to save the rules in a self-describing format along with the data.
    * If there are new rule classes created, they will need to be described
    * here.
    */
-  Serialization _ruleSerialization() {
+  Serialization ruleSerialization() {
     // TODO(alanknight): There's an extensibility issue here with new rules.
     // TODO(alanknight): How to handle rules with closures? They have to
     // exist on the other side, but we might be able to hook them up by name,
     // or we might just be able to validate that they're correctly set up
     // on the other side.
 
     // Make some bogus rule instances so we have something to feed rule creation
     // and get their types. If only we had class literals implemented...
    var basicRule = new BasicRule(reflect(null).type, '', [], [], []);
 
@@ skipping 31 matching lines @@
   }
 }
 
 /**
  * An exception class for errors during serialization.
  */
 class SerializationException implements Exception {
   final String message;
   const SerializationException([this.message]);
 }