Basic support for reading and writing JSON objects.

runtime/vm/json.h

Index: runtime/vm/json.h
===================================================================
--- runtime/vm/json.h	(revision 0)
+++ runtime/vm/json.h	(revision 0)
@@ -0,0 +1,118 @@
+// Copyright (c) 2012, the Dart project authors.  Please see the AUTHORS file
+// for details. All rights reserved. Use of this source code is governed by a
+// BSD-style license that can be found in the LICENSE file.
+
+#ifndef VM_JSON_H_
+#define VM_JSON_H_
+
+#include "vm/allocation.h"
+#include "vm/globals.h"
+
+namespace dart {
+
+
+// A low level interface to tokenize JSON strings.

[turnidge, 2012/04/09 22:26:42]

Will anybody ever use this directly, or will they always use JSONReader instead?

It would be nice to remove this from the header entirely, but then it would be
hard to write direct scanner tests.  You could move this to the bottom of the
header file with a comment saying that it is in the header only so the unit test
can access it.  Your call.

[hausner, 2012/04/10 16:26:51]

The JSONReader declarations in this file need it. And since these are
ValueObject subtypes, not pointers, the order is dictated as well. (A forward
declaration doesn't work.)

On 2012/04/09 22:26:42, turnidge wrote:

+class JSONScanner : ValueObject {
+ public:
+  enum Token {
+    TokenIllegal = 0,
+    TokenLBrace,
+    TokenRBrace,
+    TokenLBrack,
+    TokenRBrack,
+    TokenColon,
+    TokenComma,
+    TokenString,
+    TokenInteger,
+    TokenTrue,
+    TokenFalse,
+    TokenNull,
+    TokenEOM
+  };
+  explicit JSONScanner(const char* msg);

[turnidge, 2012/04/09 22:26:42]

For consistency, consider renaming msg -> json_object?  Or perhaps json_text.

[hausner, 2012/04/10 16:26:51]

Done.

+
+  void Set(const char* str);

[turnidge, 2012/04/09 22:26:42]

str -> json_object

[hausner, 2012/04/10 16:26:51]

Done.

+  void Scan();
+  Token CurrentToken() const { return token_; }
+  bool EOM() const { return token_ == TokenEOM; }
+  const char* TokenChars() const { return token_start_; }
+  int TokenLen() const { return token_length_; }
+  bool IsStringLiteral(const char* literal) const;
+  void Skip(Token matching_token);
+
+ private:
+  bool IsLetter(char ch) const;
+  bool IsDigit(char ch) const;
+  bool IsLiteral(const char* literal);
+  void ScanNumber();
+  void ScanString();
+  void Recognize(Token t);
+
+  const char* current_pos_;
+  const char* token_start_;
+  int token_length_;
+  Token token_;
+  const char* string_;

[turnidge, 2012/04/09 22:26:42]

string_ => json_object_

[hausner, 2012/04/10 16:26:51]

Actuall, string_ wasn't even used anywhere. Removed.

+};
+
+
+// JSONReader is a higher level interface that allows for lookup of
+// name-value pairs in JSON objects.

[turnidge, 2012/04/09 22:26:42]

 JSONReader reparses the whole json text on every call to Seek().  This would be
good if we expected to only look up one value.  But I think that we will
probably care about all of the name-value pairs in a given command.

What do you think of returning a parse tree for the json object?

Right now, usage looks like this:

  reader.Set(rawText);
  if (reader.Seek("foo")) {
    reader.Set(reader.ValueChars());
    if (reader.Seek("subfield")) {
      // handle foo/subfield.
      ....
    }
  }
  if (reader.Error()) {
     ...error stuff...
  }
  // What if I forget to call Set here?
  reader.Set(rawText)
  if (reader.Seek("bar")) {
    ....other stuff
  }
  if (reader.Error()) {
     ...
  }

It's sort of awkward to use because you need to remember to call Set again every
time you pop back out to the outer level.  You also need to check for Error()
over and over again (I've left some of this out) because you may or may not
encounter the error on each call to Seek().  It is also awkward to iterate over
array elements in order.

If we just parsed the whole thing at once, we could check for errors once.  I
guess that would look something like this:

JSONNode* top = reader.Parse();
if (top.Error()) {
}
JSONNode* foo_node = top.Child("foo");
if (foo_node) {
  JSONNode* subfield = foo_node.Child("subfield");
  if (subfield) {
     ....
  }
}
...

Thoughts?

[hausner, 2012/04/10 16:26:51]

As we discussed in person, this is a low-level interface where I didn't want to
allocate anything. The interface/class isn't complete, by all means. I expect
this to evolve according to our needs. If we find that for another use case, it
would make sense to create a value tree from the text buffer, we can add that as
an alternative interface, either within JSONReader or as a new class that uses
JSONScanner.


On 2012/04/09 22:26:42, turnidge wrote:

+class JSONReader : ValueObject {
+ public:
+  enum JSONType {
+    kString,
+    kInteger,
+    kObject,
+    kArray,
+    kLiteral,
+    kNone
+  };
+
+  explicit JSONReader(const char* json_object);
+  void Set(const char* json_object);

[turnidge, 2012/04/09 22:26:42]

Set is a pretty generic name.  Consider something more explanatory like
"SetText" or something like that.

[hausner, 2012/04/10 16:26:51]

Done.

+
+  // Returns true if the a pair with the given name was found.

[turnidge, 2012/04/09 22:26:42]

typo: "the a"

[hausner, 2012/04/10 16:26:51]

Done.

+  bool Seek(const char* name);
+
+  // Returns true if a syntax error was found.
+  bool Error() const { return error_; }
+
+  JSONType Type() const;
+  const char* ValueChars() const {
+    return (Type() != kNone) ? scanner_.TokenChars() : NULL;
+  }
+  int ValueLen() const {
+    return (Type() != kNone) ? scanner_.TokenLen() : 0;
+  }
+  bool IsStringLiteral(const char* literal) const {
+    return scanner_.IsStringLiteral(literal);
+  }
+
+ private:
+  JSONScanner scanner_;
+  const char* json_object_;
+  bool error_;
+};
+
+

[turnidge, 2012/04/09 22:26:42]

Add a class-level comment here?

[hausner, 2012/04/10 16:26:51]

Done.

+class JSONWriter : ValueObject {

[turnidge, 2012/04/09 22:26:42]

It's confusing to call this a JSONWriter since it doesn't have any json specific
functionality.  I would prefer giving this a more generic name and putting this
in its own file.  Maybe "PrintBuffer" or "CharBuffer" or something.

If it did have json-specific functionality, then the name would be less
confusing.  A JSONWriter would have functions like "WritePair", etc.

[hausner, 2012/04/10 16:26:51]

I agree, this has nothing to do with JSON at this point. It's a formatter with
built-in buffer management. Renamed. 

+ public:
+  explicit JSONWriter(intptr_t buf_size);
+  ~JSONWriter();
+
+  intptr_t Printf(const char* format, ...);
+  void Clear();
+
+  char* buf() { return buf_; }
+
+ private:
+  void GrowBuffer(intptr_t len);
+  char* buf_;
+  intptr_t buf_size_;
+  intptr_t msg_len_;
+};
+
+}  // namespace dart
+
+#endif  // VM_JSON_H_