Fix bug when the scanner does a pushback at the end of input.

src/scanner-base.h

Index: src/scanner-base.h
diff --git a/src/scanner-base.h b/src/scanner-base.h
index b668df5046dae63903c4a2fd6445336235a24024..1024ad18582e3c078793d2295fe3cb8a92d23b18 100644
--- a/src/scanner-base.h
+++ b/src/scanner-base.h
@@ -64,10 +64,10 @@ class UC16CharacterStream {
   // Returns and advances past the next UC16 character in the input
   // stream. If there are no more characters, it returns a negative
   // value.
-  inline int32_t Advance() {
+  inline uc32 Advance() {
     if (buffer_cursor_ < buffer_end_ || ReadBlock()) {
       pos_++;
-      return *(buffer_cursor_++);
+      return static_cast<uc32>(*(buffer_cursor_++));
     }
     // Note: currently the following increment is necessary to avoid a
     // parser problem! The scanner treats the final kEndOfInput as
@@ -97,13 +97,14 @@ class UC16CharacterStream {
     return SlowSeekForward(character_count);
   }
 
-  // Pushes back the most recently read UC16 character, i.e.,
-  // the value returned by the most recent call to Advance.
+  // Pushes back the most recently read UC16 character (or negative

[Søren Thygesen Gjesse, 2011/01/14 10:37:16]

negative value -> kEndOfInput?

[Lasse Reichstein, 2011/01/14 10:49:04]

The kEndOfInput name isn't publicly visible. All we guarantee is that we return
negative for end-of-input.
The constant is used internally only to give meaningful name to the value
returned in those case.

+  // value if at end of input), i.e., the value returned by the most recent
+  // call to Advance.
   // Must not be used right after calling SeekForward.
-  virtual void PushBack(uc16 character) = 0;
+  virtual void PushBack(int32_t character) = 0;
 
  protected:
-  static const int32_t kEndOfInput = -1;
+  static const uc32 kEndOfInput = -1;
 
   // Ensures that the buffer_cursor_ points to the character at
   // position pos_ of the input, if possible. If the position