Use new hash table template for compressing token streams.

Use new hash table template for compressing token streams.

Uses the flexible low-level interface to minimize lookups and allocation.

BUG=dart:13747
R=asiva@google.com

Committed: https://code.google.com/p/dart/source/detail?r=38294

[siva, 2014-07-16 01:19:51]

One question:

Did you notice a size difference of the token stream after
this change?

One of the reasons for having separate literal token and ident tables was that
the index value would be a lower number and compress well.

[koda, 2014-07-16 03:27:36]

On 2014/07/16 01:19:51, siva wrote:
> One question:
> 
> Did you notice a size difference of the token stream after
> this change?
> 
> One of the reasons for having separate literal token and ident tables was that
> the index value would be a lower number and compress well.

No, the separate tables were only used for the hashing part. All indices
referred to the same array of objects.

The object indices I now write to the stream have all actually *decreased* by
exactly one, since I removed the (legacy?) unused null_string at the beginning.
(So, in theory, the stream can be slightly shorter.)

Or did I misunderstand your question?

[siva, 2014-07-16 17:23:57]

Got it (forgot the code, it has been a while).

LGTM.

[koda, 2014-07-16 17:39:16]

Committed patchset #6 manually as r38294 (presubmit successful).