Update of the input stream interface

Update of the input stream interface

This combines the StringInputStream and the InputStream interfaces.

BUG=
TEST=

[Søren Gjesse, 2011-11-28 14:18:37]

This is a RFC on consolidation of the input stream interface


I have tried to get rid of the StringInputStream and move all out
basic stream input to the InputStream interface. This involved
adding an encoding property to the stream making the data that
can be read to strings instead of byte arrays (List<int>). The
methods readLine and readUntil have also been added.

However this also involved adding the readLine method which have
some issues with the way the data handler is triggered. In the
current implementation the data handler is triggered when there
is data available (level triggered). If readLine is called in the
data handler it might return null as no full line is actually
available. However, this will cause the data handler to be
triggered again as there is still data available.

In order to address this the properties chunkSize and dataPattern
have been added. If chunkSize is set the data handler will only
be triggered if chunk size number of bytes/characters are
available. If dataPattern is set the data handler will only be
triggered if a match for the specified pattern is in the
available data. To support readLine (which is just a special case
of readUntil) there will be a special pattern for line end.

Finally I have added the method pipe to pipe the input stream to
an output stream.

My concern with this input stream interface is that it gets too
complicated. There more properties and methods, and some methods
can only be used if certain perperties have been set. However the
requirement of having to wrap one input stream into another have
also been a concern.

Another issue is that all implementers of an input stream will
have to support this advanced interface. It should probably be
possible to handle most of it in an abstract superclass though.

[Mads Ager (google), 2011-11-30 10:44:07]

This gets pretty complicated.

Søren and I discussed this today and we would prefer to keep InputStream really,
really simple. The only thing that it can do is tell you when there are bytes
ready to be read. On top of that you can build StringInputStreams and
ChunkedInputStreams and any other type of interpretation of the raw bytes with
only one level of buffering. Additionally it has the advantage that someone can
write an InputStream which is just a byte provider very quickly and get
StringInputStream and ChunkedInputStreams for free.

The down side is that you have to do wrapping. However, I think that is worth
the simplicity in the implementation and for the programmer wanting to provide
an InputStream:

var input = new StringInputStream(process.stdout, 'UTF-8');
input.lineHandler = () {
  // Do stuff with the lines that you get from readLine.
  // This handler is only called when a full line is available.
}

var input = new ChunkedInputStream(process.stdout, 1024);
input.chunkHandler = () {
  // Do stuff with the chunks you get from readChunk.
  // This handler is only called when a full chunk is available.
  // You can change the chunk size with assignments: chunkSize = 512;
}

http://codereview.chromium.org/8698018/diff/1/runtime/bin/input_stream.dart
File runtime/bin/input_stream.dart (right):

http://codereview.chromium.org/8698018/diff/1/runtime/bin/input_stream.dart#n...
runtime/bin/input_stream.dart:10: * Reads data from the stream. If no encoding
is set fir the stream
fir -> for

http://codereview.chromium.org/8698018/diff/1/runtime/bin/input_stream.dart#n...
runtime/bin/input_stream.dart:38: Object readUntil(Object pattern);
Should pattern have type Pattern?

http://codereview.chromium.org/8698018/diff/1/runtime/bin/input_stream.dart#n...
runtime/bin/input_stream.dart:50: * Returns the number of bytes available for
immediate reading.
Hmm, bytes or characters depending on the encoding?

http://codereview.chromium.org/8698018/diff/1/runtime/bin/input_stream.dart#n...
runtime/bin/input_stream.dart:58: * the encodings "UTF-8", "ISO-8859-1" and
"ASCII" are supported.
Maybe there should always be an encoding? A default 'RAW' encoding that means
that you just want to read the bytes?

http://codereview.chromium.org/8698018/diff/1/runtime/bin/input_stream.dart#n...
runtime/bin/input_stream.dart:64: * Sets the minimum number of bytes required to
trigger a the
Again, is this bytes or characters depending on the encoding or is it just
bytes?

[Ivan Posva, 2011-12-01 18:23:52]

I fully agree with Mads that the lowest level input stream should only provide
bytes to make it easy on other sources of input to implement the bottleneck
interface and leave all the handling of encodings et al. to higher levels.
Otherwise I suspect there will develop a parallel interface that is simpler to
implement, which will only cause fragmentation.

-Ivan


On 2011/11/30 10:44:07, Mads Ager wrote:
> This gets pretty complicated.
> 
> Søren and I discussed this today and we would prefer to keep InputStream
really,
> really simple. The only thing that it can do is tell you when there are bytes
> ready to be read. On top of that you can build StringInputStreams and
> ChunkedInputStreams and any other type of interpretation of the raw bytes with
> only one level of buffering. Additionally it has the advantage that someone
can
> write an InputStream which is just a byte provider very quickly and get
> StringInputStream and ChunkedInputStreams for free.
> 
> The down side is that you have to do wrapping. However, I think that is worth
> the simplicity in the implementation and for the programmer wanting to provide
> an InputStream:
> 
> var input = new StringInputStream(process.stdout, 'UTF-8');
> input.lineHandler = () {
>   // Do stuff with the lines that you get from readLine.
>   // This handler is only called when a full line is available.
> }
> 
> var input = new ChunkedInputStream(process.stdout, 1024);
> input.chunkHandler = () {
>   // Do stuff with the chunks you get from readChunk.
>   // This handler is only called when a full chunk is available.
>   // You can change the chunk size with assignments: chunkSize = 512;
> }
> 
> http://codereview.chromium.org/8698018/diff/1/runtime/bin/input_stream.dart
> File runtime/bin/input_stream.dart (right):
> 
>
http://codereview.chromium.org/8698018/diff/1/runtime/bin/input_stream.dart#n...
> runtime/bin/input_stream.dart:10: * Reads data from the stream. If no encoding
> is set fir the stream
> fir -> for
> 
>
http://codereview.chromium.org/8698018/diff/1/runtime/bin/input_stream.dart#n...
> runtime/bin/input_stream.dart:38: Object readUntil(Object pattern);
> Should pattern have type Pattern?
> 
>
http://codereview.chromium.org/8698018/diff/1/runtime/bin/input_stream.dart#n...
> runtime/bin/input_stream.dart:50: * Returns the number of bytes available for
> immediate reading.
> Hmm, bytes or characters depending on the encoding?
> 
>
http://codereview.chromium.org/8698018/diff/1/runtime/bin/input_stream.dart#n...
> runtime/bin/input_stream.dart:58: * the encodings "UTF-8", "ISO-8859-1" and
> "ASCII" are supported.
> Maybe there should always be an encoding? A default 'RAW' encoding that means
> that you just want to read the bytes?
> 
>
http://codereview.chromium.org/8698018/diff/1/runtime/bin/input_stream.dart#n...
> runtime/bin/input_stream.dart:64: * Sets the minimum number of bytes required
to
> trigger a the
> Again, is this bytes or characters depending on the encoding or is it just
> bytes?