raw_io module: Add text_input and text_output

raw_io module: Add text_input and text_output

This CLs adds text_input and text_output to the raw io module. These are the
same as input and output, except that they assume the data they handle is
utf-8 encoded, and will decode or encode it as appropriate. If any errors are
encountered while doing this, they will be replaced with an appropriate
replacement character, as defined in https://docs.python.org/2/library/codecs.html#codec-base-classes

BUG=686280, 664352
Review-Url: https://codereview.chromium.org/2672593002
Committed: https://github.com/luci/recipes-py/commit/9456147555cc76adec014709e02928b620d7d8d9

[martiniss, 2017-02-01 23:03:46]

Description was changed from

==========
raw_io module: Try to decode before encoding

Manually try to decode data we are trying to write out to disk. We
assume users want to write out utf-8 to disk.

BUG=686280
==========

to

==========
raw_io module: Try to decode before encoding

Manually try to decode data we are trying to write out to disk. We
assume users want to write out utf-8 to disk.

BUG=686280,664352
==========

[martiniss, 2017-02-01 23:03:46]

martiniss@chromium.org changed reviewers:
+ dnj@chromium.org, iannucci@chromium.org

[martiniss, 2017-02-01 23:04:04]

PTAL

It does require valid utf-8...

[martiniss, 2017-02-01 23:04:48]

https://codereview.chromium.org/2106333004 was the change to force calling
encode, but there's no bug number on it :(

[dnj, 2017-02-02 00:38:41]

lgtm in spirit. WDYT about raising an exception if it's not UTF-8? I don't like
this silent response, and if the stream is off by a byte or something, the rest
of the data could be junk anyway.

[iannucci, 2017-02-02 01:45:39]

Uh... this looks really wrong to me. raw_io should be dealing with unencoded
bytes. If people want utf8_input or something, we should add another placehoder
for that instead.

[martiniss, 2017-02-02 02:16:57]

Description was changed from

==========
raw_io module: Try to decode before encoding

Manually try to decode data we are trying to write out to disk. We
assume users want to write out utf-8 to disk.

BUG=686280,664352
==========

to

==========
raw_io module: Add text_input and text_output

BUG=686280,664352
==========

[iannucci, 2017-02-02 02:23:30]

this approach lgtm

https://codereview.chromium.org/2672593002/diff/20001/recipe_modules/raw_io/a...
File recipe_modules/raw_io/api.py (right):

https://codereview.chromium.org/2672593002/diff/20001/recipe_modules/raw_io/a...
recipe_modules/raw_io/api.py:161: def input_text(data, suffix=''):
docstring (and for input above)

https://codereview.chromium.org/2672593002/diff/20001/recipe_modules/raw_io/a...
recipe_modules/raw_io/api.py:184: """Returns a Placeholder for use as a step
argument, or for std{out,err}.
document encoding difference here and text_input.

[martiniss, 2017-02-06 20:01:58]

Description was changed from

==========
raw_io module: Add text_input and text_output

BUG=686280,664352
==========

to

==========
raw_io module: Add text_input and text_output

This CLs adds text_input and text_output to the raw io module. These are the
same as input and output, except that they assume the data they handle is
utf-8 encoded, and will decode or encode it as appropriate. If any errors are
encountered while doing this, they will be replaced with an appropriate
replacement character, as defined in
https://docs.python.org/2/library/codecs.html#codec-base-classes

BUG=686280,664352
==========

[martiniss, 2017-02-06 20:04:52]

PTAL again.

[martiniss, 2017-02-06 21:20:12]

On 2017/02/06 at 20:04:52, martiniss wrote:
> PTAL again.

Need to add more tests, but api.py shouldn't need to change.

[martiniss, 2017-02-07 19:22:31]

On 2017/02/06 at 21:20:12, martiniss wrote:
> On 2017/02/06 at 20:04:52, martiniss wrote:
> > PTAL again.
> 
> Need to add more tests, but api.py shouldn't need to change.

Ready for review

[iannucci, 2017-02-07 22:18:24]

lgtm

https://codereview.chromium.org/2672593002/diff/80001/recipe_modules/raw_io/a...
File recipe_modules/raw_io/api.py (right):

https://codereview.chromium.org/2672593002/diff/80001/recipe_modules/raw_io/a...
recipe_modules/raw_io/api.py:47: def encode(self, data): # pragma: no cover
why uncovered?

https://codereview.chromium.org/2672593002/diff/80001/recipe_modules/raw_io/a...
recipe_modules/raw_io/api.py:105: def decode(self, result): # pragma: no cover
why uncovered?

[martiniss, 2017-02-08 00:25:55]

The CQ bit was checked by martiniss@chromium.org

[martiniss, 2017-02-08 00:25:56]

https://codereview.chromium.org/2672593002/diff/80001/recipe_modules/raw_io/a...
File recipe_modules/raw_io/api.py (right):

https://codereview.chromium.org/2672593002/diff/80001/recipe_modules/raw_io/a...
recipe_modules/raw_io/api.py:47: def encode(self, data): # pragma: no cover
On 2017/02/07 at 22:18:23, iannucci wrote:
> why uncovered?

They are called from uncovered code; line 32. I'll add a comment, since it isn't
obvious.

[martiniss, 2017-02-08 00:25:56]

The patchset sent to the CQ was uploaded after l-g-t-m from dnj@chromium.org,
iannucci@chromium.org
Link to the patchset: https://codereview.chromium.org/2672593002/#ps100001
(title: "Add comment about no cover")

[commit-bot: I haz the power, 2017-02-08 00:26:02]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-02-08 00:28:57]

CQ is committing da patch.

Bot data: {"patchset_id": 100001, "attempt_start_ts": 1486513555755760,
"parent_rev": "5d4666c5340082e7bfad85ee01d47ebbc48aeab6", "commit_rev":
"9456147555cc76adec014709e02928b620d7d8d9"}

[commit-bot: I haz the power, 2017-02-08 00:29:02]

Description was changed from

==========
raw_io module: Add text_input and text_output

This CLs adds text_input and text_output to the raw io module. These are the
same as input and output, except that they assume the data they handle is
utf-8 encoded, and will decode or encode it as appropriate. If any errors are
encountered while doing this, they will be replaced with an appropriate
replacement character, as defined in
https://docs.python.org/2/library/codecs.html#codec-base-classes

BUG=686280,664352
==========

to

==========
raw_io module: Add text_input and text_output

This CLs adds text_input and text_output to the raw io module. These are the
same as input and output, except that they assume the data they handle is
utf-8 encoded, and will decode or encode it as appropriate. If any errors are
encountered while doing this, they will be replaced with an appropriate
replacement character, as defined in
https://docs.python.org/2/library/codecs.html#codec-base-classes

BUG=686280,664352

Review-Url: https://codereview.chromium.org/2672593002
Committed:
https://github.com/luci/recipes-py/commit/9456147555cc76adec014709e02928b620d...
==========

[commit-bot: I haz the power, 2017-02-08 00:29:03]

Committed patchset #6 (id:100001) as
https://github.com/luci/recipes-py/commit/9456147555cc76adec014709e02928b620d...