create anyfile_utils.py for copying files between HTTP/GS/local filesystem

create anyfile_utils.py for copying files between HTTP/GS/local filesystem

COMMIT=false

[epoger, 2014-08-04 13:00:44]

This is not yet ready for commit, more of an exploratory thing...

As I have been working on https://codereview.chromium.org/424263005/ ('teach
rebaseline_server to generate diffs of rendered SKPs'), I found myself wishing
we had a library to hide the differences between copying files from GS, HTTP, or
local filesystem.

Do you guys think a layer like this would be useful?

If so, give me any further tweaking advice, and I'll clean this up for real
review...

[epoger, 2014-08-05 13:43:44]

On 2014/08/04 13:00:44, epoger wrote:
> Do you guys think a layer like this would be useful?

Bueller?

[epoger, 2014-08-05 14:22:41]

On 2014/08/05 13:43:44, epoger wrote:
> On 2014/08/04 13:00:44, epoger wrote:
> > Do you guys think a layer like this would be useful?
> 
> Bueller?

I think this is related to discussions Eric and I have been having about merging
our gs_utils wrapper into recipes (or not).

[borenet, 2014-08-05 14:30:06]

On 2014/08/05 14:22:41, epoger wrote:
> On 2014/08/05 13:43:44, epoger wrote:
> > On 2014/08/04 13:00:44, epoger wrote:
> > > Do you guys think a layer like this would be useful?
> > 
> > Bueller?
> 
> I think this is related to discussions Eric and I have been having about
merging
> our gs_utils wrapper into recipes (or not).

Yeah, I think this would be useful (assuming that it can be used in our future
buildbot code in some way).  At some point I think I did some exploration of
this type of thing, where each class of file had a CopyTo and CopyFrom method
and used local temporary files as an intermediate stage.  There's a copy
function like this:

def copy(src, dst):
  tmp_file = make_tmp_file()
  src.CopyTo(tmp_file)
  dst.CopyFrom(tmp_file)

The cost, of course, is that you have to use the temporary file.  You might be
able to do that in memory though.

[epoger, 2014-08-05 21:38:44]

Abandoning this CL, at least for now, because it will likely be wasted work
depending on how we access Google Storage in a recipes world.