Support files larger than 2GB

Define _FILE_OFFSET_BITS=64 in order to support large files (>2GB).
_FILE_OFFSET_BITS=64 is incompatible with fts (issue 17492), so file_util::Delete, file_util::CopyDirectory  file_util::FileEnumerator are reimplemented without fts. Delete and CopyDirectory are now implemented using FileEnumerator.

BUG=13718, 17492
TEST=none

committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=22670

[vandebo.gmail, 2009-07-31 21:34:32]

A small code review to allow downloads of >2G files on Linux.

[Evan Martin, 2009-07-31 21:37:49]

I think agl needs to review this.  He looked into solving this before and didn't
go this route for some reason.

[Lei Zhang, 2009-07-31 21:38:14]

On 2009/07/31 21:34:32, vandebo wrote:
> A small code review to allow downloads of >2G files on Linux.

The mac trybot looks unhappy:
/b/slave/mac/build/src/base/file_util_posix.cc:446: error: 'mkstemp64' was not
declared in this scope
...

[vandebo.gmail, 2009-07-31 21:51:23]

That may be why agl didn't go this way... It seems (and I was unaware) that mac
doesn't support the foo64 variants.

On 2009/07/31 21:38:14, Lei Zhang wrote:
> On 2009/07/31 21:34:32, vandebo wrote:
> > A small code review to allow downloads of >2G files on Linux.
> 
> The mac trybot looks unhappy:
> /b/slave/mac/build/src/base/file_util_posix.cc:446: error: 'mkstemp64' was not
> declared in this scope
> ...

[vandebo.gmail, 2009-08-06 04:13:26]

This change discontinues the use of FTS in order to globally enable large file
support.

[agl, 2009-08-06 17:34:57]

Seem comments below, also make sure that it passes the try bots first.
Generally, LGTM.

http://codereview.chromium.org/160479/diff/2012/3011
File base/file_util_posix.cc (right):

http://codereview.chromium.org/160479/diff/2012/3011#newcode178
Line 178: current = traversal.Next()) {
one more space of indent needed

http://codereview.chromium.org/160479/diff/2012/3011#newcode238
Line 238: if (real_to_path.value().compare(0, from_path.value().size(),
You should probably check that real_to_path.size() >= from_path first. Otherwise
you're taking an invalid substring and the STL docs don't say if that's valid or
not.

http://codereview.chromium.org/160479/diff/2012/3011#newcode635
Line 635: while (readdir_r(dir, &dent_buf, &dent) == 0 && dent) {
Since this is a POSIX file, not a Linux file, we should be portable:

  len = offsetof(struct dirent, d_name) +
                 pathconf(dirpath, _PC_NAME_MAX) + 1
  dent_buf = (struct dirent*) malloc(len);

[vandebo.gmail, 2009-08-06 20:03:46]

Updated version passes try servers.

http://codereview.chromium.org/160479/diff/2012/3011
File base/file_util_posix.cc (right):

http://codereview.chromium.org/160479/diff/2012/3011#newcode178
Line 178: current = traversal.Next()) {
On 2009/08/06 17:34:57, agl wrote:
> one more space of indent needed

Done.

http://codereview.chromium.org/160479/diff/2012/3011#newcode238
Line 238: if (real_to_path.value().compare(0, from_path.value().size(),
On 2009/08/06 17:34:57, agl wrote:
> You should probably check that real_to_path.size() >= from_path first.
Otherwise
> you're taking an invalid substring and the STL docs don't say if that's valid
or
> not.

Done.
I also realized that this test must canonicalize the destination path as well,
done.

http://codereview.chromium.org/160479/diff/2012/3011#newcode635
Line 635: while (readdir_r(dir, &dent_buf, &dent) == 0 && dent) {
On 2009/08/06 17:34:57, agl wrote:
> Since this is a POSIX file, not a Linux file, we should be portable:
> 
>   len = offsetof(struct dirent, d_name) +
>                  pathconf(dirpath, _PC_NAME_MAX) + 1
>   dent_buf = (struct dirent*) malloc(len);

Both Linux and Mac define struct dirent with enough space for the name.  This
file already has a use of readdir_r done in the way I did it.  To make it
portable, how about:
#if !defined(OS_LINUX) && !defined(OS_MACOSX)
  #error Depending on the definition of struct dirent, additional space for \
      pathname may be needed
#endif

[agl, 2009-08-06 20:04:53]

LGTM

http://codereview.chromium.org/160479/diff/2012/3011
File base/file_util_posix.cc (right):

http://codereview.chromium.org/160479/diff/2012/3011#newcode635
Line 635: while (readdir_r(dir, &dent_buf, &dent) == 0 && dent) {
On 2009/08/06 20:03:46, vandebo wrote:
> portable, how about:

Sure thing.