Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(23)

Issue 394313002: Add support for loading pak files from arbitrary file regions. (Closed)

Created:
6 years, 5 months ago by Primiano Tucci (use gerrit)
Modified:
6 years, 4 months ago
CC:
chromium-reviews, erikwright+watch_chromium.org, Torne, mkosiba (inactive)
Base URL:
https://chromium.googlesource.com/chromium/src.git@master
Project:
chromium
Visibility:
Public.

Description

Add support for loading pak files from arbitrary file regions. This is to support a new use case for Android: mmap a pak file directly from the APK (where it will be stored uncompressed) without extracting it first. This would save both precious space on the flash and startup time on the first run. This CL introduces: - the necessary changes to base::File to memory map arbitrary regions of a file. - The corresponding changes (plus unittests) in DataPack and ResourceBundle to take advantage of the new support. At present state, this CL is not intended to introduce any behavioral change. BUG=394502 Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=285945

Patch Set 1 #

Patch Set 2 : #

Patch Set 3 : Win implementation #

Total comments: 3

Patch Set 4 : Support arbitrary alignments #

Patch Set 5 : Refactor + fix win #

Patch Set 6 : Fix return value check + speculative fix on windows #

Patch Set 7 : Add base unittest and fix win #

Total comments: 15

Patch Set 8 : Address first willchan@ comments #

Total comments: 9

Patch Set 9 : remove IsWholeFile, use kWholeFile; other nits #

Total comments: 4

Patch Set 10 : Ah, remove <algorithm> #

Total comments: 6

Patch Set 11 : Fix test + DCHECK on mask #

Patch Set 12 : Move Region to MemoryMappedFile #

Total comments: 2
Unified diffs Side-by-side diffs Delta from patch set Stats (+403 lines, -24 lines) Patch
M base/base.gyp View 1 2 3 4 5 6 1 chunk +1 line, -0 lines 0 comments Download
M base/files/memory_mapped_file.h View 1 2 3 4 5 6 7 8 9 10 11 3 chunks +36 lines, -1 line 2 comments Download
M base/files/memory_mapped_file.cc View 1 2 3 4 5 6 7 8 9 10 11 4 chunks +37 lines, -2 lines 0 comments Download
M base/files/memory_mapped_file_posix.cc View 1 2 3 4 5 6 7 8 9 10 11 1 chunk +51 lines, -10 lines 0 comments Download
A base/files/memory_mapped_file_unittest.cc View 1 2 3 4 5 6 7 8 9 10 11 1 chunk +166 lines, -0 lines 0 comments Download
M base/files/memory_mapped_file_win.cc View 1 2 3 4 5 6 7 8 9 10 11 2 chunks +44 lines, -8 lines 0 comments Download
M ui/base/resource/data_pack.h View 1 2 3 4 5 6 7 8 9 10 11 3 chunks +5 lines, -1 line 0 comments Download
M ui/base/resource/data_pack.cc View 1 2 3 4 5 6 7 8 9 10 11 1 chunk +8 lines, -1 line 0 comments Download
M ui/base/resource/data_pack_unittest.cc View 1 2 3 4 5 6 7 8 9 10 11 1 chunk +40 lines, -0 lines 0 comments Download
M ui/base/resource/resource_bundle.h View 1 2 3 4 5 6 7 8 9 10 11 2 chunks +6 lines, -0 lines 0 comments Download
M ui/base/resource/resource_bundle.cc View 1 2 3 4 5 6 7 8 9 10 11 1 chunk +9 lines, -1 line 0 comments Download

Messages

Total messages: 28 (0 generated)
Primiano Tucci (use gerrit)
Nico, are you a good OWNER for this or do you prefer me to ping ...
6 years, 5 months ago (2014-07-16 23:14:39 UTC) #1
mkosiba (inactive)
lgtm but I'm not an owner :) https://codereview.chromium.org/394313002/diff/70001/base/files/memory_mapped_file_win.cc File base/files/memory_mapped_file_win.cc (right): https://codereview.chromium.org/394313002/diff/70001/base/files/memory_mapped_file_win.cc#newcode28 base/files/memory_mapped_file_win.cc:28: if (len ...
6 years, 5 months ago (2014-07-17 00:38:01 UTC) #2
Primiano Tucci (use gerrit)
https://codereview.chromium.org/394313002/diff/70001/base/files/memory_mapped_file_win.cc File base/files/memory_mapped_file_win.cc (right): https://codereview.chromium.org/394313002/diff/70001/base/files/memory_mapped_file_win.cc#newcode28 base/files/memory_mapped_file_win.cc:28: if (len <= 0 || len > kint32max) On ...
6 years, 5 months ago (2014-07-17 01:15:17 UTC) #3
Primiano Tucci (use gerrit)
willchan, tony, could you PTAL (respectively to base/ and ui/). Many thanks!
6 years, 5 months ago (2014-07-17 22:21:12 UTC) #4
tony
This may result in some unaligned reads from memory, but maybe that's OK? It sounds ...
6 years, 5 months ago (2014-07-17 22:55:26 UTC) #5
Primiano Tucci (use gerrit)
On 2014/07/17 22:55:26, tony wrote: > This may result in some unaligned reads from memory, ...
6 years, 5 months ago (2014-07-17 23:02:16 UTC) #6
willchan no longer on Chromium
I must apologize, I didn't get a chance to finish this review today. But I ...
6 years, 5 months ago (2014-07-21 23:55:40 UTC) #7
Primiano Tucci (use gerrit)
Many thanks for your quick response Will! Replies inline below, looking forward your next batch ...
6 years, 5 months ago (2014-07-22 10:32:01 UTC) #8
willchan no longer on Chromium
There are more places I didn't mention that unnecessarily do base::. Other than that, this ...
6 years, 5 months ago (2014-07-22 20:45:47 UTC) #9
Jeffrey Yasskin
https://codereview.chromium.org/394313002/diff/280001/base/files/file.h File base/files/file.h (right): https://codereview.chromium.org/394313002/diff/280001/base/files/file.h#newcode160 base/files/file.h:160: static Region WholeFile(); Maybe-dumb drive-by suggestion: Maybe make this ...
6 years, 5 months ago (2014-07-22 21:22:29 UTC) #10
Primiano Tucci (use gerrit)
Sorry for the delay, busy day. Will / Jeffrey, I hope I covered all the ...
6 years, 5 months ago (2014-07-23 23:17:39 UTC) #11
willchan no longer on Chromium
https://codereview.chromium.org/394313002/diff/260001/base/files/file.h File base/files/file.h (right): https://codereview.chromium.org/394313002/diff/260001/base/files/file.h#newcode159 base/files/file.h:159: struct BASE_EXPORT Region { On 2014/07/23 23:17:38, Primiano Tucci ...
6 years, 4 months ago (2014-07-25 21:21:24 UTC) #12
tony
On 2014/07/25 21:21:24, willchan wrote: > https://codereview.chromium.org/394313002/diff/260001/base/files/file.h > File base/files/file.h (right): > > https://codereview.chromium.org/394313002/diff/260001/base/files/file.h#newcode159 > ...
6 years, 4 months ago (2014-07-26 16:44:58 UTC) #13
Primiano Tucci (use gerrit)
https://codereview.chromium.org/394313002/diff/260001/base/files/file.h File base/files/file.h (right): https://codereview.chromium.org/394313002/diff/260001/base/files/file.h#newcode159 base/files/file.h:159: struct BASE_EXPORT Region { On 2014/07/25 21:21:23, willchan wrote: ...
6 years, 4 months ago (2014-07-28 12:52:11 UTC) #14
Primiano Tucci (use gerrit)
Ok, as promised I created the variant of this CL in https://codereview.chromium.org/426553005/. I organized the ...
6 years, 4 months ago (2014-07-28 14:01:22 UTC) #15
willchan no longer on Chromium
https://codereview.chromium.org/394313002/diff/260001/base/files/file.h File base/files/file.h (right): https://codereview.chromium.org/394313002/diff/260001/base/files/file.h#newcode159 base/files/file.h:159: struct BASE_EXPORT Region { On 2014/07/28 12:52:11, Primiano Tucci ...
6 years, 4 months ago (2014-07-28 14:52:17 UTC) #16
Primiano Tucci (use gerrit)
Got it. Thanks for the quick response! By LGTemming the other CL I assume you ...
6 years, 4 months ago (2014-07-28 15:08:30 UTC) #17
willchan no longer on Chromium
LGTM On Mon, Jul 28, 2014 at 8:08 AM, <primiano@chromium.org> wrote: > Got it. Thanks ...
6 years, 4 months ago (2014-07-28 15:11:31 UTC) #18
willchan no longer on Chromium
lgtm
6 years, 4 months ago (2014-07-28 15:12:52 UTC) #19
Primiano Tucci (use gerrit)
The CQ bit was checked by primiano@chromium.org
6 years, 4 months ago (2014-07-28 15:13:24 UTC) #20
commit-bot: I haz the power
CQ is trying da patch. Follow status at https://chromium-status.appspot.com/cq/primiano@chromium.org/394313002/370014
6 years, 4 months ago (2014-07-28 15:14:29 UTC) #21
commit-bot: I haz the power
Change committed as 285945
6 years, 4 months ago (2014-07-28 17:57:47 UTC) #22
sky
https://codereview.chromium.org/394313002/diff/370014/base/files/memory_mapped_file.h File base/files/memory_mapped_file.h (right): https://codereview.chromium.org/394313002/diff/370014/base/files/memory_mapped_file.h#newcode29 base/files/memory_mapped_file.h:29: static const Region kWholeFile; Doesn't this result in a ...
6 years, 4 months ago (2014-07-28 22:07:43 UTC) #23
Primiano Tucci (use gerrit)
https://codereview.chromium.org/394313002/diff/370014/base/files/memory_mapped_file.h File base/files/memory_mapped_file.h (right): https://codereview.chromium.org/394313002/diff/370014/base/files/memory_mapped_file.h#newcode29 base/files/memory_mapped_file.h:29: static const Region kWholeFile; On 2014/07/28 22:07:43, sky wrote: ...
6 years, 4 months ago (2014-07-29 09:05:49 UTC) #24
willchan no longer on Chromium
Yeah, I was assuming our static initializer tests would barf if Primiano got it wrong ...
6 years, 4 months ago (2014-07-29 15:17:35 UTC) #25
sky
On 2014/07/29 15:17:35, willchan wrote: > Yeah, I was assuming our static initializer tests would ...
6 years, 4 months ago (2014-07-29 17:52:54 UTC) #26
Nico
It should be fine in practice, but we use base::LINKER_INITIALIZED in other places where we ...
6 years, 4 months ago (2014-07-29 18:12:07 UTC) #27
Primiano Tucci (use gerrit)
6 years, 4 months ago (2014-07-29 19:00:19 UTC) #28
Message was sent while issue was closed.
On 2014/07/29 18:12:07, Nico (away) wrote:
> It should be fine in practice, but we use base::LINKER_INITIALIZED in other
> places where we do this for documentation purposes, so you might want to
> use that here too.
> 
> 
> On Tue, Jul 29, 2014 at 10:52 AM, <mailto:sky@chromium.org> wrote:
> 
> > On 2014/07/29 15:17:35, willchan wrote:
> >
> >> Yeah, I was assuming our static initializer tests would barf if Primiano
> >> got it wrong :) I agree, however we do it, adding a static initializer is
> >> unacceptable. I think it's fine, but if it does add a static initializer
> >> somewhere, let's fix that.
> >>
> >
> >
> >  On Tue, Jul 29, 2014 at 2:05 AM, <mailto:primiano@chromium.org> wrote:
> >>
> >
> >  >
> >> > https://codereview.chromium.org/394313002/diff/370014/
> >> > base/files/memory_mapped_file.h
> >> > File base/files/memory_mapped_file.h (right):
> >> >
> >> > https://codereview.chromium.org/394313002/diff/370014/
> >> > base/files/memory_mapped_file.h#newcode29
> >> > base/files/memory_mapped_file.h:29: static const Region kWholeFile;
> >> > On 2014/07/28 22:07:43, sky wrote:
> >> >
> >> >> Doesn't this result in a static constructor, which is disallowed?
> >> >>
> >> > We had an offline discussion with Will on this point (I had the same
> >> > doubt), and my understanding is:  if the only thing the ctor does is
> >> > initializing POD fields (in this specific case, to 0), it would not be a
> >> > problem. Essentially here kWholeFile is a struct with two 0-initialized
> >> > fields. I'd expect kWholeFile to be just a carve-out of .bss.
> >> >
> >> > I did a quick test by turning kWholeFile into a static method which
> >> > returns a new Region every time, and if I diff the program headers, I
> >> > can see a 16 bytes delta in bss (for libbase.so):
> >> >
> >> > With kWholeFile being a static const:
> >> > [27] .bss              NOBITS          00000000003e3660 3e2660 002714 00
> >> >  WA  0   0 16
> >> >
> >> > With kWholeFile being a method which retuns a new object
> >> > [27] .bss              NOBITS          00000000003e3660 3e2660 002704 00
> >> >  WA  0   0 16
> >> >
> >> > (Size column is the 002714, 002704)
> >> > Also, I assume we have checks in place for this (I am sure I've seen
> >> > clang barfing for at-exit destructors).
> >> >
> >> > Does it sound right?
> >> >
> >> > https://codereview.chromium.org/394313002/
> >> >
> >>
> >
> >  To unsubscribe from this group and stop receiving emails from it, send an
> >>
> > email
> >
> >> to mailto:chromium-reviews+unsubscribe@chromium.org.
> >>
> >
> > I thought the right thing here was to have no constructor. That way no
> > static
> > initialize. I'm a bit hazy on this though. Nico knows this more than I
> > though,
> > so +nico.
> >
> > https://codereview.chromium.org/394313002/
> >
> 
> To unsubscribe from this group and stop receiving emails from it, send an
email
> to mailto:chromium-reviews+unsubscribe@chromium.org.

FTR I checked with  tools/linux/dump-static-initializers.py and I confirm that
the Release binary (I tried just doing a component build of libbase.so) doesn't
have any static initializer.
The debug version, instead, shows it. I think that's because the compiler
doesn't try to inline the empty body of the ctor and doesn't realize that there
isn't anything to call.

Anyways, I'm adding the LINKER_INITIALIZED as Nico suggested here:
https://codereview.chromium.org/428883012/

Powered by Google App Engine
This is Rietveld 408576698