Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(479)

Issue 1968993002: Compressing .pak resources with new option: "type=GZIPPABLE_BINDATA" (Closed)

Created:
4 years, 7 months ago by smaier
Modified:
4 years, 6 months ago
CC:
arv+watch_chromium.org, chromium-reviews, darin-cc_chromium.org, jam
Base URL:
https://chromium.googlesource.com/chromium/src.git@master
Target Ref:
refs/pending/heads/master
Project:
chromium
Visibility:
Public.

Description

Compress .pak resources with new option: "type=GZIPPABLE_BINDATA" Allows compression via gzip for rarely used .pak resources. These resources will only be uncompressed into memory upon request of the resource. We are compressing all "internals" pages, with a savings of 472KB. TBR=sdefresne BUG=609219 CQ_INCLUDE_TRYBOTS=tryserver.chromium.linux:closure_compilation Committed: https://crrev.com/e3a06a21aa1a7adfaef988488cdbcccbfa9a2ef6 Cr-Commit-Position: refs/heads/master@{#397168}

Patch Set 1 #

Patch Set 2 : Fixed DEPS files #

Total comments: 15

Patch Set 3 : Reformatted based on review suggestions #

Total comments: 11

Patch Set 4 : Gzip string added as new file #

Patch Set 5 : Updated using new ResourceBundle #

Patch Set 6 : Added CHECK to ensure callers are not getting compressed resource #

Patch Set 7 : Changing internals pages over to be compressed #

Patch Set 8 : Updating comments #

Total comments: 6

Patch Set 9 : Addressing agrieve's comments #

Total comments: 11

Patch Set 10 : Addressed flackr's comments #

Total comments: 15

Patch Set 11 : Addressing sky's comments #

Patch Set 12 : Fixing copy on vector to swap #

Total comments: 2

Patch Set 13 : Fixing some of the try bots #

Total comments: 9

Patch Set 14 : Changes suggested from review #

Total comments: 4

Patch Set 15 : Fixed Python formatting #

Unified diffs Side-by-side diffs Delta from patch set Stats (+368 lines, -75 lines) Patch
M chrome/browser/browser_resources.grd View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 2 chunks +12 lines, -12 lines 0 comments Download
M chrome/browser/resources/net_internals_resources.grd View 1 2 3 4 1 chunk +2 lines, -2 lines 0 comments Download
M chrome/browser/resources/password_manager_internals_resources.grd View 1 2 3 4 5 6 1 chunk +3 lines, -3 lines 0 comments Download
M chrome/browser/resources/translate_internals_resources.grd View 1 2 3 4 5 6 1 chunk +2 lines, -2 lines 0 comments Download
M chrome/test/data/webui/webui_resource_browsertest.cc View 1 2 3 4 5 6 7 8 9 10 11 12 1 chunk +4 lines, -2 lines 0 comments Download
M components/resources/gcm_driver_resources.grdp View 1 2 3 4 5 6 1 chunk +3 lines, -3 lines 0 comments Download
M components/resources/signin_resources.grdp View 1 2 3 4 5 6 1 chunk +3 lines, -2 lines 0 comments Download
M content/content_resources.grd View 1 2 3 4 5 6 2 chunks +15 lines, -15 lines 0 comments Download
M content/public/test/browser_test_utils.cc View 1 2 3 4 5 6 7 8 9 10 11 12 1 chunk +3 lines, -2 lines 0 comments Download
M tools/grit/grit/exception.py View 1 2 3 4 5 6 7 8 9 2 chunks +4 lines, -1 line 0 comments Download
M tools/grit/grit/format/data_pack_unittest.py View 1 2 3 4 5 6 7 8 9 2 chunks +34 lines, -0 lines 0 comments Download
A tools/grit/grit/format/gzip_string.py View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 1 chunk +43 lines, -0 lines 0 comments Download
A tools/grit/grit/format/gzip_string_unittest.py View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 1 chunk +62 lines, -0 lines 0 comments Download
M tools/grit/grit/node/include.py View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 4 chunks +26 lines, -4 lines 0 comments Download
M tools/grit/grit/node/include_unittest.py View 1 2 3 4 5 6 7 8 9 10 11 12 13 14 2 chunks +16 lines, -1 line 0 comments Download
M tools/grit/grit/test_suite_all.py View 1 2 3 4 5 6 7 2 chunks +2 lines, -0 lines 0 comments Download
A tools/grit/grit/testdata/test_text.txt View 1 2 3 4 5 6 7 8 9 1 chunk +1 line, -0 lines 0 comments Download
M ui/base/BUILD.gn View 1 2 3 4 1 chunk +1 line, -0 lines 0 comments Download
M ui/base/DEPS View 1 2 3 4 1 chunk +1 line, -0 lines 0 comments Download
M ui/base/resource/resource_bundle.h View 1 2 3 4 5 6 7 8 9 10 3 chunks +12 lines, -1 line 0 comments Download
M ui/base/resource/resource_bundle.cc View 1 2 3 4 5 6 7 8 9 10 11 5 chunks +79 lines, -25 lines 0 comments Download
M ui/base/resource/resource_bundle_unittest.cc View 1 2 3 4 5 6 7 8 9 10 11 12 13 2 chunks +39 lines, -0 lines 0 comments Download
M ui/base/ui_base.gyp View 1 2 3 4 5 6 7 8 9 10 11 12 1 chunk +1 line, -0 lines 0 comments Download

Messages

Total messages: 53 (19 generated)
smaier
flackr@chromium.org: Please review changes in tools/grit/grit/node/include.py estade@chromium.org: Please review changes in content/browser/webui, chrome/browser/resources/net_internals_resources.grd
4 years, 7 months ago (2016-05-11 14:55:34 UTC) #3
agrieve
https://codereview.chromium.org/1968993002/diff/20001/content/browser/webui/web_ui_data_source_impl.cc File content/browser/webui/web_ui_data_source_impl.cc (right): https://codereview.chromium.org/1968993002/diff/20001/content/browser/webui/web_ui_data_source_impl.cc#newcode245 content/browser/webui/web_ui_data_source_impl.cc:245: if (response_array[0] == 0x1f && response_array[1] == 0x8b) { ...
4 years, 7 months ago (2016-05-11 18:29:52 UTC) #6
flackr
https://codereview.chromium.org/1968993002/diff/20001/tools/grit/grit/node/include.py File tools/grit/grit/node/include.py (right): https://codereview.chromium.org/1968993002/diff/20001/tools/grit/grit/node/include.py#newcode113 tools/grit/grit/node/include.py:113: raise On 2016/05/11 at 18:29:52, agrieve wrote: > nit: ...
4 years, 7 months ago (2016-05-11 18:54:33 UTC) #7
smaier
https://codereview.chromium.org/1968993002/diff/20001/content/browser/webui/web_ui_data_source_impl.cc File content/browser/webui/web_ui_data_source_impl.cc (right): https://codereview.chromium.org/1968993002/diff/20001/content/browser/webui/web_ui_data_source_impl.cc#newcode245 content/browser/webui/web_ui_data_source_impl.cc:245: if (response_array[0] == 0x1f && response_array[1] == 0x8b) { ...
4 years, 7 months ago (2016-05-13 15:27:25 UTC) #8
flackr
https://codereview.chromium.org/1968993002/diff/40001/content/browser/webui/web_ui_data_source_impl.cc File content/browser/webui/web_ui_data_source_impl.cc (right): https://codereview.chromium.org/1968993002/diff/40001/content/browser/webui/web_ui_data_source_impl.cc#newcode275 content/browser/webui/web_ui_data_source_impl.cc:275: if (response_array[0] == 0x1f && response_array[1] == 0x8b) { ...
4 years, 7 months ago (2016-05-13 23:02:17 UTC) #9
smaier
https://codereview.chromium.org/1968993002/diff/40001/tools/grit/grit/node/include.py File tools/grit/grit/node/include.py (right): https://codereview.chromium.org/1968993002/diff/40001/tools/grit/grit/node/include.py#newcode44 tools/grit/grit/node/include.py:44: def _GzipString(self, data): On 2016/05/13 23:02:17, flackr wrote: > ...
4 years, 7 months ago (2016-05-19 14:17:57 UTC) #10
agrieve
lgtm https://codereview.chromium.org/1968993002/diff/140001/tools/grit/grit/format/gzip_string.py File tools/grit/grit/format/gzip_string.py (right): https://codereview.chromium.org/1968993002/diff/140001/tools/grit/grit/format/gzip_string.py#newcode7 tools/grit/grit/format/gzip_string.py:7: import StringIO nit: cStringIO is probably faster. https://codereview.chromium.org/1968993002/diff/140001/ui/base/resource/resource_bundle.cc ...
4 years, 6 months ago (2016-05-27 17:40:00 UTC) #13
smaier
https://codereview.chromium.org/1968993002/diff/140001/tools/grit/grit/format/gzip_string.py File tools/grit/grit/format/gzip_string.py (right): https://codereview.chromium.org/1968993002/diff/140001/tools/grit/grit/format/gzip_string.py#newcode7 tools/grit/grit/format/gzip_string.py:7: import StringIO On 2016/05/27 17:40:00, agrieve wrote: > nit: ...
4 years, 6 months ago (2016-05-27 17:48:47 UTC) #14
smaier
sdefresne@chromium.org: Please review changes in components/ sievers@chromium.org: Please review changes in content/ sky@chromium.org: Please review ...
4 years, 6 months ago (2016-05-27 17:51:54 UTC) #16
flackr
https://codereview.chromium.org/1968993002/diff/160001/tools/grit/grit/node/include.py File tools/grit/grit/node/include.py (right): https://codereview.chromium.org/1968993002/diff/160001/tools/grit/grit/node/include.py#newcode53 tools/grit/grit/node/include.py:53: 'compress': 'false', We should still have a test of ...
4 years, 6 months ago (2016-05-27 19:42:15 UTC) #17
smaier
https://codereview.chromium.org/1968993002/diff/160001/tools/grit/grit/node/include.py File tools/grit/grit/node/include.py (right): https://codereview.chromium.org/1968993002/diff/160001/tools/grit/grit/node/include.py#newcode53 tools/grit/grit/node/include.py:53: 'compress': 'false', On 2016/05/27 19:42:14, flackr wrote: > We ...
4 years, 6 months ago (2016-05-27 21:20:27 UTC) #18
sky
https://codereview.chromium.org/1968993002/diff/180001/ui/base/resource/resource_bundle.cc File ui/base/resource/resource_bundle.cc (right): https://codereview.chromium.org/1968993002/diff/180001/ui/base/resource/resource_bundle.cc#newcode118 ui/base/resource/resource_bundle.cc:118: base::RefCountedBytes* DecodeGzipData( I'm not familiar with zlib. Is another ...
4 years, 6 months ago (2016-05-27 22:14:36 UTC) #19
no sievers
content lgtm
4 years, 6 months ago (2016-05-27 22:40:37 UTC) #20
smaier
agl - Please review changes in ui/base/resource/resource_bundle.cc for zlib check https://codereview.chromium.org/1968993002/diff/180001/ui/base/resource/resource_bundle.cc File ui/base/resource/resource_bundle.cc (right): https://codereview.chromium.org/1968993002/diff/180001/ui/base/resource/resource_bundle.cc#newcode118 ...
4 years, 6 months ago (2016-05-30 14:10:04 UTC) #22
sdefresne
On 2016/05/30 14:10:04, smaier wrote: > agl - Please review changes in ui/base/resource/resource_bundle.cc for zlib ...
4 years, 6 months ago (2016-05-30 14:18:09 UTC) #23
smaier
On 2016/05/30 14:18:09, sdefresne wrote: > On 2016/05/30 14:10:04, smaier wrote: > > agl - ...
4 years, 6 months ago (2016-05-30 14:23:23 UTC) #24
smaier
https://codereview.chromium.org/1968993002/diff/180001/ui/base/resource/resource_bundle.cc File ui/base/resource/resource_bundle.cc (right): https://codereview.chromium.org/1968993002/diff/180001/ui/base/resource/resource_bundle.cc#newcode141 ui/base/resource/resource_bundle.cc:141: return new base::RefCountedBytes(output); On 2016/05/30 14:10:03, smaier wrote: > ...
4 years, 6 months ago (2016-05-30 15:45:16 UTC) #25
sdefresne
components l-g-t-m but why don't I see any try bot results?
4 years, 6 months ago (2016-05-30 17:02:03 UTC) #26
commit-bot: I haz the power
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/1968993002/220001 View timeline at https://chromium-cq-status.appspot.com/patch-timeline/1968993002/220001
4 years, 6 months ago (2016-05-30 17:14:26 UTC) #28
commit-bot: I haz the power
Dry run: Try jobs failed on following builders: mac_chromium_compile_dbg_ng on tryserver.chromium.mac (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.mac/builders/mac_chromium_compile_dbg_ng/builds/211245)
4 years, 6 months ago (2016-05-30 17:35:02 UTC) #30
flackr
https://codereview.chromium.org/1968993002/diff/160001/tools/grit/grit/node/include.py File tools/grit/grit/node/include.py (right): https://codereview.chromium.org/1968993002/diff/160001/tools/grit/grit/node/include.py#newcode97 tools/grit/grit/node/include.py:97: if 'compress' in self.attrs and self.attrs['compress'] == 'gzip': On ...
4 years, 6 months ago (2016-05-30 18:55:33 UTC) #31
commit-bot: I haz the power
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/1968993002/240001 View timeline at https://chromium-cq-status.appspot.com/patch-timeline/1968993002/240001
4 years, 6 months ago (2016-05-30 20:05:05 UTC) #33
flackr
https://codereview.chromium.org/1968993002/diff/160001/ui/base/resource/resource_bundle.cc File ui/base/resource/resource_bundle.cc (right): https://codereview.chromium.org/1968993002/diff/160001/ui/base/resource/resource_bundle.cc#newcode141 ui/base/resource/resource_bundle.cc:141: return new base::RefCountedBytes(output); On 2016/05/30 at 18:55:33, flackr wrote: ...
4 years, 6 months ago (2016-05-30 20:23:48 UTC) #34
commit-bot: I haz the power
Dry run: This issue passed the CQ dry run.
4 years, 6 months ago (2016-05-30 21:10:26 UTC) #36
sky
LGTM https://codereview.chromium.org/1968993002/diff/240001/ui/base/resource/resource_bundle_unittest.cc File ui/base/resource/resource_bundle_unittest.cc (right): https://codereview.chromium.org/1968993002/diff/240001/ui/base/resource/resource_bundle_unittest.cc#newcode279 ui/base/resource/resource_bundle_unittest.cc:279: char compressedEntryPakContents[] = { kCompressed... https://codereview.chromium.org/1968993002/diff/240001/ui/base/resource/resource_bundle_unittest.cc#newcode294 ui/base/resource/resource_bundle_unittest.cc:294: size_t ...
4 years, 6 months ago (2016-05-31 18:09:15 UTC) #37
agl
LGTM assuming that the source data is fully trusted. If not, see comments. https://codereview.chromium.org/1968993002/diff/240001/ui/base/resource/resource_bundle.cc File ...
4 years, 6 months ago (2016-05-31 18:18:06 UTC) #38
commit-bot: I haz the power
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/1968993002/260001 View timeline at https://chromium-cq-status.appspot.com/patch-timeline/1968993002/260001
4 years, 6 months ago (2016-05-31 21:40:45 UTC) #40
smaier
@agl - this function runs only on resources inserted by our build process. Therefore, we ...
4 years, 6 months ago (2016-05-31 21:44:17 UTC) #41
commit-bot: I haz the power
Dry run: This issue passed the CQ dry run.
4 years, 6 months ago (2016-05-31 23:26:38 UTC) #43
flackr
tools/grit lgtm with a few nits https://codereview.chromium.org/1968993002/diff/260001/tools/grit/grit/format/gzip_string.py File tools/grit/grit/format/gzip_string.py (right): https://codereview.chromium.org/1968993002/diff/260001/tools/grit/grit/format/gzip_string.py#newcode9 tools/grit/grit/format/gzip_string.py:9: nit: two newlines ...
4 years, 6 months ago (2016-06-01 14:58:30 UTC) #44
commit-bot: I haz the power
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/patch-status/1968993002/280001 View timeline at https://chromium-cq-status.appspot.com/patch-timeline/1968993002/280001
4 years, 6 months ago (2016-06-01 15:52:24 UTC) #48
commit-bot: I haz the power
Committed patchset #15 (id:280001)
4 years, 6 months ago (2016-06-01 17:16:36 UTC) #50
commit-bot: I haz the power
Patchset 15 (id:??) landed as https://crrev.com/e3a06a21aa1a7adfaef988488cdbcccbfa9a2ef6 Cr-Commit-Position: refs/heads/master@{#397168}
4 years, 6 months ago (2016-06-01 17:18:20 UTC) #52
hshi1
4 years, 6 months ago (2016-06-02 17:11:24 UTC) #53
Message was sent while issue was closed.
On 2016/06/01 17:18:20, commit-bot: I haz the power wrote:
> Patchset 15 (id:??) landed as
> https://crrev.com/e3a06a21aa1a7adfaef988488cdbcccbfa9a2ef6
> Cr-Commit-Position: refs/heads/master@{#397168}

This CL is causing cros pfq to fail on all boards.
https://uberchromegw.corp.google.com/i/chromeos/builders/master-chromium-pfq/...

chromeos-chrome-53.0.2756.0_rc-r1: FAILED: cd
../../../../../../../home/chrome-bot/chrome_root/src/content; python
../tools/grit/grit.py -i content_resources.grd build -f
../tools/gritsettings/resource_ids -o ../c/Release/gen/content
"--write-only-new=1" -D _chromium -E "CHROMIUM_BUILD=chromium" -D chromeos -D
"scale_factors=2x" -D toolkit_views -D use_aura -D use_ash -D use_nss_certs -D
use_ozone -D image_loader_extension -D enable_extensions -D enable_plugins -D
enable_printing -D enable_print_preview -D enable_themes -D enable_app_list -D
use_concatenated_impulse_responses -D enable_media_router -D enable_webrtc -D
enable_task_manager -D enable_notifications -D enable_service_discovery -E
"root_out_dir=../c/Release"
chromeos-chrome-53.0.2756.0_rc-r1: Error processing node <?xml version="1.0"
encoding="UTF-8"?>
chromeos-chrome-53.0.2756.0_rc-r1: <include allowexternalscript="true"
compress="gzip" file="browser/resources/appcache/appcache_internals.html"
flattenhtml="true" name="IDR_APPCACHE_INTERNALS_HTML" type="BINDATA" />
chromeos-chrome-53.0.2756.0_rc-r1: Traceback (most recent call last):
chromeos-chrome-53.0.2756.0_rc-r1:   File "../tools/grit/grit.py", line 15, in
<module>
chromeos-chrome-53.0.2756.0_rc-r1:     grit.grit_runner.Main(sys.argv[1:])
chromeos-chrome-53.0.2756.0_rc-r1:   File
"/home/chrome-bot/chrome_root/src/tools/grit/grit/grit_runner.py", line 268, in
Main
chromeos-chrome-53.0.2756.0_rc-r1:     toolobject.Run(options, args[1:])
chromeos-chrome-53.0.2756.0_rc-r1:   File
"/home/chrome-bot/chrome_root/src/tools/grit/grit/tool/build.py", line 230, in
Run
chromeos-chrome-53.0.2756.0_rc-r1:     self.Process()
chromeos-chrome-53.0.2756.0_rc-r1:   File
"/home/chrome-bot/chrome_root/src/tools/grit/grit/tool/build.py", line 360, in
Process
chromeos-chrome-53.0.2756.0_rc-r1:     self.ProcessNode(self.res, output,
outfile)
chromeos-chrome-53.0.2756.0_rc-r1:   File
"/home/chrome-bot/chrome_root/src/tools/grit/grit/tool/build.py", line 297, in
ProcessNode
chromeos-chrome-53.0.2756.0_rc-r1:     formatted = formatter(node,
output_node.GetLanguage(), output_dir=base_dir)
chromeos-chrome-53.0.2756.0_rc-r1:   File
"/home/chrome-bot/chrome_root/src/tools/grit/grit/format/data_pack.py", line 45,
in Format
chromeos-chrome-53.0.2756.0_rc-r1:     id, value = node.GetDataPackPair(lang,
UTF8)
chromeos-chrome-53.0.2756.0_rc-r1:   File
"/home/chrome-bot/chrome_root/src/tools/grit/grit/node/include.py", line 102, in
GetDataPackPair
chromeos-chrome-53.0.2756.0_rc-r1:     data =
grit.format.gzip_string.GzipStringRsyncable(data)
chromeos-chrome-53.0.2756.0_rc-r1:   File
"/home/chrome-bot/chrome_root/src/tools/grit/grit/format/gzip_string.py", line
26, in GzipStringRsyncable
chromeos-chrome-53.0.2756.0_rc-r1:     stderr)
chromeos-chrome-53.0.2756.0_rc-r1: subprocess.CalledProcessError: Command 'gzip'
returned non-zero exit status 1

A revert will land soon.

Powered by Google App Engine
This is Rietveld 408576698