Chromium Code Reviews
|
|
Chromium Code Reviews
Issue
2422443002:
isolate client: get back faster compression library. (Closed)
|
Created:
4 years, 2 months ago by tandrii(chromium) Modified:
4 years, 2 months ago Reviewers:
M-A Ruel CC:
chromium-reviews, infra-reviews+luci-go_chromium.org, andrew.wang, todd, tandrii+luci-go_chromium.org Target Ref:
refs/heads/master Project:
luci-go Visibility:
Public. |
Descriptionisolate client: get back faster compression library.
On my workstation, isolate archive 1GB file with fake isolate server
is reduced from 3m 48s to 1m 9s (almost 3x).
Depends on https://chromium-review.googlesource.com/397624 to modify
Go deps in infra/infra.
BUG=653723
R=maruel@chromium.org
Committed: https://github.com/luci/luci-go/commit/14ac2e2b1844dada4337c300b9d68896bda9668a
Patch Set 1 #
Messages
Total messages: 19 (11 generated)
The CQ bit was checked by tandrii@chromium.org to run a CQ dry run
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Dry run: Try jobs failed on following builders: Luci-go Linux Precise 32 Tester on luci.infra.try (JOB_FAILED, https://luci-milo.appspot.com/swarming/task/31d93b8725290f10) Luci-go Mac Tester on luci.infra.try (JOB_FAILED, https://luci-milo.appspot.com/swarming/task/31d93b882524f610)
Description was changed from ========== isolate client: get back faster compression library. On my workstation, isolate archive 1GB file with fake isolate server is reduced from 3m 48s to 1m 9s (almost 3x). BUG=653723 R=maruel@chromium.org ========== to ========== isolate client: get back faster compression library. On my workstation, isolate archive 1GB file with fake isolate server is reduced from 3m 48s to 1m 9s (almost 3x). Depends on https://chromium-review.googlesource.com/397624 to modify Go deps in infra/infra. BUG=653723 R=maruel@chromium.org ==========
What are the existing requirements on compression? Should we consider using a faster (prod-hardened) algorithm such as https://godoc.org/github.com/golang/snappy ?
On 2016/10/14 00:07:04, djd wrote: > What are the existing requirements on compression? 1) the algorithm cannot change for now 2) it must work on stock python without third party library > Should we consider using a faster (prod-hardened) algorithm such as > https://godoc.org/github.com/golang/snappy ? Eventually but that's out of scope for here.
On 2016/10/14 01:36:53, M-A Ruel wrote: > On 2016/10/14 00:07:04, djd wrote: > > What are the existing requirements on compression? > > 1) the algorithm cannot change for now > 2) it must work on stock python without third party library Does it have requirements in terms of size? Are we relying on a certain compression ratio? (For example – would a "no-op" compression break everything assuming it was valid zlib?) > > > Should we consider using a faster (prod-hardened) algorithm such as > > https://godoc.org/github.com/golang/snappy ? > > Eventually but that's out of scope for here.
On 2016/10/14 03:12:40, djd wrote: > On 2016/10/14 01:36:53, M-A Ruel wrote: > > On 2016/10/14 00:07:04, djd wrote: > > > What are the existing requirements on compression? > > > > 1) the algorithm cannot change for now > > 2) it must work on stock python without third party library > > Does it have requirements in terms of size? Are we relying on a certain > compression ratio? (For example – would a "no-op" compression break everything > assuming it was valid zlib?) No, we aren't strictly relying on specific ratio, though I can imagine [1] that would implicitly rely on it, but we should discourage such jobs. [1] I can imagine a swarming job that just downloads 10GB of very compressible isolates, so as actual download is say ~a few MB, and then has a very harsh timeout of say 2 minutes. This way no compression could result in task failure.
and btw, PTAL :)
The CQ bit was checked by tandrii@chromium.org to run a CQ dry run
Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
The CQ bit was unchecked by commit-bot@chromium.org
Dry run: This issue passed the CQ dry run.
lgtm
The CQ bit was checked by tandrii@chromium.org
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
Message was sent while issue was closed.
Description was changed from ========== isolate client: get back faster compression library. On my workstation, isolate archive 1GB file with fake isolate server is reduced from 3m 48s to 1m 9s (almost 3x). Depends on https://chromium-review.googlesource.com/397624 to modify Go deps in infra/infra. BUG=653723 R=maruel@chromium.org ========== to ========== isolate client: get back faster compression library. On my workstation, isolate archive 1GB file with fake isolate server is reduced from 3m 48s to 1m 9s (almost 3x). Depends on https://chromium-review.googlesource.com/397624 to modify Go deps in infra/infra. BUG=653723 R=maruel@chromium.org Committed: https://github.com/luci/luci-go/commit/14ac2e2b1844dada4337c300b9d68896bda9668a ==========
Message was sent while issue was closed.
Committed patchset #1 (id:1) as https://github.com/luci/luci-go/commit/14ac2e2b1844dada4337c300b9d68896bda9668a |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
