Break metadata into smaller chunks (number of chunks=10) for faster delta generation.

Break metadata into smaller chunks (number of chunks=10) for faster delta generation.

Here are some data for comparison.

Without any metadata processing:
 Run Time:
   real 2m22.331s
   user 1m56.040s
   sys  0m6.170s
 Payload Size:
   756162 bytes

With metadata processing (number of chunks=4):
 Run Time:
   real 13m48.292s
   user 13m10.120s
   sys  0m6.920s
 Payload Size:
   749713 bytes

With metadata processing (number of chunks=10):
 Run Time:
   real 3m46.242s
   user 3m17.880s
   sys  0m6.640s
 Payload Size:
   748464 bytes

BUG=chromium-os:10669
TEST=Unit tests and generated delta payloads between two images that do
not have their metadata preserved.  Ran the delta generator with and
without metadata processing and also used different number of chunks for
comparison purposes.

Change-Id: I15ff831b95f43cbebbcca6ecc6a3fbc404b6f065

Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=f650219

[thieule, 2011-01-05 22:06:25]

This time with unit test updated...

PTAL

[petkov, 2011-01-05 22:09:33]

LGTM

[adlr, 2011-01-05 22:31:50]

LGTM

On Wed, Jan 5, 2011 at 2:06 PM, <thieule@chromium.org> wrote:

> Reviewers: adlr, petkov,
>
> Message:
> This time with unit test updated...
>
> PTAL
>
> Description:
> Break metadata into smaller chunks (number of chunks=10) for faster delta
> generation.
>
> Here are some data for comparison.
>
> Without any metadata processing:
>  Run Time:
>   real 2m22.331s
>   user 1m56.040s
>   sys  0m6.170s
>  Payload Size:
>   756162 bytes
>
> With metadata processing (number of chunks=4):
>  Run Time:
>   real 13m48.292s
>   user 13m10.120s
>   sys  0m6.920s
>  Payload Size:
>   749713 bytes
>
> With metadata processing (number of chunks=10):
>  Run Time:
>   real 3m46.242s
>   user 3m17.880s
>   sys  0m6.640s
>  Payload Size:
>   748464 bytes
>
> BUG=chromium-os:10669
> TEST=Unit tests and generated delta payloads between two images that do
> not have their metadata preserved.  Ran the delta generator with and
> without metadata processing and also used different number of chunks for
> comparison purposes.
>
> Change-Id: I15ff831b95f43cbebbcca6ecc6a3fbc404b6f065
>
> Please review this at http://codereview.chromium.org/6060012/
>
> SVN Base: http://git.chromium.org/git/update_engine.git@master
>
> Affected files:
>  M metadata.cc
>  M metadata_unittest.cc
>
>
> Index: metadata.cc
> diff --git a/metadata.cc b/metadata.cc
> index
>
20c0a0d9b69bf2eb69a0f78b423fa5c69380abdc..7dfc787210f0e27c09cc6ca272b768ba1a26f008
> 100644
> --- a/metadata.cc
> +++ b/metadata.cc
> @@ -220,7 +220,7 @@ bool ReadFilesystemMetadata(Graph* graph,
>
>     // Due to bsdiff slowness, we're going to break each block group down
>     // into metadata chunks and feed them to bsdiff.
> -    __u32 num_chunks = 4;
> +    __u32 num_chunks = 10;
>     __u32 blocks_per_chunk = num_metadata_blocks / num_chunks;
>     __u32 curr_block = bg_start_block;
>     for (__u32 chunk = 0; chunk < num_chunks; chunk++) {
> Index: metadata_unittest.cc
> diff --git a/metadata_unittest.cc b/metadata_unittest.cc
> index
>
fde0b989cbb76ab61da1849f4ea90f54dac6a051..885197d0fe7251a7cbd7701e251dadcc8c7aa2c5
> 100644
> --- a/metadata_unittest.cc
> +++ b/metadata_unittest.cc
> @@ -113,14 +113,26 @@ TEST_F(MetadataTest, RunAsRootReadMetadata) {
>     off_t start_block; // Set to -1 to skip start block verification
>     off_t num_blocks; // Set to -1 to skip num blocks verification
>   } exp_results[] =
> -      {{"<rootfs-bg-0-0-metadata>", 0, 260},
> -       {"<rootfs-bg-0-1-metadata>", 260, 260},
> -       {"<rootfs-bg-0-2-metadata>", 520, 260},
> -       {"<rootfs-bg-0-3-metadata>", 780, 263},
> -       {"<rootfs-bg-1-0-metadata>", 32768, 260},
> -       {"<rootfs-bg-1-1-metadata>", 33028, 260},
> -       {"<rootfs-bg-1-2-metadata>", 33288, 260},
> -       {"<rootfs-bg-1-3-metadata>", 33548, 263},
> +      {{"<rootfs-bg-0-0-metadata>", 0, 104},
> +       {"<rootfs-bg-0-1-metadata>", 104, 104},
> +       {"<rootfs-bg-0-2-metadata>", 208, 104},
> +       {"<rootfs-bg-0-3-metadata>", 312, 104},
> +       {"<rootfs-bg-0-4-metadata>", 416, 104},
> +       {"<rootfs-bg-0-5-metadata>", 520, 104},
> +       {"<rootfs-bg-0-6-metadata>", 624, 104},
> +       {"<rootfs-bg-0-7-metadata>", 728, 104},
> +       {"<rootfs-bg-0-8-metadata>", 832, 104},
> +       {"<rootfs-bg-0-9-metadata>", 936, 107},
> +       {"<rootfs-bg-1-0-metadata>", 32768, 104},
> +       {"<rootfs-bg-1-1-metadata>", 32872, 104},
> +       {"<rootfs-bg-1-2-metadata>", 32976, 104},
> +       {"<rootfs-bg-1-3-metadata>", 33080, 104},
> +       {"<rootfs-bg-1-4-metadata>", 33184, 104},
> +       {"<rootfs-bg-1-5-metadata>", 33288, 104},
> +       {"<rootfs-bg-1-6-metadata>", 33392, 104},
> +       {"<rootfs-bg-1-7-metadata>", 33496, 104},
> +       {"<rootfs-bg-1-8-metadata>", 33600, 104},
> +       {"<rootfs-bg-1-9-metadata>", 33704, 107},
>        {"<rootfs-inode-2-metadata>", -1, 1},
>        {"<rootfs-inode-8-metadata>", -1, 4101},
>        {"<rootfs-inode-11-metadata>", -1, 4},
>
>
>