AU: Speed up full update payload generation by running multiple threads.

AU: Speed up full update payload generation by running multiple threads.

Full end-to-end payload generation drops from 140 to 40 seconds on my machine.

BUG=8747
TEST=unit tests, updated through dev server

Change-Id: Ib3934b38a55e043abfa49abb25b2931e529ff768

Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=7a22d79

[adlr, 2010-11-08 21:27:08]

LGTM!

Great stuff!

http://codereview.chromium.org/4610001/diff/7001/8005
File full_update_generator.cc (right):

http://codereview.chromium.org/4610001/diff/7001/8005#newcode158
full_update_generator.cc:158: threads.pop_front();
might be faster to wait on all of the threads, but that can get tricky, so i
think this is fine for now. just wanted to mention it.

[petkov, 2010-11-08 21:45:21]

http://codereview.chromium.org/4610001/diff/7001/8005
File full_update_generator.cc (right):

http://codereview.chromium.org/4610001/diff/7001/8005#newcode158
full_update_generator.cc:158: threads.pop_front();
On 2010/11/08 21:27:08, adlr wrote:
> might be faster to wait on all of the threads, but that can get tricky, so i
> think this is fine for now. just wanted to mention it.

Maybe... We could also make the separate threads write the output directly. This
may have performance benefits but it will make us generate unreproducable
payloads. Or, we just rely on reorder data blobs later to generate an ordered
payload. Right now, ReorderDataBlobs doesn't really need to run for full updates
(i.e., another way to improve performance).

Anyway, this seems good enough as is...

[adlr, 2010-11-08 22:48:51]

agree! check it in!

On Mon, Nov 8, 2010 at 1:45 PM, <petkov@chromium.org> wrote:

>
> http://codereview.chromium.org/4610001/diff/7001/8005
> File full_update_generator.cc (right):
>
> http://codereview.chromium.org/4610001/diff/7001/8005#newcode158
> full_update_generator.cc:158: threads.pop_front();
> On 2010/11/08 21:27:08, adlr wrote:
>
>> might be faster to wait on all of the threads, but that can get
>>
> tricky, so i
>
>> think this is fine for now. just wanted to mention it.
>>
>
> Maybe... We could also make the separate threads write the output
> directly. This may have performance benefits but it will make us
> generate unreproducable payloads. Or, we just rely on reorder data blobs
> later to generate an ordered payload. Right now, ReorderDataBlobs
> doesn't really need to run for full updates (i.e., another way to
> improve performance).
>
> Anyway, this seems good enough as is...
>
>
> http://codereview.chromium.org/4610001/show
>