Add SerializeNull and SerializeSmi benchmark for benchmarking serialization of the messages used in…

Add SerializeNull and SerializeSmi benchmark for benchmarking serialization of the messages used inside of dart:io.

BUG=
R=asiva@google.com

Committed: https://code.google.com/p/dart/source/detail?r=34108

[siva, 2014-02-11 23:50:41]

https://codereview.chromium.org/159723003/diff/1/runtime/vm/benchmark_test.cc
File runtime/vm/benchmark_test.cc (right):

https://codereview.chromium.org/159723003/diff/1/runtime/vm/benchmark_test.cc...
runtime/vm/benchmark_test.cc:538: MessageWriter writer(&buffer,
&malloc_allocator);
sing malloc_allocator here means that you are measuring the cost
of malloc/free in this loop.

Changing this allocator to message_allocator and getting rid
of the free in the loop cuts SerializeSmi from
513644 micorseconds to 356110 microseconds

static uint8_t message_buffer[64];
static uint8_t* message_allocator(
    uint8_t* ptr, intptr_t old_size, intptr_t new_size) {
  return message_buffer;
}

[Anders Johnsen, 2014-02-12 07:23:26]

https://codereview.chromium.org/159723003/diff/1/runtime/vm/benchmark_test.cc
File runtime/vm/benchmark_test.cc (right):

https://codereview.chromium.org/159723003/diff/1/runtime/vm/benchmark_test.cc...
runtime/vm/benchmark_test.cc:538: MessageWriter writer(&buffer,
&malloc_allocator);
On 2014/02/11 23:50:41, siva wrote:
> sing malloc_allocator here means that you are measuring the cost
> of malloc/free in this loop.
> 
> Changing this allocator to message_allocator and getting rid
> of the free in the loop cuts SerializeSmi from
> 513644 micorseconds to 356110 microseconds
> 
> static uint8_t message_buffer[64];
> static uint8_t* message_allocator(
>     uint8_t* ptr, intptr_t old_size, intptr_t new_size) {
>   return message_buffer;
> }

This was actually on purpose. I'm trying to understand where all the time goes,
and measuring without the allocation, it's not really measuring what we do in
the the MessageHandler.

On the other hand, if we do want to speed up the serialization part of this, it
may be easier to not include the allocation.

[siva, 2014-02-12 19:29:22]

https://codereview.chromium.org/159723003/diff/1/runtime/vm/benchmark_test.cc
File runtime/vm/benchmark_test.cc (right):

https://codereview.chromium.org/159723003/diff/1/runtime/vm/benchmark_test.cc...
runtime/vm/benchmark_test.cc:538: MessageWriter writer(&buffer,
&malloc_allocator);
I agree that you want to measure what goes on in the MessageHandler and that is
why I find this benchmark a bit misleading.

From this benchmark we would tend to infer that it takes 500 ns to
serialize/deserialize a Smi which is not quite right. If all we are interested
in is measuring the cost of serializing/deserializing Smi we should create a
SnapshotWriter write about 10000 smis and read back the 10000 Smis and measure
the cost for each Smi.

You should probably add a different benchmark SimpleMessage or something like
that which does a loop of creating a typical message (maybe more than just a Smi
or Null) and reading it back.

On 2014/02/12 07:23:26, Anders Johnsen wrote:
> On 2014/02/11 23:50:41, siva wrote:
> > sing malloc_allocator here means that you are measuring the cost
> > of malloc/free in this loop.
> > 
> > Changing this allocator to message_allocator and getting rid
> > of the free in the loop cuts SerializeSmi from
> > 513644 micorseconds to 356110 microseconds
> > 
> > static uint8_t message_buffer[64];
> > static uint8_t* message_allocator(
> >     uint8_t* ptr, intptr_t old_size, intptr_t new_size) {
> >   return message_buffer;
> > }
> 
> This was actually on purpose. I'm trying to understand where all the time
goes,
> and measuring without the allocation, it's not really measuring what we do in
> the the MessageHandler.
> 
> On the other hand, if we do want to speed up the serialization part of this,
it
> may be easier to not include the allocation.

[Anders Johnsen, 2014-03-18 19:26:46]

PTAL, added a benchmark where we measure the full cost of creating a message,
not only serializing.

https://codereview.chromium.org/159723003/diff/1/runtime/vm/benchmark_test.cc
File runtime/vm/benchmark_test.cc (right):

https://codereview.chromium.org/159723003/diff/1/runtime/vm/benchmark_test.cc...
runtime/vm/benchmark_test.cc:538: MessageWriter writer(&buffer,
&malloc_allocator);
On 2014/02/12 19:29:23, siva wrote:
> I agree that you want to measure what goes on in the MessageHandler and that
is
> why I find this benchmark a bit misleading.
> 
> From this benchmark we would tend to infer that it takes 500 ns to
> serialize/deserialize a Smi which is not quite right. If all we are interested
> in is measuring the cost of serializing/deserializing Smi we should create a
> SnapshotWriter write about 10000 smis and read back the 10000 Smis and measure
> the cost for each Smi.
> 
> You should probably add a different benchmark SimpleMessage or something like
> that which does a loop of creating a typical message (maybe more than just a
Smi
> or Null) and reading it back.
> 
> On 2014/02/12 07:23:26, Anders Johnsen wrote:
> > On 2014/02/11 23:50:41, siva wrote:
> > > sing malloc_allocator here means that you are measuring the cost
> > > of malloc/free in this loop.
> > > 
> > > Changing this allocator to message_allocator and getting rid
> > > of the free in the loop cuts SerializeSmi from
> > > 513644 micorseconds to 356110 microseconds
> > > 
> > > static uint8_t message_buffer[64];
> > > static uint8_t* message_allocator(
> > >     uint8_t* ptr, intptr_t old_size, intptr_t new_size) {
> > >   return message_buffer;
> > > }
> > 
> > This was actually on purpose. I'm trying to understand where all the time
> goes,
> > and measuring without the allocation, it's not really measuring what we do
in
> > the the MessageHandler.
> > 
> > On the other hand, if we do want to speed up the serialization part of this,
> it
> > may be easier to not include the allocation.
> 

Done.

[siva, 2014-03-18 21:18:22]

LGTM

(you will have to add these to the list of benchmarks under VMBenchmarks)

[Anders Johnsen, 2014-03-19 08:54:12]

Committed patchset #2 manually as r34108 (presubmit successful).