Make several cpu-profile tests more stable

Make several cpu-profile tests more stable

10 samples they collected were not enough to ensure the expected
call tree is covered. Increase the samples count to 100.

[yurys, 2014-05-26 14:34:04]

This still looks fragile. Let's not go this way and change the tests to avoid
this non-determinism: let's feed the stack iterator with some mock data and
check that the frames it returns match our expectations.

[alph, 2014-05-26 15:13:00]

On 2014/05/26 14:34:04, yurys wrote:
> This still looks fragile. Let's not go this way and change the tests to avoid
> this non-determinism: let's feed the stack iterator with some mock data and
> check that the frames it returns match our expectations.

Let me advocate.

Assuming the current test failure probability (caused by small population size)
is p. By increasing the number of samples 10 times the probability becomes p^10.

The v8 waterfall suggests the rough value of p is about 1/100. Thus the patch
makes the probability to be 10^(-20), i.e. one false positive failure per 10^20
runs. Provided the test is run every 3 seconds, we shouldn't see a failure next
10^13 years. ;-)
I think we may consider it as stable. wdyt?

The benefit of this approach is that it covers the same execution paths as they
run in the user code. The mock data won't let us test e.g. sampler and
inter-thread communications.

[yurys, 2014-05-27 07:32:22]

On 2014/05/26 15:13:00, alph wrote:
> On 2014/05/26 14:34:04, yurys wrote:
> > This still looks fragile. Let's not go this way and change the tests to
avoid
> > this non-determinism: let's feed the stack iterator with some mock data and
> > check that the frames it returns match our expectations.
> 
> Let me advocate.
> 
> Assuming the current test failure probability (caused by small population
size)
> is p. By increasing the number of samples 10 times the probability becomes
p^10.
> 
Correct. But p seems to depend on how busy the bot is.

> The v8 waterfall suggests the rough value of p is about 1/100. Thus the patch
> makes the probability to be 10^(-20), i.e. one false positive failure per
10^20
> runs. Provided the test is run every 3 seconds, we shouldn't see a failure
next
> 10^13 years. ;-)
> I think we may consider it as stable. wdyt?
>
We can try but I doubt that playing with these numbers while still having the
test
run for a reasonable time we will make the test stable.

> The benefit of this approach is that it covers the same execution paths as
they
> run in the user code. The mock data won't let us test e.g. sampler and
> inter-thread communications.
Of cause that would test only part of the functionality the test currently
covers.
On the other hand the test seems to be testing too many things where the most
problematic
one has always been the stack iterator. That's why I'd rather have a stable test
for
the stack iterator than one covering the real path but flaky. Btw, there are
already some
tests for the inter-thread communication.

[yurys, 2014-05-27 11:42:56]

All three tests call startProfiling in javascript. It should be enough to change
RunProfiler method to simply stop that profile instead of starting/stopping a
new one. That profile should always contain at least one sample that we check.