Add HTML parsing benchmark.

Add HTML parsing benchmark.

For background and motivation see bugs.

BUG=625986, 595492
CQ_EXTRA_TRYBOTS=tryserver.chromium.perf:android_s5_perf_cq;tryserver.chromium.perf:mac_retina_perf_cq

[ulan, 2016-07-06 17:45:35]

Description was changed from

==========
Add HTML parsing benchmark.

For background and motivation see bugs.

BUG=625986,595492
==========

to

==========
Add HTML parsing benchmark.

For background and motivation see bugs.

BUG=625986,595492
CQ_EXTRA_TRYBOTS=tryserver.chromium.perf:android_s5_perf_cq;tryserver.chromium.perf:mac_retina_perf_cq
==========

[ulan, 2016-07-06 17:47:48]

ulan@chromium.org changed reviewers:
+ dtu@chromium.org, esprehn@chromium.org

[ulan, 2016-07-06 17:47:49]

ptal

[dtu, 2016-07-06 19:25:09]

dtu@chromium.org changed reviewers:
+ eakuefner@chromium.org

[dtu, 2016-07-06 19:25:09]

--> eakuefner, the TBM expert. TBM is now preferred for writing metrics

[eakuefner, 2016-07-06 21:59:28]

Hi Ulan,

As Dave says LegacyPageTest is deprecated, so we'd like to avoid adding new
tests that don't use TBM. That said, this does bring up some technical issues
that Telemetry team have discussed in the past but never reached consensus on,
but basically everyone is OOO this week due to no meetings. So just to let you
know, I'll not have forgotten about this CL, but probably won't get to it until
next week when I can manage to discuss the approach with folks.

[ulan, 2016-07-07 09:52:59]

On 2016/07/06 21:59:28, eakuefner wrote:
> Hi Ulan,
> 
> As Dave says LegacyPageTest is deprecated, so we'd like to avoid adding new
> tests that don't use TBM. That said, this does bring up some technical issues
> that Telemetry team have discussed in the past but never reached consensus on,
> but basically everyone is OOO this week due to no meetings. So just to let you
> know, I'll not have forgotten about this CL, but probably won't get to it
until
> next week when I can manage to discuss the approach with folks.

Thanks for the heads-up, Ethan. I'll be happy to change to the benchmark to TBM
(if there is a way to fetch results from JS).

[ulan, 2016-07-27 09:11:39]

On 2016/07/06 21:59:28, eakuefner wrote:
> Hi Ulan,
> 
> As Dave says LegacyPageTest is deprecated, so we'd like to avoid adding new
> tests that don't use TBM. That said, this does bring up some technical issues
> that Telemetry team have discussed in the past but never reached consensus on,
> but basically everyone is OOO this week due to no meetings. So just to let you
> know, I'll not have forgotten about this CL, but probably won't get to it
until
> next week when I can manage to discuss the approach with folks.

Ethan, did you have chance to discuss the approach with folks?

[nduca, 2016-07-27 22:50:44]

nduca@chromium.org changed reviewers:
+ nduca@chromium.org

[nduca, 2016-07-27 22:50:45]

i have a few framing questions about this benchmark that i'd like to take to an
email thread. stay tuned. :)

[esprehn, 2016-07-27 22:53:48]

Is it better to have this in perf/ instead of the blink PerformanceTests
directory? :)

[ulan, 2016-07-28 15:36:32]

On 2016/07/27 22:53:48, esprehn wrote:
> Is it better to have this in perf/ instead of the blink PerformanceTests
> directory? :)

PerformanceTests directory seems to be better fit. Uploaded CL:
https://codereview.chromium.org/2189533004/

[haraken, 2016-08-01 07:39:25]

haraken@chromium.org changed reviewers:
+ haraken@chromium.org, peria@chromium.org

[haraken, 2016-08-01 07:39:26]

peria-san: Is it worth adding the fake binding tests? As far as we discussed
before, the performance of the fake binding is highly affected by JS subtleties
and thus doesn't make much sense.

[peria, 2016-08-01 08:06:08]

On 2016/08/01 07:39:26, haraken wrote:
> peria-san: Is it worth adding the fake binding tests? As far as we discussed
> before, the performance of the fake binding is highly affected by JS
subtleties
> and thus doesn't make much sense.

IMO, the fake bindings test is not useful.
As we can see in the first result in issue 595492, the fake bindings test takes
longer than the real bindings test on Firefox.
So I think the fake bindings test and the difference with the real bindings
can be mis-leading metrics.

[esprehn, 2016-08-05 21:30:26]

On 2016/08/01 at 08:06:08, peria wrote:
> On 2016/08/01 07:39:26, haraken wrote:
> > peria-san: Is it worth adding the fake binding tests? As far as we discussed
> > before, the performance of the fake binding is highly affected by JS
subtleties
> > and thus doesn't make much sense.
> 
> IMO, the fake bindings test is not useful.
> As we can see in the first result in issue 595492, the fake bindings test
takes
> longer than the real bindings test on Firefox.
> So I think the fake bindings test and the difference with the real bindings
> can be mis-leading metrics.

They make sense to me when compared relative to the other 3 numbers. Comparing
them back to Chrome might not be useful.

What makes you think the fake bindings test is not useful?

[peria, 2016-08-08 05:10:52]

What I concern is that it is not clear what the fake bindings test measures.

IIUC, "the fake bindings test" shows time for (parsing HTML) + (operations
on JS objects),
while "the real bindings test" shows time for (parsing HTML) + (Blink
binding) + (DOM operations).

If (operations on JS objects) takes very short time, the fake bindings test
makes sense to me.
However as I said, it takes longer time than the real bindings test on
Firefox, so it is difficult
to think it takes ignorablly short time on Chrome and Safari.

Then, what does (the real binding test) - (the fake binding test) metric
figure?
Can we compare this metric between different browsers?


2016-08-06 6:30 GMT+09:00 <esprehn@chromium.org>:

> On 2016/08/01 at 08:06:08, peria wrote:
> > On 2016/08/01 07:39:26, haraken wrote:
> > > peria-san: Is it worth adding the fake binding tests? As far as we
> discussed
> > > before, the performance of the fake binding is highly affected by JS
> subtleties
> > > and thus doesn't make much sense.
> >
> > IMO, the fake bindings test is not useful.
> > As we can see in the first result in issue 595492, the fake bindings test
> takes
> > longer than the real bindings test on Firefox.
> > So I think the fake bindings test and the difference with the real
> bindings
> > can be mis-leading metrics.
>
> They make sense to me when compared relative to the other 3 numbers.
> Comparing
> them back to Chrome might not be useful.
>
> What makes you think the fake bindings test is not useful?
>
> https://codereview.chromium.org/2119413003/
>



-- 
Hitoshi Yoshida (Peria)
email: peria@chromium.org

-- 
You received this message because you are subscribed to the Google Groups
"Chromium-reviews" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to chromium-reviews+unsubscribe@chromium.org.