Make isVisible() faster and more accurate

Make isVisible() faster and more accurate

Function isVisible() checked whether display is none, but it is not
inherited. This is OK when doing DOM traversal, since we skip children
of invisible elements. However, this doesn't work in general. For
example, when checking whether an anchor element is visible in next page
detection.

We check properties like offsetParent, offsetWidth, and offsetHeight
instead to amend this.

Function isVisible() checked for display, opacity and visibility, which
requires slow getComputedStyle(). Opacity and visibility are rarely used
to hide elements, and display is replaced by offset*, so that we can
stop getting computed styles.

** Score changes:

Multi-page dataset:
Minor changes in content extraction. Most entries here are not
distillable anyway.
Average F1 of content: 0.561 -> 0.561
https://x20web.corp.google.com/~wychen/domdistillerscore/visible/multi-page.html

Reader-image dataset:
Average F1 of image: 0.633 -> 0.634
https://x20web.corp.google.com/~wychen/domdistillerscore/visible/reader-images.html

No changes on other data sets:
- cjk-golden-data
- cleaneval-golden-data
- golden_data_with_knowledge
- page-links-golden-data
- reader-mode-golden-data

** Performance impact:

The average time reported by eval server for dataset
"reader-mode-golden-data" is used as the benchmark. To reduce noise, it
is rerun for 100 times.

The total time spent is 4.8% shorter.

BUG=431067

[wychen, 2015-12-07 23:23:51]

wychen@chromium.org changed reviewers:
+ mdjones@chromium.org

[wychen, 2015-12-07 23:23:51]

PTAL

[mdjones, 2015-12-08 00:04:33]

https://codereview.chromium.org/1508963003/diff/1/java/org/chromium/distiller...
File java/org/chromium/distiller/DomUtil.java (right):

https://codereview.chromium.org/1508963003/diff/1/java/org/chromium/distiller...
java/org/chromium/distiller/DomUtil.java:100: return !(e.getOffsetParent() ==
null && e.getOffsetHeight() == 0 && e.getOffsetWidth() == 0);
why not:
return e.getOffsetParent() != null || ... || ...;

https://codereview.chromium.org/1508963003/diff/1/java/org/chromium/distiller...
java/org/chromium/distiller/DomUtil.java:103: // Currently not used by anyone.
I think it's okay if we remove this if not used.

[wychen, 2015-12-08 00:41:46]

https://codereview.chromium.org/1508963003/diff/1/java/org/chromium/distiller...
File java/org/chromium/distiller/DomUtil.java (right):

https://codereview.chromium.org/1508963003/diff/1/java/org/chromium/distiller...
java/org/chromium/distiller/DomUtil.java:100: return !(e.getOffsetParent() ==
null && e.getOffsetHeight() == 0 && e.getOffsetWidth() == 0);
On 2015/12/08 00:04:33, mdjones wrote:
> why not:
> return e.getOffsetParent() != null || ... || ...;

Done.

https://codereview.chromium.org/1508963003/diff/1/java/org/chromium/distiller...
java/org/chromium/distiller/DomUtil.java:103: // Currently not used by anyone.
On 2015/12/08 00:04:33, mdjones wrote:
> I think it's okay if we remove this if not used.

Done.

[mdjones, 2015-12-08 01:15:27]

lgtm

[wychen, 2015-12-08 22:54:46]

Description was changed from

==========
Make isVisible() faster and more accurate

Function isVisible() checked whether display is none, but it is not inherited.
This is OK when doing DOM traversal, since we skip children of invisible
elements. However, this doesn't work in general. For example, when checking
whether an anchor element is visible in next page detection.

We check properties like offsetParent, offsetWidth, and offsetHeight
instead to amend this.

Function isVisible() checked for display, opacity and visibility, which requires
slow getComputedStyle(). Opacity and visibility are rarely used to hide
elements,
and display is replaced by offset*, so that we can stop getting computed styles.

** Score changes:

Multi-page dataset:
Minor changes in content extraction. Most entries here are not distillable
anyway.
Average F1 of content: 0.561 -> 0.561
https://x20web.corp.google.com/~wychen/domdistillerscore/visible/multi-page.html

Reader-image dataset:
Average F1 of image: 0.633 -> 0.634
https://x20web.corp.google.com/~wychen/domdistillerscore/visible/reader-image...

No changes on other data sets:
- cjk-golden-data
- cleaneval-golden-data
- golden_data_with_knowledge
- page-links-golden-data
- reader-mode-golden-data

** Performance impact:

The average time reported by eval server for dataset
"reader-mode-golden-data" is used as the benchmark.
To reduce noise, it is rerun for 100 times

The total time spent is 4.8% shorter.

BUG=
==========

to

==========
Make isVisible() faster and more accurate

Function isVisible() checked whether display is none, but it is not inherited.
This is OK when doing DOM traversal, since we skip children of invisible
elements. However, this doesn't work in general. For example, when checking
whether an anchor element is visible in next page detection.

We check properties like offsetParent, offsetWidth, and offsetHeight
instead to amend this.

Function isVisible() checked for display, opacity and visibility, which requires
slow getComputedStyle(). Opacity and visibility are rarely used to hide
elements,
and display is replaced by offset*, so that we can stop getting computed styles.

** Score changes:

Multi-page dataset:
Minor changes in content extraction. Most entries here are not distillable
anyway.
Average F1 of content: 0.561 -> 0.561
https://x20web.corp.google.com/~wychen/domdistillerscore/visible/multi-page.html

Reader-image dataset:
Average F1 of image: 0.633 -> 0.634
https://x20web.corp.google.com/~wychen/domdistillerscore/visible/reader-image...

No changes on other data sets:
- cjk-golden-data
- cleaneval-golden-data
- golden_data_with_knowledge
- page-links-golden-data
- reader-mode-golden-data

** Performance impact:

The average time reported by eval server for dataset
"reader-mode-golden-data" is used as the benchmark.
To reduce noise, it is rerun for 100 times

The total time spent is 4.8% shorter.

BUG=431067
==========

[wychen, 2016-03-11 09:29:01]

Description was changed from

==========
Make isVisible() faster and more accurate

Function isVisible() checked whether display is none, but it is not inherited.
This is OK when doing DOM traversal, since we skip children of invisible
elements. However, this doesn't work in general. For example, when checking
whether an anchor element is visible in next page detection.

We check properties like offsetParent, offsetWidth, and offsetHeight
instead to amend this.

Function isVisible() checked for display, opacity and visibility, which requires
slow getComputedStyle(). Opacity and visibility are rarely used to hide
elements,
and display is replaced by offset*, so that we can stop getting computed styles.

** Score changes:

Multi-page dataset:
Minor changes in content extraction. Most entries here are not distillable
anyway.
Average F1 of content: 0.561 -> 0.561
https://x20web.corp.google.com/~wychen/domdistillerscore/visible/multi-page.html

Reader-image dataset:
Average F1 of image: 0.633 -> 0.634
https://x20web.corp.google.com/~wychen/domdistillerscore/visible/reader-image...

No changes on other data sets:
- cjk-golden-data
- cleaneval-golden-data
- golden_data_with_knowledge
- page-links-golden-data
- reader-mode-golden-data

** Performance impact:

The average time reported by eval server for dataset
"reader-mode-golden-data" is used as the benchmark.
To reduce noise, it is rerun for 100 times

The total time spent is 4.8% shorter.

BUG=431067
==========

to

==========
Make isVisible() faster and more accurate

Function isVisible() checked whether display is none, but it is not
inherited. This is OK when doing DOM traversal, since we skip children
of invisible elements. However, this doesn't work in general. For
example, when checking whether an anchor element is visible in next page
detection.

We check properties like offsetParent, offsetWidth, and offsetHeight
instead to amend this.

Function isVisible() checked for display, opacity and visibility, which
requires slow getComputedStyle(). Opacity and visibility are rarely used
to hide elements, and display is replaced by offset*, so that we can
stop getting computed styles.

** Score changes:

Multi-page dataset:
Minor changes in content extraction. Most entries here are not
distillable anyway.
Average F1 of content: 0.561 -> 0.561
https://x20web.corp.google.com/~wychen/domdistillerscore/visible/multi-page.html

Reader-image dataset:
Average F1 of image: 0.633 -> 0.634
https://x20web.corp.google.com/~wychen/domdistillerscore/visible/reader-image...

No changes on other data sets:
- cjk-golden-data
- cleaneval-golden-data
- golden_data_with_knowledge
- page-links-golden-data
- reader-mode-golden-data

** Performance impact:

The average time reported by eval server for dataset
"reader-mode-golden-data" is used as the benchmark. To reduce noise, it
is rerun for 100 times.

The total time spent is 4.8% shorter.

BUG=431067
==========