Reland: Make the HTMLDocumentParser yield more aggressively

Reland: Make the HTMLDocumentParser yield more aggressively

The patch caused failure in Android DumpAccessibilityTreeTest.ACcessibilityANoText.
However, it turns out the expectation of the test is currently wrong.
The test is disabled in Chromium tree as of now, and I'll rebaseline the test after this patch is landed.

---

Based on eseidel@'s CL: https://codereview.chromium.org/258013009/

Before this CL,
HTMLDocumentParser::didReceiveParsedChunkFromBackgroundParser
parsed the received chunk inline.
This was problematic because the function is invoked in IPC priority,
and DOM construction can block other time critical tasks.

This CL stops inline parsing and didReceiveParsedChunkFromBackgroundParser
is changed to always defer the processing.

Note that this also changes how HTMLPreloader is scheduled.
Before this CL, the preloader was not invoked when pump was done inline
from didReceiveParsedChunkFromBackgroundParser.
However, this CL ensures that the preloader is always invoked for all
confirmed tokens so that resource loading would not be delayed from
running slow synchronous scripts. This CL also delays the preloader
invocation until appcache is initialized, so that preloader requests would
respect it.

Rebaselines due to preloader being invoked more eagerly:
inspector/tracing/timeline-script-tag-1.html
fast/dom/HTMLLinkElement/link-and-subresource-test.html
permissionclient/image-permissions.html
tables/mozilla_expected_failures/bugs/bug128876.html

BUG=356292, 421300
Committed: https://src.chromium.org/viewvc/blink?view=rev&revision=188275

Committed: https://src.chromium.org/viewvc/blink?view=rev&revision=188369

[kouhei (in TOK), 2014-11-19 04:40:55]

Patchset #10 (id:180001) has been deleted

[kouhei (in TOK), 2014-11-19 04:42:53]

kouhei@chromium.org changed reviewers:
+ haraken@chromium.org, skyostil@google.com

[kouhei (in TOK), 2014-11-19 04:42:54]

Finally done with yak shaving HTMLPreloader bugs, so we can hopefully land this
now.

Would you take a look?

https://codereview.chromium.org/673603002/diff/160001/Source/core/html/parser...
File Source/core/html/parser/HTMLPreloadScanner.h (right):

https://codereview.chromium.org/673603002/diff/160001/Source/core/html/parser...
Source/core/html/parser/HTMLPreloadScanner.h:77: Checkpoint(const KURL&
predictedBaseElementURL, bool inStyle, bool isAppCacheEnabled, bool
isCSPEnabled, size_t templateCount)
Please ignore these changes in preloader. These are included in other CL.

[kouhei (in TOK), 2014-11-19 05:44:45]

kouhei@chromium.org changed reviewers:
+ tyoshino@chromium.org

[kouhei (in TOK), 2014-11-19 05:44:46]

tyoshino-san: Would you take a look?

[kouhei (in TOK), 2014-11-21 08:56:23]

Sorry. Please hold on review. This is passing on my local mac/linux, but trybots
seems unhappy.

[kouhei (in TOK), 2014-11-27 04:00:10]

Patchset #21 (id:420001) has been deleted

[kouhei (in TOK), 2014-12-02 05:09:51]

Yay! Bots are green finally.
I'm splitting the CLs so that we can land this incrementally.

[kouhei (in TOK), 2014-12-03 03:25:46]

Ready for review now. PTAL

[tyoshino (SeeGerritForStatus), 2014-12-04 06:33:33]

Please write something in the CL description about the appcache related change
in didReceiveParsedChunkFromBackgroundParser().

[tyoshino (SeeGerritForStatus), 2014-12-04 06:48:21]

lg. some comment writing requests

https://codereview.chromium.org/673603002/diff/580001/Source/core/html/parser...
File Source/core/html/parser/HTMLDocumentParser.cpp (right):

https://codereview.chromium.org/673603002/diff/580001/Source/core/html/parser...
Source/core/html/parser/HTMLDocumentParser.cpp:341:
ASSERT(m_queuedPreloads.isEmpty());
please write the reason that we're sure that m_queuedPreloads is already
consumed when we arrive here.

https://codereview.chromium.org/673603002/diff/580001/Source/core/html/parser...
Source/core/html/parser/HTMLDocumentParser.cpp:471: 
before this CL we were calling chunk->preloads.clear(). Now, you're calling
takeAndPreload(). If this is intentional change, please write the purpose in the
CL description.

[kouhei (in TOK), 2014-12-04 08:00:38]

On 2014/12/04 06:33:33, tyoshino wrote:
> Please write something in the CL description about the appcache related change
> in didReceiveParsedChunkFromBackgroundParser().

Done.

On 2014/12/04 06:48:21, tyoshino wrote:
> lg. some comment writing requests
> 
>
https://codereview.chromium.org/673603002/diff/580001/Source/core/html/parser...
> File Source/core/html/parser/HTMLDocumentParser.cpp (right):
> 
>
https://codereview.chromium.org/673603002/diff/580001/Source/core/html/parser...
> Source/core/html/parser/HTMLDocumentParser.cpp:341:
> ASSERT(m_queuedPreloads.isEmpty());
> please write the reason that we're sure that m_queuedPreloads is already
> consumed when we arrive here.
Done.

>
https://codereview.chromium.org/673603002/diff/580001/Source/core/html/parser...
> Source/core/html/parser/HTMLDocumentParser.cpp:471: 
> before this CL we were calling chunk->preloads.clear(). Now, you're calling
> takeAndPreload(). If this is intentional change, please write the purpose in
the
> CL description.
Done.

[tyoshino (SeeGerritForStatus), 2014-12-04 08:48:58]

lgtm

In CL description:

timeline-script-tag -> timeline-script-tag-1
image-permissions-expected.txt -> image-permissions.html

[kouhei (in TOK), 2014-12-05 04:02:16]

Thanks. Will send to CQ once https://codereview.chromium.org/774033002/ is
landed.

[kouhei (in TOK), 2015-01-07 07:11:10]

The CQ bit was checked by kouhei@chromium.org

[commit-bot: I haz the power, 2015-01-07 07:11:36]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/673603002/660001

[commit-bot: I haz the power, 2015-01-07 08:25:16]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2015-01-07 08:25:17]

Try jobs failed on following builders:
  linux_blink_rel on tryserver.blink
(http://build.chromium.org/p/tryserver.blink/builders/linux_blink_rel/builds/4...)

[kouhei (in TOK), 2015-01-09 04:28:26]

The CQ bit was checked by kouhei@chromium.org

[commit-bot: I haz the power, 2015-01-09 04:29:32]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/673603002/700001

[commit-bot: I haz the power, 2015-01-09 05:46:14]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2015-01-09 05:46:16]

Try jobs failed on following builders:
  linux_blink_rel on tryserver.blink
(http://build.chromium.org/p/tryserver.blink/builders/linux_blink_rel/builds/4...)

[kouhei (in TOK), 2015-01-13 05:55:35]

The CQ bit was checked by kouhei@chromium.org

[commit-bot: I haz the power, 2015-01-13 05:56:08]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/673603002/720001

[commit-bot: I haz the power, 2015-01-13 07:10:25]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2015-01-13 07:10:26]

Try jobs failed on following builders:
  linux_blink_rel on tryserver.blink
(http://build.chromium.org/p/tryserver.blink/builders/linux_blink_rel/builds/4...)

[kouhei (in TOK), 2015-01-13 07:43:41]

The CQ bit was checked by kouhei@chromium.org

[commit-bot: I haz the power, 2015-01-13 07:44:23]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/673603002/740001

[commit-bot: I haz the power, 2015-01-13 08:55:08]

Committed patchset #36 (id:740001) as
https://src.chromium.org/viewvc/blink?view=rev&revision=188275

[Zhenyao Mo, 2015-01-13 17:44:08]

A revert of this CL (patchset #36 id:740001) has been created in
https://codereview.chromium.org/847193002/ by zmo@chromium.org.

The reason for reverting is: This might break
DumpAccessibilityTreeTest.AccessibilityANoText on Android
http://build.chromium.org/p/chromium.webkit/builders/Android%20Tests%20%28dbg%29

[ RUN      ] DumpAccessibilityTreeTest.AccessibilityANoText
[WARNING:dns_config_service_posix.cc(293)] Failed to read DnsConfig.
[ERROR:shell_android.cc(69)] Not implemented reached in void
content::Shell::PlatformSetTitle(const string16&):
screen_orientation_lock_smoke.html
[ERROR:unix_domain_server_socket_posix.cc(106)] Not implemented reached in
virtual int net::UnixDomainServerSocket::GetLocalAddress(net::IPEndPoint*) const
[WARNING:proxy_service.cc(899)] PAC support disabled because there is no system
implementation
Testing: /storage/emulated/0/content/test/data/accessibility/html/a-no-text.html
../../content/browser/accessibility/dump_accessibility_browsertest_base.cc:202:
Failure
Value of: is_different
  Actual: true
Expected: false
[WARNING:channel.cc(136)] WriteMessage() while shutting down
[WARNING:channel.cc(136)] WriteMessage() while shutting down
* Line Expected
- ---- --------
     1 android.webkit.WebView focusable focused scrollable
     2     android.view.View
     3         android.view.View clickable focusable name='dest1'
     4         android.view.View clickable focusable name='dest2'
     5         android.view.View clickable focusable name='dest3'
     6         android.view.View clickable focusable name='dest4'
     7         android.view.View clickable focusable name='dest5'
     8         android.view.View clickable focusable name='dest6'
     9         android.view.View clickable focusable name='dest7'
    10         android.view.View clickable focusable name='dest.8'
*   11         android.view.View clickable focusable name='dest9'
*   12         android.view.View clickable focusable name='dest10'
    13 <-- End-of-file -->

Actual
------
android.webkit.WebView focusable focused scrollable
    android.view.View
        android.view.View clickable focusable name='dest1'
        android.view.View clickable focusable name='dest2'
        android.view.View clickable focusable name='dest3'
        android.view.View clickable focusable name='dest4'
        android.view.View clickable focusable name='dest5'
        android.view.View clickable focusable name='dest6'
        android.view.View clickable focusable name='dest7'
        android.view.View clickable focusable name='dest.8'
        android.view.View clickable focusable name='greenbox'
        android.view.View clickable focusable name='greenbox'
<-- End-of-file -->
[  FAILED  ] DumpAccessibilityTreeTest.AccessibilityANoText, where TypeParam = 
and GetParam() =  (601 ms)
[----------] 1 test from DumpAccessibilityTreeTest (604 ms total).

[kouhei (in TOK), 2015-01-14 07:43:47]

kouhei@chromium.org changed reviewers:
+ dmazzoni@chromium.org

[kouhei (in TOK), 2015-01-14 07:43:48]

On 2015/01/13 17:44:08, Zhenyao Mo wrote:
> A revert of this CL (patchset #36 id:740001) has been created in
> https://codereview.chromium.org/847193002/ by mailto:zmo@chromium.org.
> 
> The reason for reverting is: This might break
> DumpAccessibilityTreeTest.AccessibilityANoText on Android
>
http://build.chromium.org/p/chromium.webkit/builders/Android%20Tests%20%28dbg%29
> 
> [ RUN      ] DumpAccessibilityTreeTest.AccessibilityANoText
> [WARNING:dns_config_service_posix.cc(293)] Failed to read DnsConfig.
> [ERROR:shell_android.cc(69)] Not implemented reached in void
> content::Shell::PlatformSetTitle(const string16&):
> screen_orientation_lock_smoke.html
> [ERROR:unix_domain_server_socket_posix.cc(106)] Not implemented reached in
> virtual int net::UnixDomainServerSocket::GetLocalAddress(net::IPEndPoint*)
const
> [WARNING:proxy_service.cc(899)] PAC support disabled because there is no
system
> implementation
> Testing:
/storage/emulated/0/content/test/data/accessibility/html/a-no-text.html
>
../../content/browser/accessibility/dump_accessibility_browsertest_base.cc:202:
> Failure
> Value of: is_different
>   Actual: true
> Expected: false
> [WARNING:channel.cc(136)] WriteMessage() while shutting down
> [WARNING:channel.cc(136)] WriteMessage() while shutting down
> * Line Expected
> - ---- --------
>      1 android.webkit.WebView focusable focused scrollable
>      2     android.view.View
>      3         android.view.View clickable focusable name='dest1'
>      4         android.view.View clickable focusable name='dest2'
>      5         android.view.View clickable focusable name='dest3'
>      6         android.view.View clickable focusable name='dest4'
>      7         android.view.View clickable focusable name='dest5'
>      8         android.view.View clickable focusable name='dest6'
>      9         android.view.View clickable focusable name='dest7'
>     10         android.view.View clickable focusable name='dest.8'
> *   11         android.view.View clickable focusable name='dest9'
> *   12         android.view.View clickable focusable name='dest10'
>     13 <-- End-of-file -->
> 
> Actual
> ------
> android.webkit.WebView focusable focused scrollable
>     android.view.View
>         android.view.View clickable focusable name='dest1'
>         android.view.View clickable focusable name='dest2'
>         android.view.View clickable focusable name='dest3'
>         android.view.View clickable focusable name='dest4'
>         android.view.View clickable focusable name='dest5'
>         android.view.View clickable focusable name='dest6'
>         android.view.View clickable focusable name='dest7'
>         android.view.View clickable focusable name='dest.8'
>         android.view.View clickable focusable name='greenbox'
>         android.view.View clickable focusable name='greenbox'
> <-- End-of-file -->
> [  FAILED  ] DumpAccessibilityTreeTest.AccessibilityANoText, where TypeParam =

> and GetParam() =  (601 ms)
> [----------] 1 test from DumpAccessibilityTreeTest (604 ms total).

I think the new result is actually "correct", as it shows the filename as the
alttext as in:
http://www.w3.org/TR/1998/REC-html40-19980424/appendix/notes.html#altgen

It looks like the expectation file was changed from recent rebaseline:
https://codereview.chromium.org/805403003

dmazzoni: Can I land this patch and rebaseline the test result?

[dmazzoni, 2015-01-14 07:45:30]

On 2015/01/14 07:43:48, kouhei wrote:
> dmazzoni: Can I land this patch and rebaseline the test result?

Yes, okay to rebaseline!

[kouhei (in TOK), 2015-01-14 09:22:08]

On 2015/01/14 07:45:30, dmazzoni wrote:
> On 2015/01/14 07:43:48, kouhei wrote:
> > dmazzoni: Can I land this patch and rebaseline the test result?
> 
> Yes, okay to rebaseline!

Thanks! It looks like the test is already disabled as for now, so let me reland
this first in Blink and re-enable the test with the rebaseline in Chromium.

[kouhei (in TOK), 2015-01-14 09:26:41]

The CQ bit was checked by kouhei@chromium.org

[kouhei (in TOK), 2015-01-14 09:26:48]

The CQ bit was unchecked by kouhei@chromium.org

[kouhei (in TOK), 2015-01-14 09:28:39]

The CQ bit was checked by kouhei@chromium.org

[commit-bot: I haz the power, 2015-01-14 09:29:05]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/673603002/740001

[commit-bot: I haz the power, 2015-01-14 09:30:05]

Committed patchset #36 (id:740001) as
https://src.chromium.org/viewvc/blink?view=rev&revision=188369