|
|
Created:
6 years, 8 months ago by Primiano Tucci (use gerrit) Modified:
6 years, 8 months ago Reviewers:
jam CC:
chromium-reviews, darin-cc_chromium.org, jam, rmcilroy, bulach, jochen (gone - plz use gerrit) Base URL:
svn://svn.chromium.org/chrome/trunk/src Visibility:
Public. |
DescriptionSplit content_shell dependencies for layout tests.
This CL is isolating the minimum dependencies required to make layout
tests work in content_shell without requiring it to depend on the entire
test_support_content_target.
crrev.com/11275277 has introduced a dependency on test_support_content
to cope with layout test. test_support_content, however, was intended
for unit/browser-tests and adding its dependency to content_shell
causes a lot of unnecessary sources to be linked in.
The linker --gc-sections has hidden the problem so far, but that doesn't
seem to be an option for arm64 (at least with the current toolchain).
BUG=159847, 354405
Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=261741
Patch Set 1 #
Total comments: 5
Patch Set 2 : Keep layout-test files in content_shell #Patch Set 3 : Break other dependency from test_support_content (needs re-review) #Patch Set 4 : Fix iOS #
Total comments: 16
Patch Set 5 : rename test_support_content_jni_headers #Patch Set 6 : Move mock_webclipboard_impl and break support vs laouttest dep #Patch Set 7 : Move WebKit test files to proper place #
Messages
Total messages: 28 (0 generated)
John, as we agreed yesterday during our offline chat, I'm isolating the dependencies for layout tests, so that content_shell doesn't have to depend anymore on content_test_support_content. Can you PTAL?
https://codereview.chromium.org/221993002/diff/1/content/content_tests.gypi File content/content_tests.gypi (right): https://codereview.chromium.org/221993002/diff/1/content/content_tests.gypi#n... content/content_tests.gypi:8: 'target_name': 'layout_tests_support_content', call this target layouttest_support_content to match the header and cc file below. also same for the android target, that should be layouttest_support_content_jni_headers https://codereview.chromium.org/221993002/diff/1/content/content_tests.gypi#n... content/content_tests.gypi:29: 'shell/browser/shell_layout_tests_android.cc', these two files in shell/ should be in content_shell's gypi file as before https://codereview.chromium.org/221993002/diff/1/content/content_tests.gypi#n... content/content_tests.gypi:1617: 'shell/android/java/src/org/chromium/content_shell/ShellLayoutTestUtils.java', ditto: this should stay in content_shell's gypi
https://codereview.chromium.org/221993002/diff/1/content/content_tests.gypi File content/content_tests.gypi (right): https://codereview.chromium.org/221993002/diff/1/content/content_tests.gypi#n... content/content_tests.gypi:8: 'target_name': 'layout_tests_support_content', On 2014/04/02 17:43:20, jam wrote: > call this target layouttest_support_content to match the header and cc file > below. also same for the android target, that should be > layouttest_support_content_jni_headers Done. https://codereview.chromium.org/221993002/diff/1/content/content_tests.gypi#n... content/content_tests.gypi:29: 'shell/browser/shell_layout_tests_android.cc', On 2014/04/02 17:43:20, jam wrote: > these two files in shell/ should be in content_shell's gypi file as before Done.
lgtm
John, I have to ask you to take another look to patchset 3. Sorry for the double review, but I realized only later that there was another dependency between content_shell and test_support_content. Essentially these dependencies were, IMHO, screwing the boundaries of content layers in component build, that's why I had to include these extra dependencies to content, like webkit_storage_browser. If my understanding is correct, what was happening was that: - some files in content_shell (or in the *tests) had dependencies on some component targets (for instance cc.so). - These component dependency were NOT expressed in the gyp files. - However, content_shell was always depending on test_support_content, which is a static_library (even in component build) - depending on the static test_support_content, in essence was causing many of these dependencies to be statically linked into content_shell, even in shared_library component build, essentially duplicating dozens of symbols, which were supposed to reside in their own .so libs, inside the content_shell exe. A concrete example of this: shell_message_filter.cc depends on webkit/browser/fileapi/isolated_context.h and isolated_context.cc is defined in webkit_storage_browser (webkit/storage_browser.gyp) From the gyp viewpoint, dependencies are apparently fine since: content_shell -> content_shell_lib -> content_common -> webkit_storage_browser However, when doing a component build, it ends-up in the following lib chain: content_shell(exe) -> content.so -> libwebkit_storage_browser.so (webkit_storage_browser is defined as 'type': '<(component)',) This is now wrong, since the code in the content_shell exe (specifically shell_message_filter.cc) has a direct dependency on libwebkit_storage_browser.so, which is not expressed in the gyp files. However, depending on test_support_content hides the problem, re-likning shell_message_filter.cc inside content_shell. Similarly content_shell depends on compositor because of shell/renderer/test_runner/TestPlugin.cpp, but the cc dependency is only in test_support_content. With this further patchset, I did an overall cleanup of the content targets, so now there is no more subtle dependencies between content_shell and the test files (with the only exceptions of layout_tests which need to be part of content_shell) https://codereview.chromium.org/221993002/diff/280001/content/content_tests.gypi File content/content_tests.gypi (right): https://codereview.chromium.org/221993002/diff/280001/content/content_tests.g... content/content_tests.gypi:987: 'test_support_content', This is needed because previously content_unittests were relying on content_shell to depend on test_support_content.
https://codereview.chromium.org/221993002/diff/280001/content/content_tests.gypi File content/content_tests.gypi (right): https://codereview.chromium.org/221993002/diff/280001/content/content_tests.g... content/content_tests.gypi:16: ['OS!="ios"', { this whole target, since its not used on ios, should be inside an OS != "ios" condition https://codereview.chromium.org/221993002/diff/280001/content/content_tests.g... content/content_tests.gypi:250: 'layouttest_support_content', this is odd to see, why does test_support_content need this? if they're files just for layout tests, only content_shell should need them https://codereview.chromium.org/221993002/diff/280001/content/content_tests.g... content/content_tests.gypi:1615: 'target_name': 'layouttest_support_content_jni_headers', why the rename? this is also used in content_browsertests right?
I'm attaching a drawing of the refactoring, which should make the situation more clear. Unfortunatelly the dependencies here are very entangled. https://docs.google.com/a/chromium.org/drawings/d/1KYpxfQKwBIzN-qvLqVvjWcivkB... https://codereview.chromium.org/221993002/diff/280001/content/content_tests.gypi File content/content_tests.gypi (right): https://codereview.chromium.org/221993002/diff/280001/content/content_tests.g... content/content_tests.gypi:16: ['OS!="ios"', { On 2014/04/03 17:40:20, jam wrote: > this whole target, since its not used on ios, should be inside an OS != "ios" > condition Hmm, if I nest the entire target inside a conditions section I get: gyp: Dependency '/src/chrome/src/content/content_shell_and_tests.gyp:layouttest_support_content#target' not found while trying to load target /src/chrome/src/content/content_shell_and_tests.gyp:content_shell_lib#target https://codereview.chromium.org/221993002/diff/280001/content/content_tests.g... content/content_tests.gypi:250: 'layouttest_support_content', On 2014/04/03 17:40:20, jam wrote: > this is odd to see, why does test_support_content need this? if they're files > just for layout tests, only content_shell should need them See the attached drawing. Essentially some of the 3 files being moved from test_support_content to layouttest_support_content (e.g, mock_webclipboard_impl.cc) are needed by both layouttests and contentshell itself. But they can't be recompiled twice and linked twice. Hence the extra dependency here. Furthermore, webkit_unit_tests need test_support_content (cause they depend on mock_webclipboard_impl.cc) but they don't depend on content-shell. That's why I need this (see the drawing) https://codereview.chromium.org/221993002/diff/280001/content/content_tests.g... content/content_tests.gypi:1615: 'target_name': 'layouttest_support_content_jni_headers', On 2014/04/03 17:40:20, jam wrote: > why the rename? this is also used in content_browsertests right? Hmm, not really. THis is only needed by nested_message_pump_android.cc, which is for layouttest_support_content
https://codereview.chromium.org/221993002/diff/280001/content/content_tests.gypi File content/content_tests.gypi (right): https://codereview.chromium.org/221993002/diff/280001/content/content_tests.g... content/content_tests.gypi:16: ['OS!="ios"', { On 2014/04/03 19:13:44, Primiano Tucci wrote: > On 2014/04/03 17:40:20, jam wrote: > > this whole target, since its not used on ios, should be inside an OS != "ios" > > condition > > Hmm, if I nest the entire target inside a conditions section I get: > gyp: Dependency > '/src/chrome/src/content/content_shell_and_tests.gyp:layouttest_support_content#target' > not found while trying to load target > /src/chrome/src/content/content_shell_and_tests.gyp:content_shell_lib#target hmm, are these targets built on ios? they shouldn't be https://codereview.chromium.org/221993002/diff/280001/content/content_tests.g... content/content_tests.gypi:250: 'layouttest_support_content', On 2014/04/03 19:13:44, Primiano Tucci wrote: > On 2014/04/03 17:40:20, jam wrote: > > this is odd to see, why does test_support_content need this? if they're files > > just for layout tests, only content_shell should need them > > See the attached drawing. > Essentially some of the 3 files being moved from test_support_content to > layouttest_support_content (e.g, mock_webclipboard_impl.cc) are needed by both > layouttests and contentshell itself. But they can't be recompiled twice and > linked twice. Hence the extra dependency here. > > Furthermore, webkit_unit_tests need test_support_content (cause they depend on > mock_webclipboard_impl.cc) but they don't depend on content-shell. That's why I > need this (see the drawing) let's look at each of the cc files one by one test/mock_webclipboard_impl.cc -this is used by content_shell and the unit tests. so we can add that cc file in the layout test target, and in this target test_media_stream_client.cc -TestMediaStreamClient is only used by layouttest_support.cc, so it should be fine to keep in the layout test target only test_video_frame_provider.cc -TestVideoFrameProvider is only used by test_media_stream_client.cc, so it can go to the layout test target https://codereview.chromium.org/221993002/diff/280001/content/content_tests.g... content/content_tests.gypi:1615: 'target_name': 'layouttest_support_content_jni_headers', On 2014/04/03 19:13:44, Primiano Tucci wrote: > On 2014/04/03 17:40:20, jam wrote: > > why the rename? this is also used in content_browsertests right? > > Hmm, not really. THis is only needed by nested_message_pump_android.cc, which is > for layouttest_support_content hmmm, this s used by browser_tests afaik because they need a nested message loop on Android to run the tests. NestedMessagePumpAndroid is used by content_test_launcher.cc, which is in content_browsertests
https://codereview.chromium.org/221993002/diff/280001/content/content_tests.gypi File content/content_tests.gypi (right): https://codereview.chromium.org/221993002/diff/280001/content/content_tests.g... content/content_tests.gypi:16: ['OS!="ios"', { On 2014/04/03 19:39:33, jam wrote: > On 2014/04/03 19:13:44, Primiano Tucci wrote: > > On 2014/04/03 17:40:20, jam wrote: > > > this whole target, since its not used on ios, should be inside an OS != > "ios" > > > condition > > > > Hmm, if I nest the entire target inside a conditions section I get: > > gyp: Dependency > > > '/src/chrome/src/content/content_shell_and_tests.gyp:layouttest_support_content#target' > > not found while trying to load target > > /src/chrome/src/content/content_shell_and_tests.gyp:content_shell_lib#target > > hmm, are these targets built on ios? they shouldn't be On 2014/04/03 19:39:33, jam wrote: > On 2014/04/03 19:13:44, Primiano Tucci wrote: > > On 2014/04/03 17:40:20, jam wrote: > > > this whole target, since its not used on ios, should be inside an OS != > "ios" > > > condition > > > > Hmm, if I nest the entire target inside a conditions section I get: > > gyp: Dependency > > > '/src/chrome/src/content/content_shell_and_tests.gyp:layouttest_support_content#target' > > not found while trying to load target > > /src/chrome/src/content/content_shell_and_tests.gyp:content_shell_lib#target > > hmm, are these targets built on ios? they shouldn't be all.gyp is including content_shell_and_tests.gyp:* (which in turn includes this file). Essentially, only a small part of test_support_content is intended to be built for iOS (if you look at line 190, it has an horrible rule which excludes all .cc files and then reincludes only a dozen on iOS ..['exclude', '\\.(cc|mm)$']). This layouttest target, however, is not required for iOS. But unfortunately is implicitly included by all.gyp (and I don't feel adventuring in refactoring also all.gyp at this point, at least not in this CL). I agree with you that ideally the entire target should be masked by target!=ios, but unfortunatelly GYP is not liking that (and I feel I reached the borders of my GYP knowledge... any hints?) https://codereview.chromium.org/221993002/diff/280001/content/content_tests.g... content/content_tests.gypi:250: 'layouttest_support_content', On 2014/04/03 19:39:33, jam wrote: > On 2014/04/03 19:13:44, Primiano Tucci wrote: > > On 2014/04/03 17:40:20, jam wrote: > > > this is odd to see, why does test_support_content need this? if they're > files > > > just for layout tests, only content_shell should need them > > > > See the attached drawing. > > Essentially some of the 3 files being moved from test_support_content to > > layouttest_support_content (e.g, mock_webclipboard_impl.cc) are needed by both > > layouttests and contentshell itself. But they can't be recompiled twice and > > linked twice. Hence the extra dependency here. > > > > Furthermore, webkit_unit_tests need test_support_content (cause they depend on > > mock_webclipboard_impl.cc) but they don't depend on content-shell. That's why > I > > need this (see the drawing) > > let's look at each of the cc files one by one > > test/mock_webclipboard_impl.cc > -this is used by content_shell and the unit tests. so we can add that cc file in > the layout test target, and in this target Unfortunately we can't. If I include the .cc file both in layouttest and in this target, content_shell_lib will fail to link because the symbols in webclipboard.cc will end'up being defined twice. It would work if I could add the mock_webclipboard file to both AND remove this dependency on line 250. However, the game here is more complicated because: webkit_unit_tests (via webkit_test_runner.cc) depend on both: - test_support_content - layouttest_support So, if I remove the dep on line 250, I'd break the functional dependency between webkit_test_runner.cc and layouttest_support.cc (Sorry I missed that in the drawing) Agree for test_media_stream_client and tesT_video_frame_provider (but they are already in the right place) https://codereview.chromium.org/221993002/diff/280001/content/content_tests.g... content/content_tests.gypi:1615: 'target_name': 'layouttest_support_content_jni_headers', On 2014/04/03 19:39:33, jam wrote: > On 2014/04/03 19:13:44, Primiano Tucci wrote: > > On 2014/04/03 17:40:20, jam wrote: > > > why the rename? this is also used in content_browsertests right? > > > > Hmm, not really. THis is only needed by nested_message_pump_android.cc, which > is > > for layouttest_support_content > > hmmm, this s used by browser_tests afaik because they need a nested message loop > on Android to run the tests. NestedMessagePumpAndroid is used by > content_test_launcher.cc, which is in content_browsertests Ahh sorry, you're right here. NestedMessagePumpAndroid is used also by other parties, not only layouttest. Will rename.
https://codereview.chromium.org/221993002/diff/280001/content/content_tests.gypi File content/content_tests.gypi (right): https://codereview.chromium.org/221993002/diff/280001/content/content_tests.g... content/content_tests.gypi:16: ['OS!="ios"', { On 2014/04/03 19:52:59, Primiano Tucci wrote: > On 2014/04/03 19:39:33, jam wrote: > > On 2014/04/03 19:13:44, Primiano Tucci wrote: > > > On 2014/04/03 17:40:20, jam wrote: > > > > this whole target, since its not used on ios, should be inside an OS != > > "ios" > > > > condition > > > > > > Hmm, if I nest the entire target inside a conditions section I get: > > > gyp: Dependency > > > > > > '/src/chrome/src/content/content_shell_and_tests.gyp:layouttest_support_content#target' > > > not found while trying to load target > > > /src/chrome/src/content/content_shell_and_tests.gyp:content_shell_lib#target > > > > hmm, are these targets built on ios? they shouldn't be > On 2014/04/03 19:39:33, jam wrote: > > On 2014/04/03 19:13:44, Primiano Tucci wrote: > > > On 2014/04/03 17:40:20, jam wrote: > > > > this whole target, since its not used on ios, should be inside an OS != > > "ios" > > > > condition > > > > > > Hmm, if I nest the entire target inside a conditions section I get: > > > gyp: Dependency > > > > > > '/src/chrome/src/content/content_shell_and_tests.gyp:layouttest_support_content#target' > > > not found while trying to load target > > > /src/chrome/src/content/content_shell_and_tests.gyp:content_shell_lib#target > > > > hmm, are these targets built on ios? they shouldn't be > > all.gyp is including content_shell_and_tests.gyp:* (which in turn includes this > file). Essentially, only a small part of test_support_content is intended to be > built for iOS (if you look at line 190, it has an horrible rule which excludes > all .cc files and then reincludes only a dozen on iOS ..['exclude', > '\\.(cc|mm)$']). > > This layouttest target, however, is not required for iOS. But unfortunately is > implicitly included by all.gyp (and I don't feel adventuring in refactoring also > all.gyp at this point, at least not in this CL). > I agree with you that ideally the entire target should be masked by target!=ios, > but unfortunatelly GYP is not liking that (and I feel I reached the borders of > my GYP knowledge... any hints?) ugh, it's unfortunate that these other targets are built on ios. ok, that's not your fault, so this is fine to workaround that for now. i agree fixing that is outside the scope of this cl. https://codereview.chromium.org/221993002/diff/280001/content/content_tests.g... content/content_tests.gypi:250: 'layouttest_support_content', On 2014/04/03 19:52:59, Primiano Tucci wrote: > On 2014/04/03 19:39:33, jam wrote: > > On 2014/04/03 19:13:44, Primiano Tucci wrote: > > > On 2014/04/03 17:40:20, jam wrote: > > > > this is odd to see, why does test_support_content need this? if they're > > files > > > > just for layout tests, only content_shell should need them > > > > > > See the attached drawing. > > > Essentially some of the 3 files being moved from test_support_content to > > > layouttest_support_content (e.g, mock_webclipboard_impl.cc) are needed by > both > > > layouttests and contentshell itself. But they can't be recompiled twice and > > > linked twice. Hence the extra dependency here. > > > > > > Furthermore, webkit_unit_tests need test_support_content (cause they depend > on > > > mock_webclipboard_impl.cc) but they don't depend on content-shell. That's > why > > I > > > need this (see the drawing) > > > > let's look at each of the cc files one by one > > > > test/mock_webclipboard_impl.cc > > -this is used by content_shell and the unit tests. so we can add that cc file > in > > the layout test target, and in this target > > Unfortunately we can't. If I include the .cc file both in layouttest and in this > target, content_shell_lib will fail to link because the symbols in > webclipboard.cc will end'up being defined twice. > It would work if I could add the mock_webclipboard file to both AND remove this > dependency on line 250. huh? why would it be defined twice? content_shell_lib doesn't depend on test_support_content? > > However, the game here is more complicated because: > webkit_unit_tests (via webkit_test_runner.cc) depend on both: > - test_support_content > - layouttest_support why does webkit_unit_tests need layouttest_support? seems that layouttest_support should only contain stuff that content_shell needs for running layout tests > > So, if I remove the dep on line 250, I'd break the functional dependency between > webkit_test_runner.cc and layouttest_support.cc > (Sorry I missed that in the drawing) > > Agree for test_media_stream_client and tesT_video_frame_provider (but they are > already in the right place)
https://codereview.chromium.org/221993002/diff/280001/content/content_tests.gypi File content/content_tests.gypi (right): https://codereview.chromium.org/221993002/diff/280001/content/content_tests.g... content/content_tests.gypi:250: 'layouttest_support_content', On 2014/04/03 20:08:29, jam wrote: > On 2014/04/03 19:52:59, Primiano Tucci wrote: > > On 2014/04/03 19:39:33, jam wrote: > > > On 2014/04/03 19:13:44, Primiano Tucci wrote: > > > > On 2014/04/03 17:40:20, jam wrote: > > > > > this is odd to see, why does test_support_content need this? if they're > > > files > > > > > just for layout tests, only content_shell should need them > > > > > > > > See the attached drawing. > > > > Essentially some of the 3 files being moved from test_support_content to > > > > layouttest_support_content (e.g, mock_webclipboard_impl.cc) are needed by > > both > > > > layouttests and contentshell itself. But they can't be recompiled twice > and > > > > linked twice. Hence the extra dependency here. > > > > > > > > Furthermore, webkit_unit_tests need test_support_content (cause they > depend > > on > > > > mock_webclipboard_impl.cc) but they don't depend on content-shell. That's > > why > > > I > > > > need this (see the drawing) > > > > > > let's look at each of the cc files one by one > > > > > > test/mock_webclipboard_impl.cc > > > -this is used by content_shell and the unit tests. so we can add that cc > file > > in > > > the layout test target, and in this target > > > > Unfortunately we can't. If I include the .cc file both in layouttest and in > this > > target, content_shell_lib will fail to link because the symbols in > > webclipboard.cc will end'up being defined twice. > > It would work if I could add the mock_webclipboard file to both AND remove > this > > dependency on line 250. > > huh? why would it be defined twice? content_shell_lib doesn't depend on > test_support_content? Twice: in this target (test_support_content) and layouttest_support_content (according to the suggestion in your comment) > > > > > However, the game here is more complicated because: > > webkit_unit_tests (via webkit_test_runner.cc) depend on both: > > - test_support_content > > - layouttest_support > > why does webkit_unit_tests need layouttest_support? seems that > layouttest_support should only contain stuff that content_shell needs for > running layout tests Unfortunatelly there is yet another dependency between content_shell and mock_webclipboard_impl.cc via shell_content_renderer_client.cc. I dont' see other solutions then keeping also shell_content_renderer_client.cc into layout_test... (hence keeping this dependency here between test_support_content and layouttest_support_content)
lgtm, thanks!
The CQ bit was checked by primiano@chromium.org
CQ is trying da patch. Follow status at https://chromium-status.appspot.com/cq/primiano@chromium.org/221993002/320001
The CQ bit was unchecked by commit-bot@chromium.org
Try jobs failed on following builders: tryserver.chromium on linux_chromium_chromeos_rel
The CQ bit was checked by primiano@chromium.org
CQ is trying da patch. Follow status at https://chromium-status.appspot.com/cq/primiano@chromium.org/221993002/340001
The CQ bit was unchecked by commit-bot@chromium.org
Try jobs failed on following builders: tryserver.chromium on win_chromium_compile_dbg
The CQ bit was checked by primiano@chromium.org
CQ is trying da patch. Follow status at https://chromium-status.appspot.com/cq/primiano@chromium.org/221993002/360001
The CQ bit was unchecked by commit-bot@chromium.org
Try jobs failed on following builders: tryserver.chromium on win_chromium_compile_dbg
The CQ bit was checked by primiano@chromium.org
CQ is trying da patch. Follow status at https://chromium-status.appspot.com/cq/primiano@chromium.org/221993002/380001
Message was sent while issue was closed.
Change committed as 261741
Message was sent while issue was closed.
A revert of this CL has been created in https://codereview.chromium.org/226123003/ by pfeldman@chromium.org. The reason for reverting is: Speculative rollout due to Linux DEPS builder compilation failure: http://build.chromium.org/p/chromium.webkit/builders/WebKit%20Linux%20%28deps.... |