Chromium Code Reviews
|
|
Chromium Code Reviews
Issue
2857743004:
Workaround for UCRT deadlock between gpu's main and watchdog threads (Closed)
|
Created:
3 years, 7 months ago by mmarkowicz Modified:
3 years, 7 months ago CC:
chromium-reviews, piman+watch_chromium.org, piman Target Ref:
refs/heads/master Project:
chromium Visibility:
Public. |
DescriptionWorkaround for UCRT deadlock between gpu's main and watchdog threads
Deadlock is likely caused by UCRT lib bug that can be reproduced with
low-frequency in Opera Browser on Windows 7. Deadlock will permanently freeze
gpu process (watchdog will not intervene as, ironically, it's involved in a
deadlock).
BUG=718388
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel
Review-Url: https://codereview.chromium.org/2857743004
Cr-Commit-Position: refs/heads/master@{#469617}
Committed: https://chromium.googlesource.com/chromium/src/+/abecaf4066bdffab7af3e26102d2e2644835d8a9
Patch Set 1 #
Total comments: 1
Messages
Total messages: 14 (8 generated)
Description was changed from ========== Workaround for UCRT deadlock between gpu's main and watchdog threads Deadlock is likely caused by UCRT lib bug that can be reproduced with low-frequency in Opera Browser on Windows 7. Deadlock will permanently freeze gpu process (watchdog will not intervene as, ironically, it's involved in a deadlock). BUG=718388 ========== to ========== Workaround for UCRT deadlock between gpu's main and watchdog threads Deadlock is likely caused by UCRT lib bug that can be reproduced with low-frequency in Opera Browser on Windows 7. Deadlock will permanently freeze gpu process (watchdog will not intervene as, ironically, it's involved in a deadlock). BUG=718388 CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel ==========
Description was changed from ========== Workaround for UCRT deadlock between gpu's main and watchdog threads Deadlock is likely caused by UCRT lib bug that can be reproduced with low-frequency in Opera Browser on Windows 7. Deadlock will permanently freeze gpu process (watchdog will not intervene as, ironically, it's involved in a deadlock). BUG=718388 CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel ========== to ========== Workaround for UCRT deadlock between gpu's main and watchdog threads Deadlock is likely caused by UCRT lib bug that can be reproduced with low-frequency in Opera Browser on Windows 7. Deadlock will permanently freeze gpu process (watchdog will not intervene as, ironically, it's involved in a deadlock). BUG=718388 CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel ==========
mmarkowicz@opera.com changed reviewers: + kbr@chromium.org
We couldn't reproduce this freeze in Chrome, but in Opera it isn't easy to reproduce it either. Solution (workaround) seems to make sense for any chromium-based product. Here's chromium issue that looks somewhat similar to what we've seen in opera: https://bugs.chromium.org/p/chromium/issues/detail?id=701103. Solution will introduce a slight delay in gpu process initialization (on my machine it was measured at around 0.1ms).
kbr@chromium.org changed reviewers: + jbauman@chromium.org
Thanks for measuring the startup time impact. The 0.1 ms delay sounds acceptable to me, but I'd like to defer to jbauman@ for review in case he can think of another way to work around this. https://codereview.chromium.org/2857743004/diff/1/gpu/ipc/service/gpu_init.cc File gpu/ipc/service/gpu_init.cc (right): https://codereview.chromium.org/2857743004/diff/1/gpu/ipc/service/gpu_init.cc... gpu/ipc/service/gpu_init.cc:174: // but we might have come accross a not-yet-covered scenario. Please refer to crbug.com/718388 in the comment, in case this code's later changed in bulk and git blame doesn't find it easily.
lgtm. Do you know what's calling setlocale? Something in the driver?
On 2017/05/04 18:15:52, jbauman wrote:
> lgtm. Do you know what's calling setlocale? Something in the driver?
It's called while in ostringstream constructor that's running somewhere in the
context of gl::DriverEGL::GetClientExtensions(). It's not really anything
specific to gpu process and, seems to me, you can imagine similar scenarios
anytime you start a new thread. Here's the main thread callstack:
[External Code]
libGLESv2.dll!try_get_function(const `anonymous-namespace'::function_id id,
const char * const name, const `anonymous-namespace'::module_id * const
first_module_id, const `anonymous-namespace'::module_id * const last_module_id)
Line 346 C++
[Inline Frame] libGLESv2.dll!try_get_AreFileApisANSI() Line 396 C++
libGLESv2.dll!__acrt_eagerly_load_locale_apis() Line 830 C++
libGLESv2.dll!setlocale(int _category, const char * _locale) Line 38 C++
[External Code]
libGLESv2.dll!std::_Locinfo::_Locinfo(const char * _Pch) Line 79 C++
libGLESv2.dll!std::ctype<char>::_Getcat(const std::locale::facet * * _Ppf,
const std::locale * _Ploc) Line 2432 C++
libGLESv2.dll!std::use_facet<std::ctype<char> >(const std::locale & _Loc) Line
517 C++
libGLESv2.dll!std::basic_ios<char,std::char_traits<char> >::widen(char _Byte)
Line 130 C++
[Inline Frame] libGLESv2.dll!std::basic_ios<char,std::char_traits<char>
>::init(std::basic_streambuf<char,std::char_traits<char> > *) Line 173 C++
libGLESv2.dll!std::basic_ostream<char,std::char_traits<char>
>::basic_ostream<char,std::char_traits<char>
>(std::basic_streambuf<char,std::char_traits<char> > * _Strbuf, bool _Isstd)
Line 54 C++
libGLESv2.dll!std::basic_ostringstream<char,std::char_traits<char>,std::allocator<char>
>::basic_ostringstream<char,std::char_traits<char>,std::allocator<char> >(int
_Mode) Line 494 C++
libGLESv2.dll!egl::GenerateExtensionsString<egl::ClientExtensions>(const
egl::ClientExtensions & extensions) Line 926 C++
libGLESv2.dll!egl::Display::getClientExtensionString() Line 938 C++
libGLESv2.dll!egl::QueryString(void * dpy, int name) Line 152 C++
gl::RealEGLApi::eglQueryStringFn(void * dpy, int name) Line 94 C++
gl::DriverEGL::GetClientExtensions() Line 53 C++
gl::DriverEGL::InitializeClientExtensionBindings() Line 127 C++
gl::GLSurfaceEGL::InitializeOneOff(HDC__ * native_display) Line 515 C++
gl::init::InitializeGLOneOffPlatform() Line 239 C++
gl::init::InitializeGLOneOffImplementation(gl::GLImplementation impl, bool
fallback_to_software_gl, bool gpu_service_logging, bool disable_gl_drawing) Line
68 C++
gl::init::InitializeGLOneOff() Line 59 C++
gpu::GpuInit::InitializeAndStartSandbox(const base::CommandLine &
command_line) Line 198 C++
content::GpuMain(const content::MainFunctionParams & parameters) Line 270 C++
Stack in the watchdog thread:
[External Code]
[Inline Frame]
libGLESv2.dll!__acrt_lock_and_call::__l2::<lambda_aa87e3671a710a21b5dc78c0bdf72e11>::operator()()
Line 909 C++
libGLESv2.dll!__crt_seh_guarded_call<void>::operator() Line 221 C++
libGLESv2.dll!construct_ptd_array(__acrt_ptd * const ptd) Line 127 C++
libGLESv2.dll!__acrt_getptd_noexit() Line 278 C++
libGLESv2.dll!__acrt_thread_attach() Line 300 C++
[External Code]
The CQ bit was checked by mmarkowicz@opera.com
CQ is trying da patch. Follow status at: https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...
CQ is committing da patch.
Bot data: {"patchset_id": 1, "attempt_start_ts": 1493975712760770, "parent_rev":
"023275881c6e064bfd5a87e29e75b90070280603", "commit_rev":
"abecaf4066bdffab7af3e26102d2e2644835d8a9"}
CQ is committing da patch.
Bot data: {"patchset_id": 1, "attempt_start_ts": 1493975712760770, "parent_rev":
"023275881c6e064bfd5a87e29e75b90070280603", "commit_rev":
"abecaf4066bdffab7af3e26102d2e2644835d8a9"}
Message was sent while issue was closed.
Description was changed from ========== Workaround for UCRT deadlock between gpu's main and watchdog threads Deadlock is likely caused by UCRT lib bug that can be reproduced with low-frequency in Opera Browser on Windows 7. Deadlock will permanently freeze gpu process (watchdog will not intervene as, ironically, it's involved in a deadlock). BUG=718388 CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel ========== to ========== Workaround for UCRT deadlock between gpu's main and watchdog threads Deadlock is likely caused by UCRT lib bug that can be reproduced with low-frequency in Opera Browser on Windows 7. Deadlock will permanently freeze gpu process (watchdog will not intervene as, ironically, it's involved in a deadlock). BUG=718388 CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel Review-Url: https://codereview.chromium.org/2857743004 Cr-Commit-Position: refs/heads/master@{#469617} Committed: https://chromium.googlesource.com/chromium/src/+/abecaf4066bdffab7af3e26102d2... ==========
Message was sent while issue was closed.
Committed patchset #1 (id:1) as https://chromium.googlesource.com/chromium/src/+/abecaf4066bdffab7af3e26102d2... |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
