Roll ICU to ICU-59-to-be (97b9daaf8)

Roll ICU to ICU-59-to-be (97b9daaf8)

It has fixes/work-arounds for:

1) PNaCl does not like inline assembly
2) Goma issue about the first 4 byte non-ASCII check
3) A few Chrome-specific changes 

BUG=

[jungshik at Google, 2017-03-13 23:54:48]

The CQ bit was checked by jshin@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2017-03-13 23:55:22]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-03-14 00:02:54]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-03-14 00:02:55]

Dry run: Try jobs failed on following builders:
  cast_shell_linux on master.tryserver.chromium.linux (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.linux/builders/cast_shell_linu...)
  linux_chromium_compile_dbg_ng on master.tryserver.chromium.linux (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_...)
  linux_chromium_rel_ng on master.tryserver.chromium.linux (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_...)
  mac_chromium_compile_dbg_ng on master.tryserver.chromium.mac (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.mac/builders/mac_chromium_comp...)
  mac_chromium_rel_ng on master.tryserver.chromium.mac (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.mac/builders/mac_chromium_rel_...)

[jungshik at Google, 2017-03-14 00:24:36]

I have a better luck locally than trybots here, but came across this issue. 

It looks like PNaCl does not like U_ALIASING_BARRIER.

[20503/46350] LINK newlib_pnacl/remoting_client_plugin_newlib.pexe
FAILED: newlib_pnacl/remoting_client_plugin_newlib.pexe
newlib_pnacl/exe.unstripped/remoting_client_plugin_newlib.pexe 
python "../../build/toolchain/gcc_link_wrapper.py"
--output="newlib_pnacl/remoting_client_plugin_newlib.pexe" --strip="../../nat
ive_client/toolchain/linux_x86/pnacl_newlib/bin/pnacl-finalize"
--unstripped-file="newlib_pnacl/exe.unstripped/remoting_client_p
lugin_newlib.pexe" --
../../native_client/toolchain/linux_x86/pnacl_newlib/bin/pnacl-clang++
-Wl,--fatal-warnings -Werror -Wl,-O
1 -Wl,--gc-sections -o
"newlib_pnacl/exe.unstripped/remoting_client_plugin_newlib.pexe"
-Wl,--start-group @"newlib_pnacl/remotin
g_client_plugin_newlib.pexe.rsp"  -Wl,--end-group   
Function uidna_nameToASCII_59 disallowed: inline assembly:    call void asm
sideeffect "", "rm,~{memory}"(i16* %name.asptr)
Function uidna_nameToASCII_59 disallowed: inline assembly:    call void asm
sideeffect "", "rm,~{memory}"(i16* %dest.asptr26)
LLVM ERROR: PNaCl ABI verification failed

[jungshik at Google, 2017-03-14 06:24:47]

My attempt to make PNaCl does not work (yet).

https://codereview.chromium.org/2747973002/diff/20001/url/url_canon_icu.cc
File url/url_canon_icu.cc (right):

https://codereview.chromium.org/2747973002/diff/20001/url/url_canon_icu.cc#ne...
url/url_canon_icu.cc:179: output->Append(icu::toUCharPtr(ascii.getBuffer()),
ascii.length());
PNaCl now complains here about inline assembly.

[jungshik at Google, 2017-03-14 06:26:31]

On 2017/03/14 06:24:47, jungshik at Google wrote:
> My attempt to make PNaCl does not work (yet).
> 
> https://codereview.chromium.org/2747973002/diff/20001/url/url_canon_icu.cc
> File url/url_canon_icu.cc (right):
> 
>
https://codereview.chromium.org/2747973002/diff/20001/url/url_canon_icu.cc#ne...
> url/url_canon_icu.cc:179: output->Append(icu::toUCharPtr(ascii.getBuffer()),
> ascii.length());
> PNaCl now complains here about inline assembly.

python "../../build/toolchain/gcc_link_wrapper.py"
--output="newlib_pnacl/remoting_client_plugin_newlib.pexe"
--strip="../../native_client/toolchain/linux_x86/pnacl_newlib/bin/pnacl-finalize"
--unstripped-file="newlib_pnacl/exe.unstripped/remoting_client_plugin_newlib.pexe"
-- ../../native_client/toolchain/linux_x86/pnacl_newlib/bin/pnacl-clang++
-Wl,--fatal-warnings -Werror -Wl,-O1 -Wl,--gc-sections -o
"newlib_pnacl/exe.unstripped/remoting_client_plugin_newlib.pexe"
-Wl,--start-group @"newlib_pnacl/remoting_client_plugin_newlib.pexe.rsp" 
-Wl,--end-group   
Function
_ZN3url12_GLOBAL__N_16DoHostIchEEvPKT_RKNS_9ComponentEPNS_12CanonOutputTIcEEPNS_13CanonHostInfoE
disallowed: inline assembly:    call void asm sideeffect "", "rm,~{memory}"(i16*
%.asptr347)
Function
_ZN3url12_GLOBAL__N_16DoHostIchEEvPKT_RKNS_9ComponentEPNS_12CanonOutputTIcEEPNS_13CanonHostInfoE
disallowed: inline assembly:    call void asm sideeffect "", "rm,~{memory}"(i16*
%.asptr355)
Function
_ZN3url12_GLOBAL__N_16DoHostIchEEvPKT_RKNS_9ComponentEPNS_12CanonOutputTIcEEPNS_13CanonHostInfoE
disallowed: inline assembly:    call void asm sideeffect "", "rm,~{memory}"(i16*
%retval.0.i.i.i.i.asptr)
LLVM ERROR: PNaCl ABI verification failed

[jungshik at Google, 2017-03-14 17:13:12]

On 2017/03/14 06:26:31, jungshik at Google wrote:
> On 2017/03/14 06:24:47, jungshik at Google wrote:
> > My attempt to make PNaCl does not work (yet).
> > 
> > https://codereview.chromium.org/2747973002/diff/20001/url/url_canon_icu.cc
> > File url/url_canon_icu.cc (right):
> > 
> >
>
https://codereview.chromium.org/2747973002/diff/20001/url/url_canon_icu.cc#ne...
> > url/url_canon_icu.cc:179: output->Append(icu::toUCharPtr(ascii.getBuffer()),
> > ascii.length());
> > PNaCl now complains here about inline assembly.
> 
> python "../../build/toolchain/gcc_link_wrapper.py"
> --output="newlib_pnacl/remoting_client_plugin_newlib.pexe"
>
--strip="../../native_client/toolchain/linux_x86/pnacl_newlib/bin/pnacl-finalize"
>
--unstripped-file="newlib_pnacl/exe.unstripped/remoting_client_plugin_newlib.pexe"
> -- ../../native_client/toolchain/linux_x86/pnacl_newlib/bin/pnacl-clang++
> -Wl,--fatal-warnings -Werror -Wl,-O1 -Wl,--gc-sections -o
> "newlib_pnacl/exe.unstripped/remoting_client_plugin_newlib.pexe"
> -Wl,--start-group @"newlib_pnacl/remoting_client_plugin_newlib.pexe.rsp" 
> -Wl,--end-group   
> Function
>
_ZN3url12_GLOBAL__N_16DoHostIchEEvPKT_RKNS_9ComponentEPNS_12CanonOutputTIcEEPNS_13CanonHostInfoE
> disallowed: inline assembly:    call void asm sideeffect "",
"rm,~{memory}"(i16*
> %.asptr347)
> Function
>
_ZN3url12_GLOBAL__N_16DoHostIchEEvPKT_RKNS_9ComponentEPNS_12CanonOutputTIcEEPNS_13CanonHostInfoE
> disallowed: inline assembly:    call void asm sideeffect "",
"rm,~{memory}"(i16*
> %.asptr355)
> Function
>
_ZN3url12_GLOBAL__N_16DoHostIchEEvPKT_RKNS_9ComponentEPNS_12CanonOutputTIcEEPNS_13CanonHostInfoE
> disallowed: inline assembly:    call void asm sideeffect "",
"rm,~{memory}"(i16*
> %retval.0.i.i.i.i.asptr)
> LLVM ERROR: PNaCl ABI verification failed

At this point, there are two alternatives.

1) Do not use Aliasing Barrier. I tested this out locally and PNaCl is happy.  

Markus, can you add an ICU configuration option for NO_U_ALIASING_BARRIER?  When
it's set, I want aliasing barrier not to be used even with gcc/clang. I'll file
a icu ticket on that. 

2) I'll try to change the call site to use C++ UTF-8 IDNA API. That way, I'm
hoping to avoid a code path requiring U_ALIASING_BARRIER. 
It's in my TODO list any way, but requires some code juggling in Chrome's url
canonicalization code.

[mscherer, 2017-03-14 17:23:05]

Description was changed from

==========
Roll ICU to ICU-59-to-be (2bccace)

BUG=
==========

to

==========
Roll ICU to ICU-59-to-be (2bccace)

BUG=
==========

[mscherer, 2017-03-14 17:26:51]

On 2017/03/14 17:13:12, jungshik at Google wrote:
> 1) Do not use Aliasing Barrier. I tested this out locally and PNaCl is happy. 

> 
> Markus, can you add an ICU configuration option for NO_U_ALIASING_BARRIER? 
When
> it's set, I want aliasing barrier not to be used even with gcc/clang. I'll
file
> a icu ticket on that. 

I would prefer to automatically turn off the aliasing barrier in PNaCl
compilation.
For example, don't use the aliasing barrier if defined(__pnacl__) or maybe
defined(__native_client__).
Please let me know which of these we should test for.

https://developer.chrome.com/native-client/reference/pnacl-c-cpp-language-sup...

[jungshik at Google, 2017-03-14 17:54:26]

On 2017/03/14 17:26:51, mscherer wrote:
> On 2017/03/14 17:13:12, jungshik at Google wrote:
> > 1) Do not use Aliasing Barrier. I tested this out locally and PNaCl is
happy. 
> 
> > 
> > Markus, can you add an ICU configuration option for NO_U_ALIASING_BARRIER? 
> When
> > it's set, I want aliasing barrier not to be used even with gcc/clang. I'll
> file
> > a icu ticket on that. 
> 
> I would prefer to automatically turn off the aliasing barrier in PNaCl
> compilation.
> For example, don't use the aliasing barrier if defined(__pnacl__) or maybe
> defined(__native_client__).
> Please let me know which of these we should test for.
> 
>
https://developer.chrome.com/native-client/reference/pnacl-c-cpp-language-sup...

Yup.  That's a good idea. Filed https://ssl.icu-project.org/trac/ticket/13032
Locally confirmed with a patch there that it works fine. 

Now at least on Linux, *all* targets in Chromium tree are built fine. The only
remaining issue is a goma mystery ! Unfortunately, it's a blocker.

[jungshik at Google, 2017-03-14 19:48:21]

The CQ bit was checked by jshin@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2017-03-14 19:49:08]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[jungshik at Google, 2017-03-14 19:49:39]

On 2017/03/14 17:54:26, jungshik at Google wrote:
> On 2017/03/14 17:26:51, mscherer wrote:
> > On 2017/03/14 17:13:12, jungshik at Google wrote:
> > > 1) Do not use Aliasing Barrier. I tested this out locally and PNaCl is
> happy. 
> > 
> > > 
> > > Markus, can you add an ICU configuration option for NO_U_ALIASING_BARRIER?

> > When
> > > it's set, I want aliasing barrier not to be used even with gcc/clang. I'll
> > file
> > > a icu ticket on that. 
> > 
> > I would prefer to automatically turn off the aliasing barrier in PNaCl
> > compilation.
> > For example, don't use the aliasing barrier if defined(__pnacl__) or maybe
> > defined(__native_client__).
> > Please let me know which of these we should test for.
> > 
> >
>
https://developer.chrome.com/native-client/reference/pnacl-c-cpp-language-sup...
> 
> Yup.  That's a good idea. Filed https://ssl.icu-project.org/trac/ticket/13032
> Locally confirmed with a patch there that it works fine. 
> 
> Now at least on Linux, *all* targets in Chromium tree are built fine. The only
> remaining issue is a goma mystery ! Unfortunately, it's a blocker.

The goma issue is also worked around ( b/36108313 ) by inserting a space right
before the copyright sign in all the 
ICU source files.

[jungshik at Google, 2017-03-14 19:56:46]

Description was changed from

==========
Roll ICU to ICU-59-to-be (2bccace)

BUG=
==========

to

==========
Roll ICU to ICU-59-to-be (97b9daaf8)

It has fixes/work-arounds for:

1) PNaCl does not like inline assembly
2) Goma issue about the first 4 byte non-ASCII check
3) A few Chrome-specific changes 

BUG=
==========

[commit-bot: I haz the power, 2017-03-14 20:03:59]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-03-14 20:04:00]

Dry run: Try jobs failed on following builders:
  ios-device-xcode-clang on master.tryserver.chromium.mac (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.mac/builders/ios-device-xcode-...)
  ios-simulator-xcode-clang on master.tryserver.chromium.mac (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.mac/builders/ios-simulator-xco...)
  win_chromium_compile_dbg_ng on master.tryserver.chromium.win (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_comp...)

[jungshik at Google, 2017-03-14 20:54:17]

The CQ bit was checked by jshin@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2017-03-14 20:54:49]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[jungshik at Google, 2017-03-14 20:58:40]

Windows bots (both clang and msvc) failed with the following link error. Well,
both clang and msvc bots use link.exe for linking, I believe. Hmm... why can't
it resolve toUCharPtr?  


[6488/29616] LINK(DLL) base_i18n.dll base_i18n.dll.lib base_i18n.dll.pdb
FAILED: base_i18n.dll base_i18n.dll.lib base_i18n.dll.pdb 
E:/b/depot_tools/python276_bin/python.exe
../../build/toolchain/win/tool_wrapper.py link-wrapper environment.x64 False
link.exe /nologo /IMPLIB:./base_i18n.dll.lib /DLL /OUT:./base_i18n.dll
/PDB:./base_i18n.dll.pdb @./base_i18n.dll.rsp
message_formatter.obj : error LNK2019: unresolved external symbol
"__declspec(dllimport) wchar_t const * __cdecl icu_59::toUCharPtr(char16_t const
*)" (__imp_?toUCharPtr@icu_59@@YAPEB_WPEB_S@Z) referenced in function "class
std::basic_string<wchar_t,struct std::char_traits<wchar_t>,class
std::allocator<wchar_t> > __cdecl base::i18n::UnicodeStringToString16(class
icu_59::UnicodeString const &)"
(?UnicodeStringToString16@i18n@base@@YA?AV?$basic_string@_WU?$char_traits@_W@std@@V?$allocator@_W@2@@std@@AEBVUnicodeString@icu_59@@@Z)

number_formatting.obj : error LNK2001: unresolved external symbol
"__declspec(dllimport) wchar_t const * __cdecl icu_59::toUCharPtr(char16_t const
*)" (__imp_?toUCharPtr@icu_59@@YAPEB_WPEB_S@Z)

time_formatting.obj : error LNK2001: unresolved external symbol
"__declspec(dllimport) wchar_t const * __cdecl icu_59::toUCharPtr(char16_t const
*)" (__imp_?toUCharPtr@icu_59@@YAPEB_WPEB_S@Z)

./base_i18n.dll : fatal error LNK1120: 1 unresolved externals

[commit-bot: I haz the power, 2017-03-14 21:34:06]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-03-14 21:34:07]

Dry run: Try jobs failed on following builders:
  win_clang on master.tryserver.chromium.win (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.win/builders/win_clang/builds/...)

[jungshik at Google, 2017-03-14 23:04:36]

On 2017/03/14 20:58:40, jungshik at Google wrote:
> Windows bots (both clang and msvc) failed with the following link error. Well,
> both clang and msvc bots use link.exe for linking, I believe. Hmm... why can't
> it resolve toUCharPtr?  

toUCharPtr was defined to be inline but with U_COMMON_API
https://chromium.googlesource.com/chromium/deps/icu/+/chromium/59_ucharptr2/s...

I'll try adding U_EXPORT2. If it does not work, I'll also try dropping
U_COMMON_API. 

 
> [6488/29616] LINK(DLL) base_i18n.dll base_i18n.dll.lib base_i18n.dll.pdb
> FAILED: base_i18n.dll base_i18n.dll.lib base_i18n.dll.pdb 
> E:/b/depot_tools/python276_bin/python.exe
> ../../build/toolchain/win/tool_wrapper.py link-wrapper environment.x64 False
> link.exe /nologo /IMPLIB:./base_i18n.dll.lib /DLL /OUT:./base_i18n.dll
> /PDB:./base_i18n.dll.pdb @./base_i18n.dll.rsp
> message_formatter.obj : error LNK2019: unresolved external symbol
> "__declspec(dllimport) wchar_t const * __cdecl icu_59::toUCharPtr(char16_t
const
> *)" (__imp_?toUCharPtr@icu_59@@YAPEB_WPEB_S@Z) referenced in function "class
> std::basic_string<wchar_t,struct std::char_traits<wchar_t>,class
> std::allocator<wchar_t> > __cdecl base::i18n::UnicodeStringToString16(class
> icu_59::UnicodeString const &)"
>
(?UnicodeStringToString16@i18n@base@@YA?AV?$basic_string@_WU?$char_traits@_W@std@@V?$allocator@_W@2@@std@@AEBVUnicodeString@icu_59@@@Z)
> 
> number_formatting.obj : error LNK2001: unresolved external symbol
> "__declspec(dllimport) wchar_t const * __cdecl icu_59::toUCharPtr(char16_t
const
> *)" (__imp_?toUCharPtr@icu_59@@YAPEB_WPEB_S@Z)
> 
> time_formatting.obj : error LNK2001: unresolved external symbol
> "__declspec(dllimport) wchar_t const * __cdecl icu_59::toUCharPtr(char16_t
const
> *)" (__imp_?toUCharPtr@icu_59@@YAPEB_WPEB_S@Z)
> 
> ./base_i18n.dll : fatal error LNK1120: 1 unresolved externals

[jungshik at Google, 2017-03-14 23:17:17]

The CQ bit was checked by jshin@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2017-03-14 23:18:04]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-03-14 23:43:51]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-03-14 23:43:52]

Dry run: Try jobs failed on following builders:
  win_chromium_compile_dbg_ng on master.tryserver.chromium.win (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_comp...)

[jungshik at Google, 2017-03-15 08:44:14]

The CQ bit was checked by jshin@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2017-03-15 08:44:24]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[jungshik at Google, 2017-03-15 08:46:08]

Dropping both U_COMMON_API and U_EXPORT2 for to(Old)UCharPtr makes Windows bots
happy.

[commit-bot: I haz the power, 2017-03-15 09:46:27]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-03-15 09:46:28]

Dry run: Try jobs failed on following builders:
  android_n5x_swarming_rel on master.tryserver.chromium.android (JOB_FAILED,
https://build.chromium.org/p/tryserver.chromium.android/builders/android_n5x_...)

[jungshik at Google, 2017-03-15 18:36:44]

On 2017/03/15 08:46:08, jungshik at Google wrote:
> Dropping both U_COMMON_API and U_EXPORT2 for to(Old)UCharPtr makes Windows
bots
> happy.

Red bots are red not because of a compile failure but because of test failures.
So, as far as builds are concerned, just using inline works on all platforms.