Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(27)

Issue 839713003: ICU update to 54 step 3 (Closed)

Created:
5 years, 11 months ago by jungshik at Google
Modified:
5 years, 11 months ago
Reviewers:
Mark Mentovai, jsbell
CC:
chromium-reviews
Base URL:
https://chromium.googlesource.com/chromium/deps/icu.git@master
Target Ref:
refs/heads/master
Visibility:
Public.

Description

ICU update to 54 step 3 A. Converter update per HTML encoding spec along with changes in the encoding name alias table. B. Remove all the codes for converters Blink and Chromium do not need (SCSU, Lotus, ISO-2022-xx other than JP, BOCU, UTF-7, etc). This is reapplying the following CLs (that we used for ICU 52.1) to ICU 54.1 : https://codereview.chromium.org/598383002 https://codereview.chromium.org/654153002 We have two upstream bugs filed for A and B above: http://www.icu-project.org/trac/ticket/11296 http://www.icu-project.org/trac/ticket/10303 In addiition to A and B, we unified Big5 and Big5-HKSCS per the encoding spec (bug 277868). That also includes properly supporting the four 2-character sequences ( see http://crbug.com/277868#c3 ). big5_gen.sh deviates from the current spec to work around a bug in the spec. (see https://www.w3.org/Bugs/Public/show_bug.cgi?id=27878) Moreover, ucmlocal.mk is added to list only encodings we want to support. Also, tighten the state table for windows-946-2000.ucm that we use for EUC-KR for now. And, drop 'base' map for windows-{936,949}-2000.ucm. Finally, add euc-kr-html.ucm along with scripts/euckr_gen.sh, but it is not yet used pending the resolution of bug 450312. Data size checkpoint: 20,566,864 bytes (the original ICU 54=25,343,024) BUG=277868, 428145, 450312 TEST=net_unittests --gtest_filter="*ilenameUtil*" TEST=base_unittests --gtest_filter="*Conv*" TEST=browser_tests --gtest_filter="*ncoding*" TEST=Blink: fast/encoding/* R=jsbell@chromium.org, mark@chromium.org Committed: https://chromium.googlesource.com/chromium/deps/icu/+/afd723ba594ba3e0d00cc10bc8ca8dbae57bb04a

Patch Set 1 #

Patch Set 2 : update the description #

Patch Set 3 : add euc-kr-html.ucm (not yet used) #

Total comments: 4

Patch Set 4 : update windows-949/936 and euc-kr #

Patch Set 5 : address jsbell's comments #

Patch Set 6 : fix big5 mapping #

Unified diffs Side-by-side diffs Delta from patch set Stats (+13641 lines, -18188 lines) Patch
M README.chromium View 1 2 3 4 5 3 chunks +13 lines, -9 lines 0 comments Download
M patches/uconv.patch View 1 2 3 4 5 55 chunks +284 lines, -285 lines 0 comments Download
A scripts/big5_gen.sh View 1 2 3 4 5 1 chunk +89 lines, -0 lines 0 comments Download
M scripts/eucjp_gen.sh View 1 2 3 4 5 1 chunk +1 line, -1 line 0 comments Download
A scripts/euckr_gen.sh View 1 2 3 4 5 1 chunk +67 lines, -0 lines 0 comments Download
M scripts/ibm866_gen.sh View 1 2 3 4 5 1 chunk +1 line, -1 line 0 comments Download
M scripts/single_byte_gen.sh View 1 2 3 4 5 1 chunk +1 line, -1 line 0 comments Download
M scripts/sjis_gen.sh View 1 2 3 4 5 1 chunk +1 line, -1 line 0 comments Download
M source/common/ucnv2022.cpp View 1 2 3 4 5 33 chunks +65 lines, -10 lines 0 comments Download
M source/common/ucnv_bld.cpp View 1 2 3 4 5 4 chunks +31 lines, -6 lines 0 comments Download
M source/common/ucnv_cnv.h View 1 2 3 4 5 1 chunk +5 lines, -1 line 0 comments Download
M source/common/ucnv_ct.c View 1 2 3 4 5 1 chunk +1 line, -1 line 0 comments Download
M source/common/ucnv_lmb.c View 1 2 3 4 5 1 chunk +1 line, -1 line 0 comments Download
M source/common/ucnv_u7.c View 1 2 3 4 5 1 chunk +1 line, -1 line 0 comments Download
M source/common/ucnv_u8.c View 1 2 3 4 5 5 chunks +14 lines, -5 lines 0 comments Download
M source/common/ucnvbocu.cpp View 1 2 3 4 5 1 chunk +1 line, -1 line 0 comments Download
M source/common/ucnvhz.c View 1 2 3 4 5 2 chunks +2 lines, -2 lines 0 comments Download
M source/common/ucnvisci.c View 1 2 3 4 5 1 chunk +1 line, -1 line 0 comments Download
M source/common/ucnvscsu.c View 1 2 3 4 5 1 chunk +1 line, -1 line 0 comments Download
M source/common/unicode/uconfig.h View 1 2 3 4 5 1 chunk +8 lines, -0 lines 0 comments Download
M source/common/unicode/urename.h View 1 2 3 4 5 2 chunks +6 lines, -0 lines 0 comments Download
A + source/data/mappings/big5-html.ucm View 1 2 3 4 5 744 chunks +4287 lines, -5145 lines 0 comments Download
M source/data/mappings/convrtrs.txt View 1 2 3 4 5 1 chunk +301 lines, -1132 lines 0 comments Download
A + source/data/mappings/euc-jp-html.ucm View 1 2 3 4 5 149 chunks +5795 lines, -5868 lines 0 comments Download
A + source/data/mappings/euc-kr-html.ucm View 1 2 3 4 5 11 chunks +18 lines, -370 lines 0 comments Download
A + source/data/mappings/ibm866-html.ucm View 1 2 3 4 5 5 chunks +9 lines, -145 lines 0 comments Download
A + source/data/mappings/iso-8859-10-html.ucm View 1 2 3 4 5 1 chunk +13 lines, -46 lines 0 comments Download
A + source/data/mappings/iso-8859-13-html.ucm View 1 2 3 4 5 2 chunks +6 lines, -103 lines 0 comments Download
A + source/data/mappings/iso-8859-14-html.ucm View 1 2 3 4 5 1 chunk +13 lines, -43 lines 0 comments Download
A + source/data/mappings/iso-8859-15-html.ucm View 1 2 3 4 5 2 chunks +6 lines, -103 lines 0 comments Download
A + source/data/mappings/iso-8859-16-html.ucm View 1 2 3 4 5 4 chunks +38 lines, -135 lines 0 comments Download
A + source/data/mappings/iso-8859-2-html.ucm View 1 2 3 4 5 4 chunks +6 lines, -172 lines 0 comments Download
A + source/data/mappings/iso-8859-3-html.ucm View 1 2 3 4 5 2 chunks +6 lines, -103 lines 0 comments Download
A + source/data/mappings/iso-8859-4-html.ucm View 1 2 3 4 5 2 chunks +6 lines, -103 lines 0 comments Download
A + source/data/mappings/iso-8859-5-html.ucm View 1 2 3 4 5 2 chunks +6 lines, -103 lines 0 comments Download
A + source/data/mappings/iso-8859-6-html.ucm View 1 2 3 4 5 2 chunks +54 lines, -158 lines 0 comments Download
A + source/data/mappings/iso-8859-7-html.ucm View 1 2 3 4 5 4 chunks +6 lines, -106 lines 0 comments Download
A + source/data/mappings/iso-8859-8-html.ucm View 1 2 3 4 5 2 chunks +6 lines, -105 lines 0 comments Download
A + source/data/mappings/koi8-r-html.ucm View 1 2 3 4 5 2 chunks +6 lines, -105 lines 0 comments Download
A + source/data/mappings/koi8-u-html.ucm View 1 2 3 4 5 2 chunks +6 lines, -104 lines 0 comments Download
A + source/data/mappings/macintosh-html.ucm View 1 2 3 4 5 5 chunks +6 lines, -77 lines 0 comments Download
A + source/data/mappings/shift_jis-html.ucm View 1 2 3 4 5 251 chunks +2241 lines, -2479 lines 0 comments Download
A source/data/mappings/ucmlocal.mk View 1 2 3 4 5 1 chunk +57 lines, -0 lines 0 comments Download
A + source/data/mappings/windows-1250-html.ucm View 1 2 3 4 5 2 chunks +6 lines, -103 lines 0 comments Download
A + source/data/mappings/windows-1251-html.ucm View 1 2 3 4 5 2 chunks +6 lines, -103 lines 0 comments Download
A + source/data/mappings/windows-1252-html.ucm View 1 2 3 4 5 2 chunks +6 lines, -103 lines 0 comments Download
A + source/data/mappings/windows-1253-html.ucm View 1 2 3 4 5 4 chunks +6 lines, -105 lines 0 comments Download
A + source/data/mappings/windows-1254-html.ucm View 1 2 3 4 5 2 chunks +6 lines, -103 lines 0 comments Download
A + source/data/mappings/windows-1255-html.ucm View 1 2 3 4 5 2 chunks +6 lines, -103 lines 0 comments Download
A + source/data/mappings/windows-1256-html.ucm View 1 2 3 4 5 5 chunks +70 lines, -167 lines 0 comments Download
A + source/data/mappings/windows-1257-html.ucm View 1 2 3 4 5 2 chunks +6 lines, -103 lines 0 comments Download
A + source/data/mappings/windows-1258-html.ucm View 1 2 3 4 5 2 chunks +6 lines, -103 lines 0 comments Download
A + source/data/mappings/windows-874-html.ucm View 1 2 3 4 5 2 chunks +6 lines, -103 lines 0 comments Download
M source/data/mappings/windows-936-2000.ucm View 1 2 3 4 5 1 chunk +2 lines, -1 line 0 comments Download
M source/data/mappings/windows-949-2000.ucm View 1 2 3 4 5 1 chunk +5 lines, -3 lines 0 comments Download
A + source/data/mappings/x-mac-cyrillic-html.ucm View 1 2 3 4 5 4 chunks +6 lines, -55 lines 0 comments Download
M source/i18n/csdetect.cpp View 1 2 3 4 5 2 chunks +2 lines, -0 lines 0 comments Download
M source/i18n/csr2022.h View 1 2 3 4 5 2 chunks +2 lines, -0 lines 0 comments Download
M source/i18n/csr2022.cpp View 1 2 3 4 5 4 chunks +4 lines, -0 lines 0 comments Download
M source/i18n/csrsbcs.h View 1 2 3 4 5 4 chunks +4 lines, -0 lines 0 comments Download
M source/i18n/csrsbcs.cpp View 1 2 3 4 5 6 chunks +6 lines, -0 lines 0 comments Download

Messages

Total messages: 5 (1 generated)
jungshik at Google
Can you take a look? Thanks
5 years, 11 months ago (2015-01-20 19:48:52 UTC) #2
Mark Mentovai
LGTM
5 years, 11 months ago (2015-01-20 19:57:29 UTC) #3
jsbell
cursory scan lgtm with some script nits https://codereview.chromium.org/839713003/diff/20001/scripts/big5_gen.sh File scripts/big5_gen.sh (right): https://codereview.chromium.org/839713003/diff/20001/scripts/big5_gen.sh#newcode2 scripts/big5_gen.sh:2: # Copyright ...
5 years, 11 months ago (2015-01-20 21:47:22 UTC) #4
jungshik at Google
5 years, 11 months ago (2015-01-21 21:24:11 UTC) #5
Message was sent while issue was closed.
Committed patchset #6 (id:50001) manually as
afd723ba594ba3e0d00cc10bc8ca8dbae57bb04a (presubmit successful).

Powered by Google App Engine
This is Rietveld 408576698