Index: README.chromium |
diff --git a/README.chromium b/README.chromium |
index b83aa828791a9c09890901d74e710ebb2c2c85c4..596a520cf37d74e65ff82669b1a452cde908e1b5 100644 |
--- a/README.chromium |
+++ b/README.chromium |
@@ -63,14 +63,6 @@ This directory contains the source code of ICU 54.1 for C/C++. |
patch: patches/cjdict.patch |
upstream bug: http://bugs.icu-project.org/trac/ticket/10888 |
- |
- - android/brkitr.patch (to be applied for Android build only) : |
- Do not use the C+J dictionary for Chinese/Japanese segmentation |
- to reduce the data size. Adjust word.txt and a few other files. |
- |
- - source/data/brkitr/word_ja.txt (used only on Android) |
- Added for Japanese-specific word-breaking without the C+J dictionary. |
- |
4. Converter changes : |
- convrtrs.txt : Replaced the original by our own that only lists encodings |
@@ -128,15 +120,6 @@ This directory contains the source code of ICU 54.1 for C/C++. |
and language menus. Also change the English display name |
for ckb to 'Kurdish (Arabic)'. |
- - android/patch_locale.sh (to be run for Android build only): |
- a. Make changes to source/data/{region,lang} to exclude these data |
- except the language and script names of zh_Hans and zh_Hant. |
- b. Remove exemplar cities in timezone data (data/zone). |
- c. Keep only the minimal calendar data in data/locales. |
- d. Include currency display names for a smaller subset of currencies. |
- e. Minimize the locale data for 9 locales to which Chrome on Android |
- is not localized. |
- |
6. Timezone data update |
- Grab the latest version of the following timezone data files and |
put them in source/data/misc. |
@@ -183,61 +166,61 @@ This directory contains the source code of ICU 54.1 for C/C++. |
Put ICU_DATA_ENTRY_POINT(icudtXX_dat) in common when we use |
the icu data file or icudt.dll |
-9. Pre-built data files are checked in with the following steps on Linux: |
- |
- a. Make a icu data build directory outside the Chromium source tree |
- and cd to that directory. |
- b. Run |
- |
- ${CHROME_ICU_TREE_TOP}/source/runConfigureICU Linux --disable-layout |
- |
- c. Run 'make' |
- d. 'make' will fail when pkgdata looks for css3transform.res. Edit |
- data/out/tmp/icudata.lst to replace 'css3transform.res' with 'root.res'. |
- (see http://bugs.icu-project.org/trac/ticket/10570 ) and run 'make' again. |
- |
- |
- - source/data/in/icudtl.dat : Built on Linux with all the patches |
- above applied. icudt54l.dat is generated in |
- {BUILD_DIR_ROOT}/data/out/tmp and copied to the above location with a |
- version number (54) dropped. |
- |
- |
- - {mac,linux}/icudtl_dat.S : Built on Linux with all the |
- patches above (except android/brkitr.patch) applied and checked in. |
- This file will be generated in {BUILD_DIR_ROOT}/data/out/tmp as |
- icudt54l_dat.S, but '54' is dropped while copying. |
- |
- mac/icudtl_dat.S is identical to linux/icudtl_dat.S except for |
- the header portion. With "linux/icudtl_dat.S" in its place, |
- run scripts/make_mac_assembly.sh to generate it. |
- |
- - android/icudtl_dat.S : Built on Linux with all the patches above and |
- android/brkitr.patch applied and android/patch_locale.sh executed. |
- '54' is dropped from the name generated in the build tree. |
- |
- - android/icudtl.dat : Generated as icudt54l.dat in |
- {BUILD_DIR_ROOT}/data/out/tmp along with icudt54l_dat.S and |
- copied to the above location with '54' dropped in its name. |
- |
- - windows/icudt.dll (by default, we set icu_use_icu_data_flag to 1 |
- and don't use this file.) |
- |
- a. check out a clean copy of icu54 from the upstream on Windows |
- outside the Chrome tree. |
- |
- $ svn export --native-eol LF http://source.icu-project.org/repos/icu/icu/tags/release-54-1 ${SEPARATE_ICU_ROOT}/icu54 |
- |
- b. copy ${CHROME_ICU_ROOT}/source/data/in/icudtl.dat to |
- ${SEPARATE_ICU_ROOT}/source/data/in/icudt54l.dat |
- c. copy ${CHROME_ICU_ROOT}/source/data/makedata.mak to |
- ${SEPARATE_ICU_ROOT}/source/data/makedata.mak |
- c. In Visual Studio, open source/allinone/allinone.sln solution |
- in ${SEPARATE_ICU_ROOT} |
- d. Build 'makedata' target |
- e. icudt54.dll will be generated in ${SEPARATE_ICU_ROOT}/bin |
- f. Copy that icudt54.dll to ${CHROME_ICU_ROOT}/windows/icudt.dll |
- and check that in. |
+#9. Pre-built data files are checked in with the following steps on Linux: |
+# |
+# a. Make a icu data build directory outside the Chromium source tree |
+# and cd to that directory. |
+# b. Run |
+# |
+# ${CHROME_ICU_TREE_TOP}/source/runConfigureICU Linux --disable-layout |
+# |
+# c. Run 'make' |
+# d. 'make' will fail when pkgdata looks for css3transform.res. Edit |
+# data/out/tmp/icudata.lst to replace 'css3transform.res' with 'root.res'. |
+# (see http://bugs.icu-project.org/trac/ticket/10570 ) and run 'make' again. |
+# |
+# |
+# - source/data/in/icudtl.dat : Built on Linux with all the patches |
+# above applied. icudt54l.dat is generated in |
+# {BUILD_DIR_ROOT}/data/out/tmp and copied to the above location with a |
+# version number (54) dropped. |
+# |
+# |
+# - {mac,linux}/icudtl_dat.S : Built on Linux with all the |
+# patches above (except android/brkitr.patch) applied and checked in. |
+# This file will be generated in {BUILD_DIR_ROOT}/data/out/tmp as |
+# icudt54l_dat.S, but '54' is dropped while copying. |
+# |
+# mac/icudtl_dat.S is identical to linux/icudtl_dat.S except for |
+# the header portion. With "linux/icudtl_dat.S" in its place, |
+# run scripts/make_mac_assembly.sh to generate it. |
+# |
+# - android/icudtl_dat.S : Built on Linux with all the patches above and |
+# android/brkitr.patch applied and android/patch_locale.sh executed. |
+# '54' is dropped from the name generated in the build tree. |
+# |
+# - android/icudtl.dat : Generated as icudt54l.dat in |
+# {BUILD_DIR_ROOT}/data/out/tmp along with icudt54l_dat.S and |
+# copied to the above location with '54' dropped in its name. |
+# |
+# - windows/icudt.dll (by default, we set icu_use_icu_data_flag to 1 |
+# and don't use this file.) |
+# |
+# a. check out a clean copy of icu54 from the upstream on Windows |
+# outside the Chrome tree. |
+# |
+# $ svn export --native-eol LF http://source.icu-project.org/repos/icu/icu/tags/release-54-1 ${SEPARATE_ICU_ROOT}/icu54 |
+# |
+# b. copy ${CHROME_ICU_ROOT}/source/data/in/icudtl.dat to |
+# ${SEPARATE_ICU_ROOT}/source/data/in/icudt54l.dat |
+# c. copy ${CHROME_ICU_ROOT}/source/data/makedata.mak to |
+# ${SEPARATE_ICU_ROOT}/source/data/makedata.mak |
+# c. In Visual Studio, open source/allinone/allinone.sln solution |
+# in ${SEPARATE_ICU_ROOT} |
+# d. Build 'makedata' target |
+# e. icudt54.dll will be generated in ${SEPARATE_ICU_ROOT}/bin |
+# f. Copy that icudt54.dll to ${CHROME_ICU_ROOT}/windows/icudt.dll |
+# and check that in. |
10. Apply the following patches for regex |
- patches/regex.patch (a combined patch of 3 revisions below) |
@@ -262,3 +245,69 @@ This directory contains the source code of ICU 54.1 for C/C++. |
- patches/cmemory.patch |
- upstream bug (fixed in the upstream 55 RC) |
http://www.icu-project.org/trac/ticket/11538 |
+ |
+ |
+General information for maintenance |
+----------------------------------- |
+ |
+ * Build system in gyp/gn + ninja to generate the data packages: |
+ We build the necessary tools (genrb, makeconv, icupkg, ...) and then |
+ run those tools on all data files we want to generate. |
+ |
+ This process is slightly complicated because some data files we want to |
+ modify (shrink) before the build them. This is implemented with a filter |
+ program that takes files that we want to modify and put them in a modified |
+ shape in a special directory. |
+ |
+ The whole chain is (dir names can vary): |
+ source/data/* <- raw/pure data |
+ out/gen/tmp_icudt54l/* <- filtered/stripped data |
+ out/gen/icudt54l/* <- compiled data |
+ out/icudt54l.dat <- packaged data |
+ |
+ This multi step chain means that files are named twice or three times in |
+ the build system with different paths (this might change if we can drop |
+ gyp) so to add or remove one data chunk it will have to be removed |
+ from these lists in icu_data.gypi: |
+ *_raw_source |
+ *_filtered_source |
+ *_generated |
+ where * is something like icu_curr_res or icu_lang_res. |
+ |
+ Some special files are also handled specifically in the filtering system |
+ and in the build system. |
+ |
+ * Build system for tools: makeconv, pkgdata, icupkg, genrb, gendict, |
+ genbrk, ... |
+ |
+ These are pretty straight forward. ICU is built as a static lib and used by |
+ the tools that often are just a single file implementing main() and some |
+ utility code. Whatever works for building the rest of ICU typically works |
+ for building the data tools. |
+ |
+ * Filtering tool (filter_data_for_size.py): |
+ Based on: |
+ - android/patch_locale.sh (to be run for Android build only): |
+ a. Make changes to source/data/{region,lang} to exclude these data |
+ except the language and script names of zh_Hans and zh_Hant. |
+ b. Remove exemplar cities in timezone data (data/zone). |
+ c. Keep only the minimal calendar data in data/locales. |
+ d. Include currency display names for a smaller subset of currencies. |
+ e. Minimize the locale data for 9 locales to which Chrome on Android |
+ is not localized. |
+ |
+ - android/brkitr.patch (to be applied for Android build only) : |
+ Do not use the C+J dictionary for Chinese/Japanese segmentation |
+ to reduce the data size. Adjust word.txt and a few other files. |
+ |
+ - source/data/brkitr/word_ja.txt (used only on Android) |
+ Added for Japanese-specific word-breaking without the C+J dictionary. |
+ |
+ * The build system is not warning clean. In particular: |
+ - ICU tools warn about empty input files. - Check --quiet flags? |
+ - The gyp Makefile generator generates duplicate rules (affects |
+ v8-standalone) - Reported as gyp issue 484. |
+ |
+ * Updating to a newer ICU: Diff all Makefile.* and *.mk files before |
+ and after and mimic necessary changes (if any) in BUILD.gn, icu_data.gyp |
+ and icu_data.gypi. |