ICU 56 step 6: Check in the pre-built ICU data

ICU 56 step 6:Check in the pre-built ICU data

* Update the pre-built ICU data files for all platforms

  source/data/in/icudtl.dat for non-Android platforms
  {linux,mac}/icudt*.S for linux/mac
  android/icudtl.dat and android/icudt*.S for Android
  windows/icudt.dll for Windows


* Update Android data trimming script

  1. Make sure that 'default' calendar is kept in locales where it's relevant
    : root, th, fa, ar_SA, etc.
  2. Add a minimal region data to work around a bug in ICU with pool.res
     handling

* Update gn and gyp files
* And add a TODO comment to update.sh to automate the build file update.
* Add it_CH to the locale list.
* Add sr_Latn to unit/reslocal.mk (required by sh) and
  line_normal_fi to brkitr/brklocal.mk (referred to in brkitr/fi.txt) in
  place of line_fi.

* Update and add scripts for data building
* Completely rewrite README.chromium
* Check-in the prebuilt ICU data files/assembly sources for
  Linux,Mac,Windows,Chrome OS and Android.

BUG=575007
TEST=Blink layout tests, webkit unittests
TEST=All bots can build successfully
TEST=net_unittests --gtest_filter="*ilenameUtil*"
TEST=net_unittests --gtest_filter="*IDN*" (pending bug 336973)
TEST=base_unittests --gtest_filter="*Conv*"
TEST=browser_tests --gtest_filter="*ncoding*"
TEST=base_unittests --gtest_filter="*essage*"
TEST=ui_base_unittests --gtest_filter="*ormat*"
TEST=ui_base_unittests --gtest_filter="L10n*"
R=mark@chromium.org

Committed: https://chromium.googlesource.com/chromium/deps/icu/+/d2c1830066314b3c06289a70d67f8411f7702579

[jungshik at Google, 2016-01-27 01:12:36]

Description was changed from

==========
Check in the pre-built ICU data

* Update Android data trimming script

  1. Make sure that 'default' calendar is kept in locales where it's relevant
    : root, th, fa, ar_SA, etc.
  2. Add a minimal region data to work around a bug in ICU with pool.res
     handling

* Update gn and gyp files
* And add a TODO comment to update.sh to automate the build file update.
* Add it_CH to the locale list.

* Update and add scripts for data building
* Completely rewrite README.chromium
* Check-in the prebuilt ICU data files/assembly sources for
  Linux,Mac,Windows,Chrome OS and Android.

BUG=
==========

to

==========
Check in the pre-built ICU data

* Update Android data trimming script

  1. Make sure that 'default' calendar is kept in locales where it's relevant
    : root, th, fa, ar_SA, etc.
  2. Add a minimal region data to work around a bug in ICU with pool.res
     handling

* Update gn and gyp files
* And add a TODO comment to update.sh to automate the build file update.
* Add it_CH to the locale list.

* Update and add scripts for data building
* Completely rewrite README.chromium
* Check-in the prebuilt ICU data files/assembly sources for
  Linux,Mac,Windows,Chrome OS and Android.

BUG=
==========

[jungshik at Google, 2016-01-27 09:48:01]

Description was changed from

==========
Check in the pre-built ICU data

* Update Android data trimming script

  1. Make sure that 'default' calendar is kept in locales where it's relevant
    : root, th, fa, ar_SA, etc.
  2. Add a minimal region data to work around a bug in ICU with pool.res
     handling

* Update gn and gyp files
* And add a TODO comment to update.sh to automate the build file update.
* Add it_CH to the locale list.

* Update and add scripts for data building
* Completely rewrite README.chromium
* Check-in the prebuilt ICU data files/assembly sources for
  Linux,Mac,Windows,Chrome OS and Android.

BUG=
==========

to

==========
Check in the pre-built ICU data

* Update the pre-built ICU data files for all platforms

  source/data/in/icudtl.dat for non-Android platforms
  {linux,mac}/icudt*.S for linux/mac
  android/icudtl.dat and android/icudt*.S for Android
  windows/icudt.dll for Windows


* Update Android data trimming script

  1. Make sure that 'default' calendar is kept in locales where it's relevant
    : root, th, fa, ar_SA, etc.
  2. Add a minimal region data to work around a bug in ICU with pool.res
     handling

* Update gn and gyp files
* And add a TODO comment to update.sh to automate the build file update.
* Add it_CH to the locale list.
* Add sr_Latn to unit/reslocal.mk (required by sh) and
  line_normal_fi to brkitr/brklocal.mk (referred to in brkitr/fi.txt) in
  place of line_fi.

* Update and add scripts for data building
* Completely rewrite README.chromium
* Check-in the prebuilt ICU data files/assembly sources for
  Linux,Mac,Windows,Chrome OS and Android.

BUG=
==========

[jungshik at Google, 2016-01-27 17:59:43]

Description was changed from

==========
Check in the pre-built ICU data

* Update the pre-built ICU data files for all platforms

  source/data/in/icudtl.dat for non-Android platforms
  {linux,mac}/icudt*.S for linux/mac
  android/icudtl.dat and android/icudt*.S for Android
  windows/icudt.dll for Windows


* Update Android data trimming script

  1. Make sure that 'default' calendar is kept in locales where it's relevant
    : root, th, fa, ar_SA, etc.
  2. Add a minimal region data to work around a bug in ICU with pool.res
     handling

* Update gn and gyp files
* And add a TODO comment to update.sh to automate the build file update.
* Add it_CH to the locale list.
* Add sr_Latn to unit/reslocal.mk (required by sh) and
  line_normal_fi to brkitr/brklocal.mk (referred to in brkitr/fi.txt) in
  place of line_fi.

* Update and add scripts for data building
* Completely rewrite README.chromium
* Check-in the prebuilt ICU data files/assembly sources for
  Linux,Mac,Windows,Chrome OS and Android.

BUG=
==========

to

==========
icu 56 step6:Check in the pre-built ICU data

* Update the pre-built ICU data files for all platforms

  source/data/in/icudtl.dat for non-Android platforms
  {linux,mac}/icudt*.S for linux/mac
  android/icudtl.dat and android/icudt*.S for Android
  windows/icudt.dll for Windows


* Update Android data trimming script

  1. Make sure that 'default' calendar is kept in locales where it's relevant
    : root, th, fa, ar_SA, etc.
  2. Add a minimal region data to work around a bug in ICU with pool.res
     handling

* Update gn and gyp files
* And add a TODO comment to update.sh to automate the build file update.
* Add it_CH to the locale list.
* Add sr_Latn to unit/reslocal.mk (required by sh) and
  line_normal_fi to brkitr/brklocal.mk (referred to in brkitr/fi.txt) in
  place of line_fi.

* Update and add scripts for data building
* Completely rewrite README.chromium
* Check-in the prebuilt ICU data files/assembly sources for
  Linux,Mac,Windows,Chrome OS and Android.

BUG=
==========

[jungshik at Google, 2016-01-27 18:11:38]

Description was changed from

==========
icu 56 step6:Check in the pre-built ICU data

* Update the pre-built ICU data files for all platforms

  source/data/in/icudtl.dat for non-Android platforms
  {linux,mac}/icudt*.S for linux/mac
  android/icudtl.dat and android/icudt*.S for Android
  windows/icudt.dll for Windows


* Update Android data trimming script

  1. Make sure that 'default' calendar is kept in locales where it's relevant
    : root, th, fa, ar_SA, etc.
  2. Add a minimal region data to work around a bug in ICU with pool.res
     handling

* Update gn and gyp files
* And add a TODO comment to update.sh to automate the build file update.
* Add it_CH to the locale list.
* Add sr_Latn to unit/reslocal.mk (required by sh) and
  line_normal_fi to brkitr/brklocal.mk (referred to in brkitr/fi.txt) in
  place of line_fi.

* Update and add scripts for data building
* Completely rewrite README.chromium
* Check-in the prebuilt ICU data files/assembly sources for
  Linux,Mac,Windows,Chrome OS and Android.

BUG=
==========

to

==========
ICU 56 step6:Check in the pre-built ICU data

* Update the pre-built ICU data files for all platforms

  source/data/in/icudtl.dat for non-Android platforms
  {linux,mac}/icudt*.S for linux/mac
  android/icudtl.dat and android/icudt*.S for Android
  windows/icudt.dll for Windows


* Update Android data trimming script

  1. Make sure that 'default' calendar is kept in locales where it's relevant
    : root, th, fa, ar_SA, etc.
  2. Add a minimal region data to work around a bug in ICU with pool.res
     handling

* Update gn and gyp files
* And add a TODO comment to update.sh to automate the build file update.
* Add it_CH to the locale list.
* Add sr_Latn to unit/reslocal.mk (required by sh) and
  line_normal_fi to brkitr/brklocal.mk (referred to in brkitr/fi.txt) in
  place of line_fi.

* Update and add scripts for data building
* Completely rewrite README.chromium
* Check-in the prebuilt ICU data files/assembly sources for
  Linux,Mac,Windows,Chrome OS and Android.

BUG=575007
TEST=Blink layout tests, webkit unittests
TEST=All bots can build successfully
TEST=net_unittests --gtest_filter="*ilenameUtil*"
TEST=net_unittests --gtest_filter="*IDN*" (pending bug 336973)
TEST=base_unittests --gtest_filter="*Conv*"
TEST=browser_tests --gtest_filter="*ncoding*"
TEST=base_unittests --gtest_filter="*essage*"
TEST=ui_base_unittests --gtest_filter="*ormat*"
TEST=ui_base_unittests --gtest_filter="L10n*"
==========

[jungshik at Google, 2016-01-27 18:12:00]

Description was changed from

==========
ICU 56 step6:Check in the pre-built ICU data

* Update the pre-built ICU data files for all platforms

  source/data/in/icudtl.dat for non-Android platforms
  {linux,mac}/icudt*.S for linux/mac
  android/icudtl.dat and android/icudt*.S for Android
  windows/icudt.dll for Windows


* Update Android data trimming script

  1. Make sure that 'default' calendar is kept in locales where it's relevant
    : root, th, fa, ar_SA, etc.
  2. Add a minimal region data to work around a bug in ICU with pool.res
     handling

* Update gn and gyp files
* And add a TODO comment to update.sh to automate the build file update.
* Add it_CH to the locale list.
* Add sr_Latn to unit/reslocal.mk (required by sh) and
  line_normal_fi to brkitr/brklocal.mk (referred to in brkitr/fi.txt) in
  place of line_fi.

* Update and add scripts for data building
* Completely rewrite README.chromium
* Check-in the prebuilt ICU data files/assembly sources for
  Linux,Mac,Windows,Chrome OS and Android.

BUG=575007
TEST=Blink layout tests, webkit unittests
TEST=All bots can build successfully
TEST=net_unittests --gtest_filter="*ilenameUtil*"
TEST=net_unittests --gtest_filter="*IDN*" (pending bug 336973)
TEST=base_unittests --gtest_filter="*Conv*"
TEST=browser_tests --gtest_filter="*ncoding*"
TEST=base_unittests --gtest_filter="*essage*"
TEST=ui_base_unittests --gtest_filter="*ormat*"
TEST=ui_base_unittests --gtest_filter="L10n*"
==========

to

==========
ICU 56 step 6:Check in the pre-built ICU data

* Update the pre-built ICU data files for all platforms

  source/data/in/icudtl.dat for non-Android platforms
  {linux,mac}/icudt*.S for linux/mac
  android/icudtl.dat and android/icudt*.S for Android
  windows/icudt.dll for Windows


* Update Android data trimming script

  1. Make sure that 'default' calendar is kept in locales where it's relevant
    : root, th, fa, ar_SA, etc.
  2. Add a minimal region data to work around a bug in ICU with pool.res
     handling

* Update gn and gyp files
* And add a TODO comment to update.sh to automate the build file update.
* Add it_CH to the locale list.
* Add sr_Latn to unit/reslocal.mk (required by sh) and
  line_normal_fi to brkitr/brklocal.mk (referred to in brkitr/fi.txt) in
  place of line_fi.

* Update and add scripts for data building
* Completely rewrite README.chromium
* Check-in the prebuilt ICU data files/assembly sources for
  Linux,Mac,Windows,Chrome OS and Android.

BUG=575007
TEST=Blink layout tests, webkit unittests
TEST=All bots can build successfully
TEST=net_unittests --gtest_filter="*ilenameUtil*"
TEST=net_unittests --gtest_filter="*IDN*" (pending bug 336973)
TEST=base_unittests --gtest_filter="*Conv*"
TEST=browser_tests --gtest_filter="*ncoding*"
TEST=base_unittests --gtest_filter="*essage*"
TEST=ui_base_unittests --gtest_filter="*ormat*"
TEST=ui_base_unittests --gtest_filter="L10n*"
==========

[jungshik at Google, 2016-01-28 09:01:42]

jshin@chromium.org changed reviewers:
+ mark@chromium.org

[jungshik at Google, 2016-01-28 09:01:42]

This is the last in the series of CLs for ICU 56.1 update. 

I updated and added several scripts to pre-build ICU data files. On purpose, I
use multiple scripts instead of a single script to facilitate the examination of
the result of each step. README.chromium section B explains how to use them to
build ICU data/assembly files. 

 
Besides, README.chromium was completely overhauled and divided into 4 sections. 

I followed the instruction in README.chromium to build icu data files/assembly
files for all platforms.

[jungshik at Google, 2016-01-28 09:02:37]

Thank you in advance for taking a look !

[Mark Mentovai, 2016-01-28 14:23:33]

LGTM otherwise

https://codereview.chromium.org/1639543006/diff/30001/README.chromium
File README.chromium (right):

https://codereview.chromium.org/1639543006/diff/30001/README.chromium#newcode18
README.chromium:18: This will download the ICU from the upstream svn repository.
the ICU→ICU

https://codereview.chromium.org/1639543006/diff/30001/README.chromium#newcode291
README.chromium:291: 
You don’t need these blank lines at EOF.

https://codereview.chromium.org/1639543006/diff/30001/icu.gypi
File icu.gypi (right):

https://codereview.chromium.org/1639543006/diff/30001/icu.gypi#newcode395
icu.gypi:395: }
scripts/big5_gen.sh shows up as empty on both sides, can you check?

https://codereview.chromium.org/1639543006/diff/30001/scripts/clean_up_data_s...
File scripts/clean_up_data_source.sh (right):

https://codereview.chromium.org/1639543006/diff/30001/scripts/clean_up_data_s...
scripts/clean_up_data_source.sh:4: cd "$(dirname $0)/.."
dirname "$0"

https://codereview.chromium.org/1639543006/diff/30001/scripts/copy_data.sh
File scripts/copy_data.sh (right):

https://codereview.chromium.org/1639543006/diff/30001/scripts/copy_data.sh#ne...
scripts/copy_data.sh:9: TOPSRC="$(dirname $0)/.."
Same, and in other scripts, too.

https://codereview.chromium.org/1639543006/diff/30001/scripts/data_common.sh
File scripts/data_common.sh (right):

https://codereview.chromium.org/1639543006/diff/30001/scripts/data_common.sh#...
scripts/data_common.sh:24: echo "The major version of ICU is $VERSION"
scripts/euckr_gen.sh shows up as empty on both sides.

https://codereview.chromium.org/1639543006/diff/30001/scripts/update.sh
File scripts/update.sh (right):

https://codereview.chromium.org/1639543006/diff/30001/scripts/update.sh#newco...
scripts/update.sh:54: # TODO(jshin): Automatically update BUILd.gn and icu.gypi
with the updated
BUILD.gn

[jungshik at Google, 2016-01-29 08:11:19]

Thank you. I addressed all the comments (and I also added '>&2' for other error
output).

https://codereview.chromium.org/1639543006/diff/30001/README.chromium
File README.chromium (right):

https://codereview.chromium.org/1639543006/diff/30001/README.chromium#newcode18
README.chromium:18: This will download the ICU from the upstream svn repository.
On 2016/01/28 14:23:33, Mark Mentovai wrote:
> the ICU→ICU

Done.

https://codereview.chromium.org/1639543006/diff/30001/README.chromium#newcode291
README.chromium:291: 
On 2016/01/28 14:23:33, Mark Mentovai wrote:
> You don’t need these blank lines at EOF.

Done.

https://codereview.chromium.org/1639543006/diff/30001/icu.gypi
File icu.gypi (right):

https://codereview.chromium.org/1639543006/diff/30001/icu.gypi#newcode395
icu.gypi:395: }
On 2016/01/28 14:23:33, Mark Mentovai wrote:
> scripts/big5_gen.sh shows up as empty on both sides, can you check?

Hmm... it's strange. I just chmod'ed (made it executable). The same is true of
euckr_gen.sh

Running git diff locally, I got the following:

git diff dafa8443b5513 scripts/euckr_gen.sh
diff --git a/scripts/euckr_gen.sh b/scripts/euckr_gen.sh
old mode 100644
new mode 100755

I guess it should be ok (perhaps, I'll file a bug against rietveld to handle
cases like this better).

https://codereview.chromium.org/1639543006/diff/30001/scripts/clean_up_data_s...
File scripts/clean_up_data_source.sh (right):

https://codereview.chromium.org/1639543006/diff/30001/scripts/clean_up_data_s...
scripts/clean_up_data_source.sh:4: cd "$(dirname $0)/.."
On 2016/01/28 14:23:33, Mark Mentovai wrote:
> dirname "$0"

Done.

https://codereview.chromium.org/1639543006/diff/30001/scripts/copy_data.sh
File scripts/copy_data.sh (right):

https://codereview.chromium.org/1639543006/diff/30001/scripts/copy_data.sh#ne...
scripts/copy_data.sh:9: TOPSRC="$(dirname $0)/.."
On 2016/01/28 14:23:33, Mark Mentovai wrote:
> Same, and in other scripts, too.

Done.

[jungshik at Google, 2016-01-29 09:18:17]

Description was changed from

==========
ICU 56 step 6:Check in the pre-built ICU data

* Update the pre-built ICU data files for all platforms

  source/data/in/icudtl.dat for non-Android platforms
  {linux,mac}/icudt*.S for linux/mac
  android/icudtl.dat and android/icudt*.S for Android
  windows/icudt.dll for Windows


* Update Android data trimming script

  1. Make sure that 'default' calendar is kept in locales where it's relevant
    : root, th, fa, ar_SA, etc.
  2. Add a minimal region data to work around a bug in ICU with pool.res
     handling

* Update gn and gyp files
* And add a TODO comment to update.sh to automate the build file update.
* Add it_CH to the locale list.
* Add sr_Latn to unit/reslocal.mk (required by sh) and
  line_normal_fi to brkitr/brklocal.mk (referred to in brkitr/fi.txt) in
  place of line_fi.

* Update and add scripts for data building
* Completely rewrite README.chromium
* Check-in the prebuilt ICU data files/assembly sources for
  Linux,Mac,Windows,Chrome OS and Android.

BUG=575007
TEST=Blink layout tests, webkit unittests
TEST=All bots can build successfully
TEST=net_unittests --gtest_filter="*ilenameUtil*"
TEST=net_unittests --gtest_filter="*IDN*" (pending bug 336973)
TEST=base_unittests --gtest_filter="*Conv*"
TEST=browser_tests --gtest_filter="*ncoding*"
TEST=base_unittests --gtest_filter="*essage*"
TEST=ui_base_unittests --gtest_filter="*ormat*"
TEST=ui_base_unittests --gtest_filter="L10n*"
==========

to

==========
ICU 56 step 6:Check in the pre-built ICU data

* Update the pre-built ICU data files for all platforms

  source/data/in/icudtl.dat for non-Android platforms
  {linux,mac}/icudt*.S for linux/mac
  android/icudtl.dat and android/icudt*.S for Android
  windows/icudt.dll for Windows


* Update Android data trimming script

  1. Make sure that 'default' calendar is kept in locales where it's relevant
    : root, th, fa, ar_SA, etc.
  2. Add a minimal region data to work around a bug in ICU with pool.res
     handling

* Update gn and gyp files
* And add a TODO comment to update.sh to automate the build file update.
* Add it_CH to the locale list.
* Add sr_Latn to unit/reslocal.mk (required by sh) and
  line_normal_fi to brkitr/brklocal.mk (referred to in brkitr/fi.txt) in
  place of line_fi.

* Update and add scripts for data building
* Completely rewrite README.chromium
* Check-in the prebuilt ICU data files/assembly sources for
  Linux,Mac,Windows,Chrome OS and Android.

BUG=575007
TEST=Blink layout tests, webkit unittests
TEST=All bots can build successfully
TEST=net_unittests --gtest_filter="*ilenameUtil*"
TEST=net_unittests --gtest_filter="*IDN*" (pending bug 336973)
TEST=base_unittests --gtest_filter="*Conv*"
TEST=browser_tests --gtest_filter="*ncoding*"
TEST=base_unittests --gtest_filter="*essage*"
TEST=ui_base_unittests --gtest_filter="*ormat*"
TEST=ui_base_unittests --gtest_filter="L10n*"
R=mark@chromium.org

Committed:
https://chromium.googlesource.com/chromium/deps/icu/+/d2c1830066314b3c06289a7...
==========

[jungshik at Google, 2016-01-29 09:18:18]

Committed patchset #5 (id:40001) manually as
d2c1830066314b3c06289a70d67f8411f7702579 (presubmit successful).