Update trim_data to deal with locale fallback failure for unit

Update trim_data to deal with locale fallback failure for units

Delete empty units,units{Narrow,Short} blocks after trimming units data.
Empty units* blocks in en_GB and a few other locales after trimming
causes ICU to fail to fall back to get the duration data for those
locales.

In addition, fix source/data/translit/root_subset.txt. Rule*Ids block has
to be present even though it's empty. When dropping Hans-Hant transform
rules, root_subset.txt was changed to be completely empty, which broke
"components_unittests --g_test_filter=AutofillProfileComparato*" .

With these changes, regenerate ICU data files. The size is slightly smaller.

android/icudtl.dat  6573872 => 6573792
common/icudt*dat    10130560 => 10130480

BUG=707515, 677043, 684609
TEST=components_unittests --gtest_filter=AutofillProfileComparato*
TEST=ui_base_unittests --gtest_filter=L10nUtilTest.TimeDurationForm*
R=derat@chromium.org

Committed: https://chromium.googlesource.com/chromium/deps/icu/+/d5c238dcc2e801210749f3bb421b9227fe1c0948

[jungshik at Google, 2017-04-11 19:51:20]

Description was changed from

==========
Update trim_data to deal with locale fallback failure for unit

Delete empty units,units{Narrow,Short} blocks after trimming units data.
Empty units* blocks in en_GB and a few other locales after trimming
causes ICU to fail to fall back to get the duration data for those
locales.

In addition, fix source/data/translit/root_subset.txt. Rule*Ids block has
to be present even though it's empty.

After that, regenerate ICU data files. The size is slightly smaller.

android/icudtl.dat  6573872 => 6573792
common/icudt*dat    10130560 => 10130480

BUG=707515,677043
==========

to

==========
Update trim_data to deal with locale fallback failure for unit

Delete empty units,units{Narrow,Short} blocks after trimming units data.
Empty units* blocks in en_GB and a few other locales after trimming
causes ICU to fail to fall back to get the duration data for those
locales.

In addition, fix source/data/translit/root_subset.txt. Rule*Ids block has
to be present even though it's empty.

After that, regenerate ICU data files. The size is slightly smaller.

android/icudtl.dat  6573872 => 6573792
common/icudt*dat    10130560 => 10130480

BUG=707515,677043
TEST=components_unittests --gtest_filter=AutoFillProf*Compar*
==========

[jungshik at Google, 2017-04-11 19:58:35]

Description was changed from

==========
Update trim_data to deal with locale fallback failure for unit

Delete empty units,units{Narrow,Short} blocks after trimming units data.
Empty units* blocks in en_GB and a few other locales after trimming
causes ICU to fail to fall back to get the duration data for those
locales.

In addition, fix source/data/translit/root_subset.txt. Rule*Ids block has
to be present even though it's empty.

After that, regenerate ICU data files. The size is slightly smaller.

android/icudtl.dat  6573872 => 6573792
common/icudt*dat    10130560 => 10130480

BUG=707515,677043
TEST=components_unittests --gtest_filter=AutoFillProf*Compar*
==========

to

==========
Update trim_data to deal with locale fallback failure for unit

Delete empty units,units{Narrow,Short} blocks after trimming units data.
Empty units* blocks in en_GB and a few other locales after trimming
causes ICU to fail to fall back to get the duration data for those
locales.

In addition, fix source/data/translit/root_subset.txt. Rule*Ids block has
to be present even though it's empty.

After that, regenerate ICU data files. The size is slightly smaller.

android/icudtl.dat  6573872 => 6573792
common/icudt*dat    10130560 => 10130480

BUG=707515,677043
TEST=components_unittests --gtest_filter=AutofillProfileComparato*
TEST=ui_base_unittests --gtest_filter=L10nUtilTest.TimeDurationForm*
==========

[jungshik at Google, 2017-04-11 20:03:22]

Description was changed from

==========
Update trim_data to deal with locale fallback failure for unit

Delete empty units,units{Narrow,Short} blocks after trimming units data.
Empty units* blocks in en_GB and a few other locales after trimming
causes ICU to fail to fall back to get the duration data for those
locales.

In addition, fix source/data/translit/root_subset.txt. Rule*Ids block has
to be present even though it's empty.

After that, regenerate ICU data files. The size is slightly smaller.

android/icudtl.dat  6573872 => 6573792
common/icudt*dat    10130560 => 10130480

BUG=707515,677043
TEST=components_unittests --gtest_filter=AutofillProfileComparato*
TEST=ui_base_unittests --gtest_filter=L10nUtilTest.TimeDurationForm*
==========

to

==========
Update trim_data to deal with locale fallback failure for units

Delete empty units,units{Narrow,Short} blocks after trimming units data.
Empty units* blocks in en_GB and a few other locales after trimming
causes ICU to fail to fall back to get the duration data for those
locales.

In addition, fix source/data/translit/root_subset.txt. Rule*Ids block has
to be present even though it's empty. When dropping Hans-Hant transform
rules, root_subset.txt was changed to be completely empty.

With these changes, regenerate ICU data files. The size is slightly smaller.

android/icudtl.dat  6573872 => 6573792
common/icudt*dat    10130560 => 10130480

BUG=707515,677043,684609
TEST=components_unittests --gtest_filter=AutofillProfileComparato*
TEST=ui_base_unittests --gtest_filter=L10nUtilTest.TimeDurationForm*
==========

[jungshik at Google, 2017-04-11 20:06:07]

jshin@chromium.org changed reviewers:
+ derat@chromium.org

[jungshik at Google, 2017-04-11 20:06:07]

Dan, can you take a look? Thanks

(btw, make_data.sh failure you experienced was already taken care of in another
CL, but DEPS was not rolled to that revision yet.)

[Daniel Erat, 2017-04-11 21:08:05]

On 2017/04/11 20:06:40, jungshik at Google wrote:

[Daniel Erat, 2017-04-11 21:18:10]

https://codereview.chromium.org/2812943003/diff/1/scripts/trim_data.sh
File scripts/trim_data.sh (right):

https://codereview.chromium.org/2812943003/diff/1/scripts/trim_data.sh#newcod...
scripts/trim_data.sh:194: /^    units(|Narrow|Short)\{\n    \}/ d
sorry, my sed is a bit rusty/nonexistent. i think that this line is deleting
opening units lines followed by closing units lines, but i'm not sure what the
earlier pattern or the 'N' ("Read/append the next line of input into the pattern
space.") does. can you explain it for me?

[jungshik at Google, 2017-04-11 21:53:18]

Thanks for taking a look.

https://codereview.chromium.org/2812943003/diff/1/scripts/trim_data.sh
File scripts/trim_data.sh (right):

https://codereview.chromium.org/2812943003/diff/1/scripts/trim_data.sh#newcod...
scripts/trim_data.sh:194: /^    units(|Narrow|Short)\{\n    \}/ d
On 2017/04/11 21:18:10, Daniel Erat wrote:
> sorry, my sed is a bit rusty/nonexistent. i think that this line is deleting
> opening units lines followed by closing units lines, but i'm not sure what the
> earlier pattern or the 'N' ("Read/append the next line of input into the
pattern
> space.") does. can you explain it for me?

    unitsNarrow{
    }

Given an input like the above, 

this line will match the first input line above and add it to the pattern space.

 
'/^    units(|Narrow|Short)\{$/ {

"N" will append the next input line ("    }") to the pattern space, making it
read like 
    unitsNarrow{\n    \}

Then, " /^    units(|Narrow|Short)\{\n    \}/" will match the pattern space and
will delete it starting a new cycle (processing the next input line. 

If the regex is not matched, the pattern space is just printed out and sed will
move on to process the next input line.

See  http://www.grymoire.com/Unix/Sed.html#uh-51 that I found after writing the
above. :-)

[Daniel Erat, 2017-04-11 21:55:05]

lgtm

thanks, makes sense! i didn't realize that sed doesn't have easier support for
multiple-line matching.

[jungshik at Google, 2017-04-11 21:55:13]

Description was changed from

==========
Update trim_data to deal with locale fallback failure for units

Delete empty units,units{Narrow,Short} blocks after trimming units data.
Empty units* blocks in en_GB and a few other locales after trimming
causes ICU to fail to fall back to get the duration data for those
locales.

In addition, fix source/data/translit/root_subset.txt. Rule*Ids block has
to be present even though it's empty. When dropping Hans-Hant transform
rules, root_subset.txt was changed to be completely empty.

With these changes, regenerate ICU data files. The size is slightly smaller.

android/icudtl.dat  6573872 => 6573792
common/icudt*dat    10130560 => 10130480

BUG=707515,677043,684609
TEST=components_unittests --gtest_filter=AutofillProfileComparato*
TEST=ui_base_unittests --gtest_filter=L10nUtilTest.TimeDurationForm*
==========

to

==========
Update trim_data to deal with locale fallback failure for units

Delete empty units,units{Narrow,Short} blocks after trimming units data.
Empty units* blocks in en_GB and a few other locales after trimming
causes ICU to fail to fall back to get the duration data for those
locales.

In addition, fix source/data/translit/root_subset.txt. Rule*Ids block has
to be present even though it's empty. When dropping Hans-Hant transform
rules, root_subset.txt was changed to be completely empty, which broke
"components_unittests --g_test_filter=AutofillProfileComparato*" .

With these changes, regenerate ICU data files. The size is slightly smaller.

android/icudtl.dat  6573872 => 6573792
common/icudt*dat    10130560 => 10130480

BUG=707515,677043,684609
TEST=components_unittests --gtest_filter=AutofillProfileComparato*
TEST=ui_base_unittests --gtest_filter=L10nUtilTest.TimeDurationForm*
==========

[jungshik at Google, 2017-04-11 21:59:06]

Description was changed from

==========
Update trim_data to deal with locale fallback failure for units

Delete empty units,units{Narrow,Short} blocks after trimming units data.
Empty units* blocks in en_GB and a few other locales after trimming
causes ICU to fail to fall back to get the duration data for those
locales.

In addition, fix source/data/translit/root_subset.txt. Rule*Ids block has
to be present even though it's empty. When dropping Hans-Hant transform
rules, root_subset.txt was changed to be completely empty, which broke
"components_unittests --g_test_filter=AutofillProfileComparato*" .

With these changes, regenerate ICU data files. The size is slightly smaller.

android/icudtl.dat  6573872 => 6573792
common/icudt*dat    10130560 => 10130480

BUG=707515,677043,684609
TEST=components_unittests --gtest_filter=AutofillProfileComparato*
TEST=ui_base_unittests --gtest_filter=L10nUtilTest.TimeDurationForm*
==========

to

==========
Update trim_data to deal with locale fallback failure for units

Delete empty units,units{Narrow,Short} blocks after trimming units data.
Empty units* blocks in en_GB and a few other locales after trimming
causes ICU to fail to fall back to get the duration data for those
locales.

In addition, fix source/data/translit/root_subset.txt. Rule*Ids block has
to be present even though it's empty. When dropping Hans-Hant transform
rules, root_subset.txt was changed to be completely empty, which broke
"components_unittests --g_test_filter=AutofillProfileComparato*" .

With these changes, regenerate ICU data files. The size is slightly smaller.

android/icudtl.dat  6573872 => 6573792
common/icudt*dat    10130560 => 10130480

BUG=707515,677043,684609
TEST=components_unittests --gtest_filter=AutofillProfileComparato*
TEST=ui_base_unittests --gtest_filter=L10nUtilTest.TimeDurationForm*
R=derat@chromium.org

Committed:
https://chromium.googlesource.com/chromium/deps/icu/+/d5c238dcc2e801210749f3b...
==========

[jungshik at Google, 2017-04-11 21:59:07]

Committed patchset #1 (id:1) manually as
d5c238dcc2e801210749f3bb421b9227fe1c0948 (presubmit successful).

[jungshik at Google, 2017-04-11 21:59:18]

On 2017/04/11 21:55:05, Daniel Erat wrote:
> lgtm
> 
> thanks, makes sense! i didn't realize that sed doesn't have easier support for
> multiple-line matching.

Thanks. :-)