Issue metadata
Sign in to add a comment
|
base::TimeDurationFormat() fails for many locales |
||||||||||||||||||||||
Issue descriptionbase::TimeDurationFormat() fails when the default locale is set to en_GB and various other locales (but not all locales). This is causing issue 677043 . This is the failing line in base/i18n/time_formatting.cc: icu::MeasureFormat measure_format(icu::Locale::getDefault(), u_width, status); ICU sets |status| to U_RESOURCE_TYPE_MISMATCH. I've uploaded a trivial test demonstrating the error at https://codereview.chromium.org/2793613004. I'm guessing that something is wrong with ICU's resource files. I tried to build them locally using the instructions in src/third_party/icu/README.chromium but was unsuccessful. Some ideas: - Maybe our resource-building scripts are removing necessary data. - Looking at the files in src/third_party/icu/source/data/locales, I see that locales like "da" and "fa" (which work) define a "generic" calendar, while en_GB only defines "gregorian". (We should be using the Gregorian calendar here, though, I assume.)
,
Apr 1 2017
Per try jobs with my test, this looks like it's broken on all platforms (and for the same locales, I think).
,
Apr 10 2017
Thank you for the report and sorry for getting to this late. I'll look into the ICU data.
,
Apr 10 2017
> en_GB only defines "gregorian" That should be fine (even if generic is necessary for this API) because the parent of en_GB is en_001 which has 'generic'. It's possible that trim_data.sh messes up something as you suspected.
,
Apr 11 2017
Measure unit data come from data/unit. Even before trimming, en_001 (the parent of en_GB) does not have entries for {units,unitsNarrow}.duration.{day,hour,month,second} while it does have unitsShort.duration.*.
en.txt does have {units, unitsNarrow}.duration.* as well as unitsShort.duration.*. I guess the fallback path is en_GB => en_001 => en.
Because unit*.duration.* are kept (not trimmed away), trimming does not make any difference (at least, should not; and I couldn't find any difference).
Nonetheless, with the full ICU data, Dan's test passes while with Chrome's trimmed data it fails.
I may be hitting a latent bug in ICU's fallback algorithm (en_GB => en_001 => en) with Chrome's trimmed ICU data file.
,
Apr 11 2017
I think I found a likely cause. With data trimming, en_001 (the parent of en_GB) ends up having an empty units and unitsNarrow. Because Chrome only needs duration and en_001 does not have any duration entry for units/unitsNarrow to start with, they become empty after trimming. And, this empty units and unitsNarrow blocks ICU from falling back to en.txt.
units{
}
unitsNarrow{
}
The same is true of all other locales listed in comment 1.
,
Apr 11 2017
That was it. I rebuilt the ICU data and Dan's test passes for all locales. > I tried to build them locally using the instructions in src/third_party/icu/README.chromium but was unsuccessful. Sorry about that. source/data/trnslit/trnslocal.mk should have root_subset.txt instead of css3transform.txt. That file was left not updated.
,
Apr 11 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/deps/icu.git/+/d5c238dcc2e801210749f3bb421b9227fe1c0948 commit d5c238dcc2e801210749f3bb421b9227fe1c0948 Author: Jungshik Shin <jshin@chromium.org> Date: Tue Apr 11 21:58:59 2017 Update trim_data to deal with locale fallback failure for units Delete empty units,units{Narrow,Short} blocks after trimming units data. Empty units* blocks in en_GB and a few other locales after trimming causes ICU to fail to fall back to get the duration data for those locales. In addition, fix source/data/translit/root_subset.txt. Rule*Ids block has to be present even though it's empty. When dropping Hans-Hant transform rules, root_subset.txt was changed to be completely empty, which broke "components_unittests --g_test_filter=AutofillProfileComparato*" . With these changes, regenerate ICU data files. The size is slightly smaller. android/icudtl.dat 6573872 => 6573792 common/icudt*dat 10130560 => 10130480 BUG= 707515 , 677043 , 684609 TEST=components_unittests --gtest_filter=AutofillProfileComparato* TEST=ui_base_unittests --gtest_filter=L10nUtilTest.TimeDurationForm* R=derat@chromium.org Review-Url: https://codereview.chromium.org/2812943003 . [modify] https://crrev.com/d5c238dcc2e801210749f3bb421b9227fe1c0948/android/icudtl.dat [modify] https://crrev.com/d5c238dcc2e801210749f3bb421b9227fe1c0948/common/icudtb.dat [modify] https://crrev.com/d5c238dcc2e801210749f3bb421b9227fe1c0948/common/icudtl.dat [modify] https://crrev.com/d5c238dcc2e801210749f3bb421b9227fe1c0948/scripts/trim_data.sh [modify] https://crrev.com/d5c238dcc2e801210749f3bb421b9227fe1c0948/source/data/translit/root_subset.txt
,
Apr 11 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/a8550bc1391772685da6e86adc72a587bd964289 commit a8550bc1391772685da6e86adc72a587bd964289 Author: derat <derat@chromium.org> Date: Tue Apr 11 22:11:31 2017 Add test for TimeDurationFormat() error. base::TimeDurationFormat() fails due to a U_RESOURCE_TYPE_MISMATCH error from ICU when called with certain locals like en_GB due to missing resource data. Add a test (disabled for now) to verify that it works for all supported locales. BUG= 677043 , 707515 Review-Url: https://codereview.chromium.org/2793613004 Cr-Commit-Position: refs/heads/master@{#463796} [modify] https://crrev.com/a8550bc1391772685da6e86adc72a587bd964289/ui/base/l10n/l10n_util_unittest.cc
,
Apr 12 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/cf65b91bc7496b3dd2c958c890984e73c26ad22c commit cf65b91bc7496b3dd2c958c890984e73c26ad22c Author: jshin <jshin@chromium.org> Date: Wed Apr 12 00:34:12 2017 Roll third_party/icu from 450be73 to b34251f http://chromium.googlesource.com/chromium/deps/icu.git/+log/450be73..b34251f Changes include (along with minor data build script updates): 1. f0449ad Update IANA timezone db to 2017a from 2016i 2. c781b5f Drops unused Hans-Hant ICU transliteration data. ( cuts down the ICU data file size by ~25kB.) 3. d5c238d Update trim_data to deal with locale fallback failure for units and fix root_subset.txt in ICU's transliteration configuration. 4. b34251f Update timezone db to 2017b BUG= 684609 ,473288, 707515 , 677043 TEST=components_unittests --gtest_filter=AutofillProfileComparato* TEST=ui_base_unittests --gtest_filter=L10nUtilTest.TimeDurationForm* TEST=For timezone change test, see CL #1 and #4 TEST lines. TBR=riesa@chromium.org,derat@chromium.org Review-Url: https://codereview.chromium.org/2755963002 Cr-Commit-Position: refs/heads/master@{#463856} [modify] https://crrev.com/cf65b91bc7496b3dd2c958c890984e73c26ad22c/DEPS [modify] https://crrev.com/cf65b91bc7496b3dd2c958c890984e73c26ad22c/ui/base/l10n/l10n_util_unittest.cc
,
Apr 12 2017
Will ask for merge to M58 after baking in canary for a day or two. Just a DEPS roll would suffice in M58 branch.
,
Apr 18 2017
Verified in a canary build (en-GB locale: in the ash-tray, I confirmed that '1% - 2:28 until full' instead of '1% - until full' is shown; see bug 677043 ). I also confirmed it in es-419 locale. Requesting for merge to M58. It's rather late in the cycle, but it has regressed and affects en-GB and es-419 users. (again, see bug 677043 ).
,
Apr 18 2017
,
Apr 18 2017
This bug requires manual review: We are only 6 days from stable. Please contact the milestone owner if you have questions. Owners: amineer@(Android), cmasso@(iOS), bhthompson@(ChromeOS), govind@(Desktop) For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Apr 18 2017
There is a huge risk in taking a change post branch as we're cutting M58 stable RC today. Can you please confirm if this is critical? Based on comment 12, seems like it's already verified in Canary. Is there enough unit test coverage for this?
,
Apr 18 2017
And also seems like this bug has been there since M56? My recommendation is to wait until M59, but if you think it's critical and it's a safe merge, please let us know.
,
Apr 18 2017
Sorry that I forgot to give how safe/complicated a cherry-pick is. It'll be just a ICU data rebuild with no code change. Dan added a unittest to verify the fix (ICU). So, I think it's pretty safe. It's unfortunate that it's been broken since M56, but it's bad for en-GB and es-419 users.(no remaining hours for battery. how long to wait until full).
,
Apr 18 2017
Since change is affecting Chrome OS more, adding bhthompson@ for merge review.
,
Apr 18 2017
As we still have a beta before stable for CrOS, if engineering is confident this is a clean cut safe fix, we can merge this if it is only for CrOS.
,
Apr 20 2017
The following revision refers to this bug: https://chrome-internal.googlesource.com/chrome/tools/buildspec/+/cd0a1e71a466bf114861203e987fb7ed07e5efd6 commit cd0a1e71a466bf114861203e987fb7ed07e5efd6 Author: Jungshik Shin <jungshik@google.com> Date: Thu Apr 20 19:47:02 2017
,
Apr 21 2017
,
Apr 21 2017
|
|||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||
Comment 1 by derat@chromium.org
, Apr 1 2017