New issue
Advanced search Search tips

Issue 817804 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Mar 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux
Pri: 3
Type: Bug



Sign in to add a comment

[css-text] Problems in some WPT tests due to fonts in Linux

Project Member Reported by r...@igalia.com, Mar 1 2018

Issue description


The following 2 tests are failing on Linux in the tryjobs:
* external/wpt/css/css-text/i18n/css3-text-line-break-opclns-004.html
* external/wpt/css/css-text/i18n/css3-text-line-break-opclns-011.html

Check the results here:
https://test-results.appspot.com/data/layout_results/linux_chromium_rel_ng/37043/site_per_process_webkit_layout_tests%20%28with%20patch%29/layout-test-results/results.html

Locally they are working fine for me.
 
css3-text-line-break-opclns-004-actual.png
8.8 KB View Download

Comment 1 by e...@chromium.org, Mar 1 2018

Cc: -kojii@chromium.org
Owner: kojii@chromium.org
Status: Assigned (was: Untriaged)
Project Member

Comment 2 by bugdroid1@chromium.org, Mar 2 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/9f75a2842405ed0f378d8a4e60d6de2568704880

commit 9f75a2842405ed0f378d8a4e60d6de2568704880
Author: Manuel Rego Casasnovas <rego@igalia.com>
Date: Fri Mar 02 03:44:44 2018

[css-text] Link css-text WPT test suite failures to specific bugs

These tests were imported in r539150, the failures have been analized
and the bugs reported accordingly.
This patch modifies TestExpectations to link to the specific issues
for each test.

BUG= 306730 , 817773 , 817776 , 817778 , 817781 , 817804 

Cq-Include-Trybots: master.tryserver.chromium.linux:linux_layout_tests_layout_ng
Change-Id: Ibfb975f825188d1693d6526fef489fd76da3ec21
Reviewed-on: https://chromium-review.googlesource.com/943104
Commit-Queue: Koji Ishii <kojii@chromium.org>
Reviewed-by: Koji Ishii <kojii@chromium.org>
Cr-Commit-Position: refs/heads/master@{#540439}
[modify] https://crrev.com/9f75a2842405ed0f378d8a4e60d6de2568704880/third_party/WebKit/LayoutTests/FlagExpectations/enable-blink-features=LayoutNG
[modify] https://crrev.com/9f75a2842405ed0f378d8a4e60d6de2568704880/third_party/WebKit/LayoutTests/TestExpectations

Comment 3 by kojii@chromium.org, Mar 5 2018

Cc: kojii@chromium.org
Components: -Blink>Layout Blink>Infra
Owner: ----
Status: Available (was: Assigned)
The discussion in CL:
https://chromium-review.googlesource.com/c/chromium/src/+/945429

discovered that there are two issues:
1. Tests rely on specific width of glyphs, which is not appropriate as cross-platform tests.
2. Somehow bots don't have fonts for these code points.

I can take 1 in the CL above, but 2 looks like a test infra issue, so re-labeled.

Comment 4 by kojii@chromium.org, Mar 5 2018

Labels: OS-Linux
Cc: robertma@chromium.org
Components: Blink>Infra>Ecosystem
Question regarding 2 in c#3: Do you mean the test assumes the system has a CJK font? I think that's actually against the general guidelines of WPT (http://web-platform-tests.org/writing-tests/general-guidelines.html):

> Aside from the Ahem font, fonts cannot be relied on to be either installed or to have specific metrics. As such, in most cases when a known font is needed Ahem should be used. In other cases, @font-face should be used.

We should add a <meta> to these tests to specify the additional font requirement (e.g. <meta name="flags" content="font" />, see http://web-platform-tests.org/writing-tests/css-metadata.html), which would make these tests manual. Or we could use a CJK webfont (albeit sizes might be a problem).

A related discussion on the font dependency upstream: https://github.com/w3c/web-platform-tests/issues/9374

Comment 7 by kojii@chromium.org, Mar 5 2018

> Question regarding 2 in c#3: Do you mean the test assumes the system has a CJK font? I think that's actually against the general guidelines of WPT

No, CJK font is included as web font, but it tests line break between CJK and Tibetan, and the test does not include Tibetan fonts.
> No, CJK font is included as web font, but it tests line break between CJK and Tibetan, and the test does not include Tibetan fonts.

Than that's the issue there. A Tibetan font should be included as web font, too.

Comment 9 by kojii@chromium.org, Mar 5 2018

> Than that's the issue there. A Tibetan font should be included as web font, too.

agreed, and that should be addressed in CL
https://chromium-review.googlesource.com/c/chromium/src/+/945429

but rego commented that:
> I'm not 100% sure if it's better to fix the tests like this, or fix the bots configuration.

and my reply:
> I think we should fix both; bots failing only these tests is strange. Apart from that, tests relying on specific widths of glyphs and hard-coded the px isn't great, it will fail anytime with new fonts or on new platforms.

IIUC his concern is that, these glyphs show up in his local Linux but not on bots, when other tests such as U+17D4 KHMER SIGN KHAN pass on bots.

I'm fine not to fix bots if rego is fine.
Cc: r...@chromium.org
+rego@

Here's the list of fonts we guarantee available across the Chromium project (on developers' workstations & bots):
https://cs.chromium.org/chromium/src/third_party/content_shell_fonts/README.chromium
(and a Khmer font is included, for example)

Most of them are legacy dependencies of (especially non-WPT) layout tests. It's likely that the glyph in question is available via some font on rego@'s personal environment that is not on that list.

We could add a Tibetan font to that list if there are many tests rely on it. That said, the best fix is still to use web font to explicitly declare the dependency (especially in WPT).

Comment 11 by r...@igalia.com, Mar 6 2018

Cc: -r...@chromium.org r...@igalia.com
Yes sorry for all the confusion, it just was strange that it only failed on Linux bots
for some reason the test seems to be working fine in Window and Mac.

But I agree that the test should use a webfont if it wants to use a Tibetan character,
so I'm fine withe not changing anything in the bots.

It's the first time we import this test suite, that's why we're finding
this kind of issues. Some of the tests are old and probably not all of them
are right, so it's expected we find some problems like this every now and then.

Thanks kojii@ for investigating this, I didn't realize of the issue when
I initially reported the bug.

I agree that we can land the patch in https://chromium-review.googlesource.com/c/chromium/src/+/945429
and close this issue.
Project Member

Comment 12 by bugdroid1@chromium.org, Mar 6 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4c7a7ae9a9908138f6c8c50ffada35d626383669

commit 4c7a7ae9a9908138f6c8c50ffada35d626383669
Author: Koji Ishii <kojii@chromium.org>
Date: Tue Mar 06 11:26:14 2018

[css-text] Fix tests that rely on the specific width of glyphs

The following 2 tests:
* external/wpt/css/css-text/i18n/css3-text-line-break-opclns-004.html
* external/wpt/css/css-text/i18n/css3-text-line-break-opclns-011.html

are failing on specific configurations because they assume some
characters are installed on the test machine, and they have specific
widths, but [blink content shell fonts] does not include Tibetan
fonts. Not having Tibetan fonts on test configuration is allowed as
per the [wpt general guidelines].

This patch uses the [css-orientation-test] font that provides a 1em
square glyph for every Unicode code points.

[blink content shell fonts]: https://cs.chromium.org/chromium/src/third_party/content_shell_fonts/README.chromium
[css-orientation-test]: https://github.com/adobe-fonts/css-orientation-test
[wpt general guidelines]: http://web-platform-tests.org/writing-tests/general-guidelines.html

Bug:  817804 
Change-Id: I17a1b7d1b34b209303f8b6684644cff3d7ce62a0
Reviewed-on: https://chromium-review.googlesource.com/945429
Commit-Queue: Koji Ishii <kojii@chromium.org>
Reviewed-by: Manuel Rego Casasnovas <rego@igalia.com>
Cr-Commit-Position: refs/heads/master@{#541073}
[modify] https://crrev.com/4c7a7ae9a9908138f6c8c50ffada35d626383669/third_party/WebKit/LayoutTests/TestExpectations
[modify] https://crrev.com/4c7a7ae9a9908138f6c8c50ffada35d626383669/third_party/WebKit/LayoutTests/external/wpt/css/css-text/i18n/css3-text-line-break-opclns-004.html
[modify] https://crrev.com/4c7a7ae9a9908138f6c8c50ffada35d626383669/third_party/WebKit/LayoutTests/external/wpt/css/css-text/i18n/css3-text-line-break-opclns-111.html
[modify] https://crrev.com/4c7a7ae9a9908138f6c8c50ffada35d626383669/third_party/WebKit/LayoutTests/external/wpt/css/css-text/i18n/reference/css3-text-line-break-opclns-004-ref.html
[modify] https://crrev.com/4c7a7ae9a9908138f6c8c50ffada35d626383669/third_party/WebKit/LayoutTests/external/wpt/css/css-text/i18n/reference/css3-text-line-break-opclns-111-ref.html

Comment 13 by r...@igalia.com, Mar 6 2018

Owner: kojii@chromium.org
Status: Fixed (was: Available)

Sign in to add a comment