Able to reproduce the issue on the latest canary(64.0.3253.0) on Windows-10 and Mac OS 10.12.6 as well. This works fine on FF. Similar behavior is seen on older chrome version: 50.0.2646.0 as well. Hence marking this Untriaged for more inputs on this.

I remember now, the problem already existed in 2016: See  Issue 504178 .

$ ../harfbuzz/util/hb-shape --language="FR" Pecita-Latin.otf "17 Z"
[one.FRA=0+470|seven.FRA=1+470|space=2+470|Zstroke=3+480]

HarfBuzz handles this correctly. So something is wrong with passing the language information for the paragraph to the shaping process. I assume the language information does not arrive at HarfBuzzShaper.

Any ideas, Koji?

We pass "fr" to hb, and hb returns glyph id 18 for "1" and 24 for "7". Can hb-shape dump the glyph id?

Thanks for investigating. hb-shape shows glyph id when using the "--no-glyph-names" parameter:

$  ../harfbuzz/util/hb-shape --language="FR" --no-glyph-names Pecita-Latin.otf "17 Z"
[504=0+470|511=1+470|1=2+470|258=3+480]

$  ../harfbuzz/util/hb-shape --no-glyph-names Pecita-Latin.otf "17 Z"
[18=0+470|24=1+470|1=2+470|59=3+480]

At which level did you see the "fr" being passed? Do we perhaps lose it somewhere and it does not reach HarfBuzzShaper::ShapeRange [1]?
 
[1] https://cs.chromium.org/chromium/src/third_party/WebKit/Source/platform/fonts/shaping/HarfBuzzShaper.cpp?q=HarfBuzzShaper&sq=package:chromium&l=192 

1. We call |ToHarfbuzLanguage| for the given "lang" attribute value ("fr" in this case,) which calls |hb_language_from_string| and caches it.
https://cs.chromium.org/chromium/src/third_party/WebKit/Source/platform/LayoutLocale.cpp?type=cs&sq=package:chromium&l=107

2. It looks like it stores "fr" as is in |hb_language_impl_t|.
https://cs.chromium.org/chromium/src/third_party/harfbuzz-ng/src/hb-common.cc?type=cs&sq=package:chromium&l=167

3. This |hb_language_impl_t| is passed to |ShapeRange|, which is passed to |hb_buffer_set_language| before we call |hb_shape|.
https://cs.chromium.org/chromium/src/third_party/WebKit/Source/platform/fonts/shaping/HarfBuzzShaper.cpp?type=cs&sq=package:chromium&l=192

I thought we may need to convert lang code to OT tag, but |hb_buffer_set_language|
https://behdad.github.io/harfbuzz/harfbuzz-Buffers.html#hb-buffer-set-language
says we should convert ISO 639 to |hb_langauge_t| using |hb_language_from_string|.

Can you identify what we miss?

BTW, we pass HB_SCRIPT_COMMON for |hb_buffer_set_script| because they are digits. Don't know if the script affects 'locl' feature but just in case.

Thank you for your efficiency.

Yes, HB_SCRIPT_COMMON is the problem.
The script contain (is hierachically superior to) the local language.
In the font the locl feature is assigned to "latin{FRA }" and I solve the issue adding "DFLT{FRA }.
I think that I am right specifying only the latin script but not sure. May be it is a user error.
Please, ask Behdad Esfahbod or Khaled Hosny to have the good answer.

Behdad, do you have answer to #9?

The script itemization should take the whole paragraph into account (just like bidi itemization), and if done correctly this should have resolved the numbers as Latin script given that the paragraph contains other Latin characters.

Khaled, thank you once again.
It's a bit of what I sensed ... Clearly expressed! The granularity of the language is at the level of the sentence, not the glyph when it is shared across scripts.
For my font, I go back by removing the French language part of the default script, that does not make sense (but I admit that this architecture is confusing).
Kojii, do you agree?

What if it's:
  <p lang="fr">17</p>
or what if:
  <p lang="fr">17!!!</p>
or even:
  <p lang="fr">Some French 17 A-FEW-CHINESE-CHARACTERS and French</p>
?

I agree the heuristic script detection algorithms in each browsers can be improved further, but heuristic is heuristic. I'm Japanese, and I know detecting Chinese and Japanese would never be 100% accurate. I think it's a question that, do you want your fonts and typography to rely on heuristic, or to make it deterministic.

As long as digits are defined as COMMON in Unicode, it can never be perfect, just like bidi algorithm is not perfect and sometimes requires author's additional markups.

We can keep this bug as a feature request to improve the heuristic algorithm. I think we had a bug a few years ago but I can't find it.

> I think it's a question that, do you want your fonts and typography to rely on heuristic, or to make it deterministic.

Without any hesitation I prefer a heuristic script detection algorithm!!!
Why ? Exactly to be able to make versatile character fonts, consequently with many features!
Adding a language in a font is a nightmare! You want to do it for three characters and you have to plug it on to all the lookups tables. If in addition it is necessary to multiply by the number of scripts it becomes grotesque. If lang = "ja" can apply to different scripts, lang = "en" always implies the latin script. The problem is thus in the articulation of the ISO 639 standard to unicode, articulation that my knowledge is not standardized and nothing prevents common sense!
Can you find the algorithm used by Firefox? That would be a good reference, right?

Changed the summary as requested.

Heuristic for Latin and digits is easy, doing a good job for all scripts is hard. I'm not a fan of heuristic for that reason and for its non-interoperable nature, but I'm not opposed to it if contributers want to add it.

I'm a bit surprised that even when we write something like abc17def the script does not get resolved to Latin, at least I could not get the right digits to show up with this experiment. We should take a look at what ScriptRunIterator does with that.

> I'm a bit surprised that even when we write something like abc17def the script does not get resolved to Latin
Attached file

Deleted: Capture du 2017-11-02 10-52-13.png 167 KB

	Capture du 2017-11-02 10-52-13.png 167 KB View Download

FYI:
https://www.microsoft.com/typography/otspec/chapter2.htm#slTbl_sRec

> The 'DFLT' script would still be used if the text contained only the neutral characters, however.

(To #19)
This link is relevant (thank you for quoting it) for the structure of the font.
I do not think that it gives a definition of what assigns a script to a text.
For example:
"A script table with the script tag 'DFLT' (default) may be used in a font to define features that are not script-specific"
or
"A Script table identifies each language system that defines how to use the glyphs in a script for a particular language. It also references a default language system that defines how to use the script's glyphs in the absence of language-specific knowledge.".
Nothing talks about the association of a language to a particular script and nothing forbids to consider that when you specify the attribute lang in an html element then you force also implicitly a corresponding script and that this applies to all the text. I think that the question is here.

On a very general level, splitting a piece of text into script runs may work very well (e.g. splitting Arabic and Devanagari, because they need completely different processing), or not really very well (because it may be overkill when just putting glyphs side-by-side, and some character interactions across scripts (e.g. kerning) may be missed).

On a more specific level, if different rendering stacks do the splitting into script runs differently (e.g. around default script characters), and fonts are built with one or the other assumption, then things won't work very well. There may be some spec pieces missing.

In the case at hand, I'd try to include French shapes of the numbers 1 and 7 labeled as default script. Then unless there is some logic somewhere that throws out the information that this is French, it will find the right shapes independent of whether the numbers are included in a Latin run or in a default script run.

I understand now the dilemma of Koji. There is an urgent need for a "script" attribute in html if for each language if it is not clear which script it belongs to (althought, except Chromium, text editors know how to cope without it).
However, for "my" "french" problem I think that it is not a update whish but a regression (see comment #3).

I just found a big argument!
From http://www.adobe.com/devnet/opentype/afdko/topic_feature_file_syntax.html
"The only permitted language tag for the 'DFLT script is 'dflt'."
If the default script (the numbers among others) can not have a language, then they must inherit of script of the current language!
Please reopen this issue as a bug.

Thank you for the investigation.

My personal opinion is that you should raise this topic to OpenType spec discussion. Its langsys system has some limitations, and I think your use case is valid.

I would like this kind of glyph choice be deterministic and interoperable across browsers. I also personally don't think "17" and "17%" and "17z" showing different glyphs is a good experience.

> Please reopen this issue as a bug.

This issue is still open. It's just no one has raised their hands to work on actively yet.

We should implement script detection heuristic that Doug Felt implemented in Google internally.  I thought drott already did that, no?

Yes, that's in, compare  issue 526095  and https://codereview.chromium.org/1323513006

Then why are we not resolving Common script to Latin?

Adding

TEST_F(ScriptRunIteratorTest, LatinNumbers) {
  CHECK_SCRIPT_RUNS({{"abc17def", USCRIPT_LATIN}});
}

to ScriptRunIteratorTest passes, and abc17def renders correctly, looking at the screenshots, my comment #16 seems a little misguided.

Script segmentation happens after the word caching mechanisms splits by spaces. So for numbers that stand on their own, surrounded by spaces, the script resolution lacks context.

#23:
> From http://www.adobe.com/devnet/opentype/afdko/topic_feature_file_syntax.html
> "The only permitted language tag for the 'DFLT script is 'dflt'."

I learned from a font expert at Adobe that this sentence was removed from the most recent [AFDKO doc], at the [afdko github  issue 438 ].

Adobe seems to be using the technique for `locl` feature for punctuation, see "Language-sensitive Features" of [Ten Mincho blog].

To clarify my point, I do agree that improving script logic is a good thing, and with our new engine under development, we will be able to handle not only "abc17" but also "abc 17". But it's unlikely that we can solve "17." or "17!" without fonts having the DFLT script table.

[AFDKO doc]: https://github.com/adobe-type-tools/afdko/blob/develop/docs/OpenTypeFeatureFileSpecification.html
[afdko github  issue 438 ]: https://github.com/adobe-type-tools/afdko/issues/438
[Ten Mincho blog]: https://blogs.adobe.com/CCJKType/2017/11/ten-mincho.html