New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 613527 link

Starred by 9 users

Issue metadata

Status: Duplicate
Merged: issue 638852
Owner:
Last visit > 30 days ago
Closed: Nov 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 1
Type: Bug-Regression

Blocking:
issue chromedriver:1383



Sign in to add a comment

getComputedStyle() in headless mode returns numbers formatted in a locale-dependent way

Reported by kont...@mateuszwozniak.pl, May 20 2016

Issue description

UserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36

Example URL:

Steps to reproduce the problem:
When running Chrome in headless mode (e.g. during tests) getComputedStyle() returns incorrect values - e.g. transform matrix set with el.style.transform = 'matrix(0.1, 0.2, 0.3, 0.4, 0.5, 0.6)' retrieved next with window.getComputedStyle(el)['transform'] gives 'matrix(0,1, 0,2, 0,3, 0,4, 0,5, 0,6)'. The same applies to other properties (see tests.js in the repository).

Steps to see it locally:
1. Get this repository https://github.com/mateuszwozniak/chrome-karma-test
2. Run npm install
3. Run npm test

What is the expected behavior?
Tests should pass - returned numbers should be incorrect format, e.g. tested matrix should look like this one: matrix(0.1, 0.2, 0.3, 0.4, 0.5, 0.6)

What went wrong?
Tests are failing - returned numbers are in wrong format (fractional parts are separated with commas): matrix(0,1, 0,2, 0,3, 0,4, 0,5, 0,6)

Does it occur on multiple sites: Yes

Is it a problem with a plugin? No 

Did this work before? N/A 

Does this work in other browsers? Yes 

Chrome version: 50.0.2661.102  Channel: stable
OS Version: OS X 10.11.4
Flash Version: Shockwave Flash 21.0 r0

I can't reproduce that on Windows machine but it was tested on few OSX machines and all failed with the same result.
 
result.png
97.1 KB View Download
Just wanted to add that it worked few versions ago, but I'm not sure when exactly it has changed.

Comment 2 by f...@opera.com, May 20 2016

Components: -Blink Blink>CSS
Labels: Needs-TestConfirmation
No repro in stable 51 on OSX or Ubuntu. May have already been fixed?
1459938493920_thumbnail
0 bytes View Download

Comment 5 by timloh@chromium.org, May 23 2016

Issue 609221 has been merged into this issue.

Comment 6 by timloh@chromium.org, May 23 2016

Cc: thakis@chromium.org rob.b...@samsung.com
Labels: -Needs-TestConfirmation
Status: Available (was: Unconfirmed)
I think this is the same as 609221 (that bug was reported by a @google.com account so google permissions are required to view it). There's some more info over there with a user data directory which is supposedly sufficient to repro this.
Just tested Chrome 51.0.2704 on Mac OS X 10.11.4, and the problem still exists.

Comment 8 by r...@opera.com, May 30 2016

You need to have a locale which uses comma as a decimal separator. I'm seeing this with opacity in two webkit_unit_tests having Norwegian locale. Running with LC_ALL=C webkit_unit_tests, and it passes. Did we introduce locale-dependent number-to-string for getComputedStyle lately? Or did we remove a forced locale-setting?

You're right! Setting LC_ALL=C fixes the problem. But why it's different for headless and normal version?

Comment 10 by r...@opera.com, May 30 2016

Cc: timloh@chromium.org
My wild guess is that this call makes it work for render processes:

https://code.google.com/p/chromium/codesearch#chromium/src/content/app/content_main_runner.cc&l=233

and that webkit_unit_tests runs in a single process and does not have its numeric locale forced. I don't know about headless otherwise.


Comment 11 by r...@opera.com, May 30 2016

Cc: r...@opera.com

Comment 12 by r...@opera.com, May 31 2016

I started to see this in webkit_unit_tests not too long ago, but the failing tests were added Apr 11th this year, so the actual problem with webkit_unit_tests might be ancient.

getComputedStyle uses WTF::String::format() which uses vsnprintf which outputs according to LC_NUMERIC locale. We either need to serialize using a method that doesn't output locale-specific formatting, or force the LC_NUMERIC locale for all targets like we apparently do for the render processes in chrome and content_shell.

I looked at this a little, seems like on a Mac setting (for example) LC_NUMERIC="de_DE.UTF-8" when spinning up Chrome is enough to show this behaviour... I guess this makes sense from the code, since we only call setlocale(LC_NUMERIC, "C") conditional on "defined(OS_POSIX) && !defined(OS_MACOSX) && !defined(OS_ANDROID)" (testing on a Mac).

So we could either update all the users of WTF::String::format which use locale-dependant or work out if we can unconditionally set LC_NUMERIC.

Comment 14 by r...@opera.com, May 31 2016

So why isn't this a huge problem on Mac? Isn't LC_NUMERIC normally localized on e.g. German Mac installations?

Comment 15 by r...@opera.com, May 31 2016

I should note that my webkit_unit_tests problem is on Ubuntu Linux with Norwegian locale.

@14: Err... I have no idea... :/
I have Polish locale set by default and in that setup it breaks. Just tested changing to UK and everything works correctly then.
Without any of the npm stuff (just opening chrome normally), everything works fine, right?
Yes, exactly. I have dual setup - I run the tests in the browser during normal development (and everything's correct there) but before pushing the code I run the tests from the console (so with headless Chrome) and it's failing there. The configuration for both is exactly the same (from test runner point of view).
Just FYI in my case, I can reproduce this with a Chrome browser that runs with a head (uh, so, not headless ;-)), but by passing these args:

      --user-data-dir= + $TEMP_DIR
      --no-default-browser-check
      --no-first-run
      --disable-default-apps
      --disable-popup-blocking
      --disable-translate
      --disable-background-timer-throttling

So does not necessarily seem to be related to headlessness?

Comment 21 by evanm@google.com, Jun 2 2016

Hey,

Many years ago I fought with this.  I might even be the author of that setlocale that is linked in comment 10.  What I discovered is that:
1) various bits of code use functions like sprintf() to format decimal numbers
2) CSS expects decimals to have periods as decimal separators
3) on Linux, sprintf obeys your locale
4) it is extremely hard in an annoying fiddly way to change the code to use a decimal-only sprintf, as you have to get all the rounding behaviors to match unless you wanna rebaseline a million tests
5) on OSX they ignore the locale for sprintf() / let you control it separately

So we "fixed" it by always running the webkit process in a locale where sprintf works as webkit expected (with periods for decimals).  (Note that you can't just run all Chrome processes in the C locale because then e.g. file open dialogs aren't localized.)

If this regressed, I imagine it's due to something about #4 changed.  It seems unlikely that OS X would randomly break backwards compat itself in this area because it would break a lot of stuff.  My memory of how OS X worked in this area is vague -- something about there being an API to control this, maybe on a per-thread basis?  You should probably bring in a Mac wizard like Nico or Scott Hess.
printf behaviour doesn't seem to have changed:

$ cat test.c
#include <stdio.h>

int main() {
 printf("%f", 0.3);
}

$ LC_NUMERIC=de ./test
0.300000

Comment 23 by evanm@google.com, Jun 2 2016

Er, in my last paragraph above I meant to refer to "something about #5", not 4.

Also yes it was me!  Wow that was a long time ago!
https://src.chromium.org/viewvc/chrome?revision=73225&view=revision
re comment #22. 'setlocale' has to be called. Otherwise, it's C locale.

$ cat numeric.c
#include <locale.h>
#include <stdio.h>

int main() {
 setlocale(LC_ALL, "");
 printf("%f", 0.3);
}

$ LC_NUMERIC=de_DE.UTF-8 ./numeric 
0,300000

Anyway, one obvious solution is NOT to use *printf* when we want a locale-independent number format. An alternative is to use ICU's NumberFormat with "en" or "root" locale. 
Cc: js...@chromium.org
> NOT to use *printf* when we want a locale-independent number format

And, even when we want a locale-sensitive number format, using *printf* is not such a great idea (for a few reasons: the lack of across-platform uniformity, incomplete implementation, OS locale is not always Chrome's UI locale). 

> An alternative is to use ICU's NumberFormat with "en" or "root" locale. 

Another alternative is to port base::{Int*,Double}ToString to blink. Hmm... I think I've seen something similar already in Blink (while working on  bug 613331  - a subpart of bug 475351 - the opposite of this bug) . If there is, that has to be used instead of *printf*. 
String::format() in Webkit/../wtf uses *sprintf* and there are a huge number of callers in Blink. 

So, what happens if we just do on Mac OS X what we have done on Linux for render process (Evan's change referred to above:  setlocale(LC_NUMERIC, "C"))

https://code.google.com/p/chromium/codesearch#chromium/src/content/app/content_main_runner.cc&l=224

Comment 27 by r...@opera.com, Jun 3 2016

jshin@: Note that this is not a Mac-only problem. I have the same issue with webkit_unit_tests on Linux with Norwegian locale. I suspect the code in content_main_runner.cc is not invoked for webkit_unit_tests.

Labels: -OS-Mac OS-All
Summary: getComputedStyle() in headless mode returns numbers formatted in a locale-dependent way (was: getComputedStyle() in headless mode returns commas instead of dots for all numbers)
rune@: I'm changing the OS to All per your comment. 

Please, excuse me for my ignorance. Why does getComputedStyle() return a "dictionary" with values as strings instead of numbers?  

Anyway, I guess there are a few different ways 

1) Make sure Blink is run in C locale (which was done for render process on Linux, but may have to be extended to other platforms and other cases - unit tests, etc)

2) Fix all callers of String::format() where we do not want locale-specific number formatting. (there are tons of them and figuring out which to fix would take long)       

3) Fix String::format() not to use *printf*. (one possibility is rewrite using libicuio - ICU's io library - in root locale). 

Comment 29 by evanm@google.com, Jun 6 2016

getComputedStyle() is a web API, so we can't change it.  (It also returns strings for things like the "matrix(0.1, 0.2, ...)" as listed above.)

Here, I found the bug for you.
https://bugs.webkit.org/show_bug.cgi?id=18994
It looks like they think they already did change it to use a locale-independent formatter?
http://trac.webkit.org/changeset/69928

I wonder if they just missed some place?
Cc: samu...@chromium.org
 Issue 620380  has been merged into this issue.
Blocking: chromedriver:1383
Labels: -Type-Bug -Pri-2 M-53 Pri-1 Type-Bug-Regression
Dupe suggests this happens in regular chrome. The last time I looked at this it seemed a regression from https://codereview.chromium.org/1120133004
Project Member

Comment 33 by sheriffbot@chromium.org, Jul 9 2016

Labels: -M-53 M-54 MovedFrom-53
Moving this nonessential bug to the next milestone.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Owner: rob.buis@chromium.org
Status: Assigned (was: Available)

Comment 36 by meade@chromium.org, Nov 22 2016

Mergedinto: 638852
Status: Duplicate (was: Assigned)
I'm duplicating this with 638852 as they seem to have the same root cause (https://codereview.chromium.org/1120133004). Please comment on that bug with any progress on this. Thanks!

Sign in to add a comment