New issue
Advanced search Search tips

Issue 811349 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Feb 2018
Cc:
EstimatedDays: ----
NextAction: ----
OS: Mac
Pri: 2
Type: ----



Sign in to add a comment

texImage-imageBitmap-from-image-resize.html flaky on mac10.11

Project Member Reported by sheriff-...@appspot.gserviceaccount.com, Feb 12 2018

Issue description

Filed by sheriff-o-matic@appspot.gserviceaccount.com on behalf of liberato@google.com

...... too many results, data snipped.... and 40 other(s) in webkit_layout_tests failing on chromium.webkit/WebKit Mac10.11

Builders failed on: 
- WebKit Mac10.11: 
  https://build.chromium.org/p/chromium.webkit/builders/WebKit%20Mac10.11

Looks like the bottom quarter of the test image has small pixel differences, sometimes. (https://test-results.appspot.com/data/layout_results/WebKit_Mac10_11/28821/layout-test-results/results.html).  first instance i see is on feb 11.  will update with revision range.
 
Labels: OS-Mac
Owner: mtklein@chromium.org
Status: Assigned (was: Available)
approximate revision range is:

https://chromium.googlesource.com/chromium/src/+log/121b6cfa09886d1dcf0ba9158d02c3555a7dbfde%5E..58525850de9c493fd4f59e008d9e9ed44534fe02?pretty=fuller&n=

mtklien, i've no idea if https://chromium-review.googlesource.com/911734 is the cause or just the unlucky messenger, but i suspect that you have more context than i do.  thanks!


Hmm.  That's definitely from that change, but it's not expected that the images are not stable.

I'm surprised that only the one Mac 10.11 bot is flaky, and not the other Mac, Linux or Windows bots.  Do you happen to know if these tests run on a variety of machines with different CPU feature sets?  I thought we disabled our runtime CPU feature detection in layout tests... I'll have to double check.

In the meantime, is it possible to disable this test or mark it as flaky?  We should be getting plenty of coverage from the other x86 layout test bots, and as you can see this  diff isn't really even visible.
looks like just one (virtual?) bot on this builder: https://ci.chromium.org/buildbot/chromium.webkit/WebKit%20Mac10.11/

does seem to flake though:
good: https://ci.chromium.org/buildbot/chromium.webkit/WebKit%20Mac10.11/28861
bad: https://ci.chromium.org/buildbot/chromium.webkit/WebKit%20Mac10.11/28843

as for disabling it, yes, it can be done.  one may add an entry to third_party/WebKit/LayoutTests/TestExpectations with the os set to "Mac10.11".

however, since there's just one bot, seems like seems like there's actually something nondeterministic going on.

i suppose it could also be some uninteresting (to chrome, anyway) differences in the vm, which the infra folks might still care about.
Cc: -liberato@google.com liber...@chromium.org
Cc: fmalita@chromium.org
Status: Fixed (was: Assigned)
I think this is fixed and the bot was just late to mention it?  I landed the first version of this CL yesterday (beginning of a long steady run of red on https://ci.chromium.org/buildbot/chromium.webkit/WebKit%20Mac10.11/?limit=100), and it was also reverted yesterday (ending that run of red).

I happened to reland the CL today after re-rebaselining, and I'm not seeing any new failures beyond one unrelated looking one (a timeout?).  Please re-open if things still seem broken at head?
I did double check and unless something's gone very wrong, we do unconditionally turn off CPU detection when running layout tests.  Each binary should produce stable images across multiple machines, unless you get real GPUs involved...

Sign in to add a comment