Example failure: https://luci-milo.appspot.com/buildbot/chromium.fyi/WebKit%20Mac%20-%20RandomOrder/37 Every build on WebKit Mac - RandomOrder up to #37 has failed because of this test. This should pass consistently on Mac: Tools/Scripts/run-webkit-tests svg/text This should fail intermittently on Mac: Tools/Scripts/run-webkit-tests svg/text --order=random We're working on removing order dependency from Blink layout tests. More context here: https://docs.google.com/document/d/11-628Ny6hY__kfIv__zVD0m3rA0OB9tQLLC_gJI_PNg/edit
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/8b8595351264b872ebf3b4c1aa13666641e3b3c6 commit 8b8595351264b872ebf3b4c1aa13666641e3b3c6 Author: jeffcarp <jeffcarp@chromium.org> Date: Wed Nov 30 19:38:57 2016 Add 2 Mac flaky tests to RandomOrderExpectations BUG= 663855 , 669693 Review-Url: https://codereview.chromium.org/2541703002 Cr-Commit-Position: refs/heads/master@{#435379} [modify] https://crrev.com/8b8595351264b872ebf3b4c1aa13666641e3b3c6/third_party/WebKit/LayoutTests/RandomOrderExpectations
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/eb52fcf8c84eb033f63d2839eb8805eb9ec137fe commit eb52fcf8c84eb033f63d2839eb8805eb9ec137fe Author: pdr <pdr@chromium.org> Date: Tue Jan 24 21:20:52 2017 Workaround platform differences in svg/text/combining-character-queries.html MacOS has consistent differences[1] in the glyphs of "ffi" with diacritics and the default cursive font (Apple Chancery). On some machines, two diacritics are drawn whereas on others only one is. This patch works around this difference by removing the ffi glyph from the test when the cursive family is used. Similarly[2], Linux has consistent differences in the space width preceding a missing glyph. On some machines, it is wider than others. This patch works around this difference by removing the space glyph in favor of a period. [1,2] http://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=webkit_tests&showExpectations=true&tests=svg%2Ftext%2Fcombining-character-queries.html BUG= 669693 Review-Url: https://codereview.chromium.org/2647293005 Cr-Commit-Position: refs/heads/master@{#445815} [modify] https://crrev.com/eb52fcf8c84eb033f63d2839eb8805eb9ec137fe/third_party/WebKit/LayoutTests/TestExpectations [modify] https://crrev.com/eb52fcf8c84eb033f63d2839eb8805eb9ec137fe/third_party/WebKit/LayoutTests/platform/linux/svg/text/combining-character-queries-expected.png [modify] https://crrev.com/eb52fcf8c84eb033f63d2839eb8805eb9ec137fe/third_party/WebKit/LayoutTests/platform/linux/svg/text/combining-character-queries-expected.txt [modify] https://crrev.com/eb52fcf8c84eb033f63d2839eb8805eb9ec137fe/third_party/WebKit/LayoutTests/platform/mac-mac10.10/svg/text/combining-character-queries-expected.png [modify] https://crrev.com/eb52fcf8c84eb033f63d2839eb8805eb9ec137fe/third_party/WebKit/LayoutTests/platform/mac-mac10.10/svg/text/combining-character-queries-expected.txt [modify] https://crrev.com/eb52fcf8c84eb033f63d2839eb8805eb9ec137fe/third_party/WebKit/LayoutTests/platform/mac-mac10.9/svg/text/combining-character-queries-expected.png [modify] https://crrev.com/eb52fcf8c84eb033f63d2839eb8805eb9ec137fe/third_party/WebKit/LayoutTests/platform/mac-mac10.9/svg/text/combining-character-queries-expected.txt [modify] https://crrev.com/eb52fcf8c84eb033f63d2839eb8805eb9ec137fe/third_party/WebKit/LayoutTests/platform/mac-retina/svg/text/combining-character-queries-expected.png [modify] https://crrev.com/eb52fcf8c84eb033f63d2839eb8805eb9ec137fe/third_party/WebKit/LayoutTests/platform/mac-retina/svg/text/combining-character-queries-expected.txt [modify] https://crrev.com/eb52fcf8c84eb033f63d2839eb8805eb9ec137fe/third_party/WebKit/LayoutTests/platform/mac/svg/text/combining-character-queries-expected.png [modify] https://crrev.com/eb52fcf8c84eb033f63d2839eb8805eb9ec137fe/third_party/WebKit/LayoutTests/platform/mac/svg/text/combining-character-queries-expected.txt [modify] https://crrev.com/eb52fcf8c84eb033f63d2839eb8805eb9ec137fe/third_party/WebKit/LayoutTests/platform/win/svg/text/combining-character-queries-expected.png [modify] https://crrev.com/eb52fcf8c84eb033f63d2839eb8805eb9ec137fe/third_party/WebKit/LayoutTests/platform/win/svg/text/combining-character-queries-expected.txt [modify] https://crrev.com/eb52fcf8c84eb033f63d2839eb8805eb9ec137fe/third_party/WebKit/LayoutTests/svg/text/combining-character-queries.html
The above patch worked for linux but I still have a failing MacOS RandomOrder bot: http://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=webkit_tests&showLargeExpectations=true&showExpectations=true&tests=svg%2Ftext%2Fcombining-character-queries.html I'll leave this patch in the tree since it's not blocking the CQ while I prepare a followup.
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/cf750a021a4e265f90b2d4a30db7cba0eb216d80 commit cf750a021a4e265f90b2d4a30db7cba0eb216d80 Author: pdr <pdr@chromium.org> Date: Thu Jan 26 01:41:05 2017 Workaround platform differences in svg/text/combining-character-queries.html (2) This is a followup to [1] which failed to fix MacOS differences [2] in glyphs with two diacritics. This patch skips the failing part of the test entirely. [1] https://chromium.googlesource.com/chromium/src/+/eb52fcf8c84eb033f63d2839eb8805eb9ec137fe [2] http://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=webkit_tests&showExpectations=true&tests=svg%2Ftext%2Fcombining-character-queries.html BUG= 669693 Review-Url: https://codereview.chromium.org/2650353005 Cr-Commit-Position: refs/heads/master@{#446197} [modify] https://crrev.com/cf750a021a4e265f90b2d4a30db7cba0eb216d80/third_party/WebKit/LayoutTests/TestExpectations [modify] https://crrev.com/cf750a021a4e265f90b2d4a30db7cba0eb216d80/third_party/WebKit/LayoutTests/platform/linux/svg/text/combining-character-queries-expected.png [modify] https://crrev.com/cf750a021a4e265f90b2d4a30db7cba0eb216d80/third_party/WebKit/LayoutTests/platform/linux/svg/text/combining-character-queries-expected.txt [modify] https://crrev.com/cf750a021a4e265f90b2d4a30db7cba0eb216d80/third_party/WebKit/LayoutTests/platform/mac-mac10.10/svg/text/combining-character-queries-expected.png [modify] https://crrev.com/cf750a021a4e265f90b2d4a30db7cba0eb216d80/third_party/WebKit/LayoutTests/platform/mac-mac10.10/svg/text/combining-character-queries-expected.txt [modify] https://crrev.com/cf750a021a4e265f90b2d4a30db7cba0eb216d80/third_party/WebKit/LayoutTests/platform/mac-mac10.9/svg/text/combining-character-queries-expected.png [modify] https://crrev.com/cf750a021a4e265f90b2d4a30db7cba0eb216d80/third_party/WebKit/LayoutTests/platform/mac-mac10.9/svg/text/combining-character-queries-expected.txt [delete] https://crrev.com/b07e16045b933d06587a8483731ffae24e21e6a3/third_party/WebKit/LayoutTests/platform/mac-retina/svg/text/combining-character-queries-expected.png [delete] https://crrev.com/b07e16045b933d06587a8483731ffae24e21e6a3/third_party/WebKit/LayoutTests/platform/mac-retina/svg/text/combining-character-queries-expected.txt [modify] https://crrev.com/cf750a021a4e265f90b2d4a30db7cba0eb216d80/third_party/WebKit/LayoutTests/platform/mac/svg/text/combining-character-queries-expected.png [modify] https://crrev.com/cf750a021a4e265f90b2d4a30db7cba0eb216d80/third_party/WebKit/LayoutTests/platform/mac/svg/text/combining-character-queries-expected.txt [modify] https://crrev.com/cf750a021a4e265f90b2d4a30db7cba0eb216d80/third_party/WebKit/LayoutTests/platform/win/svg/text/combining-character-queries-expected.png [modify] https://crrev.com/cf750a021a4e265f90b2d4a30db7cba0eb216d80/third_party/WebKit/LayoutTests/platform/win/svg/text/combining-character-queries-expected.txt [modify] https://crrev.com/cf750a021a4e265f90b2d4a30db7cba0eb216d80/third_party/WebKit/LayoutTests/svg/text/combining-character-queries.html
This appears to have worked: http://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=webkit_tests&showExpectations=true&tests=svg%2Ftext%2Fcombining-character-queries.html
Comment 1 by schenney@chromium.org
, Nov 29 2016