New issue
Advanced search Search tips

Issue 812635 link

Starred by 1 user

Issue metadata

Status: Assigned
Owner: ----
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: ----



Sign in to add a comment

flaky tests external/wpt/css/vendor-imports/mozilla/mozilla-central-reftests/flexbox/flexbox-writing-mode-0XX.html

Project Member Reported by tschumann@chromium.org, Feb 15 2018

Issue description

fails a lot on mac_chromium_rel_ng trybots: 
https://ci.chromium.org/buildbot/tryserver.chromium.mac/mac_chromium_rel_ng/?limit=200

Tests were added in https://chromium-review.googlesource.com/c/chromium/src/+/919524

Will roll-back that CL instead of disabling the flaky tests (as I don't know which ones are flaky) and want to get the try bots in shape again quickly.
 
Project Member

Comment 1 by bugdroid1@chromium.org, Feb 15 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/91ce9aa2ca57beed6cb1f819438d483688f22c85

commit 91ce9aa2ca57beed6cb1f819438d483688f22c85
Author: Tim Schumann <tschumann@chromium.org>
Date: Thu Feb 15 14:29:46 2018

Revert "Manually import wpt@08bc228af02856e05fc4263044943df751bbc114"

This reverts commit d9adce5454ff3d64725bdb45c57216fb26381a15.

Reason for revert: New tests cause flakes on trybots
Saw test failues for all flexbox-writing-mode-0??.html tests.
(https://ci.chromium.org/buildbot/tryserver.chromium.mac/mac_chromium_rel_ng/?limit=200)

BUG=812635

Original change's description:
> Manually import wpt@08bc228af02856e05fc4263044943df751bbc114
> 
> Manual import since automatic imports are blocked by  crbug.com/811976 .
> 
> Note to sheriffs: This CL imports external tests and adds
> expectations for those tests; if this CL is large and causes
> a few new failures, please fix the failures by adding new
> lines to TestExpectations rather than reverting. See:
> https://chromium.googlesource.com/chromium/src/+/master/docs/testing/web_platform_tests.md
> 
> Directory owners for changes in this CL:
> cbiesinger@chromium.org:
>   external/wpt/css/vendor-imports/mozilla/mozilla-central-reftests/flexbox
> kozyatinskiy@chromium.org:
>   external/wpt/console
> 
> TBR=kereliuk
> 
> No-Export: true
> Change-Id: Ic5ce3adf7c2b5aabd310af2c853931da441fb302
> Reviewed-on: https://chromium-review.googlesource.com/919524
> Commit-Queue: Ali Juma <ajuma@chromium.org>
> Reviewed-by: Ali Juma <ajuma@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#536866}

TBR=ajuma@chromium.org,kereliuk@chromium.org

Change-Id: I3bc675afbc10b6a3c4d970f8bc2aed4e6084c89f
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/921721
Reviewed-by: Tim Schumann <tschumann@chromium.org>
Commit-Queue: Tim Schumann <tschumann@chromium.org>
Cr-Commit-Position: refs/heads/master@{#537010}
[modify] https://crrev.com/91ce9aa2ca57beed6cb1f819438d483688f22c85/third_party/WebKit/LayoutTests/FlagExpectations/site-per-process
[modify] https://crrev.com/91ce9aa2ca57beed6cb1f819438d483688f22c85/third_party/WebKit/LayoutTests/NeverFixTests
[modify] https://crrev.com/91ce9aa2ca57beed6cb1f819438d483688f22c85/third_party/WebKit/LayoutTests/TestExpectations
[modify] https://crrev.com/91ce9aa2ca57beed6cb1f819438d483688f22c85/third_party/WebKit/LayoutTests/external/WPT_BASE_MANIFEST.json
[delete] https://crrev.com/88a96c6e1b592b920cd00655b88225334c0c4541/third_party/WebKit/LayoutTests/external/wpt/console/console-format-specifier-symbol-manual.html
[add] https://crrev.com/91ce9aa2ca57beed6cb1f819438d483688f22c85/third_party/WebKit/LayoutTests/external/wpt/css/css-backgrounds/background-331-expected.txt
[modify] https://crrev.com/91ce9aa2ca57beed6cb1f819438d483688f22c85/third_party/WebKit/LayoutTests/external/wpt/css/css-backgrounds/background-331.html
[modify] https://crrev.com/91ce9aa2ca57beed6cb1f819438d483688f22c85/third_party/WebKit/LayoutTests/external/wpt/css/css-backgrounds/background-332-expected.txt
[modify] https://crrev.com/91ce9aa2ca57beed6cb1f819438d483688f22c85/third_party/WebKit/LayoutTests/external/wpt/css/css-backgrounds/background-332.html
[add] https://crrev.com/91ce9aa2ca57beed6cb1f819438d483688f22c85/third_party/WebKit/LayoutTests/external/wpt/css/css-backgrounds/background-333-expected.txt
[modify] https://crrev.com/91ce9aa2ca57beed6cb1f819438d483688f22c85/third_party/WebKit/LayoutTests/external/wpt/css/css-backgrounds/background-333.html
[delete] https://crrev.com/88a96c6e1b592b920cd00655b88225334c0c4541/third_party/WebKit/LayoutTests/external/wpt/css/mediaqueries/viewport-script-dynamic-ref.html
[delete] https://crrev.com/88a96c6e1b592b920cd00655b88225334c0c4541/third_party/WebKit/LayoutTests/external/wpt/css/mediaqueries/viewport-script-dynamic.html
[delete] https://crrev.com/88a96c6e1b592b920cd00655b88225334c0c4541/third_party/WebKit/LayoutTests/external/wpt/css/vendor-imports/mozilla/mozilla-central-reftests/flexbox/flexbox-basic-block-horiz-001v.xhtml
[delete] https://crrev.com/88a96c6e1b592b920cd00655b88225334c0c4541/third_party/WebKit/LayoutTests/external/wpt/css/vendor-imports/mozilla/mozilla-central-reftests/flexbox/flexbox-basic-block-vert-001v.xhtml
[delete] https://crrev.com/88a96c6e1b592b920cd00655b88225334c0c4541/third_party/WebKit/LayoutTests/external/wpt/css/vendor-imports/mozilla/mozilla-central-reftests/flexbox/flexbox-basic-canvas-horiz-001v.xhtml
[delete] https://crrev.com/88a96c6e1b592b920cd00655b88225334c0c4541/third_party/WebKit/LayoutTests/external/wpt/css/vendor-imports/mozilla/mozilla-central-reftests/flexbox/flexbox-basic-canvas-vert-001v.xhtml
[delete] https://crrev.com/88a96c6e1b592b920cd00655b88225334c0c4541/third_party/WebKit/LayoutTests/external/wpt/css/vendor-imports/mozilla/mozilla-central-reftests/flexbox/flexbox-intrinsic-ratio-001v.html
[delete] https://crrev.com/88a96c6e1b592b920cd00655b88225334c0c4541/third_party/WebKit/LayoutTests/external/wpt/css/vendor-imports/mozilla/mozilla-central-reftests/flexbox/flexbox-intrinsic-ratio-002v.html
[delete] https://crrev.com/88a96c6e1b592b920cd00655b88225334c0c4541/third_party/WebKit/LayoutTests/external/wpt/css/vendor-imports/mozilla/mozilla-central-reftests/flexbox/flexbox-intrinsic-ratio-003v.html
[delete] https://crrev.com/88a96c6e1b592b920cd00655b88225334c0c4541/third_party/WebKit/LayoutTests/external/wpt/css/vendor-imports/mozilla/mozilla-central-reftests/flexbox/flexbox-intrinsic-ratio-004v.html
[delete] https://crrev.com/88a96c6e1b592b920cd00655b88225334c0c4541/third_party/WebKit/LayoutTests/external/wpt/css/vendor-imports/mozilla/mozilla-central-reftests/flexbox/flexbox-intrinsic-ratio-005v.html
[delete] https://crrev.com/88a96c6e1b592b920cd00655b88225334c0c4541/third_party/WebKit/LayoutTests/external/wpt/css/vendor-imports/mozilla/mozilla-central-reftests/flexbox/flexbox-intrinsic-ratio-006v.html
[delete] https://crrev.com/88a96c6e1b592b920cd00655b88225334c0c4541/third_party/WebKit/LayoutTests/external/wpt/css/vendor-imports/mozilla/mozilla-central-reftests/flexbox/flexbox-mbp-horiz-002v.xhtml
[delete] https://crrev.com/88a96c6e1b592b920cd00655b88225334c0c4541/third_party/WebKit/LayoutTests/external/wpt/css/vendor-imports/mozilla/mozilla-central-reftests/flexbox/flexbox-mbp-horiz-003v.xhtml
[delete] https://crrev.com/88a96c6e1b592b920cd00655b88225334c0c4541/third_party/WebKit/LayoutTests/external/wpt/css/vendor-imports/mozilla/mozilla-central-reftests/flexbox/flexbox-writing-mode-010-ref.html
[delete] https://crrev.com/88a96c6e1b592b920cd00655b88225334c0c4541/third_party/WebKit/LayoutTests/external/wpt/css/vendor-imports/mozilla/mozilla-central-reftests/flexbox/flexbox-writing-mode-010.html
[delete] https://crrev.com/88a96c6e1b592b920cd00655b88225334c0c4541/third_party/WebKit/LayoutTests/external/wpt/css/vendor-imports/mozilla/mozilla-central-reftests/flexbox/flexbox-writing-mode-011-ref.html
[delete] https://crrev.com/88a96c6e1b592b920cd00655b88225334c0c4541/third_party/WebKit/LayoutTests/external/wpt/css/vendor-imports/mozilla/mozilla-central-reftests/flexbox/flexbox-writing-mode-011.html
[delete] https://crrev.com/88a96c6e1b592b920cd00655b88225334c0c4541/third_party/WebKit/LayoutTests/external/wpt/css/vendor-imports/mozilla/mozilla-central-reftests/flexbox/flexbox-writing-mode-012-ref.html
[delete] https://crrev.com/88a96c6e1b592b920cd00655b88225334c0c4541/third_party/WebKit/LayoutTests/external/wpt/css/vendor-imports/mozilla/mozilla-central-reftests/flexbox/flexbox-writing-mode-012.html
[delete] https://crrev.com/88a96c6e1b592b920cd00655b88225334c0c4541/third_party/WebKit/LayoutTests/external/wpt/css/vendor-imports/mozilla/mozilla-central-reftests/flexbox/flexbox-writing-mode-013-ref.html
[delete] https://crrev.com/88a96c6e1b592b920cd00655b88225334c0c4541/third_party/WebKit/LayoutTests/external/wpt/css/vendor-imports/mozilla/mozilla-central-reftests/flexbox/flexbox-writing-mode-013.html
[delete] https://crrev.com/88a96c6e1b592b920cd00655b88225334c0c4541/third_party/WebKit/LayoutTests/external/wpt/css/vendor-imports/mozilla/mozilla-central-reftests/flexbox/flexbox-writing-mode-014-ref.html
[delete] https://crrev.com/88a96c6e1b592b920cd00655b88225334c0c4541/third_party/WebKit/LayoutTests/external/wpt/css/vendor-imports/mozilla/mozilla-central-reftests/flexbox/flexbox-writing-mode-014.html
[delete] https://crrev.com/88a96c6e1b592b920cd00655b88225334c0c4541/third_party/WebKit/LayoutTests/external/wpt/css/vendor-imports/mozilla/mozilla-central-reftests/flexbox/flexbox-writing-mode-015-ref.html
[delete] https://crrev.com/88a96c6e1b592b920cd00655b88225334c0c4541/third_party/WebKit/LayoutTests/external/wpt/css/vendor-imports/mozilla/mozilla-central-reftests/flexbox/flexbox-writing-mode-015.html
[delete] https://crrev.com/88a96c6e1b592b920cd00655b88225334c0c4541/third_party/WebKit/LayoutTests/external/wpt/custom-elements/Document-createElement-expected.txt
[modify] https://crrev.com/91ce9aa2ca57beed6cb1f819438d483688f22c85/third_party/WebKit/LayoutTests/external/wpt/custom-elements/Document-createElement.html
[delete] https://crrev.com/88a96c6e1b592b920cd00655b88225334c0c4541/third_party/WebKit/LayoutTests/external/wpt/custom-elements/Document-createElementNS-expected.txt
[modify] https://crrev.com/91ce9aa2ca57beed6cb1f819438d483688f22c85/third_party/WebKit/LayoutTests/external/wpt/custom-elements/Document-createElementNS.html
[delete] https://crrev.com/88a96c6e1b592b920cd00655b88225334c0c4541/third_party/WebKit/LayoutTests/external/wpt/custom-elements/builtin-coverage-expected.txt
[modify] https://crrev.com/91ce9aa2ca57beed6cb1f819438d483688f22c85/third_party/WebKit/LayoutTests/external/wpt/custom-elements/builtin-coverage.html
[delete] https://crrev.com/88a96c6e1b592b920cd00655b88225334c0c4541/third_party/WebKit/LayoutTests/external/wpt/custom-elements/parser/serializing-html-fragments-expected.txt
[delete] https://crrev.com/88a96c6e1b592b920cd00655b88225334c0c4541/third_party/WebKit/LayoutTests/external/wpt/custom-elements/parser/serializing-html-fragments.html
[delete] https://crrev.com/88a96c6e1b592b920cd00655b88225334c0c4541/third_party/WebKit/LayoutTests/external/wpt/custom-elements/upgrading/Node-cloneNode-expected.txt
[modify] https://crrev.com/91ce9aa2ca57beed6cb1f819438d483688f22c85/third_party/WebKit/LayoutTests/external/wpt/custom-elements/upgrading/Node-cloneNode.html
[modify] https://crrev.com/91ce9aa2ca57beed6cb1f819438d483688f22c85/third_party/WebKit/LayoutTests/external/wpt/lint.whitelist

Comment 2 by ajuma@chromium.org, Feb 15 2018

Cc: -kereliuk@chromium.org ajuma@chromium.org
Owner: kereliuk@chromium.org
Handing off to today's ecosystem-infra sheriff. Note that liberally adding lines to TestExpectations is still the right way to go here, since even if we don't manually import these tests, the auto-importer will import them once it's up and running again (and that'll bring back the flakiness if we don't have lines in TestExpectations).
that's good to know. Removing these tests fixed the flakiness we were seeing. Keep in mind that there's another source of flakiness going on (crbug/812632) causing tests to timeout without receiving any logs.
Labels: -Sheriff-Chromium
Status: Assigned (was: Available)
Owner: ----

Sign in to add a comment