New issue
Advanced search Search tips

Issue 670843 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Jan 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug

Blocked on:
issue 670846

Blocking:
issue 601332
issue 671805



Sign in to add a comment

Get WebKit Win - RandomOrder over 90% passing over 20 most recent builds

Project Member Reported by jeffcarp@chromium.org, Dec 2 2016

Issue description

https://luci-milo.appspot.com/buildbot/chromium.fyi/WebKit%20Win%20-%20RandomOrder/?limit=200

All recent failing tests (from the last 63 test runs), ordered in number of failures:

fast/forms/search/search-rtl.html                                                                    failures: 12
paint/invalidation/multicol-with-abspos.html                                                         failures: 11
paint/invalidation/selection-change-in-iframe-with-relative-parent.html                              failures: 11
paint/invalidation/multicol-with-inline.html                                                         failures: 11
paint/invalidation/textarea-set-disabled.html                                                        failures: 11
paint/invalidation/multicol-with-abspos-in-relpos.html                                               failures: 11
paint/invalidation/table-hover-on-link.html                                                          failures: 11
paint/invalidation/multicol-as-paint-container.html                                                  failures: 11
paint/invalidation/paint-caret-in-div-with-negative-indent.html                                      failures: 11
paint/invalidation/multicol-with-text.html                                                           failures: 11
paint/invalidation/multicol-with-block.html                                                          failures: 11
paint/invalidation/inline-outline-repaint-2.html                                                     failures: 11
paint/invalidation/inline-reflow.html                                                                failures: 11
paint/invalidation/multicol-repaint.html                                                             failures: 10
virtual/mojo-loading/http/tests/inspector/debugger/fetch-breakpoints.html                            failures: 6
fast/text/ellipsis-stroked.html                                                                      failures: 4
imported/csswg-test/css-flexbox-1/flex-minimum-width-flex-items-003.xht                              failures: 2
imported/csswg-test/css-flexbox-1/flex-minimum-width-flex-items-001.xht                              failures: 2
imported/csswg-test/css-flexbox-1/flexbox_flex-natural-mixed-basis-auto.html                         failures: 2
fast/table/percent-height-replaced-content-in-cell.html                                              failures: 2
imported/csswg-test/vendor-imports/mozilla/mozilla-central-reftests/flexbox/flexbox-baseline-multi-item-vert-001.html failures: 2
http/tests/security/suborigins/suborigin-invalid-options.html                                        failures: 2
fast/history/history-length-append-subframe-with-hash.html                                           failures: 1
imported/csswg-test/vendor-imports/mozilla/mozilla-central-reftests/flexbox/flexbox-baseline-empty-001a.html failures: 1
virtual/mojo-loading/http/tests/security/img-crossorigin-no-credentials-prompt.html                  failures: 1
imported/csswg-test/vendor-imports/mozilla/mozilla-central-reftests/flexbox/flexbox-baseline-empty-001b.html failures: 1
fast/table/percent-height-overflow-visible-content-in-cell.html                                      failures: 1
http/tests/inspector/file-system-project-mapping.html                                                failures: 1
 
Going to file bugs for these tests and put them in RandomOrderExpectations:

- fast/forms/search/search-rtl.html
- paint/invalidation/*
Status: Started (was: Untriaged)
Blockedon: 670846
Bug filed for the 13 paint/invalidation tests. I'm going to hold off on the others and keep observing since they weren't in the most recent set of failures.
Cc: jeffcarp@chromium.org
 Issue 671395  has been merged into this issue.
Summary: Get WebKit Win - RandomOrder over 75% passing over 20 most recent builds (was: Builder WebKit Win - RandomOrder has 14 tests that fail ~17% of the time)
Going after goal of getting the Windows RandomOrder bot >75% passing over the 20 most recent builds. Since  bug 670846  landed on Dec 2, here are the failures:

67.79661016949152% passing over 59 builds (19 failures)

virtual/mojo-loading/http/tests/inspector/debugger/fetch-breakpoints.html                            failures: 5
fast/table/percent-height-replaced-content-in-cell.html                                              failures: 3
http/tests/security/suborigins/suborigin-invalid-options.html                                        failures: 2
http/tests/inspector/file-system-project-mapping.html                                                failures: 2
imported/csswg-test/vendor-imports/mozilla/mozilla-central-reftests/flexbox/flexbox-baseline-multi-item-vert-001.html failures: 2
imported/csswg-test/vendor-imports/mozilla/mozilla-central-reftests/flexbox/flexbox-baseline-empty-001b.html failures: 1
http/tests/preload/single_download_preload.html                                                      failures: 1
imported/csswg-test/css-flexbox-1/flex-minimum-width-flex-items-003.xht                              failures: 1
imported/csswg-test/css-flexbox-1/flexbox_flex-natural-mixed-basis-auto.html                         failures: 1
fast/css3-text/css3-word-break/word-break-break-all-in-span.html                                     failures: 1
css3/filters/effect-reference-rename.html                                                            failures: 1
virtual/mojo-loading/http/tests/security/img-crossorigin-no-credentials-prompt.html                  failures: 1
imported/csswg-test/css-flexbox-1/flex-minimum-width-flex-items-001.xht                              failures: 1
virtual/mojo-loading/http/tests/preload/single_download_preload.html                                 failures: 1

Creating a CL to ignore all above tests that have >1 failure.
Project Member

Comment 9 by bugdroid1@chromium.org, Dec 6 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/afc995d7f069af6a41491ae4b01a935128f3f3e4

commit afc995d7f069af6a41491ae4b01a935128f3f3e4
Author: jeffcarp <jeffcarp@chromium.org>
Date: Tue Dec 06 05:09:21 2016

Add 4 Windows tests to RandomOrderExpectations

BUG= 670843 
R=qyearsley@chromium.org

Review-Url: https://codereview.chromium.org/2555613002
Cr-Commit-Position: refs/heads/master@{#436532}

[modify] https://crrev.com/afc995d7f069af6a41491ae4b01a935128f3f3e4/third_party/WebKit/LayoutTests/RandomOrderExpectations

Project Member

Comment 10 by bugdroid1@chromium.org, Dec 13 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/41dbdcc8a8f30b9ca4e7869b72a2154ebe64553e

commit 41dbdcc8a8f30b9ca4e7869b72a2154ebe64553e
Author: jeffcarp <jeffcarp@chromium.org>
Date: Tue Dec 13 02:08:11 2016

Remove [ Win ] from RandomOrderExpectations

It doesn't seem like it's being respected. See build:
https://build.chromium.org/p/chromium.fyi/builders/WebKit%20Win%20-%20RandomOrder/builds/382

It's getting close to Wednesday - I'd like to remove [ Win ] so we can get onto more important work and then circle back around when we have more time to find the correct prefix.

R=qyearsley@chromium.org
BUG= 670843 

Review-Url: https://codereview.chromium.org/2560423003
Cr-Commit-Position: refs/heads/master@{#437986}

[modify] https://crrev.com/41dbdcc8a8f30b9ca4e7869b72a2154ebe64553e/third_party/WebKit/LayoutTests/RandomOrderExpectations

Summary: Get WebKit Win - RandomOrder over 90% passing over 20 most recent builds (was: Get WebKit Win - RandomOrder over 75% passing over 20 most recent builds)
WebKit Linux Trusty : 100.0% passing
WebKit Linux - RandomOrder: 100.0% passing
WebKit Mac - RandomOrder: 100.0% passing
WebKit Win - RandomOrder: 85.0% passing

o(〃^▽^〃)o

https://luci-milo.appspot.com/buildbot/chromium.fyi/WebKit%20Win%20-%20RandomOrder/?limit=200
Windows is at 0% passing due to a couple stubborn tests. I've submitted a CL for those:
https://codereview.chromium.org/2592473002

Getting Windows to 100% and moving it to random order is my #1 priority this week.
Blocking: 671805
Project Member

Comment 14 by bugdroid1@chromium.org, Dec 19 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/94c132f8f92fc509f27b4353075177a5363980c6

commit 94c132f8f92fc509f27b4353075177a5363980c6
Author: jeffcarp <jeffcarp@chromium.org>
Date: Mon Dec 19 21:34:09 2016

Add Win as a platform to two flaky tests in TestExpectations

BUG= 670843 
R=qyearsley@chromium.org,dpranke@chromium.org

Review-Url: https://codereview.chromium.org/2592473002
Cr-Commit-Position: refs/heads/master@{#439573}

[modify] https://crrev.com/94c132f8f92fc509f27b4353075177a5363980c6/third_party/WebKit/LayoutTests/TestExpectations

Project Member

Comment 15 by bugdroid1@chromium.org, Dec 28 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3298aed7ec682d9659dd9a2215658e9ce54146c5

commit 3298aed7ec682d9659dd9a2215658e9ce54146c5
Author: jeffcarp <jeffcarp@chromium.org>
Date: Wed Dec 28 21:06:22 2016

Uncomment fast/text/ellipsis-stroked.html in TestExpectations for Win

BUG= 670843 
R=qyearsley@chromium.org

Review-Url: https://codereview.chromium.org/2604853002
Cr-Commit-Position: refs/heads/master@{#440892}

[modify] https://crrev.com/3298aed7ec682d9659dd9a2215658e9ce54146c5/third_party/WebKit/LayoutTests/TestExpectations

Project Member

Comment 16 by bugdroid1@chromium.org, Jan 3 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/2b08f5972f60cf29e9594dddf610a20685dc0ef3

commit 2b08f5972f60cf29e9594dddf610a20685dc0ef3
Author: jeffcarp <jeffcarp@chromium.org>
Date: Tue Jan 03 07:56:57 2017

Comment out two duplicate lines in TestExpectations

One of the tests in question is failing 10% of the time on Windows.

BUG= 670843 
R=qyearsley@chromium.org

Review-Url: https://codereview.chromium.org/2602183002
Cr-Commit-Position: refs/heads/master@{#441097}

[modify] https://crrev.com/2b08f5972f60cf29e9594dddf610a20685dc0ef3/third_party/WebKit/LayoutTests/TestExpectations

Status: Fixed (was: Started)
Currently at 95% passing, 100% if you count a recent change to TestExpecations to ignore the one failing test. Closing.

Sign in to add a comment