New issue
Advanced search Search tips

Issue 848747 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Jun 2018
Cc:
Components:
EstimatedDays: ----
NextAction: 2018-06-04
OS: ----
Pri: 1
Type: Task

Blocked on:
issue 848768



Sign in to add a comment

Flakey WebRTC test blocking Web Platform Test export

Project Member Reported by danyao@chromium.org, Jun 1 2018

Issue description

Filing this as Web Platform Test sheriff.

Web Platform Test exporter is blocked on https://chromium-review.googlesource.com/c/chromium/src/+/1074696 due to new flakiness in /webrtc/RTCPeerConnection-setRemoteDescription.html.

Out of 10 runs, 8 failed and 2 passed. Detailed run log here:
https://travis-ci.org/web-platform-tests/wpt/jobs/385761583

Please help investigate.


 

Comment 1 by hbos@chromium.org, Jun 1 2018

fippo do we know why this is flaking?
this should be resolved now that https://github.com/web-platform-tests/wpt/pull/11054 has been merged. See https://github.com/web-platform-tests/wpt/pull/11031 for the discussion.
https://github.com/web-platform-tests/wpt/pull/11267 passed after the same error when I reran it after 11054 had landed.

Comment 4 by hbos@chromium.org, Jun 1 2018

Status: Fixed (was: Untriaged)
Ok tentatively marking this as Fixed.
Awesome. Thanks for the quick response! The blocked PR I was looking at was https://github.com/web-platform-tests/wpt/pull/11177. I'll retry the tests there and report back if the issue is not fixed.
there still seems to be some flakyness.
Manually reloading that page on latest master I can see it alternating between 2 pass/3fail and 1 pass/4 fail.

The changing result is the "Negotiation should fire signalingsstate events" test. jan-ivar is looking into it, might be a chrome bug.

Comment 7 by hbos@chromium.org, Jun 1 2018

NextAction: 2018-06-04
Status: Started (was: Fixed)
Ok changing to Started, with a reminder to check back on the issue.
Blockedon: 848768
looks like its an actual flaky thing. Yay for tests exposing this :-)
Cc: foolip@chromium.org
The NextAction date has arrived: 2018-06-04

Comment 12 by hbos@chromium.org, Jun 4 2018

What are the implications of a flaky test blocking export?
Do failed PRs not get exported at all? Or does it sync them up later?
What happens to imports in changes to the same files?

Are the two repos out of sync?
They show up in https://foolip.github.io/ecosystem-infra-rotation/ and someone on the ecosystem infra team takes a look at how to resolve it. If it is stuck for 3-4 weeks, then for technical reasons it will be reverted from Chromium by import.

Once a fix for the flakiness is identified, then the most straightforward thing is to apply it in Chromium, and let both changes be exported. But wherever it is fixed, let me know and I'll make sure the final state is good.

Comment 14 by hbos@chromium.org, Jun 7 2018

foolip: Now https://chromium-review.googlesource.com/c/chromium/src/+/1085302 has landed (blocked-on issue fixed), can we get the export fired up again?

Comment 16 by hbos@chromium.org, Jun 11 2018

Does that mean everything has been exported, or does stuff that landed during the time the export was "blocked" require manual intervention?
danyao: is the test more stable now or do we need take more action?
Status: Fixed (was: Started)
The tests have been upstreamed to WPT.

Sign in to add a comment