RTCPeerConnection-track-stats.https.html is flaky |
|||
Issue descriptionFlakily, we can get this diff: PASS replaceTrack() after offer, before answer: new track attachment stats present PASS replaceTrack() after answer: new track attachment stats present FAIL replaceTrack(): original track attachment stats present after replacing assert_true: Has stats for original track expected true got false -PASS RTCRtpSender.getStats() contains only outbound-rtp and related stats -PASS RTCRtpReceiver.getStats() contains only inbound-rtp and related stats +FAIL RTCRtpSender.getStats() contains only outbound-rtp and related stats assert_equals: senderReport should contain candidate-pair stats expected 1 but got 0 +FAIL RTCRtpReceiver.getStats() contains only inbound-rtp and related stats assert_equals: receiverReport should contain candidate-pair stats expected 1 but got 0 PASS RTCPeerConnection.getStats(sendingTrack) is the same as RTCRtpSender.getStats() PASS RTCPeerConnection.getStats(receivingTrack) is the same as RTCRtpReceiver.getStats() PASS RTCPeerConnection.getStats(track) throws InvalidAccessError when there are zero senders or receivers for the track See https://isolateserver.appspot.com/browse?namespace=default-gzip&digest=6ee0957b1a7882bac1cea832c39b467e57f3c136&as=RTCPeerConnection-track-stats.https-diff.txt from https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Linux%20Tests/74581 Either the test is not set up properly to ensure that candidate pairs are always present or there is a bug.
,
Jan 4
Also this test has previously been marked as Timeout, is that still something that can happen?
,
Jan 4
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/07ee688b27e07d876b7b1c14f9fd4ba9fbee8bbf commit 07ee688b27e07d876b7b1c14f9fd4ba9fbee8bbf Author: Henrik Boström <hbos@chromium.org> Date: Fri Jan 04 10:02:33 2019 Fix RTCPeerConnection-track-stats.https-expected.txt files. The file is still marked as flaky due to https://crbug.com/919035 . If this still flakes we should add [ Pass Failure Timeout ] to both "track-stats" lines of the TestExpectations file and investigate in the referenced bug. TBR=hta@chromium.org Bug: 919035 Change-Id: Ief45b6504db40874439c4966eabf0e65d4363ae2 Reviewed-on: https://chromium-review.googlesource.com/c/1396117 Reviewed-by: Henrik Boström <hbos@chromium.org> Commit-Queue: Henrik Boström <hbos@chromium.org> Cr-Commit-Position: refs/heads/master@{#619903} [modify] https://crrev.com/07ee688b27e07d876b7b1c14f9fd4ba9fbee8bbf/third_party/blink/web_tests/TestExpectations [add] https://crrev.com/07ee688b27e07d876b7b1c14f9fd4ba9fbee8bbf/third_party/blink/web_tests/external/wpt/webrtc/RTCPeerConnection-track-stats.https-expected.txt
,
Jan 4
The -expected.txt file is now per #1 with a TestExpectations of [ Pass Failure ]. It is still flaky. Whether or not it ever times out is still unknown, let's see what the waterfall thinks.
,
Jan 4
I think I know why this is flaking, this import https://chromium-review.googlesource.com/c/chromium/src/+/1394189 changed the Resolver, there are several tests that are using "resolver.promise" which no longer exists. The Resolver is now extending Promise, meaning you can do "await resolver" instead of "await promise.resolver". This not only caused undefined reference errors but more sneaky flaky errors due to tests flakily skipping to wait for the connection.
,
Jan 4
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/e107337ecaa8f8fd6daaf189b3d2c4712479490d commit e107337ecaa8f8fd6daaf189b3d2c4712479490d Author: Henrik Boström <hbos@chromium.org> Date: Fri Jan 04 11:35:06 2019 Update tests that broke due to upstream changes of Resolver. Caused by import: https://chromium-review.googlesource.com/c/chromium/src/+/1394189 Fixes: - RTCPeerConnection-remote-track-mute.https.html - RTCPeerConnection-track-stats.https.html - RTCPeerConnection-setRemoteDescription-tracks.https.html The remote-track-mute case was already timing out but would have hit a problem but it was there for implementations that gets further. The track-stats problem was in a helper method, it simply caused the helper to not await for the last step, leading to flaky tests using that helper. The SRD-tracks problem had caused Failure instead of Pass, updated the -expected.txt file and fixed the test. TBR=hta@chromium.org Bug: 918768 , 919035 Change-Id: I02b855ef411ac1fd7c74d96c8c52aab6fe738201 Reviewed-on: https://chromium-review.googlesource.com/c/1396140 Commit-Queue: Henrik Boström <hbos@chromium.org> Reviewed-by: Harald Alvestrand <hta@chromium.org> Reviewed-by: Henrik Boström <hbos@chromium.org> Cr-Commit-Position: refs/heads/master@{#619911} [modify] https://crrev.com/e107337ecaa8f8fd6daaf189b3d2c4712479490d/third_party/blink/web_tests/external/wpt/webrtc/RTCPeerConnection-remote-track-mute.https.html [modify] https://crrev.com/e107337ecaa8f8fd6daaf189b3d2c4712479490d/third_party/blink/web_tests/external/wpt/webrtc/RTCPeerConnection-setRemoteDescription-tracks.https-expected.txt [modify] https://crrev.com/e107337ecaa8f8fd6daaf189b3d2c4712479490d/third_party/blink/web_tests/external/wpt/webrtc/RTCPeerConnection-setRemoteDescription-tracks.https.html [modify] https://crrev.com/e107337ecaa8f8fd6daaf189b3d2c4712479490d/third_party/blink/web_tests/external/wpt/webrtc/RTCPeerConnection-track-stats.https.html [modify] https://crrev.com/e107337ecaa8f8fd6daaf189b3d2c4712479490d/third_party/blink/web_tests/virtual/webrtc-wpt-unified-plan/external/wpt/webrtc/RTCPeerConnection-setRemoteDescription-tracks.https-expected.txt
,
Jan 4
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/f134c4d382beafc3f268e6b445c8d1bf511d57c1 commit f134c4d382beafc3f268e6b445c8d1bf511d57c1 Author: Henrik Boström <hbos@chromium.org> Date: Fri Jan 04 12:43:02 2019 Mark RTCPeerConnection-track-stats.https.html no longer flaky. Now that the bugs caused by the Resolver having updated have been fixed in https://bugs.chromium.org/p/chromium/issues/detail?id=919035, these tests are no longer expected to be flaky. TBR=hta@chromium.org Bug: 919035 Change-Id: I60c421759c69f60e620d76e862aaad5d858bc489 Reviewed-on: https://chromium-review.googlesource.com/c/1396024 Reviewed-by: Henrik Boström <hbos@chromium.org> Reviewed-by: Harald Alvestrand <hta@chromium.org> Commit-Queue: Henrik Boström <hbos@chromium.org> Cr-Commit-Position: refs/heads/master@{#619923} [modify] https://crrev.com/f134c4d382beafc3f268e6b445c8d1bf511d57c1/third_party/blink/web_tests/TestExpectations
,
Jan 4
I'm speculating this is fixed until evidence to the contrary. |
|||
►
Sign in to add a comment |
|||
Comment 1 by hbos@chromium.org
, Jan 4