Media LayoutTests using proprietary codecs are marked pass | fail |
|
Issue descriptionThis CL undoes a longstanding hack where we artificially remove support for proprietary codecs on all non-android bots In CL https://chromium-review.googlesource.com/c/chromium/src/+/849641 This allows bots with proprietary codecs to gain some coverage, but creates the unfortunate situation where we have mismatched expectations between bots with codecs vs bots without. To workaround this mismatch, the following tests are marked pass|fail: external/wpt/html/semantics/embedded-content/media-elements/mime-types/canPlayType.html external/wpt/media-source/mediasource-addsourcebuffer.html external/wpt/media-source/mediasource-buffered.html external/wpt/media-source/mediasource-config-change-mp4-a-bitrate.html external/wpt/media-source/mediasource-config-change-mp4-av-audio-bitrate.html external/wpt/media-source/mediasource-config-change-mp4-av-framesize.html external/wpt/media-source/mediasource-config-change-mp4-av-video-bitrate.html external/wpt/media-source/mediasource-config-change-mp4-v-bitrate.html external/wpt/media-source/mediasource-config-change-mp4-v-framerate.html external/wpt/media-source/mediasource-config-change-mp4-v-framesize.html external/wpt/media-source/mediasource-endofstream.html external/wpt/media-source/mediasource-is-type-supported.html external/wpt/media-source/mediasource-sequencemode-append-buffer.html external/wpt/media-source/mediasource-sourcebuffer-mode-timestamps.html Tests marked pass|fail are still not allowed to crash or timeout, so we do gain some benefit from running these. This bug serves to document the pass|fail TextExpectation and tracks any future work to improve the situation. Ideas considered include 1) Custom expecatations (like we do for Msan bots) for bots with codecs vs those without 2) Running tests requiring codecs as chrome browser_tests, and skipping as LayoutTests. Jrummell@ is exploring option 2 here: https://chromium-review.googlesource.com/c/chromium/src/+/792347 Assigning to him to update with the results of that effort.
,
Feb 16 2018
dpranke@, the [ Pass Failure ] expectation for external/wpt/media-source/mediasource-endofstream.html might be busted. I just saw this test "unexpectedly fail" in the build below? https://uberchromegw.corp.google.com/i/chromium.win/builders/Win7%20Tests%20%28dbg%29%281%29/builds/66377
,
Feb 16 2018
If you hunt around in the log files, you can eventually see that the test timed out: https://chromium-swarm.appspot.com/task?id=3bb4c23dd09fcc10&refresh=10&show_raw=1 which that expectation wouldn't be expecting. This would've been a lot clearer if the test results were actually being uploaded properly :(. That's bug 786034.
,
Feb 16 2018
I'll add "Timeout" to its expectation list.
,
Feb 16 2018
Here's the most recent build with the failure. https://ci.chromium.org/buildbot/chromium.win/Win7%20Tests%20%28dbg%29%281%29/66381
,
Feb 16 2018
Timeout is definitely a bug / failure and should be filed as another issue.
,
Feb 19 2018
Moving to bug 727252 regarding the failure of external/wpt/media-source/mediasource-endofstream.html
,
Feb 22 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/320860711ed6a447d609cbf8f5c277c9dd06bb7e commit 320860711ed6a447d609cbf8f5c277c9dd06bb7e Author: Takeshi Yoshino <tyoshino@chromium.org> Date: Thu Feb 22 15:05:26 2018 Include Timeout to the expectation for external/wpt/media-source/mediasource-endofstream.html See the bug for details. It can timeout in addition to failure. Bug: 807110, 727252 Change-Id: I7d7d10221a88e018bab768bf9e53ded20421ad9c Reviewed-on: https://chromium-review.googlesource.com/923685 Commit-Queue: Adam Rice <ricea@chromium.org> Reviewed-by: Adam Rice <ricea@chromium.org> Cr-Commit-Position: refs/heads/master@{#538435} [modify] https://crrev.com/320860711ed6a447d609cbf8f5c277c9dd06bb7e/third_party/WebKit/LayoutTests/TestExpectations
,
Mar 1 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/f3f0e3dae8bd1ccfc963c0c69810afaf1eaba22b commit f3f0e3dae8bd1ccfc963c0c69810afaf1eaba22b Author: Tsuyoshi Horo <horo@chromium.org> Date: Thu Mar 01 06:08:35 2018 Mark mediasource-config-change-mp4-v-bitrate.html [ Failure Pass Timeout ] https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=webkit_tests&tests=mediasource-config-change-mp4-v-bitrate.html TBR: jrummell@chromium.org Bug: 807110 Change-Id: Id44cc9fa07e779f03c74866b41399795dcaca7cf Reviewed-on: https://chromium-review.googlesource.com/942125 Reviewed-by: Tsuyoshi Horo <horo@chromium.org> Cr-Commit-Position: refs/heads/master@{#540069} [modify] https://crrev.com/f3f0e3dae8bd1ccfc963c0c69810afaf1eaba22b/third_party/WebKit/LayoutTests/TestExpectations
,
Mar 7 2018
mediasource-config-change-mp4-v-framerate.html times out as well. I filed https://crbug.com/819620 per the recommendation in #c6. |
|
►
Sign in to add a comment |
|
Comment 1 by bugdroid1@chromium.org
, Feb 2 2018