New issue
Advanced search Search tips

Issue 807110 link

Starred by 1 user

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

Media LayoutTests using proprietary codecs are marked pass | fail

Project Member Reported by chcunningham@chromium.org, Jan 30 2018

Issue description

This CL undoes a longstanding hack where we artificially remove support for proprietary codecs on all non-android bots
In CL https://chromium-review.googlesource.com/c/chromium/src/+/849641 

This allows bots with proprietary codecs to gain some coverage, but creates the unfortunate situation where we have mismatched expectations between bots with codecs vs bots without. To workaround this mismatch, the following tests are marked pass|fail:

external/wpt/html/semantics/embedded-content/media-elements/mime-types/canPlayType.html
external/wpt/media-source/mediasource-addsourcebuffer.html
external/wpt/media-source/mediasource-buffered.html
external/wpt/media-source/mediasource-config-change-mp4-a-bitrate.html
external/wpt/media-source/mediasource-config-change-mp4-av-audio-bitrate.html
external/wpt/media-source/mediasource-config-change-mp4-av-framesize.html
external/wpt/media-source/mediasource-config-change-mp4-av-video-bitrate.html
external/wpt/media-source/mediasource-config-change-mp4-v-bitrate.html
external/wpt/media-source/mediasource-config-change-mp4-v-framerate.html
external/wpt/media-source/mediasource-config-change-mp4-v-framesize.html
external/wpt/media-source/mediasource-endofstream.html
external/wpt/media-source/mediasource-is-type-supported.html
external/wpt/media-source/mediasource-sequencemode-append-buffer.html
external/wpt/media-source/mediasource-sourcebuffer-mode-timestamps.html

Tests marked pass|fail are still not allowed to crash or timeout, so we do gain some benefit from running these. 

This bug serves to document the pass|fail TextExpectation and tracks any future work to improve the situation. Ideas considered include

1) Custom expecatations (like we do for Msan bots) for bots with codecs vs those without
2) Running tests requiring codecs as chrome browser_tests, and skipping as LayoutTests.

Jrummell@ is exploring option 2 here:
https://chromium-review.googlesource.com/c/chromium/src/+/792347

Assigning to him to update with the results of that effort. 
 
Project Member

Comment 1 by bugdroid1@chromium.org, Feb 2 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/38370805b498c4d48646b7d4f1a521d847343df3

commit 38370805b498c4d48646b7d4f1a521d847343df3
Author: Chris Cunningham <chcunningham@chromium.org>
Date: Fri Feb 02 02:26:01 2018

Clean up proprietary codec checks/expectations in LayoutTests

This is mostly a reland of
https://chromium-review.googlesource.com/c/chromium/src/+/807604
with some improvements and now unblocked by changes to bots.

Media layout tests need repair
1) Many tests check for proprietary codec support by only asking about
   "video/mp4" without specifying a codec. This check is busted
   because MP4 is no longer considered proprietary while codecs like
   H264 and AAC still are. We currently claim "video/mp4" is entirely
   unsupported via test hacks to avoid breaking those checks.
2) Some MSE tests focus on MP4 with H264. While many bots now have the
   needed codecs, we artificially remove their support via test-only
   hacks in order to keep expectations the same for all runners.

This CL does the following
1) Remove incomplete checks for "video/mp4" - instead hardcode use of
   the .ogv files (preserving the existing behavior).
2) Update MSE mp4/h264 test expectations to PASS|FAIL. This gives us
   some coverage on bots with proprietary codecs while allowing failure
   on bots that lack codecs.
3) Remove test hacks that artificially reduce codec support.

It is also not a goal to make LayoutTests try all the supported codecs.
That is covered by unit/integration tests.

Due to 3) above, some external/wpt/media-source tests now fail because
they select mp4. Fixing these failures is tracked in Issue 794338.

Bug:  327115 , 746579 , 787575 , 568704 ,794338,807110
Change-Id: I05565d7ca9d8d5b12391785888701723810671d5
Reviewed-on: https://chromium-review.googlesource.com/849641
Commit-Queue: Chrome Cunningham <chcunningham@chromium.org>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Reviewed-by: Peter Beverloo <peter@chromium.org>
Reviewed-by: Matthew Wolenetz <wolenetz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#533924}
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/content/shell/renderer/layout_test/layout_test_content_renderer_client.cc
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/media/base/mime_util.cc
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/media/base/mime_util.h
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/media/base/mime_util_internal.cc
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/media/base/mime_util_internal.h
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/TestExpectations
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/compositing/geometry/clipped-video-controller.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/compositing/geometry/video-fixed-scrolling.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/compositing/geometry/video-opacity-overlay.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/compositing/layers-inside-overflow-scroll.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/compositing/overflow/scroll-ancestor-update.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/compositing/reflections/load-video-in-reflection.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/compositing/resources/media-testing.js
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/compositing/self-painting-layers.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/compositing/video/video-reflection.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/compositing/visibility/visibility-simple-video-layer.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/external/wpt/html/semantics/embedded-content/media-elements/mime-types/canPlayType-expected.txt
[delete] https://crrev.com/15ac173128e4e22c58e8edfc956ff4c99cca5460/third_party/WebKit/LayoutTests/external/wpt/media-source/mediasource-addsourcebuffer-expected.txt
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/external/wpt/media-source/mediasource-addsourcebuffer-mode-expected.txt
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/external/wpt/media-source/mediasource-buffered-expected.txt
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/external/wpt/media-source/mediasource-endofstream-expected.txt
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/external/wpt/media-source/mediasource-endofstream.html
[delete] https://crrev.com/15ac173128e4e22c58e8edfc956ff4c99cca5460/third_party/WebKit/LayoutTests/external/wpt/media-source/mediasource-is-type-supported-expected.txt
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/external/wpt/media-source/mediasource-sequencemode-append-buffer-expected.txt
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/external/wpt/media-source/mediasource-sourcebuffer-mode-timestamps-expected.txt
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/fast/canvas/canvas-composite-video-shadow.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/fast/canvas/canvas-composite-video.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/fast/canvas/canvas-drawImage-video.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/fast/canvas/canvas-pattern-video.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/fast/canvas/yuv-video-on-accelerated-canvas.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/fast/events/media-focus-in-standalone-media-document.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/fast/overflow/overflow-of-video-outline.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/fullscreen/resources/video.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/fullscreen/video-controls-timeline.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/fullscreen/video-specified-size.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/hdr/color-profile-video.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/hdr/video-canvas-alpha.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/appcache/video.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/canvas/webgl/origin-clean-conformance-expected.txt
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/canvas/webgl/origin-clean-conformance.html
[delete] https://crrev.com/15ac173128e4e22c58e8edfc956ff4c99cca5460/third_party/WebKit/LayoutTests/http/tests/fullscreen/resources/media-file.js
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/audio-seekable-contains-zero-without-ranges.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/audio-timeline-seek-outside-seekable.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/autoplay/document-user-activation-cross-origin-feature-policy-delegation.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/autoplay/document-user-activation-cross-origin-feature-policy-disabled.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/autoplay/document-user-activation-cross-origin-feature-policy-gesture.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/autoplay/document-user-activation-cross-origin-feature-policy-header.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/autoplay/document-user-activation-feature-policy-alternating.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/autoplay/document-user-activation-feature-policy-iframe-no-gesture.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/autoplay/document-user-activation-feature-policy-same-origin.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/autoplay/resources/autoplay-test.js
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/autoplay/resources/nested-frame-with-header.php
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/autoplay/resources/nested-iframe-1a.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/autoplay/resources/nested-iframe-1b.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/autoplay/resources/nested-iframe-2.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/autoplay/resources/test-autoplay-post-navigation.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/autoplay/resources/test-autoplay.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/controls/controls-list-add-hide.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/controls/controls-list-remove-show.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/controls/toggle-class-with-state-source-buffer.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/controls/toggle-class-with-state-source.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/controls/video-controls-overflow-menu-correct-ordering.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/controls/video-controls-overflow-menu-updates-appropriately.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/gc-while-network-loading.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/media-document.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/mixed-range-response.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/pdf-served-as-pdf.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/preload-conditions.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/progress-events-generated-correctly.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/reload-after-dialog.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/remove-while-loading.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/resources/autoplay-crossorigin-iframe.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/text-served-as-text.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/video-buffered-range-contains-currentTime.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/video-cancel-load.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/video-controls-download-button-displayed.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/video-controls-download-button-not-displayed-hide-download-ui.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/video-controls-download-button-saves-media-cross-origin.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/video-controls-download-button-saves-media.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/video-controls-overflow-menu-download-button.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/video-cookie.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/video-error-abort.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/video-load-metadata-decode-error.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/video-load-suspend.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/video-load-twice.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/video-load-with-userpass.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/video-play-progress.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/video-play-stall-before-meta-data.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/video-play-stall.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/video-preload-metadata.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/video-query-url.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/video-referer.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/video-seek-to-duration.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/video-seek-to-middle.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/video-served-as-text.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/media/video-useragent.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/security/local-video-source-from-remote-expected.txt
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/security/local-video-source-from-remote.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/security/subresourceIntegrity/integrity-attribute.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/security/video-cross-origin-readback.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/http/tests/security/video-cross-origin-via-dom.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/adopt-node-crash.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/audio-concurrent-supported.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/audio-constructor-preload.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/audio-constructor-src.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/audio-constructor.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/audio-controls-captions.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/audio-controls-do-not-fade-out.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/audio-controls-rendering.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/audio-delete-while-slider-thumb-clicked.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/audio-garbage-collect.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/audio-play-event.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/auto-play-in-sandbox-with-allow-scripts.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/autoplay-clears-autoplaying-flag.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/autoplay-document-move.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/autoplay-from-mediastream-to-src.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/autoplay-muted-conditions.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/autoplay-muted-datasaver-off.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/autoplay-muted-datasaver-on.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/autoplay-muted.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/autoplay-never-visible.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/autoplay-non-whitelisted-scope.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/autoplay-unmute-offscreen.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/autoplay-when-visible-multiple-times.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/autoplay-when-visible.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/autoplay-whitelisted-scope.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/autoplay-with-preload-none.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/autoplay.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/autoplay/document-user-activation.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/autoplay/media-document-autoplay-enabled.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/autoplay/muted-change-src.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/avtrack/addtrack.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/avtrack/audio-track-enabled.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/avtrack/audio-track-properties.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/avtrack/forget-on-load.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/avtrack/gc.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/avtrack/getTrackById.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/avtrack/track-switching.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/avtrack/video-track-properties.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/avtrack/video-track-selected.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/broken-video.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/color-profile-video-seek-filter.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/color-profile-video-seek-object-fit.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/color-profile-video-seek.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/color-profile-video.html
[delete] https://crrev.com/15ac173128e4e22c58e8edfc956ff4c99cca5460/third_party/WebKit/LayoutTests/media/content/test-par-16-9.mp4
[delete] https://crrev.com/15ac173128e4e22c58e8edfc956ff4c99cca5460/third_party/WebKit/LayoutTests/media/content/test_yuv420.mp4
[delete] https://crrev.com/15ac173128e4e22c58e8edfc956ff4c99cca5460/third_party/WebKit/LayoutTests/media/content/test_yuv422.mp4
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/controls-after-reload.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/controls-css-overload.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/controls-drag-timebar-rendering-expected-mismatch.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/controls-drag-timebar-rendering.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutTests/media/controls-drag-timebar.html
[modify] https://crrev.com/38370805b498c4d48646b7d4f1a521d847343df3/third_party/WebKit/LayoutT
dpranke@, the [ Pass Failure ] expectation for 

external/wpt/media-source/mediasource-endofstream.html

might be busted. I just saw this test "unexpectedly fail" in the build below?

https://uberchromegw.corp.google.com/i/chromium.win/builders/Win7%20Tests%20%28dbg%29%281%29/builds/66377
If you hunt around in the log files, you can eventually see that the test timed out:

https://chromium-swarm.appspot.com/task?id=3bb4c23dd09fcc10&refresh=10&show_raw=1

which that expectation wouldn't be expecting.

This would've been a lot clearer if the test results were actually being uploaded properly :(. That's bug 786034.
I'll add "Timeout" to its expectation list.
Here's the most recent build with the failure.
https://ci.chromium.org/buildbot/chromium.win/Win7%20Tests%20%28dbg%29%281%29/66381
Timeout is definitely a bug / failure and should be filed as another issue.
Moving to bug 727252 regarding the failure of external/wpt/media-source/mediasource-endofstream.html
Project Member

Comment 8 by bugdroid1@chromium.org, Feb 22 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/320860711ed6a447d609cbf8f5c277c9dd06bb7e

commit 320860711ed6a447d609cbf8f5c277c9dd06bb7e
Author: Takeshi Yoshino <tyoshino@chromium.org>
Date: Thu Feb 22 15:05:26 2018

Include Timeout to the expectation for external/wpt/media-source/mediasource-endofstream.html

See the bug for details. It can timeout in addition to failure.

Bug: 807110, 727252
Change-Id: I7d7d10221a88e018bab768bf9e53ded20421ad9c
Reviewed-on: https://chromium-review.googlesource.com/923685
Commit-Queue: Adam Rice <ricea@chromium.org>
Reviewed-by: Adam Rice <ricea@chromium.org>
Cr-Commit-Position: refs/heads/master@{#538435}
[modify] https://crrev.com/320860711ed6a447d609cbf8f5c277c9dd06bb7e/third_party/WebKit/LayoutTests/TestExpectations

Project Member

Comment 9 by bugdroid1@chromium.org, Mar 1 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f3f0e3dae8bd1ccfc963c0c69810afaf1eaba22b

commit f3f0e3dae8bd1ccfc963c0c69810afaf1eaba22b
Author: Tsuyoshi Horo <horo@chromium.org>
Date: Thu Mar 01 06:08:35 2018

Mark mediasource-config-change-mp4-v-bitrate.html [ Failure Pass Timeout ]

https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=webkit_tests&tests=mediasource-config-change-mp4-v-bitrate.html

TBR: jrummell@chromium.org
Bug: 807110
Change-Id: Id44cc9fa07e779f03c74866b41399795dcaca7cf
Reviewed-on: https://chromium-review.googlesource.com/942125
Reviewed-by: Tsuyoshi Horo <horo@chromium.org>
Cr-Commit-Position: refs/heads/master@{#540069}
[modify] https://crrev.com/f3f0e3dae8bd1ccfc963c0c69810afaf1eaba22b/third_party/WebKit/LayoutTests/TestExpectations

mediasource-config-change-mp4-v-framerate.html times out as well. I filed https://crbug.com/819620 per the recommendation in #c6.

Sign in to add a comment