New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 700572 link

Starred by 2 users
Status: Fixed
Owner:
m...@chromium.org
Closed: Jan 2018
Cc:
sko...@chromium.org
mlamouri@chromium.org
█ x...@chromium.org
mfo...@chromium.org
m...@chromium.org
taku...@chromium.org
powerb@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 2
Type: Bug-Regression



Sign in to add a comment

Media Remoting should auto-disable when captions are being used

Project Member Reported by m...@chromium.org, Mar 10 2017 
Accessibility is, as always, a huge concern. Unfortunately, Media Remoting currently lacks support for transmitting text tracks and displaying them remotely with the video. So, for the short-term, we should seek to disable Media Remoting (i.e., fallback to screen capture) wherever reliable signals are present to indicate captions are in-use.

One example:

http://iandevlin.github.io/mdn/video-player-with-captions/

Click on the 'CC' button, choose English captions.  Cast tab, then start playback.  See captions on remote device.  Go fullscreen, we switch to media remoting, but captions disappear from the remote display.  

This is using WebVTT, per https://developer.mozilla.org/en-US/Apps/Fundamentals/Audio_and_video_delivery/Adding_captions_and_subtitles_to_HTML5_video.

Comment 1 by mlamouri@chromium.org, Mar 23 2017

Cc: mlamouri@chromium.org

Comment 2 by mlamouri@chromium.org, Mar 23 2017 

The fix for this bug requires UI changes in the media element. Was it approved by UX?

Comment 3 by x...@chromium.org, Mar 23 2017 

The proposal was sent to UI. Still waiting for reply.
Project Member

Comment 4 by bugdroid1@chromium.org, Apr 13 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/df9b67eb9751650cebee13333ab125bb3f11df75

commit df9b67eb9751650cebee13333ab125bb3f11df75
Author: xjz <xjz@chromium.org>
Date: Thu Apr 13 23:28:25 2017

Media Remoting: Add interstitial elements to media element shadow dom.

This CL add the following elements to media element shadow dom when
media remoting starts.
1. Add a disable button to allow user stop remoting and fall back to
tab mirroring. This is to solve the accessiblity regression caused by
media remoting with lacking support for closed captions.
2. Add a background image, cast icon and a text message. These three
are currently rendered as a video frame in media renderer. Move them
to media element to keep consistent with the disabel button and solve
the font renderering issue on Windows 8/10.

The old interstitial (rendered as a video frame) will be removed in a
later CL.

BUG= 700572 ,  697672 
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_layout_tests_slimming_paint_v2

Review-Url: https://codereview.chromium.org/2767823002
Cr-Commit-Position: refs/heads/master@{#464604}

[modify] https://crrev.com/df9b67eb9751650cebee13333ab125bb3f11df75/content/app/strings/content_strings.grd
[modify] https://crrev.com/df9b67eb9751650cebee13333ab125bb3f11df75/content/child/blink_platform_impl.cc
[modify] https://crrev.com/df9b67eb9751650cebee13333ab125bb3f11df75/media/base/media_observer.h
[modify] https://crrev.com/df9b67eb9751650cebee13333ab125bb3f11df75/media/blink/webmediaplayer_impl.cc
[modify] https://crrev.com/df9b67eb9751650cebee13333ab125bb3f11df75/media/blink/webmediaplayer_impl.h
[modify] https://crrev.com/df9b67eb9751650cebee13333ab125bb3f11df75/third_party/WebKit/Source/core/css/CSSPrimitiveValueMappings.h
[modify] https://crrev.com/df9b67eb9751650cebee13333ab125bb3f11df75/third_party/WebKit/Source/core/css/CSSValueKeywords.json5
[modify] https://crrev.com/df9b67eb9751650cebee13333ab125bb3f11df75/third_party/WebKit/Source/core/css/mediaControls.css
[modify] https://crrev.com/df9b67eb9751650cebee13333ab125bb3f11df75/third_party/WebKit/Source/core/dom/Node.h
[modify] https://crrev.com/df9b67eb9751650cebee13333ab125bb3f11df75/third_party/WebKit/Source/core/html/BUILD.gn
[modify] https://crrev.com/df9b67eb9751650cebee13333ab125bb3f11df75/third_party/WebKit/Source/core/html/HTMLMediaElement.cpp
[modify] https://crrev.com/df9b67eb9751650cebee13333ab125bb3f11df75/third_party/WebKit/Source/core/html/HTMLMediaElement.h
[modify] https://crrev.com/df9b67eb9751650cebee13333ab125bb3f11df75/third_party/WebKit/Source/core/html/HTMLVideoElement.cpp
[modify] https://crrev.com/df9b67eb9751650cebee13333ab125bb3f11df75/third_party/WebKit/Source/core/html/HTMLVideoElement.h
[add] https://crrev.com/df9b67eb9751650cebee13333ab125bb3f11df75/third_party/WebKit/Source/core/html/shadow/MediaRemotingElements.cpp
[add] https://crrev.com/df9b67eb9751650cebee13333ab125bb3f11df75/third_party/WebKit/Source/core/html/shadow/MediaRemotingElements.h
[add] https://crrev.com/df9b67eb9751650cebee13333ab125bb3f11df75/third_party/WebKit/Source/core/html/shadow/MediaRemotingInterstitial.cpp
[add] https://crrev.com/df9b67eb9751650cebee13333ab125bb3f11df75/third_party/WebKit/Source/core/html/shadow/MediaRemotingInterstitial.h
[modify] https://crrev.com/df9b67eb9751650cebee13333ab125bb3f11df75/third_party/WebKit/Source/core/layout/LayoutMedia.cpp
[modify] https://crrev.com/df9b67eb9751650cebee13333ab125bb3f11df75/third_party/WebKit/Source/core/paint/MediaControlsPainter.cpp
[modify] https://crrev.com/df9b67eb9751650cebee13333ab125bb3f11df75/third_party/WebKit/Source/core/paint/MediaControlsPainter.h
[modify] https://crrev.com/df9b67eb9751650cebee13333ab125bb3f11df75/third_party/WebKit/Source/core/paint/ThemePainter.cpp
[modify] https://crrev.com/df9b67eb9751650cebee13333ab125bb3f11df75/third_party/WebKit/Source/modules/media_controls/MediaControlsImpl.cpp
[modify] https://crrev.com/df9b67eb9751650cebee13333ab125bb3f11df75/third_party/WebKit/Source/platform/ThemeTypes.h
[modify] https://crrev.com/df9b67eb9751650cebee13333ab125bb3f11df75/third_party/WebKit/public/blink_image_resources.grd
[add] https://crrev.com/df9b67eb9751650cebee13333ab125bb3f11df75/third_party/WebKit/public/default_100_percent/blink/mediaremoting_cast.png
[add] https://crrev.com/df9b67eb9751650cebee13333ab125bb3f11df75/third_party/WebKit/public/default_200_percent/blink/mediaremoting_cast.png
[modify] https://crrev.com/df9b67eb9751650cebee13333ab125bb3f11df75/third_party/WebKit/public/platform/WebLocalizedString.h
[modify] https://crrev.com/df9b67eb9751650cebee13333ab125bb3f11df75/third_party/WebKit/public/platform/WebMediaPlayerClient.h
Project Member

Comment 5 by bugdroid1@chromium.org, Apr 18 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/2504c4da27a272a4906a7fd1bde697c8bdf7def1

commit 2504c4da27a272a4906a7fd1bde697c8bdf7def1
Author: xjz <xjz@chromium.org>
Date: Tue Apr 18 18:50:14 2017

Media Remoting: Remove old interstitial implementation.

Remove old implementation that renders the interstitial on a video
frame. The new implemenation is committed:
https://codereview.chromium.org/2767823002/

BUG= 700572 ,  697672 

Review-Url: https://codereview.chromium.org/2801853002
Cr-Commit-Position: refs/heads/master@{#465303}

[modify] https://crrev.com/2504c4da27a272a4906a7fd1bde697c8bdf7def1/chrome/app/generated_resources.grd
[modify] https://crrev.com/2504c4da27a272a4906a7fd1bde697c8bdf7def1/chrome/common/media/media_resource_provider.cc
[modify] https://crrev.com/2504c4da27a272a4906a7fd1bde697c8bdf7def1/content/renderer/render_frame_impl.cc
[modify] https://crrev.com/2504c4da27a272a4906a7fd1bde697c8bdf7def1/media/base/localized_strings.h
[modify] https://crrev.com/2504c4da27a272a4906a7fd1bde697c8bdf7def1/media/base/media_observer.h
[modify] https://crrev.com/2504c4da27a272a4906a7fd1bde697c8bdf7def1/media/blink/webmediaplayer_impl.cc
[modify] https://crrev.com/2504c4da27a272a4906a7fd1bde697c8bdf7def1/media/blink/webmediaplayer_impl.h
[modify] https://crrev.com/2504c4da27a272a4906a7fd1bde697c8bdf7def1/media/remoting/BUILD.gn
[modify] https://crrev.com/2504c4da27a272a4906a7fd1bde697c8bdf7def1/media/remoting/DEPS
[modify] https://crrev.com/2504c4da27a272a4906a7fd1bde697c8bdf7def1/media/remoting/courier_renderer.cc
[modify] https://crrev.com/2504c4da27a272a4906a7fd1bde697c8bdf7def1/media/remoting/courier_renderer.h
[delete] https://crrev.com/e3b1b256fd85fa0c846e49240e681d4f5cee91f7/media/remoting/interstitial.cc
[delete] https://crrev.com/e3b1b256fd85fa0c846e49240e681d4f5cee91f7/media/remoting/interstitial.h
[modify] https://crrev.com/2504c4da27a272a4906a7fd1bde697c8bdf7def1/media/remoting/renderer_controller.cc
[modify] https://crrev.com/2504c4da27a272a4906a7fd1bde697c8bdf7def1/media/remoting/renderer_controller.h

Comment 6 by sko...@chromium.org, Apr 19 2017

Labels: -M-59 M-60
Project Member

Comment 7 by bugdroid1@chromium.org, Apr 20 2017

Labels: merge-merged-3071
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/2828b8c2d6fd95fccb8145129deb99c33790ec80

commit 2828b8c2d6fd95fccb8145129deb99c33790ec80
Author: xjz <xjz@chromium.org>
Date: Thu Apr 20 20:01:17 2017

Merge M59: Media Remoting: Remove old interstitial implementation.

Remove old implementation that renders the interstitial on a video
frame. The new implemenation is committed:
https://codereview.chromium.org/2767823002/

BUG= 700572 ,  697672 
NOTRY=true
NOPRESUBMIT=true
TBR=miu@chromium.org,sandersd@chromium.org,jochen@chromium.org

Review-Url: https://codereview.chromium.org/2801853002
Cr-Commit-Position: refs/heads/master@{#465303}
(cherry picked from commit 2504c4da27a272a4906a7fd1bde697c8bdf7def1)

Review-Url: https://codereview.chromium.org/2829003004
Cr-Commit-Position: refs/branch-heads/3071@{#97}
Cr-Branched-From: a106f0abbf69dad349d4aaf4bcc4f5d376dd2377-refs/heads/master@{#464641}

[modify] https://crrev.com/2828b8c2d6fd95fccb8145129deb99c33790ec80/chrome/app/generated_resources.grd
[modify] https://crrev.com/2828b8c2d6fd95fccb8145129deb99c33790ec80/chrome/common/media/media_resource_provider.cc
[modify] https://crrev.com/2828b8c2d6fd95fccb8145129deb99c33790ec80/content/renderer/render_frame_impl.cc
[modify] https://crrev.com/2828b8c2d6fd95fccb8145129deb99c33790ec80/media/base/localized_strings.h
[modify] https://crrev.com/2828b8c2d6fd95fccb8145129deb99c33790ec80/media/base/media_observer.h
[modify] https://crrev.com/2828b8c2d6fd95fccb8145129deb99c33790ec80/media/blink/webmediaplayer_impl.cc
[modify] https://crrev.com/2828b8c2d6fd95fccb8145129deb99c33790ec80/media/blink/webmediaplayer_impl.h
[modify] https://crrev.com/2828b8c2d6fd95fccb8145129deb99c33790ec80/media/remoting/BUILD.gn
[modify] https://crrev.com/2828b8c2d6fd95fccb8145129deb99c33790ec80/media/remoting/DEPS
[modify] https://crrev.com/2828b8c2d6fd95fccb8145129deb99c33790ec80/media/remoting/courier_renderer.cc
[modify] https://crrev.com/2828b8c2d6fd95fccb8145129deb99c33790ec80/media/remoting/courier_renderer.h
[delete] https://crrev.com/cb1817af2f491075b8a594ea74255da02904ace9/media/remoting/interstitial.cc
[delete] https://crrev.com/cb1817af2f491075b8a594ea74255da02904ace9/media/remoting/interstitial.h
[modify] https://crrev.com/2828b8c2d6fd95fccb8145129deb99c33790ec80/media/remoting/renderer_controller.cc
[modify] https://crrev.com/2828b8c2d6fd95fccb8145129deb99c33790ec80/media/remoting/renderer_controller.h

Comment 8 by x...@chromium.org, Apr 20 2017

Labels: -merge-merged-3071

Comment 9 by sko...@chromium.org, May 31 2017

Labels: -M-60 M-61

Comment 10 by m...@chromium.org, May 31 2017

Labels: -Pri-1 Pri-2
FWIW, a manual mechanism was put in-place to mitigate this issue (the disable button). We need to revisit that UI for better interoperability with the Remote Playback API.

Then, this issue is really about auto-detecting the use of the available web platform APIs for captions, and disabling remoting based on that.

Given users have a way to disable remoting when they want to see captions on the remote screen, we can lower the priority on this issue.

Comment 11 by sko...@chromium.org, Aug 14 2017

Labels: -M-61 M-63

Comment 12 by sko...@chromium.org, Oct 9 2017

Labels: -M-63 M-64

Comment 13 by jrw@google.com, Dec 11 2017

Blockedon: 698796

Comment 14 by jrw@google.com, Dec 11 2017

Labels: -M-64 M-65

Comment 15 by amp@chromium.org, Dec 11 2017

Blockedon: -698796

Comment 16 by m...@chromium.org, Jan 25 2018

Status: Fixed (was: Available)
We did a lot of UX work to mitigate this issue. Unfortunately, the web platform around captions is not sufficiently mature for us to meaningfully proceed with an automated caption detection solution.

So, closing this as fixed. When/if the situation improves, we can re-visit as a feature request or UX improvement.

Comment 17 by mfo...@chromium.org, Jan 25 2018 

Do you mind sharing some data that led to this conclusion?
►

Sign in to add a comment