Project: chromium Issues People Development process History Sign in
New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.
Issue 690968 Consider supporting Alpha channel recording
Starred by 9 users Project Member Reported by mcasas@chromium.org, Feb 10 Back to list
Status: Fixed
Owner:
Closed: Mar 13
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 3
Type: Feature



Sign in to add a comment
MediaRecorder does not support alpha channel in the incoming MediaStreamVideoTrack [1]
and drops it.  MSVTracks with Alpha channel are generated by <canvas> captureStream().

libvpx, used for VP8/9 sw encoding supports it (we could use it at [2]) and so does
openh264 used for H264 sw encoding (at [3]).  Note that a change in pixel format 
might be needed.


[1] https://cs.chromium.org/chromium/src/content/renderer/media/recorder/video_track_recorder.cc?l=275
[2] https://cs.chromium.org/chromium/src/content/renderer/media/recorder/video_track_recorder.cc?l=794
[3] https://cs.chromium.org/chromium/src/content/renderer/media/recorder/video_track_recorder.cc?l=1003
 
Cc: vigneshv@chromium.org
Hi,

Vignesh from the WebM team here. I added support for VP8/9 alpha channel in Chromium. Bottomline is, libvpx does NOT support encoding of alpha channel. It simply does not understand the notion of an alpha plane and ignores it.

The way we implemented it was more of a workaround in the container:
  * Encode the YUV planes separately [1].
  * Form a "fake" image with the A-plane as the Y-plane and with dummy U and V planes.
  * Encode the "fake" image [2].
  * Mux the outputs of [1] and [2] as a single frame in the container using WebM's BlockAdditional [3] element.

You could potentially do something similar in MediaRecorder, but this will mean running two instances of the encoder and sync'ing up on key frames. Sample encoder can be found in [4]. (A not very well written) Design spec is in [5].

[3] https://matroska.org/technical/specs/index.html#BlockAdditional
[4] FFmpeg can encode VPx streams with alpha channel when given an input with alpha channel: https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/libvpxenc.c#L995
[5] http://wiki.webmproject.org/alpha-channel
Owner: emir...@chromium.org
Status: Assigned
Labels: -Type-Bug Type-Feature
Project Member Comment 4 by bugdroid1@chromium.org, Mar 13
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/9d369903d6cdd84aaaf70e2a73ada508e2b34eaa

commit 9d369903d6cdd84aaaf70e2a73ada508e2b34eaa
Author: emircan <emircan@chromium.org>
Date: Mon Mar 13 07:13:39 2017

Support alpha channel recording for VPX in MediaRecorder

This CL adds support for alpha channel recording in VpxEncoder in VideoTrackRecorder.
- OnEncodedVideoCB is modified to handle extraneous alpha channel output.
- VpxEncoder is refactored to use two encoder instances and encode alpha channel padded
with dummy data.
- WebmMuxer is modified to add the optional alpha channel data using AdditionalBlock.

BUG= 690968 
TEST=Added unit tests. Demo page on
https://cdn.rawgit.com/uysalere/js-demos/master/canvascapture_alpharecording.html.
Also, working on a alpha video based content_browsertest in a follow-up CL.

Review-Url: https://codereview.chromium.org/2691373005
Cr-Commit-Position: refs/heads/master@{#456331}

[modify] https://crrev.com/9d369903d6cdd84aaaf70e2a73ada508e2b34eaa/content/renderer/media_recorder/media_recorder_handler.cc
[modify] https://crrev.com/9d369903d6cdd84aaaf70e2a73ada508e2b34eaa/content/renderer/media_recorder/media_recorder_handler.h
[modify] https://crrev.com/9d369903d6cdd84aaaf70e2a73ada508e2b34eaa/content/renderer/media_recorder/media_recorder_handler_unittest.cc
[modify] https://crrev.com/9d369903d6cdd84aaaf70e2a73ada508e2b34eaa/content/renderer/media_recorder/video_track_recorder.cc
[modify] https://crrev.com/9d369903d6cdd84aaaf70e2a73ada508e2b34eaa/content/renderer/media_recorder/video_track_recorder.h
[modify] https://crrev.com/9d369903d6cdd84aaaf70e2a73ada508e2b34eaa/content/renderer/media_recorder/video_track_recorder_unittest.cc
[modify] https://crrev.com/9d369903d6cdd84aaaf70e2a73ada508e2b34eaa/media/muxers/webm_muxer.cc
[modify] https://crrev.com/9d369903d6cdd84aaaf70e2a73ada508e2b34eaa/media/muxers/webm_muxer.h
[modify] https://crrev.com/9d369903d6cdd84aaaf70e2a73ada508e2b34eaa/media/muxers/webm_muxer_fuzzertest.cc
[modify] https://crrev.com/9d369903d6cdd84aaaf70e2a73ada508e2b34eaa/media/muxers/webm_muxer_unittest.cc

Status: Fixed
Project Member Comment 6 by bugdroid1@chromium.org, Mar 20
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/09cc93d64e1794e3ac967c25ed6e356f0aab2146

commit 09cc93d64e1794e3ac967c25ed6e356f0aab2146
Author: emircan <emircan@chromium.org>
Date: Mon Mar 20 20:47:08 2017

Add transparent video input test for WebRtcMediaRecorderBrowserTest

This CL adds a test where canvas input with alpha is recorded.

BUG= 690968 

Review-Url: https://codereview.chromium.org/2757793002
Cr-Commit-Position: refs/heads/master@{#458176}

[modify] https://crrev.com/09cc93d64e1794e3ac967c25ed6e356f0aab2146/content/browser/webrtc/webrtc_media_recorder_browsertest.cc
[modify] https://crrev.com/09cc93d64e1794e3ac967c25ed6e356f0aab2146/content/test/data/media/mediarecorder_test.html

Sign in to add a comment