webmediaplayer_ms_compositor.cc CopyFrame() should use not hardcoded ARGB nor SkBitmap |
|||
Issue description
webmediaplayer_ms_compositor.cc's static function CopyFrame()
is using libyuv::ARGBToI420() [1] and it shouldn't, it should
do use ConvertToI420() and take into account the origin
pixel format which can be ARGB or ABGR in, e.g. Android platforms,
e.g.:
const uint32 source_pixel_format =
(kN32_SkColorType == kRGBA_8888_SkColorType) ? libyuv::FOURCC_ABGR
: libyuv::FOURCC_ARGB;
If this isn't done, then essentially we get the Blue and Red
colours swapped. How is this code tested??
--------------------------------------------------------
Also, the same function is using SkBitmap, and that class is
deprecated in favour of SkSurface, which has shared ownership
semantics. The new code should do sth like what's done in
HtmlVideoElementCapturerSource::sendNewFrame() [2]
const SkImageInfo info = SkImageInfo::MakeN32(
theWidth, theHeight, kOpaque_SkAlphaType);
surface = SkSurface::MakeRaster(info);
SkPixmap pixmap;
if (!skia::GetWritablePixels(surface->getCanvas(), &pixmap)) {
DLOG(ERROR) << "Error trying to map SkSurface's pixels";
return;
}
and use SkPixmap instead for libyuv::convertToI420().
[1] https://cs.chromium.org/chromium/src/content/renderer/media/webmediaplayer_ms_compositor.cc?dr=C&q=webmediaplayer_ms_compositor&sq=package:chromium&l=64
[2] https://cs.chromium.org/chromium/src/content/renderer/media/html_video_element_capturer_source.cc?dr=C&sq=package:chromium&l=126
,
Jul 15 2016
#1: sorry, my bad, when I meant to use an example of SkSurface + SkPixmap (the new thing) versus using SkBitmap (deprecated), I meant: https://cs.chromium.org/chromium/src/content/renderer/media/image_capture_frame_grabber.cc?q=framegrabber&sq=package:chromium&dr=CSs&l=72
,
Jul 15 2016
I just worked on replacing SkBitmap usage with SkImage in video to canvas code. I think SkBitmap usage is discouraged. I'd guess SkPixmap is the new way of accessing pixel storage? Miguel's comment seems to agree with my guess.
,
Jul 15 2016
Just noticed this Copy() is a two step copy. Namely from |frame| --> |bitmap|, |bitmap| --> |new_frame| The first step is at https://cs.chromium.org/chromium/src/content/renderer/media/webmediaplayer_ms_compositor.cc?dr=C&q=webmediaplayer_ms_compositor&sq=package:chromium&l=59 Question: Has SkCanvasVideoRenderer::Copy() already resolved the color format for the input frame? dcastagna@ and emircan@ can you take a look? I am not very familiar with the Sk stuff.
,
Jul 15 2016
SkCanvasVideoRenderer doesn't know anything about videoframe until Copy is called. It has a cache in case copy is called multiple times with the same videoframe. I'm not sure what format you're referring to. The code you linked to is inside an if(video_frame->HasTextures()) branch. At that point there is only a pixel format for the textures in the videoframe, but that shouldn't affect the format you use to read back from the canvas. I think mcasas is talking about the SkCanvas format, right?
,
Jul 15 2016
qiangchen@ can you please clarify your statement in #4 "Question: Has SkCanvasVideoRenderer::Copy() already resolved the color format for the input frame?" ? Please note that: - This bug is _not_ about SkCanvasVideoRenderer at all. - (The second part of) this bug description is about using SkSurface and SkPixmap instead of SkBitmap. - But notice that in both cases, there is SkCanvas involved. - SkCanvas is _not_ deprecated.
,
Jul 15 2016
My Question is For this statement: "video_renderer->Copy(frame.get(), &canvas, context_3d);" Does this Copy call take |frame->format()| into consideration and do a conversion when painting onto |canvas|? P.S. |video_renderer| is an instance of SkCanvasVideoRenderer.
,
Jul 15 2016
I think I misunderstand this bug. I focused on the color conversion between input and output frame. But this bug is saying: 1. The color format of SkBitmap (or SkPixmap) actually depends on kN32_SkColorType; 2. Thus the copy from SkBitmap to |new_frame| should not use libyuv::ARGBToI420, but should use ConvertToI420 with correct SkBitmap color format as source format. Is this understanding correct?
,
Jul 15 2016
#8: your understanding is part 1, an is correct. There is a second part in the bug description, below the line: -------------------------------------------------------- And that says that the code should be refactored to use SkPixmap and SkSurface instead of SkBitmap.
,
Jul 15 2016
qiangchen@ never mind, I'll take this one.
,
Jul 16 2016
,
Jul 18 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/8e2816a491e5f15d5fdfc809e7bad74f0cfcb9cb commit 8e2816a491e5f15d5fdfc809e7bad74f0cfcb9cb Author: mcasas <mcasas@chromium.org> Date: Mon Jul 18 20:44:17 2016 WebMediaPlayerMsCompositor: use correct ARGB/ABGR, use SkSurface/SkPixmap ISO SkBitmap Tot has ARGB hardcoded, whereas certain platforms (e.g. Android) usually have ABGR. Make that decision runtime. Also, code in ToT uses SkBitmap, which is discouraged, this CL changes it to using SkSurface and SkPixmap instead. BUG= 628466 Review-Url: https://codereview.chromium.org/2153093003 Cr-Commit-Position: refs/heads/master@{#406086} [modify] https://crrev.com/8e2816a491e5f15d5fdfc809e7bad74f0cfcb9cb/content/renderer/media/webmediaplayer_ms_compositor.cc
,
Jul 18 2016
|
|||
►
Sign in to add a comment |
|||
Comment 1 by qiangchen@chromium.org
, Jul 15 2016