The use of SurfaceControl is to allow synchronizing changes to the rest of web content (scrolling for example) with the overlay. Right now the main web content is updated by chrome's composition while the overlay is positioned in java by re-positioning the elements in the UI. SurfaceControl gives us a way to apply these changes atomically.
Oh thats exciting ! so embedded videos could be now promoted to overlays ?
Some time back i was trying to promote content quads/tiles in Android to overlays of course for power saving reasons, that can be a possibility now ?
But arent the number of overlay still limited ?
Yes, that's the primary aim. We can promote embedded videos to overlays.
It also opens the possibility of using overlays for tiles. The limit on the number of overlays, depending on the device, might end up meaning that the framework itself has to fallback to GPU composition but at least the decision can be deferred until the end of the pipeline in the framework. And we'd expect it to be no worse than a composition in chrome with a single overlay sent to the framework.
But to begin with we're targeting video overlays which should definitely be a win and should keep us within the overlay limit for a larger number of devices. And setting up the groundwork to try it for other cases with a lot of dynamic content (like canvas) and potentially tile quads.
With https://chromium-review.googlesource.com/c/chromium/src/+/1258383, the overall framework should be in place. Making a note of a few things for follow up patches:
1) Plumbing is needed to choose the correct the format/usage for secure media when creating the AImageReader. Digging up so far makes it look like a combination of AIMAGE_FORMAT_PRIVATE and AHARDWAREBUFFER_USAGE_PROTECTED_CONTENT should work but still need to validate with the plumbing.
2) Need to make sure AVDACodecImage works. This includes plumbing to disable java overlays and pulling out the AHardwareBuffer from it similar to CodecImage.
3) We shouldn't be binding the texture in ImageReaderTextureOwner when we're using overlays. This will need a public API on TextureOwner for doing this only when the image can't be overlayed.
4) The framework is adding an additional solid color layer in the HwC when using video overlay, presumably because we mark the main surface translucent when a video underlay is present. Using OverlayStrategySingleOnTop avoids it because in that case the video overlay is placed on top of the main overlay avoiding anything to be marked translucent.
The curious part is that this is dependent on the PixelFormat we choose for the surface in java. Using PixelFormat.Opaque, which we use by default, adds it but PixelFromat.Translucent does not.
The easiest thing most likely is to use PixelFromat.Translucent with SurfaceControl but also good to understand this better first.
Comment 1 by bugdroid1@chromium.org
, Oct 2