encrypted content with non-square pixels not playing in Chrome
Reported by
tomas.bu...@prozeta.eu,
Jun 7 2016
|
|||||
Issue descriptionUserAgent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.84 Safari/537.36 Example URL: Steps to reproduce the problem: 1. Use source media that has Storage Aspect Ratio (SAR) other than 1, e.g. with non-square pixels. 2. Serve the content to Chrome via MPEG-DASH from Wowza Streaming Engine that encrypts the stream on-the-fly by using Widevine DRM Technology. 3. No MPEG-DASH player is able to render the video in Chrome, although there is no meaningful error in the browser console or in chrome://media-internals. The players load a couple of segments and then stall. What is the expected behavior? No matter what aspect ratio is used, the media should be properly decrypted and playable. What went wrong? Some component in Chrome is unable to cope with encrypted media if the mp4 container signals presence of non-square pixels. Did this work before? No Is it a problem with Flash or HTML5? HTML5 Does this work in other browsers? Yes Chrome version: 51.0.2704.84 Channel: stable OS Version: 10.0 Flash Version: Shockwave Flash 21.0 r0 I should say that this problem manifests itself in Chrome really when using encryption only. If I attempt to play the very same video file without encryption, it works as intended and the video can be played. Since Firefox Nightly now also supports Widevine DRM, I tried to verify, if it will have the same problem, but it doesn't. The same encrypted file works fine in Firefox. And it also works fine with PlayReady and Edge browser. It really boils down to just Chrome being unable to render the media under these conditions. This issue has existed since the very beginning of Encrypted Media Extensions (EME) support in Chrome and we discussed it at that time with Wowza support to find out whether it was caused by their side, but the outcome was negative. While our own encrypted content was fine in this regard, because it used "regular" dimensions (1920x1080, 1280x720, etc.), some content encoded and provided by our own clients had this exact "problem" with different aspect ratios and we were forced to look for workarounds, how to make their content work in Chrome. The reason was in the end narrowed down to Storage Aspect Ratio (SAR) metadata in the mp4 container. If SAR metadata was present, i.e. the media used non-square pixels, the playback failed. So we suggested to clients using ffmpeg to take Display Aspect Ratio (DAR) into account, create media with physical dimensions rounded according to DAR (instead of encoding, say, 720x480 with 16:9 aspect ratio) and add video filter -vf setsar=sar=0. This workaround is working perfectly and their content is now playable in Chrome. However, there might be more people running into the same issue, so Chrome should be able to process this type of content as well.
,
Jun 28 2016
,
Jun 29 2016
1. https://w01.quickmedia.tv/chrome/BigBuckBunnyAspectClear.mp4/manifest.mpd Clear content with display resolution 768x432, which is different from actual media physical resolution 640x432 (SAR 6:5, DAR 16:9), works both in Chrome and Firefox 47+ 2. https://w01.quickmedia.tv/chrome/BigBuckBunnyAspect.mp4/manifest.mpd License URL: http://widevine-proxy.appspot.com/proxy DRM content with display resolution 768x432 which is different from actual media physical resolution 640x432 (SAR 6:5, DAR 16:9), does not work in Chrome, works in Firefox 47+ 3. https://w01.quickmedia.tv/chrome/BigBuckBunnyRes.mp4/manifest.mpd License URL: http://widevine-proxy.appspot.com/proxy DRM content where display resolution 768x432 matches media physical resolution 768x432, works both in Chrome and Firefox 47+ Missing PAR and wrong SAR manifest attributes for the version that does not play are not the reason, I attempted to correct them already. However, recently I found out about this Java tool https://github.com/sannies/isoviewer and when looking at the contents of the encv box, I'm wondering whether reported value 768 for width is correct and if it shouldn't be 640.
,
Jun 29 2016
Have you tried Clear Key to narrow the issue to Widevine? There is also External Clear Key, which uses more of the same path in Chrome, though I don't know if any of the demo apps support it. kqyang, any thoughts on the encv box?
,
Jun 29 2016
The width and height in 'encv' should be codec width and height instead of display width and height. See statement below from ISO/IEC 14496-12:2015(E) 12.1.3 Sample entry 1.2.3.1: The width and height in the video sample entry document the pixel counts that the codec will deliver; this enables the allocation of buffers. Since these are counts they do not take into account pixel aspect ratio. Widevine CDM use codec resolution specified in VisualSampleEntry box to initialize the secure decoder. If wrong values are provided, the decoder may not be initialized correctly. So to fix the problem: (1) Option 1, fix the content to generate correct codec width and height in 'encv' box (derived from VisualSampleEntry box). (2) Or Option 2, ignore codec resolution specified in VisualSampleEntry box and always extract the resolution from AVCDecoderConfiguration for H264. It does not involve complicated logic to parse AVCDecoderConfiguration, so this could be a viable option. With this, we'll be able to handle non-spec-compliant streams.
,
Jun 29 2016
Ok, thanks kqyanq. Looks like my new suspicion about invalid values in encv box proved to be correct. So this will go back to Wowza. I find it interesting, though, that Widevine CDM in Firefox is already able to process such content. My general idea was that the CDM internals come from Google and Mozilla just adapted it as one of its own GMP plugins. Seems there is more to the integration than meets the eye. Also to respond to David, Wowza does not support Clear Key encryption out-of-the-box, but the same problematic content works fine with PlayReady in Edge and IE11. Anyway, the world is not perfect and not everything is always 100% standard-compliant, so some leeway based on Option 2 (that maybe Firefox already depends on) might be feasible.
,
Jun 29 2016
To answer the question on why it works in Firefox but not in Chrome, the codec resolution is provided by the browser to CDM. Chrome uses the value from VisualSampleEntry directly: https://cs.chromium.org/chromium/src/media/formats/mp4/mp4_stream_parser.cc?rcl=0&l=352. Firefox probably parses AVCDecoderConfiguration and extracts the correct codec resolution before passing the value to CDM.
,
Jun 30 2016
Thank you for providing more feedback. Adding requester "yiningc@chromium.org" for another review and adding "Needs-Review" label for tracking. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Jun 30 2016
based on #6, user find a solution. close this bug. |
|||||
►
Sign in to add a comment |
|||||
Comment 1 by yini...@chromium.org
, Jun 28 2016