New issue
Advanced search Search tips

Issue 761506 link

Starred by 9 users

Issue metadata

Status: WontFix
Owner:
Closed: Sep 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux , Windows , Chrome , Mac
Pri: 3
Type: Bug



Sign in to add a comment

Consider making HLS support cross-platform

Project Member Reported by rbyers@chromium.org, Sep 1 2017

Issue description

It's strange for Chrome Android to support Apple's HLS feature but not Chrome desktop (when Safari supports it consistently across platform).  In general we want Chrome to expose a consistent web platform feature set as much as possible across platforms.

For example, this can make it harder to have automated testing of applications where you expect desktop and mobile to behave the same.

Is there any fundamental reason HLS support can't be enabled in Chrome desktop to match Android?

Context: https://twitter.com/cramforce/status/903695613409103872
https://groups.google.com/a/chromium.org/d/msg/chromium-discuss/2iSagN5LHu0/OjzabWCvrYIJ

 
Cc: mlamouri@chromium.org
Components: -Blink>Media>Video Internals>Media>Video
Cc: markdavidscott@google.com
Owner: dalecur...@chromium.org
Status: WontFix (was: Untriaged)
If anything we'd be more prone to removing HLS support from Android than adding it for desktop. It's only supported now because of historical reasons, but it's usage is low and the underlying implementation on Android is handily outperformed by any modern JS based solution.

Fundamentally (and backed by data from many companies) we believe that adaptive streaming algorithms should be handled by the client. We have committed to providing the primitives (MSE, EME) such that clients can implement their desired streaming algorithm (SmoothStreaming, HLS, DASH, SASH, etc) using JS manifest parsers and MSE primitives. Apple has announced support for fmp4+hls which allows MSE+EME to be used with a lightweight library for manifest parsing.

Keeping this work in the client allows for standardized manifests using better algorithms, faster implementation fixes/changes (consider older iOS and Android implementations with their HLS platform bugs), and less security surface area in Chrome. For smaller clients which don't care about these benefits, there are a bevy of packaged JS solutions -- Shakaplayer being Google's own.

Additionally, HLS is patent encumbered when using mpeg2ts and thus requires royalties. Chrome is committed to open codecs, containers, and standards.

Together all of the above combine to indicate that we should not add platform support for adaptive streaming. Happy to continue debate, but at this time we won't be expanding HLS support.
Great, I'm equally happy with the "drop support on Android" path to getting us to a consistent state.  Is there a bug tracking that / discussing the usage numbers, etc?  From a compat principles perspective (http://bit.ly/blink-compat) all the points you mention are good reasons to accept some compat risk here - I'm happy to help make the argument for removal when the experts feel the time is right.
I think we are taking the easy way out. Focus on ease of implementation instead of focusing on the user.

The best known video player (shaka) is 178KB of JS and doesn't even support iOS.

That means

- using such a player makes it essentially impossible to quickly launch a video on page load
- pages with lots of video ads (say 5) would quickly need to compile 1MB of JS, which means 1s of UI thread stall time on a Nexus 5x.

"You can already do it with loading 178KB of JS" is not the same as "You can do it" if we want to be serious about performance.

For the time being we will start recommending HLS in AMP. It seems like the best trade-off for users for now.
@rbyers: Below is a longer form document on the subject; currently we don't have a tracking bug for removal. Usage metrics have been in flux while we move things around in the Android playback pipeline, but extrapolating from the older metrics usage is currently ~4% of all media loads. I'm not sure what that translates into in regards to page loads.

https://docs.google.com/a/google.com/document/d/1OarKrryRufa-hTKtkSP59SHidSdbbRnxtObBgi8f9qo/edit?usp=sharing
https://uma.googleplex.com/p/chrome/histograms/?endDate=20170905&dayCount=1&histograms=Media.Android.MediaPlayerSuccess%2CMedia.DetectedContainer&fixupData=true&showMax=true&filters=platform%2Ceq%2CA%2Csimple_version%2Ccnt%2C62.0.32%2Cisofficial%2Ceq%2CTrue&implicitFilters=isofficial

@malteubl: Happy to have a longer discussion on this, but we're definitely not taking the easy way out. It'd be way easier to cave and just add support for HLS to Chrome then continue to hash out this argument year after year. Google can certainly afford to, even though others can not. But as stated above, we have strong reasons to believe that HLS does not align with the user experience nor Chrome's values.

Obviously we're all in a tough position because Apple refuses to support open web standards such as MSE and EME on iOS, thus forcefully fragmenting the market in their preferred proprietary direction. Keep in mind that the root cause which brought us to this point is a lack of open standards compliance.

Ideals and aspirations aside... Shaka is configurable, so while the complete build is 178kb, you can create smaller custom versions. It also has HLS passthrough support for iOS on the way. FWIW, I recommend the custom build route for AMP. As noted above, Google is not a small provider. We do care about things such as load time and implementation speed. AMP could build a custom version (ideally upstreaming any needed changes) for its needs which would be highly successful and avoids the concerns you've mentioned above.

Finally, I think you're overestimating the loading costs. Below are 1080p and 480p autoplay demos that I just tried on my 5x which load near instantly on a cold cache. They use DASH and a 3rd party CDN, but fmp4+HLS on an AMP cache would probably load just as fast if not faster.

http://storage.googleapis.com/dalecurtis/shaka/shaka.html?src=buck480_dash.mpd (shortened: https://goo.gl/AWi9Cb)
http://storage.googleapis.com/dalecurtis/shaka/shaka.html?src=doom.mpd (shortened: https://goo.gl/gi3bHq)

If you must recommend HLS, you should require that it is fMP4 and not TS based so that a simple HLS manifest parser is sufficient for other platforms (otherwise transmuxing is required, and that can be expensive). You will be able to provide a better user experience on non-iOS platforms by going that route. Do not rely on src= HLS support on Android; it will not work well on older platforms since it relies on the underlying OS and may go away in the future.
I really don't care about HLS. I just want to avoid the extra download, complexity and JS execution time of a complex player dependency.

E.g. for https://goo.gl/AWi9Cb, shaka adds a full second to the time the first video byte is requested on 3G.

That means the same video would have started playing a full second earlier on Safari.
Screen Shot 2017-09-06 at 2.18.58 PM.png
20.6 KB View Download
If AMP provides the playback library, that download would be cached for all AMP pages from then on. The benefits mentioned in my previous messages then outweigh the cost of a rare (cache loss, update, etc) 50kb gzipped download; but lets run the numbers for comparison:

Native HLS:
http://storage.googleapis.com/dalecurtis/shaka/hls_native.html?src=buck_master.m3u8 (https://goo.gl/96FGt1)

TS version for Android (since Android doesn't work with fmp4 HLS):
http://storage.googleapis.com/dalecurtis/shaka/hls_native.html?src=buck_ts_master.m3u8 (https://goo.gl/z7zbk8)

Shaka Version:
http://storage.googleapis.com/dalecurtis/shaka/shaka.html?src=buck_master.m3u8 (https://goo.gl/CjSbrc)

Shaka VP9 version:
http://storage.googleapis.com/dalecurtis/shaka/shaka.html?src=buck480_vp9.mpd (https://goo.gl/fVBuHW)

Android 8.0, Pixel XL:
  Native TS HLS:   cold:   9983ms, warm:  9622ms, reload: 8337ms (Android HLS has no caching)
  Shaka fMP4 HLS:  cold:  13757ms, warm: 10711ms, reload:  229ms (caching present)
  Shaka fMP4 DASH: cold:  19217ms, warm:  9555ms, reload:  861ms
  Shaka VP9 DASH:  cold:  11988ms, warm:  6908ms, reload:  962ms

iOS 10.3.3, iPad Air 2:
  Native fMP4 HLS: cold: 262630ms (not a typo, iPad attempts to buffer ~1m15s)

Keep in mind in no test was the video watchable, i.e. constant stalling - though the iPad buffered the most, even a 480p stream is too large for 3G. Timing is computed via requestAnimationFrame checking currentTime > 0 (i.e. +/- 16ms) and computing the delta between performance.timing.fetchStart.

Three points to make on this data:
1. Cold start is definitely slower. If you're always in a cold start position, you're right that having platform support can help.
2. Warm start is equivalently fast. If you're hosting the player library and its in the cache, the JS complexity is noise.
3. Choosing HLS prevents you from using codecs like VP9+OPUS, which can dramatically reduce the media size and improve startup costs.
4. Safari/Mobile WebKit has a strange buffering strategy. Despite buffering ridiculous amounts, the content is still unplayable. During this time the page appears frozen.
In the real world cache hit rates for JS are very low (e.g. 5 streaming ads are likely to use 5 different URLs for the same video player). On Safari there is no cross domain caching (Site A + B who download the same video player from the same URL with inifite caching download it twice).

Also in the real world mobile devices are typically CPU bound not network bound. So, compiling the JS is much worse than downloading it.

I'm not arguing that you can't get decent numbers without it, but the time is non-trivial and it would be better not to do it at all.
Is that low hit rate true for AMP? I thought that all (most?) of the JS was hosted on the AMP cache, so there's no cross domain issues. In any case, I don't think it matters in the context of this debate since we're talking about Chrome.

Cold compilation of the JS as far back as a Galaxy Nexus running 4.2.2 is only 50ms. So I don't agree that in the real world you'll see material differences on pages that are not already too overloaded.

Are you not envisioning providing a common player library for AMP similar to AMPJS? Certainly if you're not controlling the library nor using a common host then yes, pages will use all sorts of different libraries and the cache rate will be poor and thus we'll always hit the cold start time. Which is frankly just the state of all non-AMP content already; i.e. we're long past the point of sites not using a micro-services model.
Cache hit rates for AMP are high. But the web is bigger than AMP.

Certainly the Galaxy Nexus can not compile more than 1KB of (obviously ungzipped) JS per millisecond. The 5X does about 1KB per ms when the temperature is right. This has gotten a bit better on modern Chrome, but work is work.
I can't tell if you're indicating disbelief in the compilation value I quoted or not, so here's the M61 profiler shots for the Galaxy Nexus and N5X; which unthrottled seems to clock in at 24ms. Obviously these are best case numbers though, so I agree your 1KB values might be more likely with common pages.
Screenshot from 2017-09-07 16:51:55.png
219 KB View Download
Screenshot from 2017-09-07 17:00:21.png
424 KB View Download
Dale,

Do you see any issue with recommending the following?

<source type="application/vnd.apple.mpegURL" src="video.m3u8">
<source type="video/mp4" src="video.mp4">

Given many of our users already have HLS versions of their video, idea is to get adaptive-bit-rate when supported and fallback properly when not supported.

My testing shows that Chrome on Android picks up the m3u8 source and Chrome desktop ignores it and fallback to the MP4. iOS of course picks up m3u8 source as well. 

Is there a version of Chrome that may decide to pick up ` type="application/vnd.apple.mpegURL"` source yet be unable to play it? If so, what's the best way to configure <video> to seamlessly switch between an HLS and MP4 sources?




You raise a good point. I think what you have might fail on out of date iOS devices and will fail on Android devices when using HLS+fMP4. Older iOS devices and Android MediaPlayer don't support HLS+fMP4 and Apple has not issued any guidance around extensions to the mime-type for fMP4 in HLS. I'll track down an old iOS device tomorrow to test my thoughts and I've filed issue 766377 to see if we can improve Chrome's fallback here.

mlamouri@: Can you reach out to your Apple contacts to see what their plans are for canPlayType() and HLS+fMP4 with older iOS devices? I.e., are they planning to add a new mime-type for this?

At this time, I think the only way to handle this case is to always use TS sources or add the sources programatically via JavaScript based on useragent && os version; which stinks.
Confirmed that a 2012 iPad and anything else stuck at iOS9 (< iOS11?) will fail playback and not proceed to the next <source> element when HLS+fMP4 is used:

http://storage.googleapis.com/dalecurtis/shaka/hls_fallback.html

Thanks!
Lack of parity between mobile and desktop also means tools such as https://github.com/GoogleChrome/puppeteer cannot accurately simulate mobile. e.g. total bytes downloaded will be incorrect, screenshots will be off.

Sign in to add a comment