Prototype for the last suggestion since it's easy to implement:
https://chromium-review.googlesource.com/c/chromium/src/+/1198342

Last one reduces data usage by 20%, to 8884957 from 10979754 on this exceedingly heavy page: https://material.io/design/motion/customization.html

+hubbe for c#2 since that feels like it's loading more data than it should for preload=metadata and adjusting FFmpegDemuxer capacity doesn't lower it.

It's possible that's just due to large server responses or poor muxing, I haven't tested.

(Tests were cold loads, i.e. fresh user data dir, rm -rf out/Release/.config)

Totals were Media.BytesReadFromNetwork summed up from UrlIndex.

Here's a hosted copy of that page in case they make changes:
http://storage.googleapis.com/dalecurtis/motionmark/motionmark.html

preload=auto + poster is pretty high:
https://uma.googleplex.com/p/chrome/histograms/?endDate=20180913&dayCount=1&histograms=Media.SRC.PreloadAutoHasPoster%2CMedia.SRC.PreloadMetaDataHasPoster&fixupData=true&showMax=true&filters=isofficial%2Ceq%2CTrue&implicitFilters=isofficial

If there's someway to apply the preload=metadata optimization there too when autoplay is disallowed / held until visible, there'd be some more benefits too.

Giving some opinions about the ideas above:

> - Allow authors to put ‘lazyload’ on video/audio tags. Loading would not 
> start until visible and upon leaving visibility (if paused?) we would evict 
> most resources (aside from last decoded video frame).

We would need to make a case that it's different from them setting preload=none then switching to preload=metadata or call load(). Sugar syntax is nice but unless we know it's a very common pattern, maybe not worth adding an HTML attribute? (I realise images have this but one can't load an image on demand very easily.)

> - As a policy video-only or muted-video elements would not be loaded until 
> visible. -- Since visibility is an async determination this could be
> problematic in performance of visible primary content.

We do this for <video autoplay muted> but not when play() is called to make sure websites that *require* playback are not blocked. We had a couple of issues where we recommended websites to switch to play().

> - Some variation of the above based on data saver enabled or “too many media elements” signal.

What would be the benefits in data usage? We could do this for Data Saver if we believe it's worth.

> - Apply lazy load to all tags marked with preload=metadata -- either by having no preload specified or an explicit preload=metadata.

For this, or launching of any form, I would like us to make sure we have clear metrics that reflect negative impact. We will see improvements in resource usage but it wouldn't be fair to ignore the rest. One metric we could use is time to first frame as it could tell us how broken the experience will be.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/7c63f2e2a405c1f7539b0b4bc3ff6991dab94303

commit 7c63f2e2a405c1f7539b0b4bc3ff6991dab94303
Author: Dale Curtis <dalecurtis@chromium.org>
Date: Wed Sep 19 21:06:27 2018

Add 'PreloadMetadataLazyLoad' experiment.

This adds an experiment where all paused preload=metadata audio and video
tags avoid decoder spool up and reduce data usage until the tag becomes
visible.

This is done by expanding the preload=metadata suspend for videos with
poster to all preload=metadata videos. It returns this status to the
HTMLMediaElement via WebMediaPlayer::DidLazyLoad(), which when true
causes the element to create an ElementVisibilityObserver. When the
element becomes visible it calls WebMediaPlayer::OnBecameVisible().

Since the preload=metadata experiment is on by default, it just replaces
that experiment with this new one.

BUG=879406
TEST=new unittests added

Change-Id: I9358f3fef7dbd8ed5a5db50c78b3694c0276c2d5
Reviewed-on: https://chromium-review.googlesource.com/1198342
Commit-Queue: Dale Curtis <dalecurtis@chromium.org>
Reviewed-by: Mounir Lamouri <mlamouri@chromium.org>
Cr-Commit-Position: refs/heads/master@{#592544}
[modify] https://crrev.com/7c63f2e2a405c1f7539b0b4bc3ff6991dab94303/media/base/media_switches.cc
[modify] https://crrev.com/7c63f2e2a405c1f7539b0b4bc3ff6991dab94303/media/base/media_switches.h
[modify] https://crrev.com/7c63f2e2a405c1f7539b0b4bc3ff6991dab94303/media/blink/webmediaplayer_impl.cc
[modify] https://crrev.com/7c63f2e2a405c1f7539b0b4bc3ff6991dab94303/media/blink/webmediaplayer_impl.h
[modify] https://crrev.com/7c63f2e2a405c1f7539b0b4bc3ff6991dab94303/media/blink/webmediaplayer_impl_unittest.cc
[modify] https://crrev.com/7c63f2e2a405c1f7539b0b4bc3ff6991dab94303/third_party/blink/public/platform/web_media_player.h
[modify] https://crrev.com/7c63f2e2a405c1f7539b0b4bc3ff6991dab94303/third_party/blink/renderer/core/html/media/html_media_element.cc
[modify] https://crrev.com/7c63f2e2a405c1f7539b0b4bc3ff6991dab94303/third_party/blink/renderer/core/html/media/html_media_element.h
[modify] https://crrev.com/7c63f2e2a405c1f7539b0b4bc3ff6991dab94303/third_party/blink/renderer/core/html/media/html_media_element_test.cc

Over 28DA on canary and dev here's a results summary:

Windows results are looking really good; ~15% reduction in Event.Latency.ScrollBegin.Touch.TimeToScrollUpdateSwapBegin4 with a 3% increase in PageLoad.PaintTiming.NavigationToFirstContentfulPaint:
https://uma.googleplex.com/p/chrome/variations/?sid=aa712f5b68370bf012dc5e7a0f347a4f

MacOS results look best, ~30% reduction in Event.Latency.ScrollUpdate.Touch.TimeToScrollUpdateSwapBegin4 at 95+ percentiles, lower percentiles seem to see gains closer to the Windows numbers, but aren't marked as statistically significant. It also sees ~2-8% reduction in Memory.Total.PrivateMemoryFootprint. There are no regressions shown (though Startup.FirstWebContents.NonEmptyPaint2 has a reduction in counts for some reason; stability is not statistically different)
https://uma.googleplex.com/p/chrome/variations/?sid=bece00e72d47d01a43fb8f6f4065e98d

ChromeOS shows a 10% reduction in Memory.Total.PrivateMemoryFootprint, but a 6% increase in Event.Latency.ScrollUpdate.Touch.TimeToScrollUpdateSwapBegin4
https://uma.googleplex.com/p/chrome/variations/?sid=d23e7186015f060e8b00dbe9f604cd80

Android doesn't show much change -- which seems a bit odd; I would have expected the most restricted view-port to have the most impact here.
https://uma.googleplex.com/p/chrome/variations/?sid=266c75d880a7308eaa617e17cb8b4167

Overall things are pretty positive and many of the strong negatives from earlier have disappeared. I think it's probably worth a 1% stable experiment in M71 to ensure we have captured population specific data -- e.g., I'm sure we have a huge long tail of Android devices which should have seen a benefit from this change. Since that's not showing up in canary, dev it's probably due to our low populations there.

We may want to reach out to the team owning these metrics, especially the ones where things get worse as we may want to have an idea of how bad an increase is.

+Speed>Metrics,tdresser per c#14 to help understand the lazy video load results from c#10.

Per my past summary, they seem good enough to try a 1% stable experiment, but we'd like the metrics owners to provide any insight into the metrics that regress relative to the metrics that got better. Specifically whether we end up with a worthy trade off.

1% experiment SGTM.

In terms of tradeoffs, I think a doc would make it easier to have platform specific discussions here.

Chrome OS looks potentially problematic - any idea why the regression in scroll latency?

Speculation would be more IntersectionObserver instances (which are not free) combined with more low end hardware. 

c#0 has a brainstorming doc if you want to leave comments there. I've cleaned it up a little bit to make it easier for discussion.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/298a2af278dbaffc6872aa00e81c3485842e97ea

commit 298a2af278dbaffc6872aa00e81c3485842e97ea
Author: Dale Curtis <dalecurtis@chromium.org>
Date: Fri Jan 11 19:07:07 2019

Enable field trial config for PreloadMetadataLazyLoad.

Prepares the way for a 50/50 beta and/or 1% stable experiment.

BUG=879406
TEST=passes cq
.

Change-Id: I99b45acf897a20abb731fca5189b75e3042c045e
Reviewed-on: https://chromium-review.googlesource.com/c/1404419
Reviewed-by: Jesse Doherty <jwd@chromium.org>
Commit-Queue: Dale Curtis <dalecurtis@chromium.org>
Cr-Commit-Position: refs/heads/master@{#622081}
[modify] https://crrev.com/298a2af278dbaffc6872aa00e81c3485842e97ea/testing/variations/fieldtrial_testing_config.json