I think this would be a bit hard to surface, since we don't get that data until ffmpeg has processed the streams. It could be accessible from WebMediaPlayerImpl via |demuxer_| though.

dale, who is the dumxer owner? can you assign appropriately?

This is a feature request, I don't think we are likely to do this w/o spec support.

Do you mean something like the cover image from a music file? Since media element metadata isn't exposed in any way at all currently, there isn't anything in HTML or Media Session about this, but I think it would make sense to treat it just like other metadata.

The way I thought about this before is that we could add a promise-vending HTMLMediaElement.prototype.getMetadata() that would resolve with a MediaMetadata object that could then be assigned to mediaSession.metadata. Perhaps this could be propagated automatically for the default media session, but just having an API for it seems OK as a start.

One problem that this gives rise to is that the MediaMetadata interface needs to be able to represent the embedded image somehow, and the MediaImage interface isn't great for this, you'd be forced to use a blob: URL I think. At some point I thought that MediaMetadata.prototype.artwork should be a Response object in order to support this kind of thing, but that's not great when you start out with a URL.

I'm not sure we would need a spec change to show a default poster image based on the metadata we find in the video file. foolip@, WDYT?

Video files usually don't have any embedded images I think, is it things like album art for audio-only files we're talking about (Just want to make sure it's not the first frame of the video or anything.)

I agree you could do this without a spec change, but I think it'd be unfortunate if you can *only* make use of this extra metadata by using the default session, so that any use of the MediaSession API would sacrifice that.

My understanding from  issue 624954  is that some files might contain a poster image in metadata. To override this, it sounds like using the poster attribute would work. Though, I'm wondering if I'm mixing multiple issues now :D

Whether or not to use <video poster="this-url-here"> is another question indeed. I'm personally a bit skeptical because people may well use something that has a play button icon baked in, and allowing that would also require a way to override that behavior without specifying some other artwork. Seems simpler to just have the APIs that allow developers to use the same URL if they want to.

To clarify, per the originating issue, the request was about the metadata contained in the video file (e.g. video file could contain the preview image). As I read it, the spec currently defines that the video element represents a blank rectangle 300x150 if there's no poster frame or video data. It seems like fetching some image from the video data would indeed require spec change (like adding another condition for what the element represents with some format/implementation specific description).

Personally, I'd suspect that not many video files on the web contain the preview image anyway.

> As I read it, the spec currently defines that the video element represents a 
> blank rectangle 300x150 if there's no poster frame or video data. It seems
> like fetching some image from the video data would indeed require spec change

Does "video data" only include the actual video stream? Isn't the cover in the file's metadata part of the "video data"?

If the data is present, the element represents the last rendered video frame per spec: https://html.spec.whatwg.org/multipage/embedded-content.html#the-video-element (scroll down to the list of conditions starting with the sentence "A video element represents what is given for the first matching condition in the list below:")

https://github.com/whatwg/html/issues/1588#issuecomment-234670697

A WHATWG member says that part of the specification is not relevant here:

>> doesn't that list, in the end, determine what is supposed to be shown?
>
> No. As explained in the definition of "represents" I linked to, it explains 
> what the element's semantics are; what it means. Usually such meaning is 
> used by humans reading source code, or search engines crawling the web, or 
> other semantic tooling of that sort. That is unrelated to how a graphical 
> web browser chooses to display it.

According to this comment by the same WHATWG member:

https://github.com/whatwg/html/issues/1588#issuecomment-234673384

there would definitely be no need for a spec change to introduce this feature.

Philip, do you agree?

I can see cross-compatibility issues if other browsers don't support it and existing websites have the metadata in the videos they haven't verified and haven't overridden (e.g. the preview image could be the wrong size or broken).

I'd rather see this added to the spec somehow than to leave it as implementation specific.

I have been confused about this issue from the very beginning. I just assumed that it was related to MediaMetadata and therefore MediaSession, but it's actually just about poster images embedded in the media resource itself.

I've commented in https://github.com/whatwg/html/issues/1588#issuecomment-235024750, in short I think a spec change would be needed to explain when the embedded poster image is used and not, in cases like <video src=video-with-embedded-poster.webm poster=poster.jpg> and whenever the dimensions of all resources don't match.

But first things first, are embedded poster images supported in all of the container formats we care about? WebM? I've honestly never heard about such a thing, but that doesn't mean it's not common :)

This issue has been Available for over a year. If it's no longer important or seems unlikely to be fixed, please consider closing it out. If it is important, please re-triage the issue.

Sorry for the inconvenience if the bug really should have been left as Available.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

This isn't something that we're going to add.