Version(s) of macOS on which we were able to reproduce:
MacOS 10.13.4
OS X 10.11.6

However, note that it was not consistent. Other users on 10.13.4 were not able to reproduce.

Also please find an image attached showing what the problem looks like.

Deleted: artifacting_demo.png 309 KB

	artifacting_demo.png 309 KB View Download

chrome://gpu for the machine running macOS 10.13.4

Deleted: chrome-gpu.rtf 74.8 KB

chrome-gpu.rtf 74.8 KB Download

Tested the issue on chrome reported version 68.0.3415.0 using Mac 10.12.6 with steps mentioned below:
1) Launched chrome reported version and navigated to URL: https://github.com/google/shaka-player/ and downloaded the zip file
2) Extracted the file to downloads and ran the command in terminal "./build/all.py"
3) It gives the output message as "ERROR:root:*** A required dependency is missing : npm

@Reporter: Please find the attached screencast for reference and let us know if we missed anything in reproducing the issue, if possible could you please provide sample Test file/URL that reproduces the issue which help in further triaging it from TE end.

Thanks!

Deleted: 844799.mp4 2.5 MB

	844799.mp4 2.5 MB View Download

Shaka Player has a setup guide here: https://shaka-player-demo.appspot.com/docs/api/tutorial-welcome.html

You should install node.js, which includes npm: https://nodejs.org/en/
The LTS version and the current version should both be fine.

You could also use the hosted demo instead: https://shaka-player-demo.appspot.com/demo/#lang=en-US;build=uncompiled

The reporter has sent us (Shaka Player team) a URL privately and asked us to not share it publicly.  I have asked him about posting here, but if someone wants to ping me internally, I can send you it so long as you keep it private.

Does this issue occur with any representation, or does it affect one in particular? (Or perhaps there is a specific transition point that is problematic?) Does it continue to occur after seeking within a buffered range of content?

Apologies in advance for any delay in responding. The affected machines are in a remote office and I need to coordinate with developers there in order to test.

The Shaka demo app allows the user to select stream bitrate under the menu Info -> Video+audio track combinations. The user selected each video track, and was able to reproduce the artifacting issue for each one. He also attempted seeking by setting currentTime:

var video = document.querySelector('video');
// seek ahead
video.currentTime = video.currentTime + 6;
// seek behind
video.currentTime = video.currentTime - 6;

He says this has no effect. He sees the artifacting problem regardless.

Thank you for providing more feedback. Adding the requester to the cc list.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

There is also no specific transition point that causes a problem. The stream is a live stream and there is problematic playback from the beginning of the stream.

I am unable to get Shaka Player to fetch any media content, and the URLs I constructed manually are 404s (perhaps I computed the times incorrectly though).

Can you provide me with the media content files directly?

I'm just leaving for the day so I will see if I can provide you a video buffer tomorrow.

Can I ask what error you are getting with Shaka? There is a companion issue filed at the Shaka GitHub page if you'd like to get support:
https://github.com/google/shaka-player/issues/1441

Also, are you based in the United States or no?

I'm seeing that Shaka repeatedly fetches the manifest but never attempts to fetch any media data. I tried seeking within the duration range (-1.5hr to live), but wasn't able to get different behavior. There is no actual error reported, though.

I am based in the USA.

As per comment# 8 by modmaker@google.com, reporter had sent URL privately to test this issue, hence routing this issue to Inhouse team to ping and get the URL privately and help in further triaging it, hence adding TE-NeedsTriageFromHYD label.

Thanks!

@fillmore.chris: Could you please update Canary to latest version #68.0.3439.0 and check if you still face the issue? If so please help us with a screencast with the issue you're facing.

@sandersd: Could you please ping or Email the URL for further triaging of the issue?

Thanks!!

We tested with Canary 68.0.3440.0 and still saw the issue. Attached is canary_issue.mp4

I've also attached canary-gpu_macos_10.11.6.rtf, which is chrome://gpu for the affected machine running macOS 10.11.6. That is the same machine that produced the video attached.

@sandersd apologies for the delay in getting a video buffer. I am working on that now. Thanks for your patience.

Deleted: canary_issue.mp4 8.5 MB

	canary_issue.mp4 8.5 MB View Download

Deleted: canary-gpu_macos_10.11.6.rtf 79.2 KB

canary-gpu_macos_10.11.6.rtf 79.2 KB Download

Thank you for providing more feedback. Adding the requester to the cc list.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

@sandersd Actually, could you please try testing with this fork of Shaka player:

https://github.com/chrisfillmore/shaka-player/tree/canaryTest

My theory is you are not able to load the content because there is a clock sync problem between your device and the server. I made a small change in this fork which should address that.

Please clone the repo and run ./build/all.py, and serve the demo application. Please let me know if you are able to play the stream this way or not. Thanks!

Thanks, this did resolve my playback issue! I won't be able to investigate further until Tuesday, however.

@modmaker: Could you please ping or email the URL as per comment #8 to proceed further with this issue from test team end?

Thanks!!

A cursory look at the bitstream suggests that the issue may be an encoding one; the initialization segment has an SPS with:

  vui.max_num_reorder_frames = 0
  vui.max_dec_frame_buffering = 0

The first of these indicates that frames are in presentation order, however the stream actually has a reorder depth of at least 2. Note that this causes ffplay to emit a warning on playback:

    [h264 @ 0x????????????] Increasing reorder buffer to 2

The second indicates that no memory needs to be allocated to hold reference frames. I didn't analyze many frames, but within the first 10 frames this value must be at least 3 (minimum number of reference pictures), and probably significantly higher than that (references as far back as 6 frames exist).

I recommend reviewing the SPS contents.

@sandersd Thank you for that feedback! I will take this to our encoding team.

While I'm discussing with our encoding and packaging teams, I have a couple questions:

1. Are you able to play the stream normally? Without any video artifacting?

2. The video plays normally in Chrome stable channel on all machines (so far as we know), and the artifacting problem is only reproducible on specific devices (which presumably share some common trait which we have not yet identified). If there is an encoding issue related to the SPS, any idea why we see different behaviour across these different device/version configurations?

1. I was not able to reproduce the artifacting with the hardware available to me, although I do not currently have a Mac device with AMD graphics as were used here.

2. I would expect the issue to be related to the precise GPU, GPU driver, and OS X versions, and not related to Chrome's version. There have not been significant changes to Chrome's implementation of hardware decode on Mac in some time. The integrated Intel device is of higher importance here, since Apple only uses Intel devices to accelerate video decoding.

You may want to try forcing Chrome to use the integrated graphics only to see if that affects things.

We have been able to reproduce on Windows as well now. A member of my team sent the following video (excuse the handycam):
https://drive.google.com/file/d/1Zpkx3-3c9JX1YN1neSCbJjIfGkwKXz99/view

On the left is Chrome v67.0.3396.62 playing normally. On the right is Canary v69.0.3446.1.

Attached is the Windows System Information and chrome://gpu for the affected machine.

Deleted: windows-sysinfo.txt 2.3 MB

windows-sysinfo.txt 2.3 MB View Download

Deleted: chrome-gpu-windows.rtf 139 KB

chrome-gpu-windows.rtf 139 KB Download

Hmm, this is a surprise. Out of wild curiosity, can you try running with --disable-low-latency-dxva on the Windows machine?

+liberato@ for DXVA expertise.

One idea that could explain this is that 50% of M68+ Dev/Canary is running with MSE Buffer by PTS.

I recommend testing with --disable-features=MseBufferByPts on machines with the issue and --enable-features=MseBufferByPts on machines that are working. You can confirm that the flag is recognized from the chrome://media-internals log.

I also wanted to share this video:
https://drive.google.com/open?id=1yM4hFkv7023MBlUNKAvEf-lScNVsMpa3

This is the same macOS 10.11.6 machine from before, showing Chrome v67.0.3396.48 playing normally on the left, and Canary v69.0.3446.0 on the right.

I will follow up with results from testing with those flags. Thank you!

Looks like MseBufferByPts was a good guess, I am now able to reproduce the issue using --enable-features=MseBufferByPts. It reproduces even with hardware decoding disabled.

joeyparrish@: I'm assuming that this is an issue with the timestamps in the container, but I'm not familiar with Shaka player's implementation.

And now after looking again at the container metadata, it's pretty clear that the problem is that every frame has been marked as a keyframe.

This is a trick that has been used in the past to reduce livestream startup latency, but it's a bad idea for exactly this sort of reason.

I'm not in control of the content itself, and we don't manipulate the container at the player level.  The only influence we have on timestamps is through MSE, and we have no influence on keyframe metadata.

@sandersd The feedback from our encoding team is:

"I'm confused at their statement that every frame is marked as a key frame.  It's definitely not an all I-frame encoding.  It's encoded as an IBBBP structure."

Could you provide some more detail about how you came to this conclusion? Thank you.

It is true that not every frame is encoded as a keyframe, but the MP4 metadata has is_sync set on every sample. You can confirm this with a tool such as http://download.tsi.telecom-paristech.fr/gpac/mp4box.js/filereader.html.

Note that vui.max_dec_frame_buffering = 0 technically implies all I-frames as well, but that doesn't seem to be the actual cause of the problem.

Deleted: is_sync.png 215 KB

	is_sync.png 215 KB View Download

@sandersd, do you know if Chrome v68 will definitely ship with MseBufferByPts enabled? We are trying to estimate how much time we have to find a resolution to this issue.

Buffer by PTS has not started launch review yet, so it is almost certainly not going to be enabled in Beta or Stable M68.

The Intent to Implement was https://groups.google.com/a/chromium.org/forum/#!topic/blink-dev/I6oGJLym-Tk

@sandersd Are you able to estimate when MseBufferByPts will be shipped? A rough estimate is fine:

2-4 months
4-6 months
6+ months

?

Thanks

I would expect somewhere in the first range, 2-4 months, assuming that no widespread issues are discovered.

Thanks. I also have some additional analysis to share from a colleague:

***

So to be able to answer your question I captured a small piece of this AUDHD-3304 stream and converted it into a static DASH stream.
Then I served it locally -- I just put it into a directory named AUD under Shaka source tree, so I can address it as /AUD/manifest_mobile.mpd from Shaka.
And I use a local http server in Shaka directory to serve Shaka itself.
 
Then I modified the stream in a way (described later), to make it work without problems in Canary Chrome.
The modified stream I placed into AUDh directory (and served as /AUDh/manifest_mobile.mpd
 
Both /AUD and /AUDh can be found here: https://drive.google.com/open?id=1h7ZfYp2phT0XwGY37-5lMEtrpQ25nac2
This zip file in addition contains file named 1526452152192item-7item_Segment-13549817614670.m4v.txt
which is the result of parsing of the similarly-named media segment (before I modified the segment).
 
In this parsing result you will find section which starts like this:
 
  sdtp
{
   V0 Fl=000000
 
   sample :   0, sample_depends_on: 2, sample_is_depended_on: 1, sample_has_redundancy: 2
   sample :   1, sample_depends_on: 0, sample_is_depended_on: 0, sample_has_redundancy: 2
   sample :   2, sample_depends_on: 0, sample_is_depended_on: 0, sample_has_redundancy: 2
 
 
This is the "sdtp" BMFF box (SampleDependencyTypeBox) containing information about samples' (frames') dependencies.
 
Based on section 8.6.4.3 "Semantics" of ISO 14496-12 the meaning of these numbers is as follows:
 
sample_depends_on takes one of the following four values:
   0: the dependency of this sample is unknown;
   1: this sample does depend on others (not an I picture);
   2: this sample does not depend on others (I picture);
   3: reserved
sample_is_depended_on takes one of the following four values:
   0: the dependency of other samples on this sample is unknown;
   1: other samples may depend on this one (not disposable);
   2: no other sample depends on this one (disposable);
   3: reserved
sample_has_redundancy takes one of the following four values:
   0: it is unknown whether there is redundant coding in this sample;
   1: there is redundant coding in this sample;
   2: there is no redundant coding in this sample;
   3: reserved
 
 
I.e. samples 1 and 2 above which have sample_depends_on: 0 claim that it is not known if they are I pictures or not.
I manually modified the value to be sample_depends_on: 1 in all the samples of this kind (this is what you can find in AUDh) and
the result of this modification works in Canary.
 
I.e. both AUD and AUDh work in Stable, while AUDh works in Canary and AUD does not.
 
 
So my conclusion is:
  Stable Chrome correctly treats sample_depends_on: 0 as "no information" i.e. ignores it.
  Canary Chrome incorrectly treats sample_depends_on: 0 as if it were sample_depends_on: 2.
 
Hence we either need to convince Harmonics to put there sample_depends_on: 1 or convince Google to fix this bug.

@sandersd Can you advise on the above analysis? Do you agree/disagree?

This is an analysis of the wrong thing; |sample_depends_on| is relevant only as a workaround for invalid content (see https://chromium.googlesource.com/chromium/src/+/07ab26dcaf24e3f91627c20bf00e4e1832c302f0/media/formats/mp4/track_run_iterator.cc#191).

The actual problem is marking all of the samples as sync samples (or, more precisely, not marking them as as non-sync samples).

Neither version of Chrome treats |sample_depends_on| incorrectly, but Canary Chrome *that is opted into the MseBufferByPts experiment* is reordering GOPs (which in MSE just means sequences of frames that start with a keyframe) into presentation order. The MseBufferByPts experiment is the correct implementation as per the MSE spec, but for correctly encoded content there should be no difference.

The experiment can be enabled and disabled independently of the Chrome release schedule. It is expected that M68 will be included when it does roll out.

Thanks again for that feedback. We have passed this information on to our packaging provider, and they'll reach out if they have any further questions.

@sandersd We would like to know the best way to keep up-to-date on upcoming video-related changes in Chrome. Although we can automate the browser, we don't know if Google is conducting some A/B feature testing. If startup flags are required, how can we know what features to enable?

Thank you for your help.

Changes that are expected to be able to affect existing content go through the Blink intent process, which includes posting on the blink-dev mailing list. A smaller selection of important developer-facing features are also mentioned in Google Chrome official blog posts.

The requirements for enabling new changes on Canary and Dev channels are relatively low; often in Canary we are looking for increased failure rates or outright crashes. Chrome variations (A/B experiments) on Canary are usually 50%, so simply testing on Canary as you are clearly doing is highly recommended.

Issues like this bug documents really help inform the Intent to Ship process which comes before a feature like this one is enabled on the Beta channel.

If you have a lot of users using Dev channel, it might start to make sense to follow bugs within the relevant Chrome components (Internals>Media>Source in this case) or commits (there are automated ways to do both of these things), but we understand that that's not reasonable for most developers.

Hi @sandersd, I don't see MseBufferByPts listed in ChromeStatus for Chrome 69 anymore. Do you know if this feature has been pushed back?

https://www.chromestatus.com/features/schedule

Thank you!

The best source for the status of MseBufferByPts that I can find is the Intent to Implement (https://groups.google.com/a/chromium.org/forum/#!msg/blink-dev/I6oGJLym-Tk/Z46le9l4CQAJ).

I'm surprised to hear that there was an entry on the Chrome Status dashboard that was removed. The Chrome feature launch process has been streamlined recently though, and MseBufferByPts may not be considered a feature launch. (That is, it's a non-trivial bug fix to an already launched feature instead.)

There was a Chrome media update on the topic (https://developers.google.com/web/updates/2018/08/chrome-69-media-updates#pts), but I don't see any official announcements.

The current status is that the variation is enabled at 50% in M69 beta, and is expected to eventually ramp to all stable users during the M69 timeframe.

Note: the (M70) fix for  bug 860420  and  bug 584384  now produces chrome://media-internals logs when the AVC-keyframe-ness mismatches that of the ISO-BMFF container in Chrome MSE's parsing of appended media.

See also  bug 879734 ; I'm optimistic it'll help, but also wary that it may risk stability.

@#52: On further consideration, I closed  bug 879734  as WontFix. Sites should fix their streams' keyframe metadata.

FWIW: With https://chromium-review.googlesource.com/c/chromium/src/+/1205640, the private repro media that sandersd@ shared with me no longer repros the decode artifacts when MseBufferByPts is enabled. That CL also doesn't regress playback of the repro media when MseBufferByPts is disabled.

I've reactivated  bug 879734  and have a fix in CQ currently. Goal is to have the fix in M-70. Sites should still fix their media to have correct keyframeness metadata in ISOBMFF.

@wolenetz
Thanks for the updates. It's possible that our test stream which previously reproduced the issue was updated by the packaging team.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1240638ead7a9b3568b4a30918aff44611c00208

commit 1240638ead7a9b3568b4a30918aff44611c00208
Author: Matt Wolenetz <wolenetz@chromium.org>
Date: Wed Sep 05 21:40:56 2018

MSE: Use video (AVC) keyframeness when it differs from MP4's

Due to prevalence of MSE support in other browsers for legacy
incorrectly muxed content where AVC keyframeness is incorrectly marked
in the MP4 container, this change expands the previous
chrome://media-internals logging when such mismatch is detected, to
trust the keyframe-ness of the AVC coded frame for buffering and
decoding.  Otherwise, such streams, especially those with frames
incorrectly marked as keyframes in the MP4, would be buffered as
out-of-order GOPS rather than out-of-order nonkeyframes, corrupting the
buffering and decode sequence on playback in the new, compliant,
MseBufferByPts logic.  Seek preroll from actual AVC nonkeyframes in
legacy MseBufferByDts logic should also be fixed by this change.

BUG= 879734 , 844799 , 229412 ,760264

Change-Id: I90f53ec333e1aeeeaf39e96e176438cb13b4a195
Reviewed-on: https://chromium-review.googlesource.com/1205640
Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Reviewed-by: Dan Sanders <sandersd@chromium.org>
Cr-Commit-Position: refs/heads/master@{#589009}
[modify] https://crrev.com/1240638ead7a9b3568b4a30918aff44611c00208/media/formats/mp4/mp4_stream_parser.cc
[modify] https://crrev.com/1240638ead7a9b3568b4a30918aff44611c00208/media/formats/mp4/mp4_stream_parser_unittest.cc

@fillmore.chris/#56: I locally tested #57 with the media that had been shared with sandersd@ privately. With #57, playback appears to work for that media now with MseBufferByPts, where it previously showed video artifacts. Please try out the next Canary with --enable-features=MseBufferByPts to see if #57 indeed fixes the problem for your real, not just test, streams.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/fc67c41a328c4564c3e39d087b1510b28b32abce

commit fc67c41a328c4564c3e39d087b1510b28b32abce
Author: Matt Wolenetz <wolenetz@chromium.org>
Date: Wed Sep 12 18:54:47 2018

To M70: MSE: Use video (AVC) keyframeness when it differs from MP4's

Due to prevalence of MSE support in other browsers for legacy
incorrectly muxed content where AVC keyframeness is incorrectly marked
in the MP4 container, this change expands the previous
chrome://media-internals logging when such mismatch is detected, to
trust the keyframe-ness of the AVC coded frame for buffering and
decoding.  Otherwise, such streams, especially those with frames
incorrectly marked as keyframes in the MP4, would be buffered as
out-of-order GOPS rather than out-of-order nonkeyframes, corrupting the
buffering and decode sequence on playback in the new, compliant,
MseBufferByPts logic.  Seek preroll from actual AVC nonkeyframes in
legacy MseBufferByDts logic should also be fixed by this change.

BUG= 879734 , 844799 , 229412 ,760264
TBR=dalecurtis@chromium.org,sandersd@chromium.org

Change-Id: I90f53ec333e1aeeeaf39e96e176438cb13b4a195
Reviewed-on: https://chromium-review.googlesource.com/1205640
Commit-Queue: Matthew Wolenetz <wolenetz@chromium.org>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Reviewed-by: Dan Sanders <sandersd@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#589009}(cherry picked from commit 1240638ead7a9b3568b4a30918aff44611c00208)
Reviewed-on: https://chromium-review.googlesource.com/1222197
Reviewed-by: Matthew Wolenetz <wolenetz@chromium.org>
Cr-Commit-Position: refs/branch-heads/3538@{#337}
Cr-Branched-From: 79f7c91a2b2a2932cd447fa6f865cb6662fa8fa6-refs/heads/master@{#587811}
[modify] https://crrev.com/fc67c41a328c4564c3e39d087b1510b28b32abce/media/formats/mp4/mp4_stream_parser.cc
[modify] https://crrev.com/fc67c41a328c4564c3e39d087b1510b28b32abce/media/formats/mp4/mp4_stream_parser_unittest.cc