New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 663095 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Last visit > 30 days ago
Closed: Jan 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Feature



Sign in to add a comment

Check 4k coverage of top uncrypted websites (YouTube, Vimeo, etc) and make sure it would catch bugs like crbug/641059 in the future

Project Member Reported by crouleau@chromium.org, Nov 7 2016

Issue description

Check 4k coverage of top uncrypted websites (YouTube, Vimeo, etc) and make sure it would catch bugs like crbug/641059 in the future.

Yining, 

Would it be reasonable to modify our manual test suite to make sure we catch bugs like this in the future? I would like to think that we have pretty good coverage of 4k on popular sites like Vimeo. Is that a realistic expectation?

cc'd johnpallett@: do you think that we should look into modifying our suite for this?

Thanks,
Caleb

 
it make sense. I have 4k video tests on youtube but not on vimeo. I'll officially add it to testtracker
Great!

It looks like crbug/641059 may still be broken. Perhaps look into the recent repro steps for guidance.
+1 to adding Vimeo to our test suite; it sounds like they're a good test case for MSE and the 4K video highlighted some buffer management code that would be good to regress.
The only problem is that I only find mp4 4k video on vimeo. If you see vp9 4k video, or know any extension/tool to force vp9 decoder on h264 video, can you let me know? 
Why is that a problem?
Vimeo doesn't support VP9. Can you just add the test for H264? Also, if we add a 4k Vimeo H264 test, I think we can drop the YouTube H264 test, since YouTube Chrome users use VP9.
*I mean we can drop the YouTube 4k H264 test.
H264 4k has already added to vimeo. Are the two site YouTube and Vimeo using the same streaming mechanism? if so, I don't object to drop 4k H264 tests on YouTube. matt@, can you confirm?
YouTube and Vimeo definitely don't use the same player, but they use the same API in Chrome to play the videos (MSE).

The point is that Chrome users don't use H264 on YouTube since YouTube always serves VP9 for Chrome. So there's no point in testing it.

If H264 4k was already added for Vimeo, then why didn't it catch crbug/641059? Basically this bug is requiring us to figure out why we didn't catch crbug/641059 ourselves and fix the problem so that we catch similar bugs in the future. It is not a problem if we sometimes miss bugs since we don't have full coverage of everything, but we still need to frequently re-evaluate our test plans to make sure we would catch similar bugs in the future.
+1 to #9 (YT and Vimeo use MSE API on Chrome, but MSE leaves much of the behavior [stream fetching & buffering, controls, app performance, etc] up to the player vs normal src= media playback.)
I'm not absolutely certain YT *always* sends VP9 to Chrome (they do experiments, at least...)
Yes, it is true that YouTube experiments. But for manual testing we should try to just see what the user sees (whether we are chosen for an experiment or not). Currently for manual H264 YouTube tests, Yining runs an extension that tells YouTube that Chrome can't play VP9, so then YouTube falls back to H264. But I don't think that kind of scenario is really what we want to test.
#9, caleb, test case for 4k H264 on Vimeo was added to test plan after you opened this bug. It was NOT tested on vimeo before. In prior time only 4k on YouTube was tested. I agree we need added re-evaluate test coverage from time to time.
Ah, okay. sounds good. Thanks for the information.
Status: Fixed (was: Assigned)

Sign in to add a comment