h264 video not playing
Reported by
sah...@hotstar.com,
Sep 13
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Issue descriptionSteps to reproduce the problem: 1. Play any video on hotstar.com or vimeo.com. 2. after the video starts, pause it and come back after a couple of hours 3. launch another video, it will fail What is the expected behavior? videos should play What went wrong? All h264 videos fail to start . The chrome browser will get in a state where h264 videos fail to start even though buffer has > 10 seconds of buffer. However, vp9 videos work as tried on youtube website. This is not specific to drm videos. Restarting the browser will fix the problem. Did this work before? N/A Does this work in other browsers? N/A Chrome version: 69.0.3497.91 Channel: n/a OS Version: Android 8.1.0. Flash Version: We are seeing that >20% of playback attempts fail to launch after getting manifest on android chrome. It seems that neither an error event is thrown not any other signal is received to indicate an error. It is NOT specific to drm videos. For our lab testing, we have found that chrome somehow gets in a state where h264 videos fail to play after a while. I have verified on the following websites and playback does not work for hls.js demo and bitmovin player demo player. But vp9 ( and vp8) videos work as confirmed on youtube.com. From our observations: readyState = 1 video.buffered() has at least 10 seconds of buffer for the current time playbackRate is 1 we did receive loadedmetadata and waiting event but no timeupdate changing the current time does not fix the issue Please let us know if this is a known issue and you guys can share some insights regarding the root cause ? Is there any tool (like chrome://media-internals) we can use to extract any more debug information ? If we restart the browser, playback works fine.
,
Sep 19
Tested the issue on android and video is playing without any fail Steps to reproduce: -------------------------- 1. Launched chrome and navigated to hotstar.com 2. Played video , paused it and came back after 2 hrs, when clicked on play video is played without any fail Chrome version: 69.0.3497.91 / 69.0.3497.100 OS: Android 8.0 Android device: Pixel 2 @ sahilb: Please check the above steps and let us know if we miss anything. Is this issue consistently reproducible, On 10 how many times this issue is reproducible? Log from chrome://media-internals would help in further debugging of the issue. Thanks!
,
Sep 20
,
Sep 20
chrome://media-internals should have the right information. vp8, vp9 might be falling back to software which would explain why they're working. if you can collect a chrome://trace of a device in this state while attempting playback that would be helpful too. Is there any specific device you're seeing this on?
,
Sep 21
this is not consistently reproducible. 2/10 times i'd say. the user agent is: Mozilla/5.0 (Linux; Android 8.1.0; Nexus 6P Build/OPM6.171019.030.B1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.91 Mobile Safari/537.36 chrome://trace does not load anything. it says : this site can't be reached. did you mean chrome://tracing?
,
Sep 21
Thank you for providing more feedback. Adding the requester to the cc list. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Sep 21
i have the device in this state now. please let me know if i can get you any additional logging. here is the output of chrome://media-internals Save log Hide players Copy all to clipboard Recent Players Player Properties Copy to clipboard Property Value audio_codec_name aac debug (Log limit reached. Further similar entries may be suppressed): Parsed audio frame has DTS 192000us, which is after the frame's PTS 158666us duration 4780.92 found_audio_stream true found_video_stream true frame_title Watch Bigg Boss episode 103 Online on hotstar.com frame_url https://www.hotstar.com/tv/bigg-boss/s-1144/samrat-roll-rida-lock-horns/1000120270 info Truncating audio buffer which overlaps append window start. PTS -12000us frame_end_timestamp 9333us append_window_start 0us origin_url https://www.hotstar.com/ pipeline_state kStarting player_id 14 render_id 14 seek_target 0.033333 url blob:https://www.hotstar.com/0801dd89-635a-4255-805a-8bccc7e0ddf3 video_codec_name h264 Log property filter Timestamp Property Value 00:00:00 00 origin_url https://www.hotstar.com/ 00:00:00 00 frame_url https://www.hotstar.com/tv/bigg-boss/s-1144/samrat-roll-rida-lock-horns/1000120270 00:00:00 00 frame_title Watch Bigg Boss episode 103 Online on hotstar.com 00:00:00 00 url blob:https://www.hotstar.com/0801dd89-635a-4255-805a-8bccc7e0ddf3 00:00:00 00 info ChunkDemuxer: buffering by DTS 00:00:00 00 pipeline_state kStarting 00:00:00 366 duration 4780.92 00:00:04 862 found_audio_stream true 00:00:04 862 audio_codec_name aac 00:00:05 364 found_video_stream true 00:00:05 364 video_codec_name h264 00:00:07 94 debug Parsed audio frame has DTS 0us, which is after the frame's PTS -33333us 00:00:07 94 info Dropping audio frame (DTS 0us PTS -33333us,-12000us) that is outside append window [0us,9223372036854775807us). 00:00:07 94 debug Parsed audio frame has DTS 21333us, which is after the frame's PTS -12000us 00:00:07 94 info Truncating audio buffer which overlaps append window start. PTS -12000us frame_end_timestamp 9333us append_window_start 0us 00:00:07 94 debug Parsed audio frame has DTS 42666us, which is after the frame's PTS 9333us 00:00:07 94 debug Parsed audio frame has DTS 64000us, which is after the frame's PTS 30666us 00:00:07 94 debug Parsed audio frame has DTS 85333us, which is after the frame's PTS 52000us 00:00:07 94 debug Parsed audio frame has DTS 106666us, which is after the frame's PTS 73333us 00:00:07 94 debug Parsed audio frame has DTS 128000us, which is after the frame's PTS 94666us 00:00:07 94 debug Parsed audio frame has DTS 149333us, which is after the frame's PTS 116000us 00:00:07 94 debug Parsed audio frame has DTS 170666us, which is after the frame's PTS 137333us 00:00:07 94 debug (Log limit reached. Further similar entries may be suppressed): Parsed audio frame has DTS 192000us, which is after the frame's PTS 158666us 00:00:12 473 seek_target 0.033333
,
Sep 21
Also, please try these titles in chrome android . it looks like these bring the decoder in that state https://www.hotstar.com/tv/bigg-boss/s-1144/samrat-roll-rida-lock-horns/1000120270 https://www.hotstar.com/tv/bigg-boss/s-1144/roll-rida-feels-disturbed/1000120258 https://www.hotstar.com/tv/bigg-boss/s-1144/fight-for-the-finale-ticket/1000120250
,
Sep 21
Yes chrome://tracing is what I meant. Please enable the media+audio categories -- if you can run the trace using Chrome Canary that'd be helpful. That said, in this case it looks like the error is that Chrome never gets enough buffered data to start playback. I can tell because there's no decoder setup or buffering state logs. You'll probably need to add logging to your application that prints out the SourceBuffer.buffered values. It looks like you may be seeking to a point that doesn't actually have any data.
,
Sep 21
I'm also unable to use any of the links. It just says go to us.hotstar.com :(
,
Sep 21
1. I am using chrome on android, so chrome://tracing does not work. it says this site can't be reached. installed chrome canary as well, still no luck. 2. there is enough buffer . Confirming the videoElement.buffered(); the range is 1200.142958 - 1224.133633 currentTime is : 1201.499996 3. as mentioned above, none of the video websites with h264 can play video once the browser is in this state. readyState is 1. I have tried vimeo and hls.js demo website.also, on closing and restarting the browser, it works. 4. the content seem to be restricted to India, I will try get it deployed to an environment where you can test in US. in the meantime, let me know how i can get chrome://tracing to work
,
Sep 21
Ah, sorry I forgot that part. It does seem like the MediaCodec is getting hosed. You need to attach to a desktop computer and do the following: https://www.chromium.org/developers/how-tos/trace-event-profiling-tool/recording-tracing-runs#TOC-Capture-a-trace-from-Chrome-on-Android-with-DevTools- If you can grab the output of "adb logcat" that would help debug this as well.
,
Sep 24
chrome tracing logs:
,
Sep 24
,
Sep 24
Don't see anything in the trace or logs -- the one crash there is issue 856976 -- which is fixed. Can you download "Chrome Canary" on Android and record the trace from that of getting the device into a bad state? It has a lot more logging which might be helpful. Please selecting "Rendering" category and then "edit categories" to also enable the "media" and "audio" categories recording the trace. You might need to select "record as much as possible" too.
,
Sep 25
i performed the steps to repro the error on chrome canary . please see attached logs.
,
Sep 25
Hmm, can you describe what happened in that trace? I don't see any problems with video playback there. It seems to play normally. Also you seem not to be on the most recent canary, so please update that for any future traces.
,
Sep 26
these logs are from executing the repro steps to bring the system in this error state when no h264 videos will play.
,
Sep 26
i have updated the chrome canary version, and attaching logs that were obtained on: 1) repro steps to bring the system in the error state 2) launching multiple playbacks that never start after playback starts. Note that in the first set of logs, you will see first 2 (or 3 playbacks) start. but somewhere the decoder seems to get in the error state and from there on no playback will start.
,
Sep 26
,
Sep 26
*typo in the above line: the 2) point above is: 2) launching multiple playbacks that never start after the system enters the "error" state.
,
Sep 26
Doesn't look like you enabled "media" category for the trace in comment#22. Did you have that enabled?
,
Sep 27
Trace #21 is good. It's showing Vimeo trying to select a decoder (which should be MediaCodecVideoDecoder), but we're not getting any reply for 13s, before it's cancelled and another attempt is made. MCVD::Initialize doesn't really do anything but return success immediately, so this suggests we may have trouble actually connecting to the MojoVideoDecoder for some reason... We don't have any traces for MojoVideoDecoder/Service though. I'll see about adding some to see what's going on.
,
Sep 28
more logs
,
Sep 28
,
Sep 29
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/1318de357be0b44219023aa4aaf9ed8a36e9922f commit 1318de357be0b44219023aa4aaf9ed8a36e9922f Author: Dale Curtis <dalecurtis@chromium.org> Date: Sat Sep 29 03:05:06 2018 Improve AVDACodecAllocator traces. This ensures we have traces for creation and deletion. This ensures we have the following traces: - CodecAllocator::CreateMediaCodec - CodecAllocator::DeleteMediaCodec Whenever a codec is created or destroyed. Previously there was no trace for when the codec is deleted. This should help find destruction hangs. This also renames one method which is only used in tests to have the ForTesting suffix. BUG= 883620 TEST=none R=liberato Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel Change-Id: Ic62b2a8b16a53c5dba0a1a914571873bf938156b Reviewed-on: https://chromium-review.googlesource.com/1252907 Reviewed-by: Frank Liberato <liberato@chromium.org> Commit-Queue: Dale Curtis <dalecurtis@chromium.org> Cr-Commit-Position: refs/heads/master@{#595290} [modify] https://crrev.com/1318de357be0b44219023aa4aaf9ed8a36e9922f/media/gpu/android/avda_codec_allocator.cc [modify] https://crrev.com/1318de357be0b44219023aa4aaf9ed8a36e9922f/media/gpu/android/avda_codec_allocator.h [modify] https://crrev.com/1318de357be0b44219023aa4aaf9ed8a36e9922f/media/gpu/android/avda_codec_allocator_unittest.cc
,
Sep 29
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/325e8189e145100d6e34bb4cba068fbadc697730 commit 325e8189e145100d6e34bb4cba068fbadc697730 Author: Dale Curtis <dalecurtis@chromium.org> Date: Sat Sep 29 19:16:02 2018 Add traces for the MojoVideoDecoderService. These traces will allow us to diagnose problems on the GPU side. Specifically the following traces are added: - MojoVideoDecoderService::Construct - MojoVideoDecoderService::GetSupportedConfigs - MojoVideoDecoderService::Initialize [async] - MojoVideoDecoderService::Decode [async] ---- [step] ReadDecoderBuffer ---- [step] Decode - MojoVideoDecoderService::Reset [async] - MojoVideoDecoderService::OnDecoderOutput - MojoVideoDecoderService::OnOverlayInfoChanged - MojoVideoDecoderService::OnDecoderRequestedOverlayInfo BUG= 883620 TEST=none R=sandersd Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel Change-Id: I24581ec28f52290856d6daf262417e632999ce21 Reviewed-on: https://chromium-review.googlesource.com/1252538 Commit-Queue: Dale Curtis <dalecurtis@chromium.org> Reviewed-by: Dan Sanders <sandersd@chromium.org> Cr-Commit-Position: refs/heads/master@{#595320} [modify] https://crrev.com/325e8189e145100d6e34bb4cba068fbadc697730/media/base/decode_status.cc [modify] https://crrev.com/325e8189e145100d6e34bb4cba068fbadc697730/media/base/decode_status.h [modify] https://crrev.com/325e8189e145100d6e34bb4cba068fbadc697730/media/filters/decoder_stream.cc [modify] https://crrev.com/325e8189e145100d6e34bb4cba068fbadc697730/media/filters/decoder_stream.h [modify] https://crrev.com/325e8189e145100d6e34bb4cba068fbadc697730/media/mojo/services/BUILD.gn [modify] https://crrev.com/325e8189e145100d6e34bb4cba068fbadc697730/media/mojo/services/mojo_video_decoder_service.cc [modify] https://crrev.com/325e8189e145100d6e34bb4cba068fbadc697730/media/mojo/services/mojo_video_decoder_service.h
,
Sep 30
Sahil if you can grab one more trace of things going wrong on the latest canary with those changes it'd be helpful. Thanks!
,
Oct 1
logs after we get into the error state. Note that even youtube does not work anymore
,
Oct 1
,
Oct 1
I don't think you sent the correct trace in c#31, it's for a desktop machine and not for the mobile device. Can you send again? The one in c#32 seems to work earlier in the trace, but later playbacks don't even instantiate a player. When you record the trace again can you include the "mojom" category?
,
Oct 3
strange. i had followed the same steps as before. Here is another set of logs on the latest canary with mojom category.
,
Oct 3
,
Oct 3
Hmm, this is still not showing anything media blocking. Can you do the following: Trigger repro (w/o tracing enabled). Start tracing with the categories we've discussed, then try to play: http://storage.googleapis.com/dalecurtis/buck2.mp4 I haven't been able to get out VPN working with the site yet.
,
Oct 4
no problem. here you go.
,
Oct 4
,
Oct 4
here is one more set of logs with all categories enabled. these were captured when the player is already in the error state. i tried the buck2.mp4 url mentioned in C36, and also youtube video. neither played
,
Oct 4
Hmm, I just don't see any reason why decoder selection isn't working. There's no attempt to even create a MojoVideoDecoder or any other video decoder once in this state -- but no threads are showing to be hung. Everything else appears to be working fine. My only guess is the thread servicing VDAs in the gpu process is shutdown or hung in some way which isn't showing up. Did you enable the disabled by default categories "task_scheduler_.*" on the right pane in the trace in c#39?
,
Oct 4
It might be worth trying chrome://crashdump/ and chrome://gpucrash/ in this state to generate crashdumps so that we can see what each thread is up to. (Note that gpucrash does actually crash the GPU process.) chrome://crashes/ should then contain crash IDs that we can look up.
,
Oct 4
(Crash reports must be enabled in the Chrome privacy settings.)
,
Oct 5
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/76bf417b04f7fce43e23551ac47426c91e05653e commit 76bf417b04f7fce43e23551ac47426c91e05653e Author: Dale Curtis <dalecurtis@chromium.org> Date: Fri Oct 05 02:59:59 2018 Add more DecoderSelector traces to debug issue. Adds a STEP trace with the decoder name being initialized as well as a STEP trace when DecryptingDemuxerStream is initialized. BUG= 883620 TEST=none Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel Change-Id: I958b4f98836e8e4d43ec59931d3cd952c52a09e1 Reviewed-on: https://chromium-review.googlesource.com/c/1263576 Commit-Queue: Dale Curtis <dalecurtis@chromium.org> Commit-Queue: Dan Sanders <sandersd@chromium.org> Reviewed-by: Dan Sanders <sandersd@chromium.org> Cr-Commit-Position: refs/heads/master@{#596973} [modify] https://crrev.com/76bf417b04f7fce43e23551ac47426c91e05653e/media/filters/decoder_selector.cc
,
Oct 8
i can't find a way to enable crash reports on chrome canary on android (version 71_0_3573_0). the option is not listed in privacy settings (https://support.google.com/chrome/answer/96817?co=GENIE.Platform%3DAndroid&oco=1) on chrome stable , i do see these settings. let me know which browser and logs would you like me to take.
,
Oct 8
in c#39, i had enabled all logs.
,
Oct 8
actually, there were all logs, on the left hand side column. I will send another round of logs with task* enabled.
,
Oct 8
,
Oct 8
,
Oct 8
Thanks for these new traces! I still can't make much sense of them though, it's as if some of the events were simply not recorded in the trace. (Did it run out of buffer at any point?) I can at least be certain now that it fails in a state of trying to bring up MojoVideoDecoder. I think we'll need to focus on finding a way to reproduce on our end.
,
Oct 9
yes, that would help a lot. our team suggested using tunnel bear , for which you can create a free account and install in on your android and then point it to India. you should be able to consistently repro it using the steps in c#8.
,
Oct 10
I am now able to access the content, but I was unable to reproduce the issue quickly. Some more information on reproduction would be useful: - Are any devices other than the Pixel 2 affected? - What Android build number are you using? Are multiple Android builds affected? - Are there adaptations (resolution or codec choices), or alternative pipelines (eg. HLS), if so, how do I ensure that I am using the correct parameters? - How long do you play the videos before the problem occurs? How do you determine that the problem has occurred? Thanks!
,
Oct 11
- we have found that this error does not happen on Android version 9.* - on android 8.*, it is consistently reproducing. - see attachments for version/model details of 2 phones that repro the error. - we are using the identical media package (dash) for all attempts. no change in parameters or codes. - i typically play the video for 5 seconds and then switch to the next one. please try these playbacks in the following order : 1. https://www.hotstar.com/tv/bigg-boss/s-1144/fight-for-the-finale-ticket/1000120250 2. https://www.hotstar.com/tv/bigg-boss/s-1144/roll-rida-feels-disturbed/1000120258 3. https://www.hotstar.com/tv/bigg-boss/s-1144/fight-for-the-finale-ticket/1000120250 4. https://www.hotstar.com/tv/bigg-boss/s-1144/roll-rida-feels-disturbed/1000120258 - for me, in 90% of the cases , the 3rd playback wont start. and rest 10% will fail at the fourth. - in canary, it will fail to start h264 + youtube videos after we are in this state - in chrome stable (69), i can play youtube videos but not h264 once we are in this state
,
Oct 13
I am able to produce playback failures in 70.0.3538.57 (current beta), but they seem to be failing before decoders are created. (Perhaps a fetch or CDM issue.) In 71.0.3578.0 (current canary), I am unable to produce a playback failure. There was a change on Oct 10 that may be of relevance (commit 4e06e7edf4b8b40930cc7b5f527bcfb2ceb6c431). Can you test Canary again to see if that has resolved the issue? Thanks!
,
Oct 16
yes, the issue is resolved on Canary. I tried 10 different playbacks that could cause the error but didn't get the error . Thats great !!! 1. Could you please share if something was special about these titles that caused the issue ? 2. Could you push this change as a patch on next chrome stable 70 ?
,
Oct 16
1. I'm not sure. The bug is triggered during teardown of a video playback, but the exact details that cause the symptoms in this bug are unknown. 2. The change is in M71. A merge was requested on Oct 10, but given that the stable release happened today, it looks like that will not be approved.
,
Oct 16
,
Oct 18
thanks. keep us posted if/when it goes out in a patch version of M70. We can monitor our error rates. If not, we will wait for the rollout of M71.
,
Nov 14
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/e2d67812c7f47150bf109334fcbf63298b03f06b commit e2d67812c7f47150bf109334fcbf63298b03f06b Author: John Rummell <jrummell@chromium.org> Date: Wed Nov 14 21:02:47 2018 Fix DCHECK to not fire on initialize failure Also add some new tests to verify Initialize() failures. BUG= 883620 TEST=new tests pass Change-Id: I8ed3ca04b37fcdca4b9c9c3483b1d7c696bf232a Reviewed-on: https://chromium-review.googlesource.com/c/1336059 Reviewed-by: Dan Sanders <sandersd@chromium.org> Commit-Queue: John Rummell <jrummell@chromium.org> Cr-Commit-Position: refs/heads/master@{#608122} [modify] https://crrev.com/e2d67812c7f47150bf109334fcbf63298b03f06b/media/mojo/services/mojo_video_decoder_service.cc [modify] https://crrev.com/e2d67812c7f47150bf109334fcbf63298b03f06b/media/mojo/test/mojo_video_decoder_integration_test.cc
,
Dec 17
Assuming this was fixed by M71 rollout. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
►
Sign in to add a comment |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Comment 1 by chelamcherla@chromium.org
, Sep 17