Interval of timeupdate events can be much bigger then required 250ms
Reported by
ddy...@vewd.com,
Dec 20 2017
|
|||||
Issue descriptionUserAgent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.108 Safari/537.36 Example URL: https://jsfiddle.net/4m7spd4e/ Steps to reproduce the problem: With JSFiddle: 1. Open provided URL (https://jsfiddle.net/4m7spd4e/) 2. Wait for video to start playing 3. Observe logs 4. Observe TC results (green = pass, red = fail) For provided local TC: 1. Run downloaded TC in Chromium/Chrome browser 2. Wait for video to start playing 3. Observe logs 4. Observe TC results (green = pass, red = fail) What is the expected behavior? Simply: Interval between two `timeupdate` fired events should be smaller then 250ms (spec says it should be in range [15ms, 250ms]). From HTML5 specification (https://dev.w3.org/html5/spec-preview/media-elements.html): "If the time was reached through the usual monotonic increase of the current playback position during normal playback, and if the user agent has not fired a timeupdate event at the element in the past 15 to 250ms and is not still running event handlers for such an event, then the user agent must queue a task to fire a simple event named timeupdate at the element. (In the other cases, such as explicit seeks, relevant events get fired as part of the overall process of changing the current playback position.)" "The event thus is not to be fired faster than about 66Hz or slower than 4Hz (assuming the event handlers don't take longer than 250ms to run). User agents are encouraged to vary the frequency of the event based on the system load and the average cost of processing the event each time, so that the UI updates are not any more frequent than the user agent can comfortably handle while decoding the video." What went wrong? Chromium/Chrome sometimes misses it's time and sends `timeupdate` after ~500ms from last `timeupdate` which is much more then what is required by the specification. Did this work before? N/A Is it a problem with Flash or HTML5? HTML5 Does this work in other browsers? N/A Chrome version: 63.0.3239.108 Channel: stable OS Version: Ubuntu 16.04 Flash Version: Contents of chrome://gpu: Note: To properly save this page, select the "Webpage, Complete" option in the Save File dialog. Graphics Feature Status Canvas: Software only, hardware acceleration unavailable CheckerImaging: Disabled Flash: Software only, hardware acceleration unavailable Flash Stage3D: Software only, hardware acceleration unavailable Flash Stage3D Baseline profile: Software only, hardware acceleration unavailable Compositing: Software only, hardware acceleration unavailable Multiple Raster Threads: Enabled Native GpuMemoryBuffers: Software only. Hardware acceleration disabled Rasterization: Software only, hardware acceleration unavailable Video Decode: Software only, hardware acceleration unavailable Video Encode: Software only, hardware acceleration unavailable WebGL: Hardware accelerated but at reduced performance WebGL2: Software only, hardware acceleration unavailable Problems Detected ATI/AMD cards with older drivers in Linux are crash-prone: 71381, 76428, 73910, 101225, 136240, 357314 Disabled Features: flash_stage3d, gpu_compositing, panel_fitting, flash3d, gpu_rasterization, accelerated_2d_canvas, accelerated_video_decode, accelerated_webgl2, accelerated_webgl, flash_stage3d_baseline, accelerated_video_encode Accelerated video decode is unavailable on Linux: 137247 Disabled Features: accelerated_video_decode Accelerated video encode is unavailable on Linux Disabled Features: accelerated_video_encode Native GpuMemoryBuffers have been disabled, either via about:flags or command line. Disabled Features: native_gpu_memory_buffers Checker-imaging has been disabled via finch trial or the command line. Disabled Features: checker_imaging Version Information Data exported 12/20/2017, 2:38:39 PM Chrome version Chrome/63.0.3239.108 Operating system Linux 4.10.0-42-generic Software rendering list version 13.13 Driver bug list version 10.34 ANGLE commit id 2ff870db3a3b 2D graphics backend Skia/63 dbae7001c9805fb0a4b18fd0cbc889941cb39db4- Command Line /usr/bin/google-chrome-stable --flag-switches-begin --flag-switches-end Driver Information Initialization time 0 In-process GPU true Passthrough Command Decoder false Supports overlays false Sandboxed false GPU0 VENDOR = 0x1002, DEVICE= 0x6610 Optimus false Optimus false AMD switchable false Driver vendor Google Inc. Driver version 3.3.0.2 Driver date 2017/04/07 Pixel shader version 3.0 Vertex shader version 3.0 Max. MSAA samples 4 Machine model name Machine model version GL_VENDOR Google Inc. GL_RENDERER Google SwiftShader GL_VERSION OpenGL ES 2.0 SwiftShader GL_EXTENSIONS Disabled Extensions Window system binding vendor Window system binding version Window system binding extensions Window manager Compiz XDG_CURRENT_DESKTOP Unity GDMSESSION ubuntu Compositing manager Yes Direct rendering Yes Reset notification strategy 0x0000 GPU process crash count 0 System visual ID 0 RGBA visual ID 0 Compositor Information Tile Update Mode One-copy Partial Raster Enabled GpuMemoryBuffers Status ATC Software only ATCIA Software only DXT1 Software only DXT5 Software only ETC1 Software only R_8 Software only R_16 Software only RG_88 Software only BGR_565 Software only RGBA_4444 Software only RGBX_8888 Software only RGBA_8888 Software only BGRX_8888 Software only BGRA_8888 Software only RGBA_F16 Software only YVU_420 Software only YUV_420_BIPLANAR Software only UYVY_422 Software only Display(s) Information Info Display[8564183066946816] bounds=0,0 1920x1080, workarea=57,24 1863x1056, scale=1, external Color space information {primaries:[[0.3590,0.4464,0.1589,],[0.1959,0.7428,0.0612,],[0.0097,0.0435,0.7717,],], transfer:0.0000*x + 0.0000 if x < 0.0000 else (1.0000*x + 0.0000)**2.2000 + 0.0000, matrix:RGB, range:FULL, icc_profile_id:10} Bits per color component 8 Bits per pixel 24 Info Display[8564183066946817] bounds=1920,0 1920x1080, workarea=1920,0 1920x1080, scale=1, external Color space information {primaries:[[0.3590,0.4464,0.1589,],[0.1959,0.7428,0.0612,],[0.0097,0.0435,0.7717,],], transfer:0.0000*x + 0.0000 if x < 0.0000 else (1.0000*x + 0.0000)**2.2000 + 0.0000, matrix:RGB, range:FULL, icc_profile_id:10} Bits per color component 8 Bits per pixel 24 There are at least two problems here: #1 - Lack of re-scheduling HTMLMediaElement::playback_progress_timer_ timer This problem can be observed when periodic `timeupdate` events are mixed with forced (by HTML5 specification) `timeupdate` events. Steps that can happen: 1. (t=0ms) Periodic `timeupdate` is queued in HTMLMediaElement::ScheduleTimeupdateEvent(true) (through HTMLMediaElement::PlaybackProgressTimerFired) 2. (t=10ms) Forced `timeupdate` is queued in HTMLMediaElement::ScheduleTimeupdateEvent(false) (from any other place) 3. (t=250ms) Periodic `timeupdate` WOULD be queued in HTMLMediaElement::ScheduleTimeupdateEvent(true) (through HTMLMediaElement::PlaybackProgressTimerFired), but it WON'T be, due to having last `timeupdate` event sent 240ms ago. Nothing is done here 4. (t=500ms) Periodic `timeupdate` is queued in HTMLMediaElement::ScheduleTimeupdateEvent(true) (through HTMLMediaElement::PlaybackProgressTimerFired) but it is 490ms from last one! Possible solution? a) HTMLMediaElement::playback_progress_timer_ should be rescheduled on every forced (`periodic_event == false`) schedule call. That way it would be invoked at t=260ms and would be called. b) HTMLMediaElement::playback_progress_timer_ should be scheduled not as automatic-retry, but after each successful `timeupdate` event queue to ~250ms from "now". #2 - Problems with time accuracy / firing event too son, so that it won't be queued when it should be (much bigger!) What happens? 1. (t=0ms) Periodic `timeupdate` is queued. 2. (t=249.95ms) Periodic `timeupdate` is NOT queued due to event being fired "too soon". 3. (t=499.87ms) Periodic `timeupdate` is queued, but after almost 500ms from last one! How it happens? I didn't debug this fully, but my guess is that: 1. At some point HTMLMediaElement::playback_progress_timer_ is started and this posts delayed task. 2. After some time, the time has come to run it. 3. TimerBase::RunInternal() is executed 3.1. `double now = ...` is computed (current time) 3.2. Next delayed task is posted with TimerBase::SetNextFireTime() 3.3. Timer is fired by TaskBase::Fired() 4. This invokes HTMLMediaElement::PlaybackProgressTimerFired 4. This invokes HTMLMediaElement::ScheduleTimeupdateEvent() 4.1. `double now = WTF::CurrentTime()` is computed At this point, `now` variable in HTMLMediaElement::ScheduleTimeupdateEvent() CAN be bigger/further then `now` in TimerBase::RunInternal. With this, the bigger value will be used in next run of HTMLMediaElement::ScheduleTimeupdateEvent() and while it really was 250ms between two runs, the value saved to HTMLMediaElement::last_time_update_event_wall_time_ is further along, so difference betwen now and it is less then 250ms (by small margin). With this, `timeupdate` event will not be fired at that go, but next one, and thus during normal playback delays can be up to 500ms instead of 250ms! ------------------------------------------------------------------------------ With this change: ``` diff --git a/third_party/WebKit/Source/core/html/HTMLMediaElement.cpp b/third_party/WebKit/Source/core/html/HTMLMediaElement.cpp index 73e9bc081cba..937329581b6c 100644 --- a/third_party/WebKit/Source/core/html/HTMLMediaElement.cpp +++ b/third_party/WebKit/Source/core/html/HTMLMediaElement.cpp @@ -26,6 +26,7 @@ #include "core/html/HTMLMediaElement.h" +#include <ios> #include <limits> #include "bindings/core/v8/ExceptionState.h" #include "bindings/core/v8/ScriptController.h" @@ -2799,6 +2800,25 @@ void HTMLMediaElement::ScheduleTimeupdateEvent(bool periodic_event) { bool media_time_has_progressed = media_time != last_time_update_event_media_time_; + if (periodic_event) { + static double first_call = now; + static double last_call = now; + + LOG(ERROR) << std::fixed << __FUNCTION__ << "(): " + << "now: " << now * 1000.0 << "ms" + << ", media_time: " << media_time * 1000.0 << "ms" + << ", from_first_call: " << (now - first_call) * 1000.0 << "ms" + << ", from_last_call: " << (now - last_call) * 1000.0 << "ms" + << ", have_not_recently_fired_timeupdate: " + << have_not_recently_fired_timeupdate + << ", media_time_has_progressed: " << media_time_has_progressed + << ", will_schedule: " + << (have_not_recently_fired_timeupdate && + media_time_has_progressed); + + last_call = now; + } + // Non-periodic timeupdate events must always fire as mandated by the spec, // otherwise we shouldn't fire duplicate periodic timeupdate events when the // movie time hasn't changed. ``` I was able to collect logs/timings in HTML5MediaElement regarding this problem and I attached them in `timeupdate_timings.log` file. This should be enough to reproduce/verify bug.
,
Dec 26 2017
Able to reproduce this issue on reported version 63.0.3239.108 using Windows 10, Ubuntu 14.04, Mac 10.13.1 with steps mentioned in comment#0. i.e; TimeUpdate event is fired without following the given range in spec. This issue is seen from M50 as well. Hence considering this issue as Non-Regression and marking as Untriaged.
,
Jan 2 2018
this bug is related to HTMLMediaElement event. give to blink team.
,
Feb 26 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/ad2b582f574ea854e8fd0f845c3675e831f3ffac commit ad2b582f574ea854e8fd0f845c3675e831f3ffac Author: chcunningham <chcunningham@chromium.org> Date: Mon Feb 26 22:33:30 2018 Improve HTMLMediaElement scheduling of timeupdate The spec allows for anywhere between 15 - 250 ms to fire periodic timeupdate events. These events trigger UI updates on custom media controls, so don't want to over trigger. Historically we _aim_ to fire every 250 ms (4x/sec) during playback. Some bugs: 1) The internal timer often fires a little early, causing frequency guards to fail and delaying the event until the next 250ms cyle. 2) Non-periodic timeupdates in the middle of the periodic cycle cause the next periodic update to be delayed a full cycle. 3) Periodic timeupdates were being scheduled by both CueTimeline and HTMLMediaElement 4) Post-seek timeupdates were being double scheduled This patch remedies (2) by resetting the timer to delay periodic updates following a non-periodic timeupdate by a consistent 250ms. Resetting the timer also allows us to remove the frequency checks, which fixes (1) (3) and (4) are fixed by removing the double scheduling. Kudos to dongheun.kang@lge.com for first addressing some of the above in https://chromium-review.googlesource.com/c/chromium/src/+/888383 Bug: 796557 Change-Id: Ia16689b2ea79c77e31b73cf96cd570ca7641bded Reviewed-on: https://chromium-review.googlesource.com/896624 Commit-Queue: Chrome Cunningham <chcunningham@chromium.org> Reviewed-by: Kentaro Hara <haraken@chromium.org> Reviewed-by: Fredrik Söderquist <fs@opera.com> Cr-Commit-Position: refs/heads/master@{#539293} [modify] https://crrev.com/ad2b582f574ea854e8fd0f845c3675e831f3ffac/third_party/WebKit/Source/core/html/media/HTMLMediaElement.cpp [modify] https://crrev.com/ad2b582f574ea854e8fd0f845c3675e831f3ffac/third_party/WebKit/Source/core/html/media/HTMLMediaElement.h [modify] https://crrev.com/ad2b582f574ea854e8fd0f845c3675e831f3ffac/third_party/WebKit/Source/core/html/media/HTMLMediaElementEventListenersTest.cpp [modify] https://crrev.com/ad2b582f574ea854e8fd0f845c3675e831f3ffac/third_party/WebKit/Source/core/html/track/CueTimeline.cpp
,
Feb 26 2018
,
Feb 27 2018
Hi, I have a question to author of this fix (@chcunningham) as above comment doesn't refer to one of the problems at all. Does this fix address the problem, where timeupdate event is scheduler to be fired after 250ms (let's say that NOW is "0" on timeline, so it should be fired at 0.25s), but at some point in time right before this event should be fired (e.g. at 0.24s) a task is started that takes 0.1s to finish. This will lead to scheduling timeupdate event after 0.24s + 0.1s (== 0.34s == 340ms) which is much more then allowed limit (250ms) causing the browser to send this even less often then it should be fired. Best regards, Damian Dyńdo.
,
Apr 27 2018
|
|||||
►
Sign in to add a comment |
|||||
Comment 1 by krajshree@chromium.org
, Dec 20 2017