New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 796557 link

Starred by 4 users

Issue metadata

Status: Fixed
Owner:
Closed: Feb 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux , Windows , Mac
Pri: 2
Type: Bug



Sign in to add a comment

Interval of timeupdate events can be much bigger then required 250ms

Reported by ddy...@vewd.com, Dec 20 2017

Issue description

UserAgent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.108 Safari/537.36

Example URL:
https://jsfiddle.net/4m7spd4e/

Steps to reproduce the problem:
With JSFiddle:

1. Open provided URL (https://jsfiddle.net/4m7spd4e/)
2. Wait for video to start playing
3. Observe logs
4. Observe TC results (green = pass, red = fail)

For provided local TC:
1. Run downloaded TC in Chromium/Chrome browser
2. Wait for video to start playing
3. Observe logs
4. Observe TC results (green = pass, red = fail)

What is the expected behavior?
Simply:
Interval between two `timeupdate` fired events should be smaller then 250ms (spec says it should be in range [15ms, 250ms]).

From HTML5 specification (https://dev.w3.org/html5/spec-preview/media-elements.html):

"If the time was reached through the usual monotonic increase of the current playback position during normal playback, and if the user agent has not fired a timeupdate event at the element in the past 15 to 250ms and is not still running event handlers for such an event, then the user agent must queue a task to fire a simple event named timeupdate at the element. (In the other cases, such as explicit seeks, relevant events get fired as part of the overall process of changing the current playback position.)"

"The event thus is not to be fired faster than about 66Hz or slower than 4Hz (assuming the event handlers don't take longer than 250ms to run). User agents are encouraged to vary the frequency of the event based on the system load and the average cost of processing the event each time, so that the UI updates are not any more frequent than the user agent can comfortably handle while decoding the video."

What went wrong?
Chromium/Chrome sometimes misses it's time and sends `timeupdate` after ~500ms from last `timeupdate` which is much more then what is required by the specification.

Did this work before? N/A 

Is it a problem with Flash or HTML5? HTML5

Does this work in other browsers? N/A

Chrome version: 63.0.3239.108  Channel: stable
OS Version: Ubuntu 16.04
Flash Version: 

Contents of chrome://gpu: 
Note: To properly save this page, select the "Webpage, Complete" option in the Save File dialog.
Graphics Feature Status
Canvas: Software only, hardware acceleration unavailable
CheckerImaging: Disabled
Flash: Software only, hardware acceleration unavailable
Flash Stage3D: Software only, hardware acceleration unavailable
Flash Stage3D Baseline profile: Software only, hardware acceleration unavailable
Compositing: Software only, hardware acceleration unavailable
Multiple Raster Threads: Enabled
Native GpuMemoryBuffers: Software only. Hardware acceleration disabled
Rasterization: Software only, hardware acceleration unavailable
Video Decode: Software only, hardware acceleration unavailable
Video Encode: Software only, hardware acceleration unavailable
WebGL: Hardware accelerated but at reduced performance
WebGL2: Software only, hardware acceleration unavailable
Problems Detected
ATI/AMD cards with older drivers in Linux are crash-prone: 71381, 76428, 73910, 101225, 136240, 357314
Disabled Features: flash_stage3d, gpu_compositing, panel_fitting, flash3d, gpu_rasterization, accelerated_2d_canvas, accelerated_video_decode, accelerated_webgl2, accelerated_webgl, flash_stage3d_baseline, accelerated_video_encode
Accelerated video decode is unavailable on Linux: 137247
Disabled Features: accelerated_video_decode
Accelerated video encode is unavailable on Linux
Disabled Features: accelerated_video_encode
Native GpuMemoryBuffers have been disabled, either via about:flags or command line.
Disabled Features: native_gpu_memory_buffers
Checker-imaging has been disabled via finch trial or the command line.
Disabled Features: checker_imaging
Version Information
Data exported	12/20/2017, 2:38:39 PM
Chrome version	Chrome/63.0.3239.108
Operating system	Linux 4.10.0-42-generic
Software rendering list version	13.13
Driver bug list version	10.34
ANGLE commit id	2ff870db3a3b
2D graphics backend	Skia/63 dbae7001c9805fb0a4b18fd0cbc889941cb39db4-
Command Line	/usr/bin/google-chrome-stable --flag-switches-begin --flag-switches-end
Driver Information
Initialization time	0
In-process GPU	true
Passthrough Command Decoder	false
Supports overlays	false
Sandboxed	false
GPU0	VENDOR = 0x1002, DEVICE= 0x6610
Optimus	false
Optimus	false
AMD switchable	false
Driver vendor	Google Inc.
Driver version	3.3.0.2
Driver date	2017/04/07
Pixel shader version	3.0
Vertex shader version	3.0
Max. MSAA samples	4
Machine model name	
Machine model version	
GL_VENDOR	Google Inc.
GL_RENDERER	Google SwiftShader
GL_VERSION	OpenGL ES 2.0 SwiftShader
GL_EXTENSIONS	
Disabled Extensions	
Window system binding vendor	
Window system binding version	
Window system binding extensions	
Window manager	Compiz
XDG_CURRENT_DESKTOP	Unity
GDMSESSION	ubuntu
Compositing manager	Yes
Direct rendering	Yes
Reset notification strategy	0x0000
GPU process crash count	0
System visual ID	0
RGBA visual ID	0
Compositor Information
Tile Update Mode	One-copy
Partial Raster	Enabled
GpuMemoryBuffers Status
ATC	Software only
ATCIA	Software only
DXT1	Software only
DXT5	Software only
ETC1	Software only
R_8	Software only
R_16	Software only
RG_88	Software only
BGR_565	Software only
RGBA_4444	Software only
RGBX_8888	Software only
RGBA_8888	Software only
BGRX_8888	Software only
BGRA_8888	Software only
RGBA_F16	Software only
YVU_420	Software only
YUV_420_BIPLANAR	Software only
UYVY_422	Software only
Display(s) Information
Info	Display[8564183066946816] bounds=0,0 1920x1080, workarea=57,24 1863x1056, scale=1, external
Color space information	{primaries:[[0.3590,0.4464,0.1589,],[0.1959,0.7428,0.0612,],[0.0097,0.0435,0.7717,],], transfer:0.0000*x + 0.0000 if x < 0.0000 else (1.0000*x + 0.0000)**2.2000 + 0.0000, matrix:RGB, range:FULL, icc_profile_id:10}
Bits per color component	8
Bits per pixel	24
Info	Display[8564183066946817] bounds=1920,0 1920x1080, workarea=1920,0 1920x1080, scale=1, external
Color space information	{primaries:[[0.3590,0.4464,0.1589,],[0.1959,0.7428,0.0612,],[0.0097,0.0435,0.7717,],], transfer:0.0000*x + 0.0000 if x < 0.0000 else (1.0000*x + 0.0000)**2.2000 + 0.0000, matrix:RGB, range:FULL, icc_profile_id:10}
Bits per color component	8
Bits per pixel	24

There are at least two problems here:

#1 - Lack of re-scheduling HTMLMediaElement::playback_progress_timer_ timer

This problem can be observed when periodic `timeupdate` events are mixed with forced (by HTML5 specification) `timeupdate` events.
Steps that can happen:

1. (t=0ms) Periodic `timeupdate` is queued in HTMLMediaElement::ScheduleTimeupdateEvent(true) (through HTMLMediaElement::PlaybackProgressTimerFired)
2. (t=10ms) Forced `timeupdate` is queued in HTMLMediaElement::ScheduleTimeupdateEvent(false) (from any other place)
3. (t=250ms) Periodic `timeupdate` WOULD be queued in HTMLMediaElement::ScheduleTimeupdateEvent(true) (through HTMLMediaElement::PlaybackProgressTimerFired), but it WON'T be, due to having last `timeupdate` event sent 240ms ago. Nothing is done here
4. (t=500ms) Periodic `timeupdate` is queued in HTMLMediaElement::ScheduleTimeupdateEvent(true) (through HTMLMediaElement::PlaybackProgressTimerFired) but it is 490ms from last one!

Possible solution?
a) HTMLMediaElement::playback_progress_timer_ should be rescheduled on every forced (`periodic_event == false`) schedule call. That way it would be invoked at t=260ms and would be called.
b) HTMLMediaElement::playback_progress_timer_ should be scheduled not as automatic-retry, but after each successful `timeupdate` event queue to ~250ms from "now".

#2 - Problems with time accuracy / firing event too son, so that it won't be queued when it should be (much bigger!)

What happens?

1. (t=0ms) Periodic `timeupdate` is queued.
2. (t=249.95ms) Periodic `timeupdate` is NOT queued due to event being fired "too soon".
3. (t=499.87ms) Periodic `timeupdate` is queued, but after almost 500ms from last one!

How it happens?
I didn't debug this fully, but my guess is that:

1. At some point HTMLMediaElement::playback_progress_timer_ is started and this posts delayed task.
2. After some time, the time has come to run it.
3. TimerBase::RunInternal() is executed
3.1. `double now = ...` is computed (current time)
3.2. Next delayed task is posted with TimerBase::SetNextFireTime()
3.3. Timer is fired by TaskBase::Fired()
4. This invokes HTMLMediaElement::PlaybackProgressTimerFired
4. This invokes HTMLMediaElement::ScheduleTimeupdateEvent()
4.1. `double now = WTF::CurrentTime()` is computed

At this point, `now` variable in HTMLMediaElement::ScheduleTimeupdateEvent() CAN be bigger/further then `now` in TimerBase::RunInternal.
With this, the bigger value will be used in next run of HTMLMediaElement::ScheduleTimeupdateEvent() and while it really was 250ms between two runs, the value saved to HTMLMediaElement::last_time_update_event_wall_time_ is further along, so difference betwen now and it is less then 250ms (by small margin).

With this, `timeupdate` event will not be fired at that go, but next one, and thus during normal playback delays can be up to 500ms instead of 250ms!

------------------------------------------------------------------------------

With this change:

```
diff --git a/third_party/WebKit/Source/core/html/HTMLMediaElement.cpp b/third_party/WebKit/Source/core/html/HTMLMediaElement.cpp
index 73e9bc081cba..937329581b6c 100644
--- a/third_party/WebKit/Source/core/html/HTMLMediaElement.cpp
+++ b/third_party/WebKit/Source/core/html/HTMLMediaElement.cpp
@@ -26,6 +26,7 @@

 #include "core/html/HTMLMediaElement.h"

+#include <ios>
 #include <limits>
 #include "bindings/core/v8/ExceptionState.h"
 #include "bindings/core/v8/ScriptController.h"
@@ -2799,6 +2800,25 @@ void HTMLMediaElement::ScheduleTimeupdateEvent(bool periodic_event) {
   bool media_time_has_progressed =
       media_time != last_time_update_event_media_time_;

+  if (periodic_event) {
+    static double first_call = now;
+    static double last_call = now;
+
+    LOG(ERROR) << std::fixed << __FUNCTION__ << "(): "
+               << "now: " << now * 1000.0 << "ms"
+               << ", media_time: " << media_time * 1000.0 << "ms"
+               << ", from_first_call: " << (now - first_call) * 1000.0 << "ms"
+               << ", from_last_call: " << (now - last_call) * 1000.0 << "ms"
+               << ", have_not_recently_fired_timeupdate: "
+               << have_not_recently_fired_timeupdate
+               << ", media_time_has_progressed: " << media_time_has_progressed
+               << ", will_schedule: "
+               << (have_not_recently_fired_timeupdate &&
+                   media_time_has_progressed);
+
+    last_call = now;
+  }
+
   // Non-periodic timeupdate events must always fire as mandated by the spec,
   // otherwise we shouldn't fire duplicate periodic timeupdate events when the
   // movie time hasn't changed.
```

I was able to collect logs/timings in HTML5MediaElement regarding this problem and I attached them in `timeupdate_timings.log` file. This should be enough to reproduce/verify bug.
 
timeupdate_test.zip
1.5 KB Download
timeupdate_timings.log
49.4 KB View Download
Labels: Needs-Triage-M63
Cc: sc00335...@techmahindra.com
Labels: M-65 Triaged-ET OS-Mac OS-Windows
Status: Untriaged (was: Unconfirmed)
Able to reproduce this issue on reported version 63.0.3239.108 using Windows 10, Ubuntu 14.04, Mac 10.13.1 with steps mentioned in comment#0. i.e; TimeUpdate event is fired without following the given range in spec.

This issue is seen from M50 as well. Hence considering this issue as Non-Regression and marking as Untriaged.
Components: -Internals>Media Blink>Media
this bug is related to HTMLMediaElement event. give to blink team.
Project Member

Comment 4 by bugdroid1@chromium.org, Feb 26 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ad2b582f574ea854e8fd0f845c3675e831f3ffac

commit ad2b582f574ea854e8fd0f845c3675e831f3ffac
Author: chcunningham <chcunningham@chromium.org>
Date: Mon Feb 26 22:33:30 2018

Improve HTMLMediaElement scheduling of timeupdate

The spec allows for anywhere between 15 - 250 ms to fire periodic
timeupdate events. These events trigger UI updates on custom media
controls, so don't want to over trigger. Historically we _aim_ to fire
every 250 ms (4x/sec) during playback.

Some bugs:
1) The internal timer often fires a little early, causing frequency
   guards to fail and delaying the event until the next 250ms cyle.
2) Non-periodic timeupdates in the middle of the periodic cycle cause
   the next periodic update to be delayed a full cycle.
3) Periodic timeupdates were being scheduled by both CueTimeline and
   HTMLMediaElement
4) Post-seek timeupdates were being double scheduled


This patch remedies (2) by resetting the timer to delay periodic updates
following a non-periodic timeupdate by a consistent 250ms. Resetting the
timer also allows us to remove the frequency checks, which fixes (1)

(3) and (4) are fixed by removing the double scheduling.

Kudos to dongheun.kang@lge.com for first addressing some of the above
in https://chromium-review.googlesource.com/c/chromium/src/+/888383


Bug:  796557 
Change-Id: Ia16689b2ea79c77e31b73cf96cd570ca7641bded
Reviewed-on: https://chromium-review.googlesource.com/896624
Commit-Queue: Chrome Cunningham <chcunningham@chromium.org>
Reviewed-by: Kentaro Hara <haraken@chromium.org>
Reviewed-by: Fredrik Söderquist <fs@opera.com>
Cr-Commit-Position: refs/heads/master@{#539293}
[modify] https://crrev.com/ad2b582f574ea854e8fd0f845c3675e831f3ffac/third_party/WebKit/Source/core/html/media/HTMLMediaElement.cpp
[modify] https://crrev.com/ad2b582f574ea854e8fd0f845c3675e831f3ffac/third_party/WebKit/Source/core/html/media/HTMLMediaElement.h
[modify] https://crrev.com/ad2b582f574ea854e8fd0f845c3675e831f3ffac/third_party/WebKit/Source/core/html/media/HTMLMediaElementEventListenersTest.cpp
[modify] https://crrev.com/ad2b582f574ea854e8fd0f845c3675e831f3ffac/third_party/WebKit/Source/core/html/track/CueTimeline.cpp

Status: Fixed (was: Untriaged)

Comment 6 by ddy...@vewd.com, Feb 27 2018

Hi,

I have a question to author of this fix (@chcunningham) as above comment doesn't refer to one of the problems at all.

Does this fix address the problem, where timeupdate event is scheduler to be fired after 250ms (let's say that NOW is "0" on timeline, so it should be fired at 0.25s), but at some point in time right before this event should be fired (e.g. at 0.24s) a task is started that takes 0.1s to finish. This will lead to scheduling timeupdate event after 0.24s + 0.1s (== 0.34s == 340ms) which is much more then allowed limit (250ms) causing the browser to send this even less often then it should be fired.

Best regards,
Damian Dyńdo.
Owner: chcunningham@chromium.org

Sign in to add a comment