Add mean time between rebuffers metric based on watch time. |
||||||
Issue descriptionWe should add a better metric for tracking rebuffers. I propose just using the YouTube one: watch time / rebuffers. This is fairly easy to add and will let us experiment more easily with increasing frame counts and measuring Mojo CDM improvements. M-60 has already passed, but the proposed change is small so we should merge back to get better metrics sooner so we can experiment more in M-61.
,
May 31 2017
,
Jun 1 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/6995b86d4c47b0de61cfc3d7417b91488e8a6b02 commit 6995b86d4c47b0de61cfc3d7417b91488e8a6b02 Author: Dale Curtis <dalecurtis@chromium.org> Date: Thu Jun 01 01:13:55 2017 Add a better underflow metric: Media.MeanTimeBetweenRebuffers. This is equivalent to YouTube's MTBR metric in that it corresponds to the total watch time for a session divided by the number of underflow events that occur during that session. The metric is recorded by having the WatchTimeReporter count the number of underflow events that occur and pass them along to MediaInternals during each watch time reporting interval. At finalization MTBR is calculated for AudioVideo.MSE, AudioVideo.SRC, AudioVideo.EME, Audio.MSE, Audio.SRC, and Audio.EME. This is nicer than the existing Media.UnderflowDuration metric since it is not as susceptible to scaling issues due to more playbacks and has no automatic zero weighting. BUG=648710, 727862 TEST=new unittests Change-Id: I144333e6e25cc599bfc89ee2be9a25bac476d156 Reviewed-on: https://chromium-review.googlesource.com/516528 Reviewed-by: Xiaohan Wang <xhwang@chromium.org> Reviewed-by: Ilya Sherman <isherman@chromium.org> Commit-Queue: Dale Curtis <dalecurtis@chromium.org> Cr-Commit-Position: refs/heads/master@{#476124} [modify] https://crrev.com/6995b86d4c47b0de61cfc3d7417b91488e8a6b02/content/browser/media/media_internals.cc [modify] https://crrev.com/6995b86d4c47b0de61cfc3d7417b91488e8a6b02/content/browser/media/media_internals_unittest.cc [modify] https://crrev.com/6995b86d4c47b0de61cfc3d7417b91488e8a6b02/media/base/media_log.cc [modify] https://crrev.com/6995b86d4c47b0de61cfc3d7417b91488e8a6b02/media/base/media_log.h [modify] https://crrev.com/6995b86d4c47b0de61cfc3d7417b91488e8a6b02/media/blink/watch_time_reporter.cc [modify] https://crrev.com/6995b86d4c47b0de61cfc3d7417b91488e8a6b02/media/blink/watch_time_reporter.h [modify] https://crrev.com/6995b86d4c47b0de61cfc3d7417b91488e8a6b02/media/blink/watch_time_reporter_unittest.cc [modify] https://crrev.com/6995b86d4c47b0de61cfc3d7417b91488e8a6b02/media/blink/webmediaplayer_impl.cc [modify] https://crrev.com/6995b86d4c47b0de61cfc3d7417b91488e8a6b02/tools/metrics/histograms/histograms.xml
,
Jun 6 2017
Merge Request for metrics.
,
Jun 6 2017
Your change meets the bar and is auto-approved for M60. Please go ahead and merge the CL to branch 3112 manually. Please contact milestone owner if you have questions. Owners: amineer@(Android), cmasso@(iOS), josafat@(ChromeOS), bustamante@(Desktop) For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Jun 6 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/20a2811bd6b7f97d5f72c308c0b8c6b3e2ddbd61 commit 20a2811bd6b7f97d5f72c308c0b8c6b3e2ddbd61 Author: Dale Curtis <dalecurtis@chromium.org> Date: Tue Jun 06 18:48:09 2017 Add a better underflow metric: Media.MeanTimeBetweenRebuffers. This is equivalent to YouTube's MTBR metric in that it corresponds to the total watch time for a session divided by the number of underflow events that occur during that session. The metric is recorded by having the WatchTimeReporter count the number of underflow events that occur and pass them along to MediaInternals during each watch time reporting interval. At finalization MTBR is calculated for AudioVideo.MSE, AudioVideo.SRC, AudioVideo.EME, Audio.MSE, Audio.SRC, and Audio.EME. This is nicer than the existing Media.UnderflowDuration metric since it is not as susceptible to scaling issues due to more playbacks and has no automatic zero weighting. BUG=648710, 727862 TEST=new unittests TBR=dalecurtis@chromium.org (cherry picked from commit 6995b86d4c47b0de61cfc3d7417b91488e8a6b02) Change-Id: I144333e6e25cc599bfc89ee2be9a25bac476d156 Reviewed-on: https://chromium-review.googlesource.com/516528 Reviewed-by: Xiaohan Wang <xhwang@chromium.org> Reviewed-by: Ilya Sherman <isherman@chromium.org> Commit-Queue: Dale Curtis <dalecurtis@chromium.org> Cr-Original-Commit-Position: refs/heads/master@{#476124} Reviewed-on: https://chromium-review.googlesource.com/526332 Reviewed-by: Dale Curtis <dalecurtis@chromium.org> Cr-Commit-Position: refs/branch-heads/3112@{#196} Cr-Branched-From: b6460e24cf59f429d69de255538d0fc7a425ccf9-refs/heads/master@{#474897} [modify] https://crrev.com/20a2811bd6b7f97d5f72c308c0b8c6b3e2ddbd61/content/browser/media/media_internals.cc [modify] https://crrev.com/20a2811bd6b7f97d5f72c308c0b8c6b3e2ddbd61/content/browser/media/media_internals_unittest.cc [modify] https://crrev.com/20a2811bd6b7f97d5f72c308c0b8c6b3e2ddbd61/media/base/media_log.cc [modify] https://crrev.com/20a2811bd6b7f97d5f72c308c0b8c6b3e2ddbd61/media/base/media_log.h [modify] https://crrev.com/20a2811bd6b7f97d5f72c308c0b8c6b3e2ddbd61/media/blink/watch_time_reporter.cc [modify] https://crrev.com/20a2811bd6b7f97d5f72c308c0b8c6b3e2ddbd61/media/blink/watch_time_reporter.h [modify] https://crrev.com/20a2811bd6b7f97d5f72c308c0b8c6b3e2ddbd61/media/blink/watch_time_reporter_unittest.cc [modify] https://crrev.com/20a2811bd6b7f97d5f72c308c0b8c6b3e2ddbd61/media/blink/webmediaplayer_impl.cc [modify] https://crrev.com/20a2811bd6b7f97d5f72c308c0b8c6b3e2ddbd61/tools/metrics/histograms/histograms.xml
,
Jun 6 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/dc04180b4874ddaaa499938eb2098856f27bbd49 commit dc04180b4874ddaaa499938eb2098856f27bbd49 Author: Dale Curtis <dalecurtis@chromium.org> Date: Tue Jun 06 18:49:25 2017 Move all watch time related keys into their own file. Per MTBR review, this list is growing too big to keep living inside MediaLog; so create a new home for all watch time related keys in a new "watch_time_keys.{h,cc} file. BUG= 727862 TEST=compiles TBR=dalecurtis@chromium.org (cherry picked from commit 5837bd9797c8dbc8c1302767398ecee98074b51e) Change-Id: Ib927f21f04d0a321dac0eec8cc81af74302334e3 Reviewed-on: https://chromium-review.googlesource.com/521804 Reviewed-by: Xiaohan Wang <xhwang@chromium.org> Commit-Queue: Dale Curtis <dalecurtis@chromium.org> Cr-Original-Commit-Position: refs/heads/master@{#476489} Reviewed-on: https://chromium-review.googlesource.com/526352 Reviewed-by: Dale Curtis <dalecurtis@chromium.org> Cr-Commit-Position: refs/branch-heads/3112@{#197} Cr-Branched-From: b6460e24cf59f429d69de255538d0fc7a425ccf9-refs/heads/master@{#474897} [modify] https://crrev.com/dc04180b4874ddaaa499938eb2098856f27bbd49/content/browser/media/media_internals.cc [modify] https://crrev.com/dc04180b4874ddaaa499938eb2098856f27bbd49/content/browser/media/media_internals_unittest.cc [modify] https://crrev.com/dc04180b4874ddaaa499938eb2098856f27bbd49/media/base/BUILD.gn [modify] https://crrev.com/dc04180b4874ddaaa499938eb2098856f27bbd49/media/base/media_log.cc [modify] https://crrev.com/dc04180b4874ddaaa499938eb2098856f27bbd49/media/base/media_log.h [add] https://crrev.com/dc04180b4874ddaaa499938eb2098856f27bbd49/media/base/watch_time_keys.cc [add] https://crrev.com/dc04180b4874ddaaa499938eb2098856f27bbd49/media/base/watch_time_keys.h [modify] https://crrev.com/dc04180b4874ddaaa499938eb2098856f27bbd49/media/blink/watch_time_reporter.cc [modify] https://crrev.com/dc04180b4874ddaaa499938eb2098856f27bbd49/media/blink/watch_time_reporter_unittest.cc
,
Jun 7 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/4a6f54c099373e4004d29e42e716310a515095cc commit 4a6f54c099373e4004d29e42e716310a515095cc Author: Dale Curtis <dalecurtis@chromium.org> Date: Wed Jun 07 02:38:44 2017 Record rebuffer count of watch time sessions to UMA. This allows us to infer a metric equivalent to YouTube's smooth rate. It records the number of rebuffer events that occur within a given watch time session under the following keys: Media.RebuffersCount.{Audio|AudioVideo}.{SRC|MSE,EME} These are the same categories recorded for MTBR. BUG=648710, 727862 TEST=new unittest. Change-Id: Ia115125c3628e8d8436255263387a68f9e225df7 Reviewed-on: https://chromium-review.googlesource.com/522860 Commit-Queue: Dale Curtis <dalecurtis@chromium.org> Reviewed-by: Xiaohan Wang <xhwang@chromium.org> Reviewed-by: Ilya Sherman <isherman@chromium.org> Cr-Commit-Position: refs/heads/master@{#477526} [modify] https://crrev.com/4a6f54c099373e4004d29e42e716310a515095cc/content/browser/media/media_internals.cc [modify] https://crrev.com/4a6f54c099373e4004d29e42e716310a515095cc/content/browser/media/media_internals_unittest.cc [modify] https://crrev.com/4a6f54c099373e4004d29e42e716310a515095cc/media/base/watch_time_keys.cc [modify] https://crrev.com/4a6f54c099373e4004d29e42e716310a515095cc/media/base/watch_time_keys.h [modify] https://crrev.com/4a6f54c099373e4004d29e42e716310a515095cc/tools/metrics/histograms/histograms.xml
,
Jun 13 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/30a7a12e3bf4bd4ba3439f4802087426eeec6291 commit 30a7a12e3bf4bd4ba3439f4802087426eeec6291 Author: Dale Curtis <dalecurtis@chromium.org> Date: Tue Jun 13 22:51:09 2017 Merge M60: "Record rebuffer count of watch time sessions to UMA." This allows us to infer a metric equivalent to YouTube's smooth rate. It records the number of rebuffer events that occur within a given watch time session under the following keys: Media.RebuffersCount.{Audio|AudioVideo}.{SRC|MSE,EME} These are the same categories recorded for MTBR. BUG=648710, 727862 TEST=new unittest. TBR=dalecurtis@chromium.org (cherry picked from commit 4a6f54c099373e4004d29e42e716310a515095cc) Change-Id: Ia115125c3628e8d8436255263387a68f9e225df7 Reviewed-on: https://chromium-review.googlesource.com/522860 Commit-Queue: Dale Curtis <dalecurtis@chromium.org> Reviewed-by: Xiaohan Wang <xhwang@chromium.org> Reviewed-by: Ilya Sherman <isherman@chromium.org> Cr-Original-Commit-Position: refs/heads/master@{#477526} Reviewed-on: https://chromium-review.googlesource.com/534597 Reviewed-by: Dale Curtis <dalecurtis@chromium.org> Cr-Commit-Position: refs/branch-heads/3112@{#329} Cr-Branched-From: b6460e24cf59f429d69de255538d0fc7a425ccf9-refs/heads/master@{#474897} [modify] https://crrev.com/30a7a12e3bf4bd4ba3439f4802087426eeec6291/content/browser/media/media_internals.cc [modify] https://crrev.com/30a7a12e3bf4bd4ba3439f4802087426eeec6291/content/browser/media/media_internals_unittest.cc [modify] https://crrev.com/30a7a12e3bf4bd4ba3439f4802087426eeec6291/media/base/watch_time_keys.cc [modify] https://crrev.com/30a7a12e3bf4bd4ba3439f4802087426eeec6291/media/base/watch_time_keys.h [modify] https://crrev.com/30a7a12e3bf4bd4ba3439f4802087426eeec6291/tools/metrics/histograms/histograms.xml
,
Jun 13 2017
|
||||||
►
Sign in to add a comment |
||||||
Comment 1 by dalecur...@chromium.org
, May 30 2017