New issue
Advanced search Search tips

Issue 884950 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Sep 20
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug

Blocked on:
issue 884952

Blocking:
issue 884851



Sign in to add a comment

Remove the discrepancy and some mean metrics

Project Member Reported by sadrul@chromium.org, Sep 18

Issue description

The various _discrepancy metrics are not super useful or easy to understand:

 . frame_times_discrepancy
 . input_event_latency_discrepancy
 . main_thread_scroll_latency_discrepancy

So these should be removed. The corresponding _std metrics display a similar trend to the _discrepancy metrics, so they can still serve as a reasonable replacement if needed.
 
Blockedon: 884952
Summary: Remove the discrepancy and some mean metrics (was: Remove the discrepancy metrics)
The following metrics are also redundant:
 . mean_frame_times
 . mean_input_event_latency
 . mean_main_thread_scroll_latency

Since there's also _avg metrics for all these metrics (i.e. frame_times_avg, input_event_latency_avg etc.): https://chromeperf.appspot.com/report?sid=794bef664b238b1a5ef34f5c2ee25211f1bb7578fde37f38a82ee45d50531ade
FYI, I don't think _avg metrics are separate metrics. _avg, _count, _max, _min, _std, and _sum of existing histograms are, for some reason, added to the perf dashboard automatically for every rendering metric we have. This is not happening on non-rendering benchmarks. Maybe there is a config file somewhere that we didn't setup correctly?

This is at least 2-month old issue (stated before in vis-metrics As previously stated in viz metrics meetings (see https://docs.google.com/document/d/1Fagz0D2y_E7Jd11OZu-2bUahQOUYVszurqIKuKycv58, notes for June 6th)
Cc: benjhayden@chromium.org
Ben, where do we change the set of aggregations displayed?
I believe it's because we don't set summaryOptions [1] for createHistogram for rendering metrics.

[1] https://cs.chromium.org/chromium/src/third_party/catapult/tracing/tracing/metrics/system_health/loading_metric.html?l=344
The default summaryOptions are here:
https://github.com/catapult-project/catapult/blob/master/tracing/tracing/value/histogram.html#L125

Metrics can control which statistics are uploaded to the chromeperf dashboard by passing a dictionary option named `summaryOptions` as the loadingMetric illustrates.

We have plans to simplify or remove summaryOptions after V2SPA launches, when it will be possible to efficiently store most or all summary statistics.
Project Member

Comment 7 by bugdroid1@chromium.org, Sep 18

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/703fd41ed04de2ad81beb5237389417b9ca0e1f0

commit 703fd41ed04de2ad81beb5237389417b9ca0e1f0
Author: Sadrul Habib Chowdhury <sadrul@chromium.org>
Date: Tue Sep 18 20:26:02 2018

rendering: Remove discrepancy metrics.

Remove the following metrics:
 . frame_times_discrepancy
 . input_event_latency_discrepancy
 . main_thread_scroll_latency_discrepancy

The corresponding standard-deviation metrics show similar trends to
these discrepancy metrics, and are easier to understand (both are
fairly noisy too).

BUG= chromium:884950 

Change-Id: I107acbb3ebddbe9ca335a7557b1b0a38681de5b8
Reviewed-on: https://chromium-review.googlesource.com/1228794
Reviewed-by: Timothy Dresser <tdresser@chromium.org>
Reviewed-by: Ben Hayden <benjhayden@chromium.org>
Reviewed-by: Annie Sullivan <sullivan@chromium.org>
Commit-Queue: Sadrul Chowdhury <sadrul@chromium.org>

[modify] https://crrev.com/703fd41ed04de2ad81beb5237389417b9ca0e1f0/telemetry/telemetry/web_perf/timeline_based_page_test_unittest.py
[modify] https://crrev.com/703fd41ed04de2ad81beb5237389417b9ca0e1f0/telemetry/telemetry/util/statistics_unittest.py
[modify] https://crrev.com/703fd41ed04de2ad81beb5237389417b9ca0e1f0/telemetry/telemetry/util/statistics.py
[modify] https://crrev.com/703fd41ed04de2ad81beb5237389417b9ca0e1f0/tracing/tracing/base/math/statistics.html
[modify] https://crrev.com/703fd41ed04de2ad81beb5237389417b9ca0e1f0/telemetry/telemetry/web_perf/metrics/smoothness_unittest.py
[modify] https://crrev.com/703fd41ed04de2ad81beb5237389417b9ca0e1f0/tracing/tracing/base/math/statistics_test.html
[modify] https://crrev.com/703fd41ed04de2ad81beb5237389417b9ca0e1f0/telemetry/telemetry/web_perf/metrics/smoothness.py

Project Member

Comment 8 by bugdroid1@chromium.org, Sep 18

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/356cb84730940a62132b55cdd4a23a2e34efab97

commit 356cb84730940a62132b55cdd4a23a2e34efab97
Author: Sadrul Habib Chowdhury <sadrul@chromium.org>
Date: Tue Sep 18 21:00:13 2018

rendering: Remove some redundant mean_ metrics.

The following metrics are redundant:
 . mean_frame_times
 . mean_input_event_latency
 . mean_main_thread_scroll_latency

These metrics already provide an _avg metric, which is equivalent. So these
metrics are redundant, and not necessary to include in the report.

https://chromeperf.appspot.com/report?sid=794bef664b238b1a5ef34f5c2ee25211f1bb7578fde37f38a82ee45d50531ade

BUG= chromium:884950 

Change-Id: Id3f16e77a7811bdb69397a1b486fcbfdf3bb8598
Reviewed-on: https://chromium-review.googlesource.com/1229335
Reviewed-by: Ben Hayden <benjhayden@chromium.org>
Reviewed-by: Annie Sullivan <sullivan@chromium.org>
Commit-Queue: Sadrul Chowdhury <sadrul@chromium.org>

[modify] https://crrev.com/356cb84730940a62132b55cdd4a23a2e34efab97/tracing/tracing/metrics/rendering/frame_time.html
[modify] https://crrev.com/356cb84730940a62132b55cdd4a23a2e34efab97/telemetry/telemetry/web_perf/metrics/smoothness_unittest.py
[modify] https://crrev.com/356cb84730940a62132b55cdd4a23a2e34efab97/tracing/tracing/metrics/rendering/frame_time_test.html
[modify] https://crrev.com/356cb84730940a62132b55cdd4a23a2e34efab97/telemetry/telemetry/web_perf/metrics/smoothness.py

Project Member

Comment 9 by bugdroid1@chromium.org, Sep 19

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/badbd1c5b6149dafb126de7cbf83fa04eb48e339

commit badbd1c5b6149dafb126de7cbf83fa04eb48e339
Author: Sadrul Habib Chowdhury <sadrul@chromium.org>
Date: Wed Sep 19 16:37:34 2018

smoothness: Update test since some metrics have been removed.

Some of the discrepancy_ and mean_ metrics have been removed in
crrev.com/c/1228794 and crrev.com/c/1229335. Update the tests
accordingly.

BUG= 884950 ,  886913 

Change-Id: If47fa9231e55fe52d6511b71cbcce1776cef4d60
Reviewed-on: https://chromium-review.googlesource.com/1234015
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Ned Nguyen <nednguyen@google.com>
Cr-Commit-Position: refs/heads/master@{#592438}
[modify] https://crrev.com/badbd1c5b6149dafb126de7cbf83fa04eb48e339/tools/perf/measurements/smoothness_unittest.py

Project Member

Comment 10 by bugdroid1@chromium.org, Sep 19

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/20136a68eadb1aaeda4252b590f0bcc047c04ebe

commit 20136a68eadb1aaeda4252b590f0bcc047c04ebe
Author: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com>
Date: Wed Sep 19 18:59:44 2018

Roll src/third_party/catapult c968ea0b6538..92ef0386b8b6 (15 commits)

https://chromium.googlesource.com/catapult.git/+log/c968ea0b6538..92ef0386b8b6


git log c968ea0b6538..92ef0386b8b6 --date=short --no-merges --format='%ad %ae %s'
2018-09-19 dtu@chromium.org [pinpoint] Data migration basic error handling.
2018-09-19 dtu@chromium.org [pinpoint] Basic stats page.
2018-09-19 dtu@chromium.org [pinpoint] Update fields in Jobs table.
2018-09-19 cbruni@chromium.org results2.html debounce search
2018-09-19 simonhatch@chromium.org Dashboard - Fix alert and comparison_magnitude.
2018-09-19 simonhatch@chromium.org Dashboard - Replace last_alerted_revision with query
2018-09-19 slobodan@google.com Fixing frames in animations following response.
2018-09-19 pasko@chromium.org androidStartupMetric: remove the FCP
2018-09-18 dtu@chromium.org [pinpoint] Remove unused <job-chart-icon> import.
2018-09-18 achuith@chromium.org oobe: Catch WebSocketException instead.
2018-09-18 dtu@chromium.org [pinpoint] Remove list of differences and reformat arguments list.
2018-09-18 sadrul@chromium.org rendering: Remove some redundant mean_ metrics.
2018-09-18 sadrul@chromium.org rendering: Remove discrepancy metrics.
2018-09-18 chiniforooshan@chromium.org Telemetry: Fix SF events
2018-09-18 mattm@chromium.org netlog_viewer/README.md: Re-order with user-focused sections at top.


Created with:
  gclient setdep -r src/third_party/catapult@92ef0386b8b6

The AutoRoll server is located here: https://autoroll.skia.org/r/catapult-autoroll

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.

CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel

BUG=chromium:878390, chromium:868073 , chromium:883079 ,chromium:883715, chromium:886621 , chromium:879353 , chromium:884950 , chromium:884950 , chromium:863926 
TBR=sullivan@chromium.org

Change-Id: I9fa8e95d9aad524c032b0bf637dfc07926ce7795
Reviewed-on: https://chromium-review.googlesource.com/1234576
Reviewed-by: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com>
Commit-Queue: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#592499}
[modify] https://crrev.com/20136a68eadb1aaeda4252b590f0bcc047c04ebe/DEPS

Status: Fixed (was: Started)

Sign in to add a comment