Remove the discrepancy and some mean metrics |
||||
Issue descriptionThe various _discrepancy metrics are not super useful or easy to understand: . frame_times_discrepancy . input_event_latency_discrepancy . main_thread_scroll_latency_discrepancy So these should be removed. The corresponding _std metrics display a similar trend to the _discrepancy metrics, so they can still serve as a reasonable replacement if needed.
,
Sep 18
The following metrics are also redundant: . mean_frame_times . mean_input_event_latency . mean_main_thread_scroll_latency Since there's also _avg metrics for all these metrics (i.e. frame_times_avg, input_event_latency_avg etc.): https://chromeperf.appspot.com/report?sid=794bef664b238b1a5ef34f5c2ee25211f1bb7578fde37f38a82ee45d50531ade
,
Sep 18
FYI, I don't think _avg metrics are separate metrics. _avg, _count, _max, _min, _std, and _sum of existing histograms are, for some reason, added to the perf dashboard automatically for every rendering metric we have. This is not happening on non-rendering benchmarks. Maybe there is a config file somewhere that we didn't setup correctly? This is at least 2-month old issue (stated before in vis-metrics As previously stated in viz metrics meetings (see https://docs.google.com/document/d/1Fagz0D2y_E7Jd11OZu-2bUahQOUYVszurqIKuKycv58, notes for June 6th)
,
Sep 18
Ben, where do we change the set of aggregations displayed?
,
Sep 18
I believe it's because we don't set summaryOptions [1] for createHistogram for rendering metrics. [1] https://cs.chromium.org/chromium/src/third_party/catapult/tracing/tracing/metrics/system_health/loading_metric.html?l=344
,
Sep 18
The default summaryOptions are here: https://github.com/catapult-project/catapult/blob/master/tracing/tracing/value/histogram.html#L125 Metrics can control which statistics are uploaded to the chromeperf dashboard by passing a dictionary option named `summaryOptions` as the loadingMetric illustrates. We have plans to simplify or remove summaryOptions after V2SPA launches, when it will be possible to efficiently store most or all summary statistics.
,
Sep 18
The following revision refers to this bug: https://chromium.googlesource.com/catapult/+/703fd41ed04de2ad81beb5237389417b9ca0e1f0 commit 703fd41ed04de2ad81beb5237389417b9ca0e1f0 Author: Sadrul Habib Chowdhury <sadrul@chromium.org> Date: Tue Sep 18 20:26:02 2018 rendering: Remove discrepancy metrics. Remove the following metrics: . frame_times_discrepancy . input_event_latency_discrepancy . main_thread_scroll_latency_discrepancy The corresponding standard-deviation metrics show similar trends to these discrepancy metrics, and are easier to understand (both are fairly noisy too). BUG= chromium:884950 Change-Id: I107acbb3ebddbe9ca335a7557b1b0a38681de5b8 Reviewed-on: https://chromium-review.googlesource.com/1228794 Reviewed-by: Timothy Dresser <tdresser@chromium.org> Reviewed-by: Ben Hayden <benjhayden@chromium.org> Reviewed-by: Annie Sullivan <sullivan@chromium.org> Commit-Queue: Sadrul Chowdhury <sadrul@chromium.org> [modify] https://crrev.com/703fd41ed04de2ad81beb5237389417b9ca0e1f0/telemetry/telemetry/web_perf/timeline_based_page_test_unittest.py [modify] https://crrev.com/703fd41ed04de2ad81beb5237389417b9ca0e1f0/telemetry/telemetry/util/statistics_unittest.py [modify] https://crrev.com/703fd41ed04de2ad81beb5237389417b9ca0e1f0/telemetry/telemetry/util/statistics.py [modify] https://crrev.com/703fd41ed04de2ad81beb5237389417b9ca0e1f0/tracing/tracing/base/math/statistics.html [modify] https://crrev.com/703fd41ed04de2ad81beb5237389417b9ca0e1f0/telemetry/telemetry/web_perf/metrics/smoothness_unittest.py [modify] https://crrev.com/703fd41ed04de2ad81beb5237389417b9ca0e1f0/tracing/tracing/base/math/statistics_test.html [modify] https://crrev.com/703fd41ed04de2ad81beb5237389417b9ca0e1f0/telemetry/telemetry/web_perf/metrics/smoothness.py
,
Sep 18
The following revision refers to this bug: https://chromium.googlesource.com/catapult/+/356cb84730940a62132b55cdd4a23a2e34efab97 commit 356cb84730940a62132b55cdd4a23a2e34efab97 Author: Sadrul Habib Chowdhury <sadrul@chromium.org> Date: Tue Sep 18 21:00:13 2018 rendering: Remove some redundant mean_ metrics. The following metrics are redundant: . mean_frame_times . mean_input_event_latency . mean_main_thread_scroll_latency These metrics already provide an _avg metric, which is equivalent. So these metrics are redundant, and not necessary to include in the report. https://chromeperf.appspot.com/report?sid=794bef664b238b1a5ef34f5c2ee25211f1bb7578fde37f38a82ee45d50531ade BUG= chromium:884950 Change-Id: Id3f16e77a7811bdb69397a1b486fcbfdf3bb8598 Reviewed-on: https://chromium-review.googlesource.com/1229335 Reviewed-by: Ben Hayden <benjhayden@chromium.org> Reviewed-by: Annie Sullivan <sullivan@chromium.org> Commit-Queue: Sadrul Chowdhury <sadrul@chromium.org> [modify] https://crrev.com/356cb84730940a62132b55cdd4a23a2e34efab97/tracing/tracing/metrics/rendering/frame_time.html [modify] https://crrev.com/356cb84730940a62132b55cdd4a23a2e34efab97/telemetry/telemetry/web_perf/metrics/smoothness_unittest.py [modify] https://crrev.com/356cb84730940a62132b55cdd4a23a2e34efab97/tracing/tracing/metrics/rendering/frame_time_test.html [modify] https://crrev.com/356cb84730940a62132b55cdd4a23a2e34efab97/telemetry/telemetry/web_perf/metrics/smoothness.py
,
Sep 19
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/badbd1c5b6149dafb126de7cbf83fa04eb48e339 commit badbd1c5b6149dafb126de7cbf83fa04eb48e339 Author: Sadrul Habib Chowdhury <sadrul@chromium.org> Date: Wed Sep 19 16:37:34 2018 smoothness: Update test since some metrics have been removed. Some of the discrepancy_ and mean_ metrics have been removed in crrev.com/c/1228794 and crrev.com/c/1229335. Update the tests accordingly. BUG= 884950 , 886913 Change-Id: If47fa9231e55fe52d6511b71cbcce1776cef4d60 Reviewed-on: https://chromium-review.googlesource.com/1234015 Reviewed-by: Ned Nguyen <nednguyen@google.com> Commit-Queue: Ned Nguyen <nednguyen@google.com> Cr-Commit-Position: refs/heads/master@{#592438} [modify] https://crrev.com/badbd1c5b6149dafb126de7cbf83fa04eb48e339/tools/perf/measurements/smoothness_unittest.py
,
Sep 19
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/20136a68eadb1aaeda4252b590f0bcc047c04ebe commit 20136a68eadb1aaeda4252b590f0bcc047c04ebe Author: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com> Date: Wed Sep 19 18:59:44 2018 Roll src/third_party/catapult c968ea0b6538..92ef0386b8b6 (15 commits) https://chromium.googlesource.com/catapult.git/+log/c968ea0b6538..92ef0386b8b6 git log c968ea0b6538..92ef0386b8b6 --date=short --no-merges --format='%ad %ae %s' 2018-09-19 dtu@chromium.org [pinpoint] Data migration basic error handling. 2018-09-19 dtu@chromium.org [pinpoint] Basic stats page. 2018-09-19 dtu@chromium.org [pinpoint] Update fields in Jobs table. 2018-09-19 cbruni@chromium.org results2.html debounce search 2018-09-19 simonhatch@chromium.org Dashboard - Fix alert and comparison_magnitude. 2018-09-19 simonhatch@chromium.org Dashboard - Replace last_alerted_revision with query 2018-09-19 slobodan@google.com Fixing frames in animations following response. 2018-09-19 pasko@chromium.org androidStartupMetric: remove the FCP 2018-09-18 dtu@chromium.org [pinpoint] Remove unused <job-chart-icon> import. 2018-09-18 achuith@chromium.org oobe: Catch WebSocketException instead. 2018-09-18 dtu@chromium.org [pinpoint] Remove list of differences and reformat arguments list. 2018-09-18 sadrul@chromium.org rendering: Remove some redundant mean_ metrics. 2018-09-18 sadrul@chromium.org rendering: Remove discrepancy metrics. 2018-09-18 chiniforooshan@chromium.org Telemetry: Fix SF events 2018-09-18 mattm@chromium.org netlog_viewer/README.md: Re-order with user-focused sections at top. Created with: gclient setdep -r src/third_party/catapult@92ef0386b8b6 The AutoRoll server is located here: https://autoroll.skia.org/r/catapult-autoroll Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel BUG=chromium:878390, chromium:868073 , chromium:883079 ,chromium:883715, chromium:886621 , chromium:879353 , chromium:884950 , chromium:884950 , chromium:863926 TBR=sullivan@chromium.org Change-Id: I9fa8e95d9aad524c032b0bf637dfc07926ce7795 Reviewed-on: https://chromium-review.googlesource.com/1234576 Reviewed-by: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com> Commit-Queue: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#592499} [modify] https://crrev.com/20136a68eadb1aaeda4252b590f0bcc047c04ebe/DEPS
,
Sep 20
|
||||
►
Sign in to add a comment |
||||
Comment 1 by sadrul@chromium.org
, Sep 18