Add Animometer / MotionMark tests to Telemetry |
|||||||||||||
Issue descriptionWe should track Animometer / MotionMark tests in Telemetry to catch inadvertent regressions. I think there are two forms. 1) Running in benchmark mode, and returning the actual Animometer score. This may good to track how we are doing directly on the score, but is likely more noisy and less useful for diagnosis. 2) The tests running at a fixed complexity with thread_times metrics. The latter is more important IMO.
,
Jun 13 2016
Because of the tests' adaptive nature in benchmarking mode, getting thread_times in benchmarking mode is low priority for me.
,
Jun 13 2016
,
Jun 13 2016
FYI: animometer-inspired canvas cases were added with this CL: https://codereview.chromium.org/2047773002/
,
Jun 13 2016
@#2: Due to the adaptive nature of the benchmarks, they were modified to make them into fixed-complexity animation loops, which is more appropriate for telemetry smoothness tests.
,
Mar 24 2017
,
Mar 24 2017
,
Apr 13 2017
,
May 2 2017
This is on hold until we make progress on updating smoothness metrics to TBMv2 and harness and reducing overall smoothness benchmark cycle times. The benchmark set-up code is here https://codereview.chromium.org/2771343003/ for when we're ready.
,
May 2 2017
For now you can still put it in tools/perf/contrib/ directory if you want to be able to run the benchmark locally or kick off trybot
,
May 2 2017
,
May 6 2017
,
Feb 5 2018
,
Jul 25
,
Sep 25
This was blocked from landing before, but we keep regressing these benchmarks and need to get them on the waterfall. Picking this up again.
,
Sep 26
📍 Pinpoint job started. https://pinpoint-dot-chromeperf.appspot.com/job/1775a4b7640000
,
Sep 26
📍 Job complete. See results below. https://pinpoint-dot-chromeperf.appspot.com/job/1775a4b7640000
,
Sep 27
📍 Pinpoint job started. https://pinpoint-dot-chromeperf.appspot.com/job/17c15c44e40000
,
Sep 27
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/3e400a746ec659116c6f381fadbf5f4938131475 commit 3e400a746ec659116c6f381fadbf5f4938131475 Author: Victor Miura <vmiura@chromium.org> Date: Thu Sep 27 16:24:32 2018 Rendering benchmarks based on MotionMark HTML and SVG tests. These tests run MotionMark with fixed complexity sufficient to saturate most mobile phones today. Excludes Canvas tests which are already included in other story sets. BUG=619680 Change-Id: Id199dc6d7c5d497941c33405a069de6a9fc26980 Reviewed-on: https://chromium-review.googlesource.com/1244544 Commit-Queue: Victor Miura <vmiura@chromium.org> Reviewed-by: Ned Nguyen <nednguyen@google.com> Reviewed-by: Sadrul Chowdhury <sadrul@chromium.org> Cr-Commit-Position: refs/heads/master@{#594746} [modify] https://crrev.com/3e400a746ec659116c6f381fadbf5f4938131475/tools/perf/benchmark.csv [modify] https://crrev.com/3e400a746ec659116c6f381fadbf5f4938131475/tools/perf/page_sets/data/rendering_mobile.json [add] https://crrev.com/3e400a746ec659116c6f381fadbf5f4938131475/tools/perf/page_sets/data/rendering_mobile_089f7a2710.wprgo.sha1 [add] https://crrev.com/3e400a746ec659116c6f381fadbf5f4938131475/tools/perf/page_sets/data/rendering_mobile_f7f3ea1d6e.wprgo.sha1 [add] https://crrev.com/3e400a746ec659116c6f381fadbf5f4938131475/tools/perf/page_sets/data/rendering_mobile_f86ce4f9f0.wprgo.sha1 [add] https://crrev.com/3e400a746ec659116c6f381fadbf5f4938131475/tools/perf/page_sets/rendering/motionmark.py [modify] https://crrev.com/3e400a746ec659116c6f381fadbf5f4938131475/tools/perf/page_sets/rendering/story_tags.py [modify] https://crrev.com/3e400a746ec659116c6f381fadbf5f4938131475/tools/perf/page_sets/rendering/tough_filters_cases.py
,
Sep 27
📍 Job complete. See results below. https://pinpoint-dot-chromeperf.appspot.com/job/17c15c44e40000
,
Jan 16
,
Jan 16
|
|||||||||||||
►
Sign in to add a comment |
|||||||||||||
Comment 1 by nedngu...@google.com
, Jun 13 2016