New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 619680 link

Starred by 3 users

Issue metadata

Status: Started
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 2
Type: Feature

Blocked on:
issue 691581

Blocking:
issue 719131



Sign in to add a comment

Add Animometer / MotionMark tests to Telemetry

Project Member Reported by vmi...@chromium.org, Jun 13 2016

Issue description

We should track Animometer / MotionMark tests in Telemetry to catch inadvertent regressions.

I think there are two forms.


1) Running in benchmark mode, and returning the actual Animometer score.  This may good to track how we are doing directly on the score, but is likely more noisy and less useful for diagnosis.

2) The tests running at a fixed complexity with thread_times metrics.

The latter is more important IMO.
 
Cc: eakuefner@chromium.org
We actually can do both at the same time. i.e: enabling trace while running the animometr/motionmark & pull out the Animometer score as the same time as thread_times metrics score computed using the trace. 
However, we would need someone's help to port thread_times metrics to use tbm2 framework.

Comment 2 by vmi...@chromium.org, Jun 13 2016

Because of the tests' adaptive nature in benchmarking mode, getting thread_times in benchmarking mode is low priority for me.

Comment 3 by vmi...@chromium.org, Jun 13 2016

Cc: junov@chromium.org

Comment 4 by junov@chromium.org, Jun 13 2016

Cc: sebastienlc@google.com
FYI: animometer-inspired canvas cases were added with this CL: https://codereview.chromium.org/2047773002/

Comment 5 by junov@chromium.org, Jun 13 2016

@#2: Due to the adaptive nature of the benchmarks, they were modified to make them into fixed-complexity animation loops, which is more appropriate for telemetry smoothness tests.

Comment 6 by vmi...@chromium.org, Mar 24 2017

Cc: esprehn@chromium.org chrishtr@chromium.org
 Issue 616273  has been merged into this issue.

Comment 7 by vmi...@chromium.org, Mar 24 2017

Components: Internals>GPU>Internals
Owner: vmi...@chromium.org
Status: Started (was: Available)
Cc: -eakuefner@chromium.org
Status: Assigned (was: Started)
This is on hold until we make progress on updating smoothness metrics to TBMv2 and harness and reducing overall smoothness benchmark cycle times.

The benchmark set-up code is here https://codereview.chromium.org/2771343003/ for when we're ready.

For now you can still put it in tools/perf/contrib/ directory if you want to be able to run the benchmark locally or kick off trybot
Blockedon: 691581
Blocking: 719131
Components: Internals>GPU>Metrics
Cc: -junov@chromium.org
Cc: sadrul@chromium.org enne@chromium.org
Status: Started (was: Assigned)
This was blocked from landing before, but we keep regressing these benchmarks and need to get them on the waterfall.  Picking this up again.
📍 Job complete. See results below.
https://pinpoint-dot-chromeperf.appspot.com/job/1775a4b7640000
Project Member

Comment 19 by bugdroid1@chromium.org, Sep 27

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3e400a746ec659116c6f381fadbf5f4938131475

commit 3e400a746ec659116c6f381fadbf5f4938131475
Author: Victor Miura <vmiura@chromium.org>
Date: Thu Sep 27 16:24:32 2018

Rendering benchmarks based on MotionMark HTML and SVG tests.

These tests run MotionMark with fixed complexity sufficient to
saturate most mobile phones today.  Excludes Canvas tests which
are already included in other story sets.

BUG=619680

Change-Id: Id199dc6d7c5d497941c33405a069de6a9fc26980
Reviewed-on: https://chromium-review.googlesource.com/1244544
Commit-Queue: Victor Miura <vmiura@chromium.org>
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Reviewed-by: Sadrul Chowdhury <sadrul@chromium.org>
Cr-Commit-Position: refs/heads/master@{#594746}
[modify] https://crrev.com/3e400a746ec659116c6f381fadbf5f4938131475/tools/perf/benchmark.csv
[modify] https://crrev.com/3e400a746ec659116c6f381fadbf5f4938131475/tools/perf/page_sets/data/rendering_mobile.json
[add] https://crrev.com/3e400a746ec659116c6f381fadbf5f4938131475/tools/perf/page_sets/data/rendering_mobile_089f7a2710.wprgo.sha1
[add] https://crrev.com/3e400a746ec659116c6f381fadbf5f4938131475/tools/perf/page_sets/data/rendering_mobile_f7f3ea1d6e.wprgo.sha1
[add] https://crrev.com/3e400a746ec659116c6f381fadbf5f4938131475/tools/perf/page_sets/data/rendering_mobile_f86ce4f9f0.wprgo.sha1
[add] https://crrev.com/3e400a746ec659116c6f381fadbf5f4938131475/tools/perf/page_sets/rendering/motionmark.py
[modify] https://crrev.com/3e400a746ec659116c6f381fadbf5f4938131475/tools/perf/page_sets/rendering/story_tags.py
[modify] https://crrev.com/3e400a746ec659116c6f381fadbf5f4938131475/tools/perf/page_sets/rendering/tough_filters_cases.py

📍 Job complete. See results below.
https://pinpoint-dot-chromeperf.appspot.com/job/17c15c44e40000
Components: Test>Telemetry
Components: -Tests>Telemetry

Sign in to add a comment