New issue
Advanced search Search tips

Issue 895565 link

Starred by 1 user
Status: Duplicate
Merged: issue 892501
Owner:
weiliangc@chromium.org
Closed: Oct 15
Cc:
vmi...@chromium.org
sadrul@chromium.org
chiniforooshan@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug

Blocking:
issue 894287



Sign in to add a comment

mean_pixels_checkerboarded result is confusing

Project Member Reported by sadrul@chromium.org, Oct 15 
Dashboard: https://v2spa-dot-chromeperf.appspot.com/#session=9e968c56cf3e6b83d51727ee875f7035587e18184b1f90dfd3330a5cb5e79777

The mean_pixels_checkerboarded reported for the rendering benchmark is consistently over 100%. This makes it difficult to understand whether a movement in the metric represents a regression/improvement or not (e.g. if the metric moves from 600% to 700%, is that a user-visible regression?)

I think we do want to keep reporting how much we are checkerboarding. But perhaps we need to change how we are measuring this? Or at least explain/document how to interpret the results and changes in the metric?

Comment 1 by sadrul@chromium.org, Oct 15 

Ugh, the URL probably doesn't work.

On https://v2spa-dot-chromeperf.appspot.com/ select the following:

test suites: rendering.mobile
measurements: mean_pixels_checkerboarded
bots: nexus 5
test cases: all
statistics: avg

Comment 3 by chiniforooshan@chromium.org, Oct 15 

Summary: the value should be divided by 100 I think. We didn't do it when we migrate the metric to TBMv2 for continuity. We should fix it sometime and ask dashboard people to fix the historical data, too.

Comment 4 by chiniforooshan@chromium.org, Oct 15

Mergedinto: 892501
Status: Duplicate (was: Assigned)

Comment 5 by sadrul@chromium.org, Oct 15 

Ah, I see, thanks!
►

Sign in to add a comment