New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 816656 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Closed: Mar 2018
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

3.8% regression in smoothness.gpu_rasterization.tough_filters_cases at 538411:538480

Project Member Reported by tdres...@chromium.org, Feb 26 2018

Issue description

See the link to graphs below.
 
Project Member

Comment 1 by 42576172...@developer.gserviceaccount.com, Feb 26 2018

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=816656

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=1e851bea05f1fbffabd325a2d1c32dfde0dcdabca1c23e3075034facf0a55c17


Bot(s) for this bug's original alert(s):

android-nexus7v2
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Feb 27 2018

Cc: brianosman@google.com fmalita@chromium.org vmi...@chromium.org bsalomon@chromium.org
Status: Assigned (was: Untriaged)
📍 Found a significant difference after 1 commit.
https://pinpoint-dot-chromeperf.appspot.com/job/16e80c28440000

Enable fSharpenMipmappedTextures in Skia by brianosman@google.com
https://chromium.googlesource.com/chromium/src/+/dbefb45d16d8544042eb143a831da738618be61b

Understanding performance regressions:
  http://g.co/ChromePerformanceRegressions
Owner: brianosman@chromium.org
brianosman: The bisect found a performance regression at your CL, can you take a look?
What does the gap in the pinpoint job mean? It looks like the metric is identical on the previous commit, so I'm not sure why my CL was selected.

For reference: My CL could have caused a minor performance regression in some cases, but it's a substantial improvement in quality, as well as an improvement in memory and performance in most cases, so any such trade-off would be worth it. I'm just not convinced this is one of those cases.
Owner: sullivan@chromium.org
That pinpoint job picked me, but the graph appears to exonerate my CL?
Cc: dtu@chromium.org
Status: WontFix (was: Assigned)
tl;dr - pinpoint didn't repro an actual regression, just a slight change in distribution. WontFix-ing because this is just a change on one device/metric/page and bisect can't reproduce.

Full details:
Sorry, didn't look closely enough last time. You can read more about how pinpoint detects performance changes at go/pinpoint-data. On the graph, the gap means that it did detect a difference in the distribution of the data points after your CL. And if you look at the values on the right while clicking on different points, you can see that the distribution of the points does seem to change, but the overall value of the metric does not. +dtu is planning to get the data from the dashboard about the size of regression that pinpoint should expect, so we can handle these cases better in the future.

Thanks for looking into this and commenting clearly!

Comment 8 by dtu@chromium.org, Mar 23 2018

The distribution changed in such a way that the mean changed, but the median did not. Unfortunately, the graph shows only the median.

Whether the overall value of the metric changed depends on what you mean by "overall value".

Comment 9 by dtu@chromium.org, Mar 23 2018

I would interpret it as, "some frames that previously took 33.3ms now take 50ms".
Since most of the frames took 50ms both before and after, the median didn't change.
dtu: Thanks, I hadn't picked up on the graph being median. That (plus the histograms) make much more sense now.

Sign in to add a comment