New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 785559 link

Starred by 0 users

Issue metadata

Status: WontFix
Owner:
Closed: Mar 2018
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

Many system_health.common_desktop perf tests are creeping up over time

Project Member Reported by m...@chromium.org, Nov 16 2017

Issue description

No clear single change causing the problem. Looks to be caused by either: a) testing environment; or b) death by 1,000 cuts.
 
Project Member

Comment 1 by 42576172...@developer.gserviceaccount.com, Nov 16 2017

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=785559

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=7f4061567a97f26db99dce4e7ab820ee960169d6e272e283e131f761651e6c83


Bot(s) for this bug's original alert(s):

chromium-rel-mac11-air

Comment 3 by m...@chromium.org, Nov 16 2017

Cc: -m...@chromium.org nednguyen@chromium.org
Owner: charliea@chromium.org
Status: Assigned (was: Untriaged)
Ignore the bisect jobs: They are unlikely to return anything.

This bug is about a general trend. Forwarding to owners of this test for further investigation...
Project Member

Comment 4 by 42576172...@developer.gserviceaccount.com, Nov 16 2017

📍 Couldn't reproduce a difference.
https://pinpoint-dot-chromeperf.appspot.com/job/12be03b9f80000
That creep is really weird, but it does seem to have evened out. Charlie, Ned, what should we do here?
If the bisect can reproduce the difference between revision (even if it might fail to pinpoint any cause), then this is a software issue.

If not, then this is a test environment issue. Possibly the device hardware degrade & consume more power?
My feeling is that this is a no-op (investigating regressions from several months ago seems really hard), but stresses the importance of having a working ref build on all benchmarks (which we don't have for system_health.common_desktop). 

The only extra data point I can provide is that the idle_platform story doesn't seem to exhibit this same creep, which seems to hint that it may have been real regressions rather than noise (https://chromeperf.appspot.com/report?sid=0de26e3eb52e89a439d2b092d0c30ff3ab41136c46d698ee5ea9e70359a98f15&start_rev=507308&end_rev=519034). Hard to say without the ref build, though, and that's only one data point.
Bisect show the same power usage of the two revision points in #4. So I highly suspect that this is due to Mac hardware degrading.

There is also sign of ref build creep up:

https://screenshot.googleplex.com/eCErB0cfL8F.png

I vote to mark this as WontFix since the creep up has stopped & stabilized. There is also evidence that this regression is not caused by software change.
Status: WontFix (was: Assigned)

Sign in to add a comment