New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 802406 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Closed: Jan 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

30.8%-455.4% regression in smoothness.tough_filters_cases at 528638:528842

Project Member Reported by m...@chromium.org, Jan 16 2018

Issue description

See the link to graphs below.
 
Project Member

Comment 1 by 42576172...@developer.gserviceaccount.com, Jan 16 2018

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=802406

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=554987ab20ff0ec17d18fb83ab4302fa282b9690de37423bf9666cc66ad9821c


Bot(s) for this bug's original alert(s):

chromium-rel-win7-gpu-intel
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Jan 16 2018

📍 Couldn't reproduce a difference.
https://pinpoint-dot-chromeperf.appspot.com/job/15973400840000
Project Member

Comment 5 by 42576172...@developer.gserviceaccount.com, Jan 16 2018

📍 Couldn't reproduce a difference.
https://pinpoint-dot-chromeperf.appspot.com/job/14cb77c0840000
Project Member

Comment 7 by 42576172...@developer.gserviceaccount.com, Jan 16 2018

📍 Couldn't reproduce a difference.
https://pinpoint-dot-chromeperf.appspot.com/job/14e75240840000

Comment 8 by m...@chromium.org, Jan 16 2018

Cc: senorblanco@chromium.org
Components: Speed>Benchmarks
Owner: cblume@chromium.org
Status: Assigned (was: Untriaged)
Tried bisecting at 2 different points, plus a 3rd wide-range bisect. In all cases, pinpoint showed no change. I'm assuming this is some kind of environmental change in the bots (the ref lines have become volatile as well).

While this probably is a WontFix, I'd like the two test owners (cblume@ and senorblanco@) to take a second look because the test results on the bots have suddenly become volatile (they were a very nice stable line before). It's possible the speed/infrastructure team should be brought into the loop here?

Comment 9 by cblume@chromium.org, Jan 16 2018

Cc: cblume@chromium.org
 Issue 802296  has been merged into this issue.
Yeah, I also tried a quick bisect. (See above)

I had only seen the decoding cases (not the filter case). Like you mentioned, the fact that the ref line changed felt odd.
IIUC, that shouldn't be changing. It represents Stable. So if both are moving, that indicates something is up with the test setup. Maybe the bots are too hot or something?

However, it seems like the ref line on the filter case stays? That makes things a bit more weird.

I'm not sure how to progress here.
If we want to speculatively revert, 3 patches look like "maybe"s:
https://chromium-review.googlesource.com/860100
https://chromium-review.googlesource.com/855587
https://chromium-review.googlesource.com/860077
Cc: vhang@chromium.org pschmidt@chromium.org
THe ref build move does look suspicious. 

Lab folks: are you aware of any hardware change to Intel GPU win 7 bot (build166-m1) recently?

Another possible causes is change to Telemetry framework or change to the test cases of smoothness.tough_filters_cases. Pinpoint would be able to tell the culprit for these cases
I did update the graphics drivers recently.   See  crbug.com/800114 


Comment 13 by m...@chromium.org, Jan 18 2018

Cc: -m...@chromium.org
Cc: majidvp@chromium.org
Status: WontFix (was: Assigned)
re #10: the ref line on filter case is also regressed (it is much more noisier than before). You can see that once you filter out all others

Give the changes in ref builds and graphics driver update in relevant dates I think we can be confident that this is an environment change issue.

Closing as wontfix but please re-open if you disagree cblume@.
Sounds good.

Since the ref bots are now noisy with these drivers it will make it more difficult to detect real perf regressions. But if these are the drives the world is using, these are the drivers we should be testing against.

Hopefully future drivers will be less noisy.

Sign in to add a comment