New issue
Advanced search Search tips

Issue 758760 link

Starred by 1 user

Issue metadata

Status: Duplicate
Merged: issue 673566
Owner: ----
Closed: Aug 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 1
Type: ----



Sign in to add a comment

Regression range on Seriff-o-Matic sometimes do not match with the actual range

Project Member Reported by zh...@chromium.org, Aug 24 2017

Issue description

Example:
https://sheriff-o-matic.appspot.com/chromium.perf/examine/chromium.perf.Android%20Nexus5X%20Perf.blink_perf.layout.


In Sheriff-o-Matic, the blink_perf.layout regression range is shown as:
chromium: 490956 - 495228

Then I look at the 1st bot "Android Nexts5X Perf" and the 1st run failing blink_perf.layout is #444:
https://uberchromegw.corp.google.com/i/chromium.perf/builders/Android%20Nexus5X%20Perf/builds/444

And #444's range is: 496768 - 497032

Notice that those 2 do not overlap.

I think it is very important to have correct range shown on sheriff-o-matic. As a sheriff myself, I often trust the range shown there and do not actually double check inside the 1st failing run. I noticed this by accident because I was checking a different failing test in similar range. A mismatch would result in a bisect failure, which results in more questions about which part went wrong and much more work.

So I am giving this pri-1. Re-adjust if needed.
 

Comment 1 by zh...@chromium.org, Aug 24 2017

Summary: Regression range on Seriff-o-Matic sometimes do not match with the actual range (was: Regression range on Seriff-o-Matic sometimes do match with the actual range)

Comment 2 by zh...@chromium.org, Aug 24 2017

Hmm, while double checking the links, I am not sure why clicking on the example link does not go to the examine page directly. Filed a separate bug for that ( issue 758764 ).
Mergedinto: 673566
Status: Duplicate (was: Available)
This unfortunately has been happening for a while. We definitely need to up the priority on this though. 

Sign in to add a comment