Dave, can you take a look at the bisects in  bug 647635 ?

(Also cc-ing Kari, current bug triage sheriff)

OK, so it looks like what happened is that multiple bisects were started on different bots/revision ranges/sub metrics. Unfortunately, all of them appear to be relatively noisy metrics.

For example:

https://chromeperf.appspot.com/buildbucket_job_status/9000908305634125424
is for the jQuery-TodoMVC/jQuery-TodoMVC metric on win_x64_perf_bisect, over revisions 418177-418999

While
https://chromeperf.appspot.com/buildbucket_job_status/9000885414096035616
is for the React-TodoMVC/React-TodoMVC metric on mac_10_11_perf_bisect, over revisions 418884-418999


The third metric bisected on, VanillaJS-TodoMVC, doesn't really even have a noticeable regression. See the third graph: https://chromeperf.appspot.com/group_report?bug_id=647635


Things to know: AutoBisect is a tool that triggers when alerts were triaged into a bug, and also when started manually by a person. I think the first three were started by the dashboard when the alerts were triaged, and the others were definitely started manually.


For 1) (Constantly bisecting to completely different revisions): The biggest issue is that we need less noisy metrics. If the regression isn't significantly larger than the variance between normal runs, it's going to be very hard to find the regression. But in addition to that we (bisect, pref sheriffs, bug assignees, etc) should be better educated when the metric is noisy and how to handle regressions on it. I filed  crbug.com/650491  to start a discussion for those changes.

For 2) (Constantly re-assigning the bug to the random people it ends up blaming):
If multiple concurrent bisects are an expected occurrence, we should handle this better. I'm going to take over this bug for discussion on that.
 
The best option I see for that at the moment is to CC the suspected author(s) instead of setting them as the owner.


I think there is also a related issue:  3 ) Dashboard should probably be better at picking what alert(s) to bisect on when a bug is filed.

I'm downgrading this to P2.

We're going to try to be better about blaming and bisect monitoring in Pinpoint.

At the same time, the Speed Operations Benchmarks team is working to reduce the number of noisy and otherwise bad benchmarks.