New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 666724 link

Starred by 1 user
Status: Available
Owner: ----
Cc:
█ phoglund@chromium.org
EstimatedDays: ----
NextAction: 2019-07-09
OS: ----
Pri: 3
Type: Feature

Blocked on:
issue 666725
issue 666726


Sign in to add a comment

[meta] Make WebRTC perf sheriff process much more pleasant

Project Member Reported by phoglund@chromium.org, Nov 18 2016 
Currently
- there are tons of false alarms
- some tests create an excessive amount of alerts even if they find real regressions
- alerts (especially improvements) frequently carry over to the next sheriff because of sloppy sheriffing, but nobody is notified
- sheriffs triage alerts for all tests, and not the ones they know

Furthermore it's easy to add new tests, but no real process to clean out tests that prove too noisy.

Comment 1 by phoglund@chromium.org, Nov 18 2016 

A few more observations:
- WebRTC rolls should happen daily; the blame lists are too large currently.
- It's really hard to triage FYI-only failures. The only way is to look at bot build pages from the before and after build, and figure out the WebRTC revision from there.
- There's no auto-bisect support for any test in the WebRTC rotation (except some telemetry tests).

Comment 2 by phoglund@chromium.org, Nov 18 2016

Blockedon: 666725

Comment 3 by phoglund@chromium.org, Nov 18 2016

Blockedon: 666726

Comment 4 by ehmaldonado@chromium.org, Nov 18 2016

Cc: ehmaldonado@chromium.org

Comment 5 by phoglund@chromium.org, Dec 14 2016

Owner: ehmaldonado@chromium.org
Edward: see sub-bugs for more details on this effort!

Comment 6 by mcasas@chromium.org, Dec 14 2016 

Re. #1: could we have something like the skia autoroll (https://autoroll.skia.org/)?

Comment 7 by phoglund@chromium.org, Dec 15 2016

Cc: kjellander@chromium.org
Re #6: kjellander@ and ehmaldonado@ will consider your request :)

Comment 8 by kjellander@chromium.org, Dec 16 2016 

That's a nice page. I can look into how reusable it is in Q1, given that we first actually implement autorolling (see  bug 666726  for more info).

Comment 9 by ehmaldonado@chromium.org, Jan 24 2018

Cc: -kjellander@chromium.org -ehmaldonado@chromium.org phoglund@chromium.org
Owner: ----
Status: Available (was: Assigned)
I think we've made some progress here, with the Autoroller and splitting the perf rotations.

Comment 10 by benhenry@google.com, Jan 10

Labels: Pri-3
NextAction: 2019-07-09
Downgrading P2s that haven't been modified in more than 6 months, which have no component or owner.
►

Sign in to add a comment