New issue
Advanced search Search tips

Issue 409693 link

Starred by 1 user

Issue metadata

Status: Archived
Owner: ----
Closed: Aug 2
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug

Blocked on:
issue 410188

Sign in to add a comment

integrate more trees into sheriff-o-matic

Project Member Reported by, Sep 1 2014

Issue description

V8 and memory teams want to use sheriff-o-matic. I think the steps involved for both of them are the same:
1. Change builder_alerts to crawl the appropriate waterfalls. Right now it crawls the ones in gatekeeper.json. We'll need to figure out what the right way to list the extra waterfalls is. Should we just hard-code into builder_alerts for now? Or we could add them to the gatekeeper.json, but I'm not sure that makes sense if gatekeeper isn't actually closing the tree for these waterfalls. Stip, WDYT?
2. Change the tree dropdown in the top-right corner of sheriff-o-matic to have the appropriate value for the tree in question. See, which will make it so that the tree name will be inferred from the master url for cases where the master isn't in gatekeeper_trees.json.

If these waterfalls run any non-standard test suites or don't upload their test results to to test-results.appspot, then there will be more work needed in order to figure out failure reasons for a given step. But, even in those cases, it should correctly identify failures in those steps as whole step failures and get you regression ranges, etc correctly.

builder_alerts code to modify:

sheriff-o-matic code to modify:
is it an issue that the v8 waterfall (client.v8) runs on master4 (and not master1 like chromium and blink).

Michael, does the layout test results upload work meanwhile, or is that still an issue (and is there a bug on file for this)?
Blockedon: chromium:410188
The v8 layout tests are trybot like. In a discussion elsewhere, we agreed that it won't make much sense for them to be uploaded to the test results server (it might be different in the sheriff-o-matic context).

As with all stand-alone v8 tests, we should upload those to the results server (added  a sub-issue).

Comment 4 by, Sep 10 2014

test-results.appspot is how builder_alerts breaks down step failures into lists of tests. FWIW, the landscape has changed a little here. We've started uploading some trybot results to test-results.appspot and plan on probably doing all of them. You should be able to get the layout tests uploading to work just by passing the right flags to run-webkit-tests.

@jochen: I don't understand your question about masters.

That all said, you can get sheriff-o-matic working without knowing which tests fail. It'll still work fine, just at the step level (i.e. it'll tell you when the step started failing and give you a regression range for that).

Comment 5 by, Sep 19 2014

Labels: backend
Skia would also like to use sheriff-o-matic: (if possible)

We don't use gtest or any of the chromium test wrappers - at the moment we'd be happy just to get a link to the build page or log for the failed step.

Comment 7 by, Oct 22 2014

Labels: -Infra-CodeYellow -CY-TreeAlwaysOpen
Project Member

Comment 8 by, Mar 19 2016

Labels: Hotlist-Recharge-Cold
This issue has been available for more than 365 days, and should be re-evaluated. Hotlist-Recharge-Cold label is added for tracking. Please re-triage this issue.

For more details visit - Your friendly Sheriffbot

Comment 9 by, Apr 27 2016

Components: Infra>SheriffOMatic
Labels: -SheriffOMatic
Components: Infra>Sheriffing>Gatekeeper
Components: -Infra>Sheriffing
Labels: Type-Bug

Comment 13 by, Feb 10 2017


Comment 14 by, Mar 7 2017

Project Member

Comment 15 by, May 1 2018

Status: Untriaged (was: Available)
This issue has been Available for over a year. If it's no longer important or seems unlikely to be fixed, please consider closing it out. If it is important, please re-triage the issue.

Sorry for the inconvenience if the bug really should have been left as Available.

For more details visit - Your friendly Sheriffbot
Status: Archived (was: Untriaged)

Sign in to add a comment