New issue
Advanced search Search tips

Issue 739522 link

Starred by 4 users
Status: Available
Owner: ----
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Feature

Blocking:
issue 674692
issue 728287



Sign in to add a comment

Support rebaselining from CQ try bot results

Project Member Reported by qyears...@chromium.org, Jul 5 2017 
The rebaseline-cl tool (and wpt-update-expectations which uses rebaseline-cl) use the set of try bots listed in config/builders.py[1].

These are used because there is one builder per supported version, and they correspond to the continuous waterfall release builders.

In most cases, baselines are the same across different versions of the same OS (or across all OSs), so results should also be able to be pulled from CQ try jobs if they're available:
 - linux_chromium_rel_ng
 - mac_chromium_rel_ng
 - win_chromium_rel_ng

The desired behavior, I believe, is that when rebaselining or updating expectations, results from the CQ builders should be fetched *if* there are no jobs for the tryserver.blink jobs (having more than one builder per port would complicate things).

So, I think the CQ bots should be added to config/builders.py[1], and tests should be added to ensure that:
 (1) optional try jobs (tryserver.blink try jobs) are preferred when both CQ jobs and optional try jobs are finished.
 (2) CQ jobs are used when optional try jobs are not available.

[1] https://chromium.googlesource.com/chromium/src/+/master/third_party/WebKit/Tools/Scripts/webkitpy/common/config/builders.py#78

Comment 1 by foolip@chromium.org, Jul 5 2017 

That sounds like a great plan. If there are situation where the choice is non-obvious, then perhaps requiring a --use-cq-results flag or similar might get around having to be too clever.

For automatic imports, would it make sense to simultaneously trigger the "waterfall" bots and CQ bots? Here's the flow I'm imagining:
1. Start all the bots that run layout tests, waterfall or CQ
2. In case any bots failed, rebaseline-cl.
3. Send the result through CQ.
4. If it fails, go to step 2.

It seems like this must eventually stabilize on something that would pass CQ, but are there dangers with trying over and over again?

Comment 2 by robertma@chromium.org, Oct 4 2017

Blockedon: 728287
This is related to issue 728287 (starting Blink try jobs and CQ jobs together).

Actually, that issue is the first step, which by itself would be beneficial (in case no rebaseline is needed).

Comment 3 by robertma@chromium.org, Oct 4 2017

Blocking: 674692
Project Member

Comment 4 by sheriffbot@chromium.org, Oct 4

Labels: Hotlist-Recharge-Cold
Status: Untriaged (was: Available)
This issue has been Available for over a year. If it's no longer important or seems unlikely to be fixed, please consider closing it out. If it is important, please re-triage the issue.

Sorry for the inconvenience if the bug really should have been left as Available.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

Comment 5 by foolip@chromium.org, Oct 9

Blockedon: -728287
Blocking: 728287
Status: Available (was: Untriaged)
►

Sign in to add a comment