Several points to discuss/decide:

1. We want to launch pre-cq builders even if they are experimental, right?
2. Do we want experimental builders to post code review comments on failure? If not, how success/failure of experimental builders are presented to users?
3. How do we record failures of experimental builders?


My rough idea:
- Launch all pre-cq builders, including experimental ones.
- Avoid sending any notification from experimental builders.
- Introduce a new CL action "fail_experimental" which is considered essentially same as "verified". If an experimental builder failed, this new CL action is inserted.

WDYT?

Sounds great; thanks for opening a bug tracking this.

Sorry for not updating for long time. I've been busy for other works these days.

So I got some time and was looking at this issue, but I found it's not going to be a simple change as I expected initially. In particular, the fact that CL rejection can happen on both pre-cq-launcher and individual trybots makes things complicated.

I'm now wondering if I should work on this now. We are trying to add 4 builders soon, but adding tens of builders (issue 842075) is expected to months away due to resource capacity, so benefit of this feature is limited for the time being. Also I remember CI team discussed removing tree status entirely.

WDYT?

Yes, I agree it's not worth doing it against Tree Status since we are shutting that down. Will leave this bug open for the larger issue of doing this through the replacement on CI API.