There were some dangling TODOs for http://crbug.com/751220 , but I felt I should open a new bug for them and re-summarize the issue.
The current state of MSVC build and test is that win-msvc-rel is a compile-only tryjob in the CQ, and chromium.win builds win_msvc_release_bot and win_msvc_debug_bot mb configs. That means we aren't running tests on the CQ or on the waterfall that sheriffs watch.
The old pinned clang bots on the chromium.clang waterfall got turned into MSVC builders as part of the clang switch, so we end up being the ones who notice when tests start failing with MSVC.
As discussed here ( https://crbug.com/799589#c2 ), the plan is to turn down these buildbots and add new tester bots to the main chromium.win waterfall so that Chromium sheriffs will see MSVC-only test failures.
We might consider making the win-msvc-rel trybot run tests, but we'd need to check if we have enough swarming capacity. We can see how much breakage Chromium sheriffs see on the chromium.win waterfall and decide later if it's worth it.
As part of the transition, I think we can drop the win_msvc_shared_release_bot configuration. We thought this might be a popular configuration used by developers who want fast builds and don't debug with VS or windbg. Clang had compatibility issues when the component build and optimizations were enabled, but I think that most people using MSVC are going to use it to work around debug info issues in clang. Therefore, I recommend against adding this config to the waterfall.
---
Here are the steps I think we need to take. I can probably send the CLs to get this done, but please check them for me, I'm not that familiar with buildbot infra:
- Create two new MSVC tester bots on chromium.win in infra/build/masters/master.chromium.win/builders.pyl. This needs two free windows VMs, I'm not sure how to allocate those.
- Categorize the new testers in milo which is done in the infra/config branch of src.git. These will show up grey until the master restart (right?).
- Schedule a master restart of chromium.win
After those are up and green, remove the old bots:
- Remove the non-LLD bots named "CrWinClang*" on chromium.clang
- Remove them from infra/config
- Schedule a master restart of chromium.clang
- After the restart, remove the win_msvc_shared_release_* configs from mb_config.pyl
Comment 1 by dpranke@chromium.org
, Jan 5 2018I wouldn't want to run the tests in the CQ, unless we're seeing a lot more breakages than I think we are (and you're right that we don't really have the capacity for it at the moment, or at least I'd rather use that capacity elsewhere). It should be straightforward to run the tests just on chromium.win first and I or someone from jbudorick's team can help with the config changes. You don't actually need new builders for this on chromium.win, we can just start running tests on the existing builders and then add new trybot builders to tryserver.chromium.win that re-use the existing CQ pool of machines, e.g., add 'win-msvc-compile-{dbg,rel}' and move the CQ to use those instead of 'win-msvc-{rel,dbg}'. This is slightly more complicated in some ways, but it would keep the names straight in the CQ.