Transition all infrastructure-related builder configuration to luci-config |
|||||
Issue descriptionToday, a builder launches from a particular point in time with a large configuration payload in JSON (config_dump.json) that’s checked-in at that same point in time. This configuration includes—among many other things—which stages to run, which child builds to kick off, and which hardware tests to execute. Most critically, it also includes a declaration of whether or not a particular child build is considered important or experimental. This item tracks moving at least: * which child builds to kick off * whether builders are critical (removing important/experimental) to luci-config. Later, additional components of config_dump.json may also transition. We should also remove any CIDB-based important status, at the same time.
,
Nov 8
This enables us to submit a CL changing the builder's critical status and then have luci-config immediately notify the parent builder that the critical status has changed. This kills off the use of Tree Status for the same.
,
Nov 8
,
Nov 8
,
Nov 12
,
Nov 13
,
Nov 15
why does a builder/build needs to know whether it is critical? usually it is the other parts of the system (that interpret build results) that decide what to do based on criticality. LUCI config files do not specify which builds to kick off from a build. Usually it is the recipe what knows what to kick off when it runs. Also Browser does not use build-triggering that much, because they don't need parent-child builds that much. I will talk about this more in the upcoming email thread about having a single axis of commits.
,
Nov 15
In the system today, the child builds need to know because they directly take action on failure. However, we'd be a lot better off if they didn't take action, and thus didn't need to know.
,
Nov 16
Only the parent builder needs to know this information. The child builder logic should be removed as Don indicated. The parent builder interprets this status and, in the proposal, is the luci-config client. Based on our conversation in SFO, I was under the impression that luci-config can be used to push any kind of configuration, not just those known to LUCI. I had intended to add a new config type specifically for CrOS builders to replace our use of branched infrastructure configuration. Will that work?
,
Nov 16
yeah, if it is a separate config file from LUCI config files, then it is simpler than I thought. However, where can I read about why parent builder needs this info? What kind of decisions does it make based on this info?
,
Nov 18
Here's what the parent does: * Performs a repo sync * Cherry-picks all changes in the current run set (including any COMMIT-QUEUE.ini changes) * Scans for COMMIT-QUEUE.ini's across all source repos and subdirectories <https://www.chromium.org/chromium-os/build/bypassing-tests-on-a-per-project-basis> * Generate a JSON dump of the configuration that can be ingested by GoldeneEye * Schedules children based on combo of newly updated checked-in config and COMMIT-QUEUE.ini. * Bumps package versions by 1 and commits. * Bumps the Android version, if available. * Exports metadata about the Android build to GS for ingestion by {unknown}. * Performs a Portage sanity check after package bumps. * Submits all CL's to Gerrit * Creates and submits any package bump CL's to Gerrit * Dumps a lot of metadata to a GS location for ingestion by GE * Writes CL history to CIDB in case of failure so that that history may be used during the next run to potentially submit an unchanged CL anyway. "* Schedules children based on combo of newly updated checked-in config and COMMIT-QUEUE.ini." is the stage that would be most affected by this change: it would allow immediately updating the configuration used by the parent. |
|||||
►
Sign in to add a comment |
|||||
Comment 1 by jclinton@chromium.org
, Nov 8