New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 755358 link

Starred by 2 users

Issue metadata

Status: WontFix
Owner:
Closed: Feb 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug

Blocking:
issue 755357



Sign in to add a comment

factor "model" -> HWTest "boards" mapping into config_lib or config_dump

Project Member Reported by akes...@chromium.org, Aug 14 2017

Issue description

Context: https://chromium-review.googlesource.com/c/526379/16/cbuildbot/builders/simple_builders.py#102 introduces what I consider to be a "run time re-writing of build config" in which cbuildbot logic runs through some non-trivial computation to map a config's boards and models to a set of HWTest stages that will be run.

This makes config changes difficult to reason about, and leads to things like hard-to-trace collisison between multiple CQ builders using the same pool (context https://chromium-review.googlesource.com/c/614568 ).


Preferred option:
 - There should be a 1 to 1 correspondence between entries in a config['hw_tests'] list and HWTest stages that are created, and each entry there should fully specify what the HWTest args will be.

Backup option:
 - If ^ not feasible, then at least the config -> set of HWTest boards logic should be factored out to config_lib, so that the logic can be shared.


Either option would make unittest coverage as in Issue 755357 easier.
 
Blocking: 755357
Cc: shapiroc@chromium.org
 Issue 755353  has been merged into this issue.
models is already factored into config_dump (option 2):

https://chromium-review.googlesource.com/c/611007/5/cbuildbot/config_dump.json

it's only used for unified builds though; the logic simply says if models are in the config, use them, else use the board (as previous).  This supports both uni and non-uni configs.

the design is at: go/cros-unibuild-hw-test

can you make comments there and I'll set up some time to review concerns






The ddoc does not seem like the right place for discussion of this implementation detail.

models is separated into it's own section of a chromeos_config entry, but the logic for mapping that and determining which hwtest stages will be created is at cbuildbot run time. That is what I don't like, because it makes it difficult to write unit tests about chromeos_config, or to reason about expected behavior simply from looking at the config. Ideally, I'd like a 1:1 mapping from hw_tests entries to actual hwtest stages.


To put it another way, I do not believe the following code should run at cbuildbot run time (simple_builders.py):

    models = [config_lib.ModelTestConfig(board)]

    if builder_run.config.models:
      models = builder_run.config.models

    for suite_config in builder_run.config.hw_tests:
      # Even for blocking stages, all models can still be run in parallel since
      # it will still block the next stage from executing.
      for model in models:
        new_stage = self._GetHWTestStage(
            builder_run, board, model, suite_config)
        if new_stage:
          parallel_stages.append(new_stage)


Instead, it should be something like:

    for suite_config in builder_run.config.hw_tests:
        new_stage = self._GetHWTestStage(
            builder_run, suite_config.board, suite_config.model, suite_config)
        if new_stage:
          parallel_stages.append(new_stage)
Labels: -Pri-1 Pri-3
this will be addressed by a longer term task of decoupling the test plan from cbuildbot
Status: WontFix (was: Assigned)

Sign in to add a comment