Telemetry regressions should be marked with Release-Block label. |
|||||
Issue descriptionCurrently telemetry regression bugs have a milestone attached to them but the bug doesn't include a Release-Block label associated with it. This can result in the regression not getting resolved before the version is released, or the fix not getting merged to the release branch. The current consensus is that we have too many telemetry regressions (up to 90 a week!) which makes it infeasible to mark them all as release blockers. But having a threshold above which the label is automatically added is a good idea. Should this threshold be a per-benchmark thing, that the benchmark owners should add when setting it?
,
Oct 4
FYI: So the milestone is automatically attached here: https://cs.chromium.org/chromium/src/third_party/catapult/dashboard/dashboard/file_bug.py?q=file_bug&sq=package:chromium&g=0&l=133 A general implementation of a Release-block label + threshold would likely want to follow the way that this info flows up to the dashboard, and get declared somewhere in the client code via a diagnostic. Ie. here's an example of where owners/components are declared: https://cs.chromium.org/chromium/src/tools/perf/benchmarks/media.py?q=media.py&sq=package:chromium&g=0&l=65. Would take some time since it has to be designed out, added to telemetry, supported by the dashboard properly, etc. A hardcoded "works for some whitelisted set of metrics/benchmarks on ChromiumPerf" version could live alongside the milestone code in file_bug and could probably be done in a few hours.
,
Oct 4
Re #2: khushalsagar suggested in the postmortem doc that pinpoint add the Release-Block label after confirming the regression, which has a lot of benefits: -Can confirm reproducible magnitude -Have a clear owner to follow up with.
,
Jan 11
Setting defect without priority to Pri-2.
,
Jan 16
(6 days ago)
,
Jan 16
(6 days ago)
|
|||||
►
Sign in to add a comment |
|||||
Comment 1 by gov...@chromium.org
, Oct 4