[Umbrella Bug] New flake detection |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Issue description1. Understand Chromium-Try-Flakes: * P0 how it detects flaky tests in CQ: https://cs.chromium.org/chromium/infra/appengine/chromium_try_flakes/status/cq_status.py?q=fetch_cq_status * P1 how it files/updates Monorail bugs of flaky tests * Ask stgao@ for docs about CTF * The task here is to make CTF report detected flakes to Findit even after CTF hits the limit of bug updates 2. Review the new BQ query to detect flaky tests: https://chromium-review.googlesource.com/c/infra/infra/+/1031017 * Current query is https://cs.chromium.org/chromium/infra/appengine/findit/services/flake_detection/detect_flakes.py?q=_CQ_FLAKES_QUERY * The task here is to define the output schema of the query that are needed to record flake occurrences, find flaky tests, file/update bugs, and show info on UI/bugs, etc. 3. Understand the data model of the new flake detection on Findit side: https://cs.chromium.org/chromium/infra/appengine/findit/cron.yaml?q=detect-cq-flakes * The task here is to refine the data model to support multiple projects. Besides Chromium, WebRTC team has feature request to support their cases as well. Later we might want to support v8 too. * A design doc might be appropriate here. 4. The end goal of this is to have a better & standalone flake detection as a appengine module/service within Findit, and then deprecate CTF. ⛆ |
|
|
,
May 5 2018
,
May 8 2018
,
May 8 2018
,
May 8 2018
,
May 8 2018
,
May 9 2018
,
May 9 2018
,
May 18 2018
,
May 18 2018
,
May 18 2018
,
May 18 2018
,
May 18 2018
,
May 19 2018
,
May 22 2018
,
May 23 2018
,
May 23 2018
,
May 24 2018
,
May 24 2018
,
May 25 2018
,
May 29 2018
,
Jun 1 2018
,
Jun 5 2018
,
Jun 5 2018
,
Jun 5 2018
,
Jun 7 2018
,
Jun 7 2018
,
Jun 14 2018
,
Jun 21 2018
,
Jun 26 2018
,
Jun 26 2018
,
Jun 28 2018
,
Jul 17
,
Jul 17
,
Jul 17
,
Jul 17
,
Aug 2
,
Aug 2
,
Aug 3
,
Aug 3
,
Aug 3
,
Aug 7
,
Aug 7
,
Aug 9
,
Aug 10
,
Aug 10
,
Aug 13
,
Aug 13
,
Aug 13
,
Aug 14
,
Aug 16
,
Aug 20
,
Aug 20
,
Aug 22
,
Aug 22
,
Aug 24
,
Aug 27
,
Aug 29
,
Aug 31
,
Aug 31
,
Sep 13
,
Dec 22
is this done now, since CTF is gone?
,
Dec 27
This is done from the perspective of replacing CTF. But there are a few more things we want to add or improve. Reassigned to chanli@ to triage the blocking bugs before closing this bug. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Comment 1 by liaoyuke@chromium.org
, May 2 2018