New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 912956 link

Starred by 1 user
Status: Verified
Merged: issue 911258
Owner:
chanli@chromium.org
Closed: Today
Cc:
chanli@chromium.org
liaoyuke@chromium.org
dpranke@chromium.org
st...@chromium.org
█ robert...@chromium.org
lijeffrey@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug

Blocking:
issue 911258



Sign in to add a comment

Feature request: metric for total # of flaky tests.

Project Member Reported by erikc...@chromium.org, Dec 7 
Apologies in advance if this metric already exists and I'm just not aware of it.

The CATS team has done a huge amount of work for analyzing, and building tools to directly and indirectly reduce the # of flaky tests in our code base. I think that we should have a metric that measures [total # of unique flaky tests in a given week] so that we can track these improvements over time.

If it isn't already, I think that this should be a top-level Ops metric.

Comment 1 by chanli@chromium.org, Dec 7 

Thank you for sharing your idea with us. 

I'm developing a new flake report UI to show information about flakes (# flaky test, # bugs, # occurrence, # impacted CLs) overall and also within each component.

My changes for this report are in review, so we'll have this report really soon.

Comment 2 by chanli@chromium.org, Dec 7

Components: Tools>Test>FindIt>Flakiness
Mergedinto: 911258
Status: Duplicate (was: Untriaged)

Comment 3 by dpranke@chromium.org, Dec 7 

Doesn't merging this into 911258 (instead of making 9122258 blocked on this) lose the feature request? Maybe we should at least copy the text over ...

Comment 4 by st...@chromium.org, Dec 7 

+1 for Drik's suggestion above

Comment 5 by st...@chromium.org, Dec 7 

Many thanks Erik for the feature request! Chan is almost done with the flake reporting feature and can show you some screenshot for further feedback.

Comment 6 by chanli@chromium.org, Dec 7

Blocking: 911258
Owner: chanli@chromium.org
Status: Assigned (was: Duplicate)
Project Member

Comment 7 by bugdroid1@chromium.org, Dec 17 

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/7c2ab11527641e8af6c498789e18e01ab1789fad

commit 7c2ab11527641e8af6c498789e18e01ab1789fad
Author: Chan <chanli@chromium.org>
Date: Mon Dec 17 21:07:12 2018

[Findit] Use component page to show reports for all components.

Bug:  912956 
Change-Id: I1d630095bb43b68e199dbca50be1eddcb9c113e3
Reviewed-on: https://chromium-review.googlesource.com/c/1370057
Commit-Queue: Chan Li <chanli@chromium.org>
Reviewed-by: Yuke Liao <liaoyuke@chromium.org>
Cr-Commit-Position: refs/heads/master@{#19611}
[modify] https://crrev.com/7c2ab11527641e8af6c498789e18e01ab1789fad/appengine/findit/handlers/flake/reporting/component_report.py
[modify] https://crrev.com/7c2ab11527641e8af6c498789e18e01ab1789fad/appengine/findit/model/flake/reporting/report.py
[modify] https://crrev.com/7c2ab11527641e8af6c498789e18e01ab1789fad/appengine/findit/templates/flake/report/component_report.html
[modify] https://crrev.com/7c2ab11527641e8af6c498789e18e01ab1789fad/appengine/findit/handlers/flake/reporting/test/component_report_test.py
[modify] https://crrev.com/7c2ab11527641e8af6c498789e18e01ab1789fad/appengine/findit/model/flake/reporting/test/report_test.py
[modify] https://crrev.com/7c2ab11527641e8af6c498789e18e01ab1789fad/appengine/findit/ui/elements/flake_report/flake_report.html
[modify] https://crrev.com/7c2ab11527641e8af6c498789e18e01ab1789fad/appengine/findit/ui/elements/flake_report/component_report.html

Comment 8 by chanli@chromium.org, Today (9 hours ago)

Status: Verified (was: Assigned)
►

Sign in to add a comment