FindIt's messages are lacking context |
|||
Issue descriptionFor example, this message on CL == https://chromium-review.googlesource.com/c/chromium/src/+/1146700 : """ Findit (https://goo.gl/kROfz5) identified this CL at revision 583220 as the culprit for introducing flakiness in the tests as shown on: https://findit-for-me.appspot.com/waterfall/flake/flake-culprit?key=ag9zfmZpbmRpdC1mb3ItbWVyQwsSDEZsYWtlQ3VscHJpdCIxY2hyb21pdW0vNzgyYzBlMzE1OTFkNTVhMTNhYTNkNDg4MzU3YmY3YmRiZTE3NGI0Ngw If the results are correct, please either revert this CL, disable, or fix the flaky test. """ While it's nice that this spotted something, this message lacks context, it should textually mention : * test name * associated bug (the analysis page has this -- two layers down for the associated bug -- but having it in the text would be much better)
,
Sep 11
Each flaky test triggers an analysis independently of the other tests, so it's possible a single culprit is responsible for multiple flaky tests thus multiple bugs. Given that we only notify a culprit once, (whichever is first to complete with high confidence), only 1 bug with 1 test would be updated. If we're ok with this we can proceed with the first test/bug. Another way to do it would be to have each analysis notify the culprit, but this can get spammy. gab@, in your view, would it be sufficient for you if multiple tests are flaky to have just 1 representative bug/test name be surfaced?
,
Sep 11
Not sure I understand but one is surely better than none Le lun. 10 sept. 2018 20 h 18, lijeffrey via monorail < monorail+v2.1545619054@chromium.org> a écrit :
,
Sep 19
,
Oct 5
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/c878b07471cb63171455cb418cdb1a6ed90c6528 commit c878b07471cb63171455cb418cdb1a6ed90c6528 Author: Jeffrey Li <lijeffrey@chromium.org> Date: Fri Oct 05 00:11:05 2018 [Findit] Flake Analyzer - Include bug id when notifying culprits Developers shouldn't have to go digging through links to get to the actionable parts of their work when it comes to fixing flakiness. This change is to fulfill a feature request to include a link to the bug in the message notification. Eventually, all bug-related operations will live in a separate layer from flake analysis, but this API shouldn't need to change much if at all when that time comes. Bug: 874999 Change-Id: Ie2df2679681a090f2e2deb3ae72690a95a3ee0a5 Reviewed-on: https://chromium-review.googlesource.com/c/1262918 Commit-Queue: Jeffrey Li <lijeffrey@chromium.org> Reviewed-by: Yuke Liao <liaoyuke@chromium.org> Cr-Commit-Position: refs/heads/master@{#18078} [modify] https://crrev.com/c878b07471cb63171455cb418cdb1a6ed90c6528/appengine/findit/services/flake_failure/culprit_util.py [modify] https://crrev.com/c878b07471cb63171455cb418cdb1a6ed90c6528/appengine/findit/pipelines/flake_failure/notify_culprit_pipeline.py [modify] https://crrev.com/c878b07471cb63171455cb418cdb1a6ed90c6528/appengine/findit/services/flake_failure/test/culprit_util_test.py
,
Dec 27
|
|||
►
Sign in to add a comment |
|||
Comment 1 by st...@chromium.org
, Sep 7Owner: lijeffrey@chromium.org
Status: Assigned (was: Untriaged)