We also break by test name when it's available (i.e. when test launcher has uploaded a valid test results JSON file to test-results.appspot.com) and file separate bugs for each test. For flakes like that we include a link to the Flakiness Dashboard that breaks flakes by platforms. We do not break by failure type yet, but that should be a separate bug filed against Infra>Flakiness>Dashboard component.

Unfortunately it is true that we do not do as good job for steps. Right we mostly focus on improving the tools detecting where most test flakiness is coming from (Flakiness Surface project), but perhaps we can do better in the future. Thanks for filing this bug and if you are willing to contribute to Flakiness Pipeline, please let me know - I'll be glad to give you some pointers on where to start.

I've learned that some tests, e.g. V8 and GPU tests, are running same tests on different platforms or using different configs and we need additional way of distinguishing these runs from each other. Using step name is usually insufficient since we normalize step name by dropping everything after first space except (with patch) suffix.

 Issue 671874  has been merged into this issue.

I wrote a proposal that will help to address this issue: https://docs.google.com/document/d/1aCf915IjQX4DlOU8kiP51ORFIJGzCaAuacILWU9A1fA (sorry, internal only). I am not planning to work on this immediately since I still focus my work on Flakiness Surface, but I've decided to document my thoughts in this doc before I forget them or if someone else decides to work on this issue.

This issue has been Available for over a year. If it's no longer important or seems unlikely to be fixed, please consider closing it out. If it is important, please re-triage the issue.

Sorry for the inconvenience if the bug really should have been left as Available.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot