flaky tests are hard to distinguish from passed tests due to their similar colors |
|||||
Issue description
,
Dec 14 2016
We don't have any other primary colors left, so all we can do is play with various shades of cyan. We should still keep distinction between flaky (light cyan) and very-flaky tests (dark cyan). How about #03b8bb for the very-flaky? Screenshot: https://screenshot.googleplex.com/Wrmn3ieuRuZ.
,
Dec 14 2016
,
Dec 14 2016
Isn't flaky a "problem"? Why that is not orange? Or red? Or purple? Or Black. But plz not green vs cyan for good vs flaky tests :/
,
Dec 14 2016
In principle these tests are just flaky - they did not cause a failure, so I think cyan/green is a good idea to keep them somewhat similar. Orange is either too similar to yellow (running) or to red (failed) depending on its intensity, purple is usually reserved for infra-related failures, black looks too spooky :-). We should probably involve UX designer here. CCing some UX designed.
,
Dec 14 2016
There are also things you can do besides colors to distinguish things (like add crosshatching patterns, text indicators, etc.).
,
Dec 14 2016
Actually I can't add Munich UX people since they only have an internal alias. I wrote an email to their group: https://goto.google.com/ux-color-pick-very-flaky (sorry, internal only).
,
Dec 14 2016
Flaky tests are arguably worse than failing tests. Black sounds good for very flaky and dark grey for flaky. If UX has opinions we can hear them out but black will solve the problem. We also reduced the number of colors recently so we should have some to spare if we didn't want to use black.
,
Dec 15 2016
UX team suggested to use Material Design colors. I've picked the darkest available color there: #212121. The text color is #fff. CL: https://chromium-review.googlesource.com/c/420605/. The other concern is that our colorful palette is not working very well for colorblind people. So we may want to review all colors, not just very-flaky. Filed issue 674506.
,
Dec 15 2016
,
Dec 15 2016
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra.git/+/63e492d7ed2d550fc09c0b83d1d0b77e8db3d09e commit 63e492d7ed2d550fc09c0b83d1d0b77e8db3d09e Author: Sergiy Byelozyorov <sergiyb@chromium.org> Date: Thu Dec 15 16:26:24 2016 Change very-flaky boxes to dark-gray color with white text BUG= 674187 Change-Id: I9b73134d6e5e93d768de22166ee011dc1273e513 Reviewed-on: https://chromium-review.googlesource.com/420605 Reviewed-by: Erik Staab <estaab@chromium.org> Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org> [modify] https://crrev.com/63e492d7ed2d550fc09c0b83d1d0b77e8db3d09e/go/src/infra/appengine/test-results/frontend/static/dashboards/css/flakiness_dashboard_tests.css
,
Dec 15 2016
|
|||||
►
Sign in to add a comment |
|||||
Comment 1 by dpranke@chromium.org
, Dec 14 2016Owner: estaab@chromium.org
Status: Assigned (was: Untriaged)