Many tests have flakiness 1.76% and 2.83% - investigate whether this makes sense |
||||||
Issue description
,
Dec 2 2016
,
Dec 5 2016
Turns out the approximation formula for true flakiness is wrong. After some thinking I think we should display raw data to the user with an explanation that we omit some flaky failures as we filter out runs where all tries fail.
,
Dec 12 2016
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra.git/+/decae1da1d563e156417de15cbee336f37e09cb3 commit decae1da1d563e156417de15cbee336f37e09cb3 Author: Sergiy Byelozyorov <sergiyb@chromium.org> Date: Mon Dec 12 14:31:20 2016 Use total tries and flaky failures in addition to flakiness percentage BUG= 670329 Change-Id: Ia2a7555db5c0772ed11b77aba23be4c7a076f58e Reviewed-on: https://chromium-review.googlesource.com/416860 Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org> [modify] https://crrev.com/decae1da1d563e156417de15cbee336f37e09cb3/go/src/infra/appengine/test-results/frontend/bower.json [modify] https://crrev.com/decae1da1d563e156417de15cbee336f37e09cb3/go/src/infra/appengine/test-results/frontend/src/test-results-flakiness.html [modify] https://crrev.com/decae1da1d563e156417de15cbee336f37e09cb3/go/src/infra/appengine/test-results/frontend/testflakiness.go [modify] https://crrev.com/decae1da1d563e156417de15cbee336f37e09cb3/go/src/infra/appengine/test-results/frontend/testflakiness_test.go
,
Dec 12 2016
CL correcting computation for flakiness: http://cl/141753209. CL adding fields for total flakes and total tries: http://cl/141563478. CL showing total flakes and tries in the Flakiness Surface page: https://chromium-review.googlesource.com/c/416860/. All CLs have been landed and deployed. Closing as Fixed. Please feel free to re-open if you'd like me to change information shown on the page.
,
Dec 16 2016
I'm going to re-open this as I'm adding total flaky-failures and skipped-tests to the chromiumdash page.
,
Dec 16 2016
We'll also need to add another table to compute data for row with "all tests", but perhaps we need to rethink the whole structure of tables to make it faster and more flexible.
,
Dec 16 2016
,
Dec 17 2016
Update: For some reason deployment to chromiumdash AE app didn't work. Also we still need to find a solution for #7.
,
Dec 17 2016
,
Jan 11 2017
> We'll also need to add another table to compute data for row with "all tests", but perhaps we need to rethink the whole structure of tables to make it faster and more flexible. I have found a relatively easy to way to achieve this in the current table configuration, but we may still need to revisit this when created in-line graphs as part of the issue 674078.
,
Jan 11 2017
Latest version of chromiumdash now shows raw values as well. |
||||||
►
Sign in to add a comment |
||||||
Comment 1 by serg...@chromium.org
, Dec 1 2016