SoM shows more failures than actual |
|||
Issue description
,
Mar 8 2018
It might be counting all the expected failures whenever there are unexpected failures too.
,
Mar 8 2018
But this is inconsistent with what Milo shows as "Unexpected Failures".
,
Mar 9 2018
I think when analyzer/step/test_step.go changed to use the test-results/model structs for parsing test results, we stopped checking "is_unexpected" (reasons escape me) and just check for PASS. This almost surely is a bug.
,
Mar 9 2018
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/c5d09780c744070a529662b9129a4cceb5113457 commit c5d09780c744070a529662b9129a4cceb5113457 Author: Sean McCullough <seanmccullough@chromium.org> Date: Fri Mar 09 17:35:26 2018 [som] Fix unexpected test failure identification Bug: 819853 Change-Id: Icdc5ac8e36acf2ebc256efe93bbe177b1ef3ef6e Reviewed-on: https://chromium-review.googlesource.com/956967 Commit-Queue: Sean McCullough <seanmccullough@chromium.org> Reviewed-by: Shuotao Gao <stgao@chromium.org> [modify] https://crrev.com/c5d09780c744070a529662b9129a4cceb5113457/go/src/infra/appengine/sheriff-o-matic/som/analyzer/step/step.infra_testing [modify] https://crrev.com/c5d09780c744070a529662b9129a4cceb5113457/go/src/infra/appengine/sheriff-o-matic/som/analyzer/step/test_step_test.go [modify] https://crrev.com/c5d09780c744070a529662b9129a4cceb5113457/go/src/infra/appengine/sheriff-o-matic/som/analyzer/step/test_step.go
,
Mar 15 2018
|
|||
►
Sign in to add a comment |
|||
Comment 1 by st...@chromium.org
, Mar 8 2018