Should not do without patch step if test failure in infra failure |
||
Issue description
,
Jul 31
Sorry for no explanation. I think without patch step is used not to prevent patch landing from deterministic failure or flaky test with high probability of failure. That sounds reasonable to me. But for the infra failure, I think it does not have strong reason to do without patch step. If we see infra failure in with patch step, it is better to block patch landing even we see infra failure in without patch step. So if there is infra failure, I think it is better to do retrying CQ without taking times for without patch steps. > Why not? It's possible that a change can cause a test to fail as "infra failure". When that happens, the signal gained from rerunning it w/o patch would be useful for the author. Such a thing can be confirmed by seeing build steps of other changes.
,
Aug 1
> If we see infra failure in with patch step, it is better to block patch landing even we see infra failure in without patch step. Do we though? I'm still having a hard time seeing *why* we wouldn't want to. That said, I don't feel strongly either way, so I'll let the higher-ups determine if this is a change we want to pursue.
,
Aug 2
Some classes of infra failures (e.g., swarming down) are probably just show-stoppers. Some classes of infra failure are only partial, but they mean you can't trust the results. In the linked CL, an entire shards' worth of "with patch" test results are missing, and so we have no idea how to compare that to the "w/o patch" results. It's possible there are infra failures that are neither of those two things (e.g., uploading to test-results), but they are probably the rarest of the three. |
||
►
Sign in to add a comment |
||
Comment 1 by bpastene@chromium.org
, Jul 31