We could go with 1 for now, for 2 we need to figure out a reasonable distance between the two commits.
For 3, it is unclear to me how to improve. Brandon, any thought here?

After some further investigation into these two tests, the confidence score of 100% is being set due to confidence.SteppinessForCommitPosition indeed identifying them as 100%.

Since there are so few data points, and they are a string of "100%" passing with a sudden dip and no subsequent points after that, this is actually a corner case where steppiness indeed comes back with a 1.0 confidence score.

Seeing as this is indeed the behavior (correctly passing the data points to the steppiness algorithm), this is not a bug, but rather an improvement needed to be made to setting confidence scores.

For now there is no action needed since this technically isn't a bug as the behavior was correct (check if the suspected data point added a new test, which this correctly saw that it was not the commit that added the test and fell back to using confidence.SteppinessForCommitPosition as expected). Rather, this bug should be repurposed to make steppiness smarter.

Removing the metabug for the september 29 release.