How about moving average of 7-days(builds), 6-weeks&/milestone-average, etc.

For 1)

The current default metric is set to compare the most recent two milestones. This would accidentally hide the regressions happened two milestones away. I think we can improve in two directions:

1. Regression management

Now we can't easily tell which regressions we observed on Crosbolt is taken care of or not. Sometimes we lose track of regressions when it past two milestones away.

As Mohan raised b/34099947, I agree we need better regression management. My first thought is to establish a table that maps (test name, flipping point) to (bug id). Bugs could be filed manually or automatically by some alert mechanism. We could know which regressions are being taken care of and which are not on Crosbolt. And also we could manage or merge existing bugs in some control panel. 

This needs adding a control panel to manage and track the bugs, UI change on Crosbolt for showing tracking bugs and adding some link or button for filing bugs on Crosbolt. I can draft a design for this later.

2. Clarify the metrics

We could change the comparison base to the 52-week best value instead of the average value of some other milestone. But sometimes the regression is expected (for example, a recent V8 change that optimized according to real world workloads would lower the score of some old benchmarks) then it would need 52-weeks to disappear. Need to figure out if such cases happens regularly.

It's not easy to tell which baseline value to use. I would say the regression management is more important here. If we won't lost track of regressions than this would not be a problem. It could be a short term workaround, though.


For 2)

I'm afraid too many colors would reduce readability. It's easy for us, the most frequent users of Crosbolt, to tell the meaning of new colors but maybe not for all of the users. Now there are five color combinations are used:

[normal] background white, text black
[warning] background yellow, text black
[problem] background red, text white
[good] background light green, text black
[great] background dark green, text white

Do we want to tweak the colors of the background or the text ? How about adding a symbol or a colored exclamation mark next to the numbers ?