Example annotator page:

https://chromiumos-build-annotator.googleplex.com/build_annotations/edit_annotations/master-paladin/2675172/?

The UI is somewhat confusing in the way that finalization is handled.  The UI state machine allows the user to make changes at will, both before and after they have been marked as finalized.  The UI does not really reflect the impact that finalization has on the system.

However, once they have been marked finalized there is a race between the human's further changes and the exonerator bot.

Possible corrective actions:
- In the "finalized" state, remove the 'edit' option in the Update column on any entries that were present when the annotation was finalized.

- Reword the "Finalize annotations?" and "(Annotations already finalized)" control/indicator to "Release to Exonerator Bot?" and "Released to Exonerator Bot".  This would emphasize the system impact rather than the internal names of the state machine.

- In the finalized state, provide a short warning that further changes will not be picked up by the exonerator bot, but may still be useful for diagnostics and metrics.


A sample discussion from the #crosoncall room, demonstrating user confusion.   Usernames were removed to protect the innocent :)

(04:17:42 PM) <A> B: thank you! that was me...I didn't see the "finalize changes" (or didn't understand). "Save changes" seemed like it should do that
(04:19:25 PM) <C> I agree, we should probably do that
(04:19:52 PM) <C> the use case of saving intermediate annotations and then coming back and finalizing them is not worth the UI confusion
(04:25:03 PM) <B> Yeah, I guess the initial design is to give you some flexibility if you're not sure what you annotate is ready for exonerator :)
(04:28:33 PM) <A> well, we can "edit" and "save" again, right?
(04:28:46 PM) <A> so I don't understand "finalize"
(04:29:51 PM) <D> How quickly does the exonerator consume the annotation?
(04:33:29 PM) <D> Looks like this time the bot marked the non-blamed CL's as of 4:03, so fairly quickly after the annotation was finalized.  If it doesn't undo its exoneration on a change, then that explains the difference between a saved draft and a finalized annotation.
(04:34:09 PM) <D> 4:03 mountain time, which was only a few min after B reported the finalization
(04:35:19 PM) <C> After you've finalized further changes will not alert the bot
(04:35:38 PM) <C> they are final
(04:36:23 PM) <C> it could probably be changed to allow updates to be informative to the bot but I expect it would be a significant undertaking
(04:44:07 PM) <B> Yeah, the exonerator kicks off exonerations every 5 minutes, and AFAIK it won't do exonerations twice for the same round of CQ. So if 'save' again, it won't be  accepted again.