Auditing on chromium/src was stuck on a commit landed during Gerrit outage |
|||
Issue descriptionThe last relevant commit was 5 days ago https://cr-audit-commits.appspot.com/view/status And it was stuck at https://chromium.googlesource.com/chromium/src/+/a944b0d7 which was landed during Gerrit outage and got purged because chromium/src master branch was reset to very last commit before the Gerrit outage. Two actions needed here: 1) Reset the checking point of audit-app to the last "good" revision. One thing to be careful here is a burst of commits to audit. 2) Set up an alert for auditing on chromium/src master branch (and maybe master branches of other audited repos): if there is a large gap between last scanned commit and the current scanning time, fire an alert.
,
Aug 14
Thanks for bringing this to my attention.
,
Aug 14
Ah, sorry for not PSA-ing you earlier. I forgot to do this on Thursday :(
,
Aug 15
The plan is as follows: Manually set the status of these 3 'RelevantCommit' entities to the new auditSkipped (i.e. 4) see this cl: https://chromium-review.googlesource.com/c/infra/infra/+/1174987 These are the commits: [hi on] (master<):robertocn@traal:~/chromium/src$ git show a944b0d7f322400a54def81b313f16b9e125167b fatal: bad object a944b0d7f322400a54def81b313f16b9e125167b [hi on] (master<):robertocn@traal:~/chromium/src$ git show 224eb4663f904ffc0c1356fe5f1d83fc4afc2c90 fatal: bad object 224eb4663f904ffc0c1356fe5f1d83fc4afc2c90 [hi on] (master<):robertocn@traal:~/chromium/src$ git show ec978fdcd324d32e1dd1ccec03900084652ebe26 fatal: bad object ec978fdcd324d32e1dd1ccec03900084652ebe26 Next, manually edit the RepoState for chromium/src to set _both_ LastKnownCommit and LastRelevantCommit to 86f8273e2fd5cd87e70f3711250d6edc12da2823 , which is the last commit audited that had a commit position before the outage began.
,
Aug 15
As discussed on the code review mentioned in #4, we'll instead delete the entries that no longer exist.
,
Aug 15
This is fixed. BTW the audit app didn't even blink when catching up. It took it about 9 seconds to scan ~1184 commits and audit 89 of them. I will file a separate bug to add an alert that will monitor the http statuses of these tasks. |
|||
►
Sign in to add a comment |
|||
Comment 1 by st...@chromium.org
, Aug 14