master.chromium.perf.fyi console is down |
||||
Issue descriptionhttps://build.chromium.org/p/chromium.perf.fyi/console responds with HTTP 500. From logs: exceptions.KeyError: u'14a3cda4337f498d01ccbf897884f343ee099805' root cause: buggy gitiles poller trigger: ? related issue: issue 626019 I've verified that the fix in that bug works correctly, so the trigger here is different
,
Aug 3 2016
the trigger is that chromium/src commit rate surpassed current change horizon configuration value (3000). We had 3967 commits since https://crrev.com/14a3cda4337f498d01ccbf897884f343ee099805 I will double the change horizon and re-populate changes table
,
Aug 3 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/build.git/+/78b745cdd539c16257999ed7a08b70b5dc91a5a4 commit 78b745cdd539c16257999ed7a08b70b5dc91a5a4 Author: nodir <nodir@chromium.org> Date: Wed Aug 03 16:41:28 2016 chromium.perf.fyi: double changeHorizon changehorizon is max number of changes that we store in the buildbot's "changes" table. This number is not enough for chromium/src commit rate as of today, at least for chromium.perf.fyi. Space is cheap. R=vadimsh@chromium.org BUG= 634014 Review-Url: https://codereview.chromium.org/2208433004 [modify] https://crrev.com/78b745cdd539c16257999ed7a08b70b5dc91a5a4/masters/master.chromium.perf.fyi/master.cfg
,
Aug 3 2016
The following revision refers to this bug: https://chrome-internal.googlesource.com/infradata/master-manager.git/+/e663e5bd4130f861a67d149f562260d014428db8 commit e663e5bd4130f861a67d149f562260d014428db8 Author: nodir <nodir@google.com> Date: Wed Aug 03 16:43:57 2016
,
Aug 3 2016
i have to manually restart master (without master manager) because I need to stop it, then modify .gitiles_poller_cursor, then start it
,
Aug 3 2016
,
Aug 3 2016
hm, no it does not
,
Aug 3 2016
it works now, but I will keep this bug open JIC the commits are being populated right now chromiumperffyi=> select count(*) from changes ; count ------- 3787 (1 row) chromiumperffyi=> select count(*) from changes ; count ------- 3920 (1 row)
,
Aug 3 2016
now I see duplicate revisions in the changes table. Looks like population did not work well.
,
Aug 3 2016
I've deleted rows from tables scheduler_changes, change_properties, sourcestamp_changes and changes and gitiles poller refilling the tables back right now starting from ae59b40c3055d9f5ef53fc0237bd236e4761833e which is ~5000 commits back
,
Aug 3 2016
number of rows in the changes table reached 5058, oldest being f67cb8db57391f982744179442638bfbd5595320 which was on July 08 |
||||
►
Sign in to add a comment |
||||
Comment 1 by no...@chromium.org
, Aug 3 2016proof that buildbucket does not create duplicate changes table rows: chromiumperffyi=> select * from (select revision, count(*) as c from changes group by revision) as c order by c.c desc limit 10; revision | c ------------------------------------------+--- 0171b362d74772f1a57a91f9fd4761569e30fa91 | 1 eea862d56d10b29a347d49bac268af0bf9f57d31 | 1 e00674ac2c822e76c19f38bc6b7c79a0d8d980d2 | 1 93c60cd3ef5908984e173112dc09345bb5714036 | 1 e82f23b3adc13c37652719585b0ddf62a514e07f | 1 138740d5a899e146fb2c3c7d5185f924ff3e8dba | 1 63942671b93caa41959910a47bc834c5c68df15c | 1 bc913a00924cc5e9072293b5d7c19cf1cc536b62 | 1 2fe4e328552fe9e4f38b66be416c6be5d113113b | 1 deb84849f66aa9db12a4c4bd8eb0bb3e18fd441d | 1 (10 rows) chromiumperffyi=> select count(*) from changes where revision = '' or revision is null; count ------- 0 (1 row)