Revive db_cleanup |
|||||
Issue descriptionLet's resurrect the db_cleanup script. Fang commented on http://crosreview.com/c/329250 that it may be possible to revive it. I imagine we could just try reducing the db_cleanup script's batch size and throttling its QPS to make sure we don't overload the master db. Chase-Pending & Pri-1 because it prevents P0 outages due to db slowness.
,
Sep 11 2017
,
Sep 12 2017
What does the db_cleanup script removed ? What impact will this have from GE perspective ?
,
Sep 12 2017
GE will no longer be able to get test results (only test results, does not affect build status / build logs / or even test logs) older than 3 months. This is not a new proposal. iirc, we already have a stated test horizon of 3 months. This is just re-instating cleanup that we were not doing for other reasons.
,
Sep 12 2017
Re #3, #4 GE is not affected. The script is trying to cleanup master scheduler database, not TKO database. GE gets data from Sponge, so it's not affected.
,
Sep 12 2017
,
Sep 13 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/4d11bc3a925dc6204c14142a4a862c645d1f7f31 commit 4d11bc3a925dc6204c14142a4a862c645d1f7f31 Author: Paul Hobbs <phobbs@google.com> Date: Wed Sep 13 22:12:01 2017 [autotest] db_cleanup: Add a "dry_run" flag Add a dry_run option to db_cleanup for sanity. BUG= chromium:762070 TEST=None Change-Id: I4e073fa98e6904a6c8e50d80279949c536d2e56f Reviewed-on: https://chromium-review.googlesource.com/657239 Commit-Ready: Paul Hobbs <phobbs@google.com> Tested-by: Paul Hobbs <phobbs@google.com> Reviewed-by: Paul Hobbs <phobbs@google.com> Reviewed-by: Dan Shi <dshi@google.com> [modify] https://crrev.com/4d11bc3a925dc6204c14142a4a862c645d1f7f31/contrib/db_cleanup.py
,
Sep 13 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/9d6c6dce76d8491b2eb3623521e5b4e7737dbb5e commit 9d6c6dce76d8491b2eb3623521e5b4e7737dbb5e Author: Paul Hobbs <phobbs@google.com> Date: Wed Sep 13 22:12:02 2017 [autotest] db_cleanup: Rate limit deletions Add a rate limiting flag to db_cleanup. This makes it safer to run on production databases, because we don't want to overload the master database. BUG= chromium:762070 TEST=None Change-Id: Ie9ed29c0a3ec11e9eabb64e32b88937db4b252f7 Reviewed-on: https://chromium-review.googlesource.com/657240 Commit-Ready: Paul Hobbs <phobbs@google.com> Tested-by: Paul Hobbs <phobbs@google.com> Reviewed-by: Paul Hobbs <phobbs@google.com> Reviewed-by: Dan Shi <dshi@google.com> [modify] https://crrev.com/9d6c6dce76d8491b2eb3623521e5b4e7737dbb5e/contrib/db_cleanup.py
,
Sep 13 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/45c53424e391613b8995993dc4baea04f4d49444 commit 45c53424e391613b8995993dc4baea04f4d49444 Author: Paul Hobbs <phobbs@google.com> Date: Wed Sep 13 22:12:02 2017 [autotest] db_cleanup: Enable on master Currently, db_cleanup exits if it is not running on a shard. Get rid of that logic. BUG= chromium:762070 TEST=None Change-Id: I86c02ea34445e858c70d2f31b321d46e597eb2bf Reviewed-on: https://chromium-review.googlesource.com/657241 Commit-Ready: Paul Hobbs <phobbs@google.com> Tested-by: Paul Hobbs <phobbs@google.com> Reviewed-by: Paul Hobbs <phobbs@google.com> Reviewed-by: Dan Shi <dshi@google.com> [modify] https://crrev.com/45c53424e391613b8995993dc4baea04f4d49444/contrib/db_cleanup.py
,
Sep 15 2017
,
Sep 25 2017
,
Feb 12 2018
The following revision refers to this bug: https://chrome-internal.googlesource.com/chromeos/chromeos-admin/+/60b0c9e25addbb8d73dab2e52cde849124318adc commit 60b0c9e25addbb8d73dab2e52cde849124318adc Author: Paul Hobbs <phobbs@google.com> Date: Mon Feb 12 18:20:11 2018
,
Feb 12 2018
The following revision refers to this bug: https://chrome-internal.googlesource.com/chromeos/chromeos-admin/+/480244e1916470885865c198e86281852c0b2ff9 commit 480244e1916470885865c198e86281852c0b2ff9 Author: Paul Hobbs <phobbs@google.com> Date: Mon Feb 12 18:46:51 2018 |
|||||
►
Sign in to add a comment |
|||||
Comment 1 by pho...@chromium.org
, Sep 7 2017