New issue
Advanced search Search tips

Issue 851127 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner: ----
Closed: Jul 11
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Task



Sign in to add a comment

Delete StoppageAlert entities

Project Member Reported by benjhayden@chromium.org, Jun 8 2018

Issue description

The StoppageAlert Model has been removed from the code, so it is not possible to get StoppageAlert entities; trying to get them raises KindError. Deleting the entities from the datastore will make get() return None instead of raising an exception. Deleting the entities will also free 700MB.
 
+1
Is there a faster way to do this than dev_console?

import time
from google.appengine.ext import ndb
deadline = time.time() + 50
total = 0
while time.time() < deadline:
  ks = ndb.Query(kind='StoppageAlert').fetch(limit=500, keys_only=True)
  if not ks: break
  total += len(ks)
  ndb.delete_multi(ks)
print total

Might be good to create a quick endpoint that does this via the taskqueue and land it, that way we can test a bit too. Think there's a few types of entities we want to delete.
Project Member

Comment 4 by bugdroid1@chromium.org, Jul 6

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/aa04e549d7fc391cb5f53b3a3b60f03c5d670471

commit aa04e549d7fc391cb5f53b3a3b60f03c5d670471
Author: benshayden <benjhayden@chromium.org>
Date: Fri Jul 06 20:39:50 2018

dashboard.DeleteAllEntities(kind)

There are currently many entities in the datastore whose models have been or can
be removed from the source.
Attempts to access these entities raises KindError, which are occasionally seen
in the logs.
This CL adds a function that deletes all entities of a given kind.
It can be called from dev console:
from dashboard import delete_all_entities
delete_all_entities.DeleteAllEntities('StoppageAlert')

Bug:  chromium:851127 
Change-Id: I01c467cb0264ff0eadf36218c960ddf4ebaf1807
Reviewed-on: https://chromium-review.googlesource.com/1127470
Commit-Queue: Ben Hayden <benjhayden@chromium.org>
Reviewed-by: Simon Hatch <simonhatch@chromium.org>

[add] https://crrev.com/aa04e549d7fc391cb5f53b3a3b60f03c5d670471/dashboard/dashboard/delete_all_entities_test.py
[add] https://crrev.com/aa04e549d7fc391cb5f53b3a3b60f03c5d670471/dashboard/dashboard/delete_all_entities.py

Project Member

Comment 5 by bugdroid1@chromium.org, Jul 6

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/77f7da163cfb91ce8b85164109b324bdd56966d3

commit 77f7da163cfb91ce8b85164109b324bdd56966d3
Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Fri Jul 06 23:33:48 2018

Roll src/third_party/catapult eca5037d8394..8a075d7c378a (3 commits)

https://chromium.googlesource.com/catapult.git/+log/eca5037d8394..8a075d7c378a


git log eca5037d8394..8a075d7c378a --date=short --no-merges --format='%ad %ae %s'
2018-07-06 simonhatch@chromium.org Dashboard - Automate TimeToCulprit calculation.
2018-07-06 benjhayden@chromium.org dashboard.DeleteAllEntities(kind)
2018-07-06 simonhatch@chromium.org Dashboard - Remove unnecessary installhooks() call


Created with:
  gclient setdep -r src/third_party/catapult@8a075d7c378a

The AutoRoll server is located here: https://catapult-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.

CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel

BUG= chromium:860334 , chromium:851127 
TBR=sullivan@chromium.org

Change-Id: I73eec976dc921e30552b6f7f604f94696b5c7122
Reviewed-on: https://chromium-review.googlesource.com/1128339
Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#573116}
[modify] https://crrev.com/77f7da163cfb91ce8b85164109b324bdd56966d3/DEPS

Status: Fixed (was: Available)
These are gone!

Sign in to add a comment