The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/1eece3601976d5a161ece11ba55a84417f4ca4af

commit 1eece3601976d5a161ece11ba55a84417f4ca4af
Author: Ningning Xia <nxia@chromium.org>
Date: Tue Oct 04 18:43:51 2016

Add build_type tags to the Buildbucket requests.

Tags can be searched by the Buildbucket.Search API. Add build_type tags
to the Buildbucket requests, so the obsolete builds can be easily
searched and cancelled by the Buildbucket APIs.

BUG= chromium:652482 
TEST=run_tryjobs

Change-Id: I607cc635bdb1194993a4784939425b7d9223b38a
Reviewed-on: https://chromium-review.googlesource.com/393046
Commit-Ready: Ningning Xia <nxia@chromium.org>
Tested-by: Ningning Xia <nxia@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>

[modify] https://crrev.com/1eece3601976d5a161ece11ba55a84417f4ca4af/cbuildbot/remote_try.py
[modify] https://crrev.com/1eece3601976d5a161ece11ba55a84417f4ca4af/cbuildbot/stages/sync_stages.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/3e39a61a97cef1add8dd1273a0f0d9694497cef4

commit 3e39a61a97cef1add8dd1273a0f0d9694497cef4
Author: Ningning Xia <nxia@chromium.org>
Date: Tue Oct 04 22:32:43 2016

Support SearchBuilds in Buildbucket_lib.

1) add 'master:False' to paladin slave builds.
2) support SearchBuilds with bucket, tags and status. Obsolete builds can
be searched and cancelled by Buildbucket to save resources.

BUG= chromium:652482 
TEST=unit_test; tryjobs

Change-Id: I229bc86e9b2c7772bffd984de0d04e9be917d4e1
Reviewed-on: https://chromium-review.googlesource.com/393307
Commit-Ready: Ningning Xia <nxia@chromium.org>
Tested-by: Ningning Xia <nxia@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>

[modify] https://crrev.com/3e39a61a97cef1add8dd1273a0f0d9694497cef4/cbuildbot/stages/sync_stages.py
[modify] https://crrev.com/3e39a61a97cef1add8dd1273a0f0d9694497cef4/cbuildbot/buildbucket_lib_unittest.py
[modify] https://crrev.com/3e39a61a97cef1add8dd1273a0f0d9694497cef4/cbuildbot/buildbucket_lib.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/339748cac82f03c3fdf63c0666c3eddfc872d194

commit 339748cac82f03c3fdf63c0666c3eddfc872d194
Author: Ningning Xia <nxia@chromium.org>
Date: Wed Oct 05 04:35:06 2016

Cancel obsolete paladin slave builds.

In the CleanUpStage in master-paladin, search and cancel all STARTED
and paladin build_type slave builds.

BUG= chromium:652482 
TEST=unit_test; cbuildbot_run

Change-Id: I7e83e059978aeb01700761de2ed536a688a5ecbe
Reviewed-on: https://chromium-review.googlesource.com/394266
Commit-Ready: Ningning Xia <nxia@chromium.org>
Tested-by: Ningning Xia <nxia@chromium.org>
Reviewed-by: Ningning Xia <nxia@chromium.org>

[modify] https://crrev.com/339748cac82f03c3fdf63c0666c3eddfc872d194/cbuildbot/stages/build_stages_unittest.py
[modify] https://crrev.com/339748cac82f03c3fdf63c0666c3eddfc872d194/cbuildbot/stages/sync_stages.py
[modify] https://crrev.com/339748cac82f03c3fdf63c0666c3eddfc872d194/cbuildbot/stages/build_stages.py
[modify] https://crrev.com/339748cac82f03c3fdf63c0666c3eddfc872d194/cbuildbot/buildbucket_lib_unittest.py
[modify] https://crrev.com/339748cac82f03c3fdf63c0666c3eddfc872d194/cbuildbot/buildbucket_lib.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/e79953d1e53d267ac2047615558357ab903ada1c

commit e79953d1e53d267ac2047615558357ab903ada1c
Author: Ningning Xia <nxia@chromium.org>
Date: Fri Oct 14 04:33:12 2016

Disable cancelling builds via Buildbucket in CleanUp stage.

CleanUpStage runs twice in a build, in the second run it cancels
the other paladins. Set dryrun to True in buildbucket CancelBuildRequest
for now, to prevent it from cancelling other running builds.

BUG= chromium:652482 
TEST=run_tests

Change-Id: If7327a33d5b021ff5f10ad53e50acb02c2a99987
Reviewed-on: https://chromium-review.googlesource.com/398582
Tested-by: Ningning Xia <nxia@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Ningning Xia <nxia@chromium.org>

[modify] https://crrev.com/e79953d1e53d267ac2047615558357ab903ada1c/cbuildbot/stages/build_stages.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/3d810584555fcc4edda9a4bdd21c7168bec176ee

commit 3d810584555fcc4edda9a4bdd21c7168bec176ee
Author: Ningning Xia <nxia@chromium.org>
Date: Fri Oct 14 17:57:40 2016

Fix and refactor SearchAllBuilds in Buildbucket_lib.

1. Raise SEARCH_LIMIT_DEFAULT to 100.
2. Introduce max_builds in the Buildbucket search requests.
3. Fix start_cursor in SearchAllBuilds.
4. Search for scheduled builds in CleanUpStage.
5. Add unit tests.

BUG= chromium:652482 
TEST=unit_tests

Change-Id: I00de8211982b53cd70005469e55de8c0b6415f62
Reviewed-on: https://chromium-review.googlesource.com/399538
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Ningning Xia <nxia@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>

[modify] https://crrev.com/3d810584555fcc4edda9a4bdd21c7168bec176ee/cbuildbot/stages/build_stages.py
[modify] https://crrev.com/3d810584555fcc4edda9a4bdd21c7168bec176ee/cbuildbot/buildbucket_lib_unittest.py
[modify] https://crrev.com/3d810584555fcc4edda9a4bdd21c7168bec176ee/cbuildbot/buildbucket_lib.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/4160b06e0b54c0676d623800026741efdd5d642d

commit 4160b06e0b54c0676d623800026741efdd5d642d
Author: Ningning Xia <nxia@chromium.org>
Date: Thu Oct 20 17:36:07 2016

Set right dryrun mode in CleanUpStage.CancelObsoleteSlaveBuilds

Enable CancelBatchBuildsRequest in CleanUpStage for master-paladin.

BUG= chromium:652482 
TEST=run_tests

Change-Id: I1043f2c3dafe81a8029e9d6b18dff6a73deb88cd
Reviewed-on: https://chromium-review.googlesource.com/401379
Commit-Ready: Ningning Xia <nxia@chromium.org>
Tested-by: Ningning Xia <nxia@chromium.org>
Reviewed-by: Ningning Xia <nxia@chromium.org>

[modify] https://crrev.com/4160b06e0b54c0676d623800026741efdd5d642d/cbuildbot/stages/build_stages.py

Want to record the canceled slave builds in the CIDB.buildTable. Saw the 'aborted' status in the CIDB.buildTable but it's not used currently.


Can I just set the status to 'aborted' for those canceled builds? Or I should add one more status (like 'aborted_by_master') to CIDB.buildTable? 

https://uberchromegw.corp.google.com/i/chromeos/builders/elm-paladin/builds/934

This build was cancelled by the master-paladin via Buildbucket, but it's marked as 'fail' status in CIDB.