New issue
Advanced search Search tips

Issue 834078 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Jun 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug

Blocked on:
issue 774251
issue 832747
issue 838472

Blocking:
issue 834068
issue 834070



Sign in to add a comment

Migrate "standalone" builders from chromeos and chromiumos waterfalls.

Project Member Reported by dgarr...@chromium.org, Apr 17 2018

Issue description

Moving all builders from the chromeos and chromiumos waterfalls is hard. However, we can migrate all "standalone" builders today.
 
Labels: Swarming
ChromeOS:
  pre-cq-launcher
  betty-asan
  link-depthcharge-full-firmware
  som-dispatcher
  betty-informational

ChromiumOS:
 *-full
  amd64-generic-asan  
  chromiumos-sdk
  amd64-generic-fuzzer 
  chromeos-infra-go

Incremental builders have additional requirements, and so will be covered by a different bug.
Some of these builders are currently driven by LUCI Scheduler, and some by build bot scheduler.

The ones driven by LUCI Scheduler need to be moved from the chromiumos-chromite project to the chromeos project.

The suggested approach is to write CLs to remove and add them. Pause the builds in chromiumos-chromite, land the Add CL.

This allows very quick fallback to buildbut by pausing in the new location, and unpausing in the old location. After we are confident in the move, we can land the "Remove" CL.
Chromite Scheduler (buildbot):
  https://luci-scheduler.appspot.com/jobs/chromiumos-chromite

ChromeOS Scheduler (swarming):
  https://luci-scheduler.appspot.com/jobs/chromeos

Project Member

Comment 4 by bugdroid1@chromium.org, Apr 17 2018

Labels: merge-merged-config
The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/manifest-internal/+/e8aca36c05fa6ed0d0af944a2b8e9028370c5798

commit e8aca36c05fa6ed0d0af944a2b8e9028370c5798
Author: Don Garrett <dgarrett@google.com>
Date: Tue Apr 17 23:15:39 2018

Comment 5 by xixuan@chromium.org, Apr 17 2018

Cc: xixuan@chromium.org
Project Member

Comment 6 by bugdroid1@chromium.org, Apr 17 2018

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/manifest-internal/+/b4b6dd14764599fa4ddf1198c6abf377649456f3

commit b4b6dd14764599fa4ddf1198c6abf377649456f3
Author: Don Garrett <dgarrett@google.com>
Date: Tue Apr 17 23:31:48 2018

Project Member

Comment 7 by bugdroid1@chromium.org, Apr 17 2018

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/manifest-internal/+/84bfc426595e0f6058aba52fb4aeabc3c71e680c

commit 84bfc426595e0f6058aba52fb4aeabc3c71e680c
Author: Don Garrett <dgarrett@google.com>
Date: Tue Apr 17 23:40:46 2018

Project Member

Comment 8 by bugdroid1@chromium.org, Apr 18 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/c4584d9381c1f4bba3cbb5d1acc1bcd8696d2eb1

commit c4584d9381c1f4bba3cbb5d1acc1bcd8696d2eb1
Author: Don Garrett <dgarrett@google.com>
Date: Wed Apr 18 00:39:24 2018

Migrate pre-cq-launcher and som-dispatcher.

This removes the scheduler for these two builders.

BUG= chromium:834078 

Change-Id: I32a426cccaf0bdd4a8701428654a82d71a8cae16
Reviewed-on: https://chromium-review.googlesource.com/1016007
Tested-by: Don Garrett <dgarrett@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>

[modify] https://crrev.com/c4584d9381c1f4bba3cbb5d1acc1bcd8696d2eb1/luci-scheduler.cfg

Project Member

Comment 9 by bugdroid1@chromium.org, Apr 18 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/1efee2d24f5bfae9bdc2276cd1cb228e348db015

commit 1efee2d24f5bfae9bdc2276cd1cb228e348db015
Author: Don Garrett <dgarrett@google.com>
Date: Wed Apr 18 02:31:23 2018

chromeos_config: Record builders moved to Legoland.

This will help remove the waterfall build columns that should no
longer exist.

BUG= chromium:834078 
TEST=chromeos_config_unittest

Change-Id: I9e38531127d1f45e1632797c288756ad91eea979
Reviewed-on: https://chromium-review.googlesource.com/1016096
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Tested-by: Don Garrett <dgarrett@chromium.org>

[modify] https://crrev.com/1efee2d24f5bfae9bdc2276cd1cb228e348db015/cbuildbot/config_dump.json
[modify] https://crrev.com/1efee2d24f5bfae9bdc2276cd1cb228e348db015/cbuildbot/waterfall_layout_dump.txt
[modify] https://crrev.com/1efee2d24f5bfae9bdc2276cd1cb228e348db015/cbuildbot/chromeos_config.py

Project Member

Comment 10 by bugdroid1@chromium.org, Apr 18 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f4b46947717b0127d9082fc3295aeca47ba714dc

commit f4b46947717b0127d9082fc3295aeca47ba714dc
Author: chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Wed Apr 18 09:13:39 2018

Roll src/third_party/chromite/ 6a93eff9e..084709505 (3 commits)

https://chromium.googlesource.com/chromiumos/chromite.git/+log/6a93eff9ef52..084709505993

$ git log 6a93eff9e..084709505 --date=short --no-merges --format='%ad %ae %s'
2018-04-17 ihf caroline-arcnext: mark CQ/PFQs bvt-arc important.
2018-04-17 dgarrett chromeos_config: Record builders moved to Legoland.
2018-04-17 dgarrett config_lib: UseBuildbucketScheduler.

Created with:
  roll-dep src/third_party/chromite
BUG= chromium:834078 ,chromium:None


The AutoRoll server is located here: https://chromite-chromium-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.


TBR=chrome-os-gardeners@chromium.org

Change-Id: Iba4ad91e214467453d3327f5cf4db291476cb558
Reviewed-on: https://chromium-review.googlesource.com/1015983
Reviewed-by: Chromite Chromium Autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Commit-Queue: Chromite Chromium Autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#551618}
[modify] https://crrev.com/f4b46947717b0127d9082fc3295aeca47ba714dc/DEPS

Remaining

ChromeOS:
  betty-asan
  link-depthcharge-full-firmware
  betty-informational

ChromiumOS:
 *-full
  amd64-generic-asan  
  chromiumos-sdk
  amd64-generic-fuzzer 
  chromeos-infra-go

Some of these builders (especially chromiumos-sdk) can not be safely run multiple times in parallel, and many of them were NOT added to the chromiumos-chromite scheduler.

So... the plan is to add them to the chromeos LUCI Scheduler and pause it right away.

Then remove them from buildbot and do a waterfall restart.

We then unpause the new scheduler.

If anything radical goes wrong, we will require another waterfall restart to recover.
Blockedon: 832747
This is now unblocked and can move forward.
Blockedon: 838472 774251
After some investigation, I believe we can drop all remaining standalones from the chromeos waterfall. Blocking on relevant bugs to be sure.
Project Member

Comment 15 by bugdroid1@chromium.org, May 18 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/17530e94d1c56da74160d26cba9aff0680929859

commit 17530e94d1c56da74160d26cba9aff0680929859
Author: Don Garrett <dgarrett@google.com>
Date: Fri May 18 21:33:14 2018

chromeos_config: Display Asan builders as informational.

Display the Asan builders as informational in Legoland, since that
seems to make more sense than incremental.

This is needed before migrating these builds to LUCI, since they need
to display correctly in Legoland.

BUG= chromium:834078 
TEST=None

Change-Id: I761b0bdd0bd1fd52f9f6647395e3a43094d059a0
Reviewed-on: https://chromium-review.googlesource.com/1062776
Commit-Ready: Don Garrett <dgarrett@chromium.org>
Tested-by: Don Garrett <dgarrett@chromium.org>
Reviewed-by: Gregory Meinke <gmeinke@chromium.org>

[modify] https://crrev.com/17530e94d1c56da74160d26cba9aff0680929859/cbuildbot/config_dump.json
[modify] https://crrev.com/17530e94d1c56da74160d26cba9aff0680929859/cbuildbot/chromeos_config.py

Project Member

Comment 16 by bugdroid1@chromium.org, May 18 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/12bf7454a6ae15b397e68c1245263bf178984ef4

commit 12bf7454a6ae15b397e68c1245263bf178984ef4
Author: chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Fri May 18 23:03:14 2018

Roll src/third_party/chromite/ 08be0a3a8..17530e94d (2 commits)

https://chromium.googlesource.com/chromiumos/chromite.git/+log/08be0a3a8469..17530e94d1c5

$ git log 08be0a3a8..17530e94d --date=short --no-merges --format='%ad %ae %s'
2018-05-16 dgarrett chromeos_config: Display Asan builders as informational.
2018-05-16 dgarrett chromeos_config: Use internal checkout for betty-asan.

Created with:
  roll-dep src/third_party/chromite
BUG= chromium:834078 , chromium:774251 


The AutoRoll server is located here: https://chromite-chromium-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.


TBR=chrome-os-gardeners@chromium.org

Change-Id: Ifea8c8a720e251e7b76931ed57162db83baf6cf9
Reviewed-on: https://chromium-review.googlesource.com/1066738
Commit-Queue: Chromite Chromium Autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Reviewed-by: Chromite Chromium Autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#560086}
[modify] https://crrev.com/12bf7454a6ae15b397e68c1245263bf178984ef4/DEPS

Project Member

Comment 17 by bugdroid1@chromium.org, May 30 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/8e76c288f2ba766dba8d7051a5777fa20be8deae

commit 8e76c288f2ba766dba8d7051a5777fa20be8deae
Author: Don Garrett <dgarrett@google.com>
Date: Wed May 30 21:35:12 2018

luci-scheduler: Move additional builders.

This change removes schedulers for:
  chromeos-infra-go
  link-depthcharge-full-firmware
  amd64-generic-fuzzer
  betty-vmtest-informational

They are being readded for Legoland in CL:*628793.

BUG= chromium:834078 
TEST=None

Change-Id: Ief1535c0f10ec65aa9df1d78bdd0e38c3e063cb4
Reviewed-on: https://chromium-review.googlesource.com/1079700
Reviewed-by: Alec Thilenius <athilenius@google.com>
Tested-by: Don Garrett <dgarrett@chromium.org>
Trybot-Ready: Don Garrett <dgarrett@chromium.org>

[modify] https://crrev.com/8e76c288f2ba766dba8d7051a5777fa20be8deae/luci-scheduler.cfg

Project Member

Comment 18 by bugdroid1@chromium.org, May 30 2018

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/manifest-internal/+/9a843b0a6ba2dabb5f3ef8795b086e4bcf6caabe

commit 9a843b0a6ba2dabb5f3ef8795b086e4bcf6caabe
Author: Don Garrett <dgarrett@google.com>
Date: Wed May 30 21:35:35 2018

Project Member

Comment 19 by bugdroid1@chromium.org, May 31 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/ac2dab3a2487faf7b802383b22a9cb88b7c8d0ee

commit ac2dab3a2487faf7b802383b22a9cb88b7c8d0ee
Author: Don Garrett <dgarrett@google.com>
Date: Thu May 31 14:32:11 2018

chromeos_config: Move migrating builders to Legoland.

Update chromeos_config to match builders moving from chromeos and
chromiumos to legoland. This change will remove the relevant build
columns from the chromeos and chromiumos waterfalls.

     chromeos-infra-go
     chromiumos-sdk
     link-depthcharge-full-firmware
     amd64-generic-asan
     amd64-generic-fuzzer
     betty-asan
     betty-vmtest-informational

BUG= chromium:834078 
TEST=None

Change-Id: Id6a6ba0c96ee81a3294e4e9677ef5607390096fb
Reviewed-on: https://chromium-review.googlesource.com/1066943
Trybot-Ready: Don Garrett <dgarrett@chromium.org>
Reviewed-by: Alec Thilenius <athilenius@google.com>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Commit-Queue: Don Garrett <dgarrett@chromium.org>
Tested-by: Don Garrett <dgarrett@chromium.org>

[modify] https://crrev.com/ac2dab3a2487faf7b802383b22a9cb88b7c8d0ee/cbuildbot/config_dump.json
[modify] https://crrev.com/ac2dab3a2487faf7b802383b22a9cb88b7c8d0ee/cbuildbot/waterfall_layout_dump.txt
[modify] https://crrev.com/ac2dab3a2487faf7b802383b22a9cb88b7c8d0ee/cbuildbot/chromeos_config.py

Project Member

Comment 20 by bugdroid1@chromium.org, Jun 1 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/d606e647f2a0f26879bd53261df5456885643f69

commit d606e647f2a0f26879bd53261df5456885643f69
Author: Don Garrett <dgarrett@google.com>
Date: Fri Jun 01 20:17:38 2018

chromeos_config: Move full builders to swarming.

This CL removes all full builders from the chromiumos waterfall, and
adds them to the swarming waterfall. CL:*634491 creates the scheduler
entries for them.

BUG= chromium:834078 
TEST=None

Change-Id: If5e97bb47ff91a958975f7047560abefca9d2785
Reviewed-on: https://chromium-review.googlesource.com/1082668
Tested-by: Don Garrett <dgarrett@chromium.org>
Trybot-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com>
Reviewed-by: Alec Thilenius <athilenius@google.com>

[modify] https://crrev.com/d606e647f2a0f26879bd53261df5456885643f69/cbuildbot/config_dump.json
[modify] https://crrev.com/d606e647f2a0f26879bd53261df5456885643f69/cbuildbot/waterfall_layout_dump.txt
[modify] https://crrev.com/d606e647f2a0f26879bd53261df5456885643f69/cbuildbot/chromeos_config.py

Project Member

Comment 21 by bugdroid1@chromium.org, Jun 1 2018

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/manifest-internal/+/d4ef470e2a3b37e8a3c5810dec49fe510d185e49

commit d4ef470e2a3b37e8a3c5810dec49fe510d185e49
Author: Don Garrett <dgarrett@google.com>
Date: Fri Jun 01 20:50:36 2018

Project Member

Comment 22 by bugdroid1@chromium.org, Jun 1 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/9b818ddbd703e8ecb782ab9d4913e114320541d1

commit 9b818ddbd703e8ecb782ab9d4913e114320541d1
Author: Chromite Chromium Autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Fri Jun 01 21:46:32 2018

Roll src/third_party/chromite ff0d51c..d606e64 (1 commits)

https://chromium.googlesource.com/chromiumos/chromite.git/+log/ff0d51c..d606e64


git log ff0d51c..d606e64 --date=short --no-merges --format='%ad %ae %s'
2018-06-01 dgarrett@google.com chromeos_config: Move full builders to swarming.


Created with:
  gclient setdep -r src/third_party/chromite@d606e64

The AutoRoll server is located here: https://chromite-chromium-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.



BUG= chromium:834078 
TBR=chrome-os-gardeners@chromium.org

Change-Id: I81c3a8fb3652b994d6f2be143057dc2cc5755eca
Reviewed-on: https://chromium-review.googlesource.com/1083123
Reviewed-by: Chromite Chromium Autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Commit-Queue: Chromite Chromium Autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#563810}
[modify] https://crrev.com/9b818ddbd703e8ecb782ab9d4913e114320541d1/DEPS

Status: Fixed (was: Started)

Sign in to add a comment