New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 773872 link

Starred by 1 user
Status: Fixed
Owner:
bpastene@chromium.org
Closed: Nov 2017
Cc:
█ phajdan.jr@chromium.org
dpranke@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 3
Type: Bug



Sign in to add a comment

Move android bots on chromium.linux over to chromium.android

Project Member Reported by bpastene@chromium.org, Oct 11 2017 
Because it's weird to have that split. Each chromium.$(platform) master contains only bots for its platform... except for linux.

Plan is to spin up a mirror of every android bot on:
https://build.chromium.org/p/chromium.linux/builders

And place them on:
https://build.chromium.org/p/chromium.android/builders

I'll let them run for a while, maybe a week, and then start pointing things (som, gatekeeper, etc) to the new bots. Once everything is switched over, I'll delete the old ones.

+ cc some people in case they have thoughts. We'd lose build & test history, and the chromium.android waterfall isn't the most stable. But IMO that shouldn't stop us from moving them.

... or maybe I should just wait for luci when the concept of a master vanishes.

Comment 1 by dpranke@chromium.org, Oct 11 2017 

I wouldn't necessarily say that this is a high-value task, but it is also nice to clean things up where we can.

Correct me if I'm wrong, but normally we expect chromium sheriffs to handle failures on chromium.linux, and android sheriffs to handle chromium.android.

If we moved the bots, would we be shifting load to the android sheriffs?

Comment 2 by bpastene@chromium.org, Oct 11 2017 

Android sheriffs (if they're using som (which they should be)) already monitor the bots on chromium.linux:
https://chromium.googlesource.com/chromium/tools/build/+/master/scripts/slave/gatekeeper_trees.json#58

And it looks like chromium sheriffs also look at them: https://chromium.googlesource.com/chromium/tools/build/+/master/scripts/slave/gatekeeper_trees.json#24

Which is another reason to move the bots. I don't think two different sheriffs should be monitoring the same set of bots.

And I don't think there'd be a lot of work required. All of the configs are already written. Just needs a lot of copy-pasting :)

Comment 3 by dpranke@chromium.org, Oct 11 2017 

Given that I am also prone to "low value" bot cleanups, I'm not going to stop you :).

I agree that we don't want two sets of sheriffs looking at the same bots.
Project Member

Comment 4 by bugdroid1@chromium.org, Oct 17 2017 

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infra/infra_internal/+/70dd1f50eeaa50b52aaafa6e343ca3878f29dba6

commit 70dd1f50eeaa50b52aaafa6e343ca3878f29dba6
Author: Benjamin Pastene <bpastene@chromium.org>
Date: Tue Oct 17 17:59:53 2017

Comment 5 by bpastene@chromium.org, Oct 17 2017

Status: Started (was: Assigned)
Barring any objections, I'm going to move ahead with this.

Comment 6 by jbudorick@chromium.org, Oct 17 2017 

Given that it seems ci.chromium.org will retain specific links beyond the move, I have no objections.
Project Member

Comment 7 by bugdroid1@chromium.org, Oct 18 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/811a67ba13016c899988300efab06cc60760159a

commit 811a67ba13016c899988300efab06cc60760159a
Author: Benjamin Pastene <bpastene@chromium.org>
Date: Wed Oct 18 23:31:53 2017

Add copies of android builders on chromium.linux to chromium.android

old bot on chromium.linux     -> new bot on chromium.android
-------------------------------------------------------------------------
"Android Builder (dbg)"       -> deduped into "Android arm Builder (dbg)"
"Android Tests (dbg)"         -> "KitKat Phone Tester (dbg)"
"Android Builder"             -> removed, merged builder and tester
"Android Tests"               -> "KitKat Phone Tester (rel)"
"Cast Android (dbg)"          -> "Cast Android (dbg)"
"Android Clang Builder (dbg)" -> "Android Clang Builder (dbg)"

src-side change:
https://chromium-review.googlesource.com/c/chromium/src/+/724176

Bug:  773872 
Change-Id: Idca6fdb537447ddef3eedcef1c15cdbb3f380cd0
Reviewed-on: https://chromium-review.googlesource.com/724164
Commit-Queue: Benjamin Pastene <bpastene@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>

[modify] https://crrev.com/811a67ba13016c899988300efab06cc60760159a/scripts/slave/recipe_modules/chromium_tests/chromium_android.py
[modify] https://crrev.com/811a67ba13016c899988300efab06cc60760159a/masters/master.chromium.android/builders.pyl
Project Member

Comment 8 by bugdroid1@chromium.org, Oct 18 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/74e47f138ca2f67aa370b15c37ba73d924f7f03d

commit 74e47f138ca2f67aa370b15c37ba73d924f7f03d
Author: Benjamin Pastene <bpastene@chromium.org>
Date: Wed Oct 18 23:50:35 2017

Revert "Add copies of android builders on chromium.linux to chromium.android"

This reverts commit 811a67ba13016c899988300efab06cc60760159a.

Reason for revert: broke android_n5x_swarming_rel: (I think) https://build.chromium.org/p/tryserver.chromium.android/builders/android_n5x_swarming_rel?numbuilds=200

Original change's description:
> Add copies of android builders on chromium.linux to chromium.android
> 
> old bot on chromium.linux     -> new bot on chromium.android
> -------------------------------------------------------------------------
> "Android Builder (dbg)"       -> deduped into "Android arm Builder (dbg)"
> "Android Tests (dbg)"         -> "KitKat Phone Tester (dbg)"
> "Android Builder"             -> removed, merged builder and tester
> "Android Tests"               -> "KitKat Phone Tester (rel)"
> "Cast Android (dbg)"          -> "Cast Android (dbg)"
> "Android Clang Builder (dbg)" -> "Android Clang Builder (dbg)"
> 
> src-side change:
> https://chromium-review.googlesource.com/c/chromium/src/+/724176
> 
> Bug:  773872 
> Change-Id: Idca6fdb537447ddef3eedcef1c15cdbb3f380cd0
> Reviewed-on: https://chromium-review.googlesource.com/724164
> Commit-Queue: Benjamin Pastene <bpastene@chromium.org>
> Reviewed-by: Dirk Pranke <dpranke@chromium.org>
> Reviewed-by: John Budorick <jbudorick@chromium.org>

TBR=dpranke@chromium.org,bpastene@chromium.org,jbudorick@chromium.org

Change-Id: I8f03d73a9a6cd337e6e086819fa4cd88e96553c0
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug:  773872 
Reviewed-on: https://chromium-review.googlesource.com/727339
Reviewed-by: Benjamin Pastene <bpastene@chromium.org>
Commit-Queue: Benjamin Pastene <bpastene@chromium.org>

[modify] https://crrev.com/74e47f138ca2f67aa370b15c37ba73d924f7f03d/scripts/slave/recipe_modules/chromium_tests/chromium_android.py
[modify] https://crrev.com/74e47f138ca2f67aa370b15c37ba73d924f7f03d/masters/master.chromium.android/builders.pyl
Project Member

Comment 9 by bugdroid1@chromium.org, Oct 19 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/d4d2d8a1f3dd3abcac53f14a49424fc2f81958e3

commit d4d2d8a1f3dd3abcac53f14a49424fc2f81958e3
Author: Benjamin Pastene <bpastene@chromium.org>
Date: Thu Oct 19 00:22:53 2017

Reland "Add copies of android builders on chromium.linux to chromium.android"

old bot on chromium.linux     -> new bot on chromium.android
-------------------------------------------------------------------------
"Android Builder (dbg)"       -> deduped into "Android arm Builder (dbg)"
"Android Tests (dbg)"         -> "KitKat Phone Tester (dbg)"
"Android Builder"             -> removed, merged builder and tester
"Android Tests"               -> "KitKat Phone Tester (rel)"
"Cast Android (dbg)"          -> "Cast Android (dbg)"
"Android Clang Builder (dbg)" -> "Android Clang Builder (dbg)"

src-side change:
https://chromium-review.googlesource.com/c/chromium/src/+/724176

TBR=jbudorick@chromium.org, dpranke@chromium.org
Bug:  773872 
Change-Id: I8eb91dc499f163e8a79ce0abb3ac30d1b463c7d6
Reviewed-on: https://chromium-review.googlesource.com/727371
Reviewed-by: Benjamin Pastene <bpastene@chromium.org>
Commit-Queue: Benjamin Pastene <bpastene@chromium.org>

[modify] https://crrev.com/d4d2d8a1f3dd3abcac53f14a49424fc2f81958e3/scripts/slave/recipe_modules/chromium_tests/chromium_android.py
[modify] https://crrev.com/d4d2d8a1f3dd3abcac53f14a49424fc2f81958e3/masters/master.chromium.android/builders.pyl
Project Member

Comment 10 by bugdroid1@chromium.org, Oct 19 2017 

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/master-manager/+/b7a777871445a5f426da3a0e5c5d63dcf8e5ce87

commit b7a777871445a5f426da3a0e5c5d63dcf8e5ce87
Author: Benjamin Pastene <bpastene@google.com>
Date: Thu Oct 19 00:44:58 2017
Project Member

Comment 11 by bugdroid1@chromium.org, Oct 19 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/9e0408c2a682b1034698e080668a01169b1b35d0

commit 9e0408c2a682b1034698e080668a01169b1b35d0
Author: Benjamin Pastene <bpastene@chromium.org>
Date: Thu Oct 19 03:06:27 2017

Add testing spec and mb config for new builders on chromium.android

build-side change:
https://chromium-review.googlesource.com/c/chromium/tools/build/+/724164

Bug:  773872 
Change-Id: Ie8bb990b3b6969cc245120bc8b3bdd0dac852c95
Reviewed-on: https://chromium-review.googlesource.com/724176
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Benjamin Pastene <bpastene@chromium.org>
Cr-Commit-Position: refs/heads/master@{#509983}
[modify] https://crrev.com/9e0408c2a682b1034698e080668a01169b1b35d0/testing/buildbot/chromium.android.json
[modify] https://crrev.com/9e0408c2a682b1034698e080668a01169b1b35d0/tools/mb/mb_config.pyl

Comment 13 by bpastene@chromium.org, Oct 31 2017 

Given that the cycle times and rate of test flake is pretty similar across each bot and its mirror, I'll plan on making the swap sometime this week (provided I get the CLs prepped and PSAs out the door before then)
Project Member

Comment 14 by bugdroid1@chromium.org, Nov 1 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/5d3fecfef13863f841f818e50eef01fda27cf845

commit 5d3fecfef13863f841f818e50eef01fda27cf845
Author: Benjamin Pastene <bpastene@chromium.org>
Date: Wed Nov 01 19:45:16 2017

Remove android_compile_rel on android tryserver from mb_config.pyl.

It's the same thing as linux_android_rel_ng without the tests.
Note that there's no test-less mirror of android_n5x_swarming_rel,
which is also on the CQ.

Need to remove it here before updating buildbucket.config

TBR=phajdan.jr@chromium.org

Bug:  773872 
Change-Id: I7a04f92f19752ab3f31cd2203249f485fe678c94
Reviewed-on: https://chromium-review.googlesource.com/748216
Commit-Queue: Benjamin Pastene <bpastene@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Cr-Commit-Position: refs/heads/master@{#513223}
[modify] https://crrev.com/5d3fecfef13863f841f818e50eef01fda27cf845/PRESUBMIT_test.py
[modify] https://crrev.com/5d3fecfef13863f841f818e50eef01fda27cf845/tools/mb/mb_config.pyl
Project Member

Comment 16 by bugdroid1@chromium.org, Nov 7 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/4cad8ab147b687f1f829296520e0a4772f421402

commit 4cad8ab147b687f1f829296520e0a4772f421402
Author: Benjamin Pastene <bpastene@chromium.org>
Date: Tue Nov 07 18:14:33 2017

Remove old android bots on chromium.linux.

After https://chromium-review.googlesource.com/748105, these bots
should be unused & unwatched.

Bug:  773872 
Change-Id: I61c0663e747fb258f5b2ce4b60edaa998254cd68
Reviewed-on: https://chromium-review.googlesource.com/755334
Commit-Queue: Benjamin Pastene <bpastene@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>

[modify] https://crrev.com/4cad8ab147b687f1f829296520e0a4772f421402/scripts/slave/recipes/chromium.expected/dynamic_swarmed_instrumentation_test.json
[modify] https://crrev.com/4cad8ab147b687f1f829296520e0a4772f421402/scripts/slave/recipes/chromium.expected/dynamic_instrumentation_test.json
[modify] https://crrev.com/4cad8ab147b687f1f829296520e0a4772f421402/masters/master.chromium.linux/master.cfg
[modify] https://crrev.com/4cad8ab147b687f1f829296520e0a4772f421402/scripts/slave/recipe_modules/chromium_tests/tests/api/wrap_chromium_tests.py
[add] https://crrev.com/4cad8ab147b687f1f829296520e0a4772f421402/scripts/slave/recipe_modules/chromium_tests/tests/api/wrap_chromium_tests.expected/no_require_device_steps_with_root.json
[modify] https://crrev.com/4cad8ab147b687f1f829296520e0a4772f421402/scripts/slave/recipe_modules/chromium/examples/full.expected/basic_out_dir_goma_module_start_failure.json
[delete] https://crrev.com/a2c95fefd01c222eddc1d1444d793891bf52eb26/masters/master.chromium.linux/master_android_cfg.py
[modify] https://crrev.com/4cad8ab147b687f1f829296520e0a4772f421402/masters/master.chromium.linux/slaves.cfg
[modify] https://crrev.com/4cad8ab147b687f1f829296520e0a4772f421402/scripts/slave/recipe_modules/chromium/examples/full.expected/basic_out_dir_with_custom_mb_config.json
[modify] https://crrev.com/4cad8ab147b687f1f829296520e0a4772f421402/scripts/slave/recipe_modules/chromium/examples/full.expected/basic_out_dir_with_goma_module.json
[modify] https://crrev.com/4cad8ab147b687f1f829296520e0a4772f421402/scripts/slave/recipe_modules/chromium/examples/full.expected/basic_out_dir_ninja_no_op_failure.json
[modify] https://crrev.com/4cad8ab147b687f1f829296520e0a4772f421402/scripts/slave/recipes/chromium.expected/dynamic_instrumentation_nodefault_build.json
[modify] https://crrev.com/4cad8ab147b687f1f829296520e0a4772f421402/scripts/slave/recipes/chromium.expected/dynamic_junit_test.json
[modify] https://crrev.com/4cad8ab147b687f1f829296520e0a4772f421402/scripts/slave/recipe_modules/chromium/examples/full.py
[modify] https://crrev.com/4cad8ab147b687f1f829296520e0a4772f421402/scripts/slave/recipe_modules/chromium_tests/tests/api/wrap_chromium_tests.expected/no_require_device_steps.json
[modify] https://crrev.com/4cad8ab147b687f1f829296520e0a4772f421402/scripts/slave/recipe_modules/chromium_tests/chromium_linux.py
[modify] https://crrev.com/4cad8ab147b687f1f829296520e0a4772f421402/scripts/slave/recipes/chromium.expected/dynamic_instrumentation_nodefault_test.json
[modify] https://crrev.com/4cad8ab147b687f1f829296520e0a4772f421402/scripts/slave/recipe_modules/chromium/examples/full.expected/basic_out_dir_ninja_build_failure.json
[modify] https://crrev.com/4cad8ab147b687f1f829296520e0a4772f421402/tests/masters_recipes_test.py
[modify] https://crrev.com/4cad8ab147b687f1f829296520e0a4772f421402/scripts/slave/recipe_modules/chromium/examples/full.expected/basic_out_dir.json
[modify] https://crrev.com/4cad8ab147b687f1f829296520e0a4772f421402/scripts/slave/recipe_modules/chromium/examples/full.expected/basic_out_dir_goma_module_build_failure.json
[modify] https://crrev.com/4cad8ab147b687f1f829296520e0a4772f421402/scripts/slave/recipes/chromium.expected/dynamic_swarmed_gn_instrumentation_test.json
[modify] https://crrev.com/4cad8ab147b687f1f829296520e0a4772f421402/scripts/slave/recipe_modules/chromium/examples/full.expected/basic_no_out_dir_with_goma_module.json
[modify] https://crrev.com/4cad8ab147b687f1f829296520e0a4772f421402/scripts/slave/recipe_modules/chromium/examples/full.expected/basic_out_dir_without_compile_py.json
[modify] https://crrev.com/4cad8ab147b687f1f829296520e0a4772f421402/scripts/slave/recipe_modules/chromium/examples/full.expected/basic_out_dir_goma_module_ping_failure.json
[modify] https://crrev.com/4cad8ab147b687f1f829296520e0a4772f421402/scripts/slave/recipes/chromium.py

Comment 17 by bpastene@chromium.org, Nov 7 2017 

Android bots have been successfully removed:
https://build.chromium.org/p/chromium.linux/builders/

I'll wait a few more days to ensure nothing needs reverting, then will close this out and reclaim the freed up slaves.
Project Member

Comment 19 by bugdroid1@chromium.org, Nov 9 2017 

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/master-manager/+/6c8cb028e643eb71fbe37496da7ea70b52f6bc6c

commit 6c8cb028e643eb71fbe37496da7ea70b52f6bc6c
Author: Benjamin Pastene <bpastene@google.com>
Date: Thu Nov 09 01:33:39 2017
Project Member

Comment 20 by bugdroid1@chromium.org, Nov 11 2017 

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chrome/tools/build_limited/scripts/slave/+/90a5764657f5cafc3862a3ce25bf710205a28ea4

commit 90a5764657f5cafc3862a3ce25bf710205a28ea4
Author: John Budorick <jbudorick@google.com>
Date: Sat Nov 11 02:38:53 2017

Comment 21 by bpastene@chromium.org, Nov 14 2017

Status: Fixed (was: Started)
This is done. (Turned out to be a lot more than just simple "config copy-pasting"... oh well)

slaves reclaimed in bug 783877 and https://chrome-internal-review.googlesource.com/c/infra/infra_internal/+/502113
Project Member

Comment 22 by bugdroid1@chromium.org, Nov 14 2017 

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infra/infra_internal/+/0db179a15e9ce59b52f9d6a4c6c2994f04c48a1f

commit 0db179a15e9ce59b52f9d6a4c6c2994f04c48a1f
Author: Benjamin Pastene <bpastene@chromium.org>
Date: Tue Nov 14 01:46:57 2017
►

Sign in to add a comment