New issue
Advanced search Search tips

Issue 818301 link

Starred by 3 users

Issue metadata

Status: Fixed
Owner:
Closed: Jan 8
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug

Blocked on:
issue 390309
issue 790265
issue 790278
issue 790286
issue 790294
issue 790303
issue 790309
issue 790340
issue 790343
issue 864817



Sign in to add a comment

We should shut off chromium.webkit and migrate the remaining tasks to chromium.{mac,win,linux,android}

Project Member Reported by dpranke@chromium.org, Mar 2 2018

Issue description

We should shut off chromium.webkit and migrate the remaining tasks to chromium.{mac,win,linux,android}.

Most of the reasons chromium.webkit is a separate waterfall are no longer relevant. The primary reason was that the layout tests took too long to run, and we didn't want them on the main waterfalls. When we moved the tests to swarming, that reason went away, and we in fact do run the layout tests on many of the main waterfall bots now.

Many of the bots are subsets of the main waterfall configs and can be shut off as soon as we point the rebaselining configs to the main bots instead. Other bots have main waterfall equivalents but the main waterfall bots aren't running the tests yet, so we need to move them.

Some of the chromium.webkit builders don't have main waterfall equivalents, and so those bots do need to be migrated over (and converted to LUCI).

Filing this bug as an overall tracking item.
 
Status: Available (was: Untriaged)
We should probably do this after migrating them to LUCI with the master name intact since it will be easier there and one less thing to break during the migration.

That might be what you were thinking but I wanted to mention it explicitly.
I'm saying the opposite: we should shut the machines off (now)instead of migrating them to swarming, since this can be done at any time. Migrating them to swarming and then shutting them off is wasted effort.
Ok, in that case is there anything you want the foundation team to do for this? We can block all chromium.webkit migrations on this bug to make sure no one starts anything but we don't know what tasks need to be moved/changed/deleted (and what are tasks in this context?).
Cc: jbudorick@chromium.org
I'm fine w/ this getting punted over to jbudorick's team (or me, or robert, or someone), I agree this isn't something I'd expect the foundation team to work through.

"tasks" meant "builders and/or tests as appropriate", based on what else I wrote in the description :).
I see the following bots already running layout tests:
* chromium.linux/Linux Tests (+Site Isolation)
* chromium.mac/Mac10.1? Tests

but can't seem to find layout tests in chromium.{android,win}.

So is the remaining work just to add layout tests to some existing or new bots in chromium.{android,win}? And then we can turn the whole chromium.webkit off?
The tests aren't running on chromium.android, we'd need to move the N4 bot over.

I think the tests need to be added to `Win7 (32) Tests` on chromium.win as well. I don't remember why I didn't do that before, but it might've been related to capacity at the time. We should have capacity now, I think.

In addition to doing the builder-side work, you'd also need to update builders.json , but, yeah, apart from that it should be straightforward.
#6: if possible, it'd be great to move the android tests from the N4s over to the N5s & just dissolve the N4 bot into swarming capacity.
Yeah I can take care of the webkitpy side of updates.

Layout tests on Android (real phones) might be a bit of an issue, but I'm not exactly familiar with it. My understanding is:
* First off, it only runs on KitKat until issue 567947 is resolved. Can we have N5 with KitKat?
* Secondly, I believe many baselines are resolution-dependant. So we have to use either N4 or N5 in all places (and rebaseline if we decide to choose N5).
Yeah, the N5s are on KTU84P. Once I resolve that, I think we'll also want to add the layout tests to the M bots.

Comment 10 by efoo@chromium.org, Mar 3 2018

Labels: LUCI-Chromium LUCI-Blocker-Chromium
Also, chromium.mac is missing a retina tester. We need to migrate chromium.webkit.WebKit Mac10.11 (retina) there.

And I just noticed WebKit Mac10.11 (retina) hasn't had any build since Feb 17. The bot is disconnected. https://ci.chromium.org/buildbot/chromium.webkit/WebKit%20Mac10.11%20%28retina%29/
This is a P1 with nobody assigned. robertma@ has said he can do the webkitpy parts of it, but is there anything to be done prior to or after that?

It sounds like this won't be just a matter of moving tasks around, but that it amounts to also doing issue 567947, which is presumably a non-trivial amount of work.

What is the minimum that could be done to just unblock LUCI? (I see LUCI-Blocker-Chromium label)
I don't think we need to block this on bug 567947, we can just move the builders over instead.
Project Member

Comment 14 by bugdroid1@chromium.org, Apr 7 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/736e26fa2b93318635af46024207ffe55b1f2390

commit 736e26fa2b93318635af46024207ffe55b1f2390
Author: Dirk Pranke <dpranke@chromium.org>
Date: Sat Apr 07 01:54:30 2018

Run Webkit's Mac 10.12 tests on Mac10.13 Tests.

Webkit's Mac 10.12 covering part of by mac_chromium_rel_ng.
To avoid migrating Webkit builder to LUCI, we need to run its tests
somewhere else, which jbudorick@ proposes to be Mac10.13 Tests.

R=jbudorick@chromium.org

Bug:  818301 ,  731543 
Change-Id: I67f575024003e890b694756f0dab804402b2983e
Reviewed-on: https://chromium-review.googlesource.com/999129
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Cr-Commit-Position: refs/heads/master@{#549013}
[modify] https://crrev.com/736e26fa2b93318635af46024207ffe55b1f2390/testing/buildbot/chromium.mac.json
[modify] https://crrev.com/736e26fa2b93318635af46024207ffe55b1f2390/testing/buildbot/chromium.webkit.json
[modify] https://crrev.com/736e26fa2b93318635af46024207ffe55b1f2390/testing/buildbot/test_suite_exceptions.pyl
[modify] https://crrev.com/736e26fa2b93318635af46024207ffe55b1f2390/testing/buildbot/waterfalls.pyl

Ping from the ecosystem-infra sheriff. There's been no activity in this bug for a month; can we downgrade its priority?
Labels: -Pri-1 Pri-2
Calling it a P2.
P2 is fine.
And, the ecosystem-infra sheriff is back!

dpranke@chromium.org does this align well enough any objectives this quarter to get assigned?
Blockedon: 790265 390309 790303 864817 790340 790278 790343 790286 790294
There was a bunch of progress on this in Jun, and I think we'll get the rest done in the next week or two. This is being done as part of the migration to LUCI, so I'll hang the bugs for the individual builders on this.
Blockedon: 790309
Cc: hinoka@chromium.org
Owner: dpranke@chromium.org
Status: Assigned (was: Available)
As per discussion with dpranke. The following should be done for these builders. Builder tracking bugs have been assigned via round robin. 

WebKit Mac10.13 (retina)  -> move to chromium.mac, trigger it from the existing "Mac Builder"
WebKit Win10              -> move to chromium.win, trigger it from existing "Win Builder"
WebKit  Win Builder       -> this can be shut off when the Win10 tests are moved as per above
WebKit Linux Trusty MSAN  ->  move to chromium.memory waterfall
WebKit Mac Builder        -> this can be shut off when the retina tests are moved
WebKit Linux Trusty Leak  -> move to chromium.memory
WebKit Linux Trusty ASAN  -> move to chromium.memory
Android Builder           -> this can be shut off when the nexus4 bot goes away (below)
WebKit Android (Nexus4)   -> jbudorick is moving these tests to android-kitkat-arm-rel as part of the work for crbug.com/824539
Cc: -efoo@chromium.org
Owner: efoo@chromium.org
@efoo - since you moved all of the actual builder migration bugs away from me, can I punt this bug over to you?
Project Member

Comment 25 by bugdroid1@chromium.org, Oct 4

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/3ebc822ac5ea701e4806f0bd6d333a8d832f724d

commit 3ebc822ac5ea701e4806f0bd6d333a8d832f724d
Author: Nodir Turakulov <nodir@google.com>
Date: Thu Oct 04 22:57:50 2018

[buildbucket] Untie migration app mastername

Users need to be able to specify different values for "mastername" property,
consumed by recipes, mastername used in luci-migration.appspot.com.
Add a custom property for the migration app's master name, and fallback to
standard "mastername" property.

Bug:  818301 
Change-Id: If20f39aa88ce0f5e3f16b1a0d72348a331af3a67
Reviewed-on: https://chromium-review.googlesource.com/c/1262931
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Auto-Submit: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#18068}
[modify] https://crrev.com/3ebc822ac5ea701e4806f0bd6d333a8d832f724d/appengine/cr-buildbucket/swarming/swarming.py
[modify] https://crrev.com/3ebc822ac5ea701e4806f0bd6d333a8d832f724d/appengine/cr-buildbucket/swarming/test/swarming_test.py

Project Member

Comment 26 by bugdroid1@chromium.org, Oct 4

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3ab24bfea119887de9b7474ac4c11af1fb1ca8d3

commit 3ab24bfea119887de9b7474ac4c11af1fb1ca8d3
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Thu Oct 04 23:22:28 2018

Mark 'WebKit Mac 10.13 (retina)' as being migrated from chromium.webkit.

On LUCI this builder is on 'chromium.mac' master, so by default Buildbucket
looks for its migration status on (chromium.mac, WebKit Mac 10.13 (retina))
page, which doesn't exist (because there are no such Buildbot builder).

With this change, Buildbucket will look at correct Buildbot builder instead.

R=jchinlee@chromium.org, nodir@chromium.org
BUG= 818301 , 864817

Change-Id: Id821857a971e36d1da9b2437950efb39f2c27cb4
Reviewed-on: https://chromium-review.googlesource.com/c/1263343
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Jao-ke Chin-Lee <jchinlee@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#596917}
[modify] https://crrev.com/3ab24bfea119887de9b7474ac4c11af1fb1ca8d3/infra/config/global/cr-buildbucket.cfg

Project Member

Comment 27 by bugdroid1@chromium.org, Oct 5

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/9c8dd25f8884c59745524e5c105bfc2b2c76d254

commit 9c8dd25f8884c59745524e5c105bfc2b2c76d254
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Fri Oct 05 01:08:38 2018

Put a copy of 'WebKit Mac10.13 (retina)' into chromium.mac.

These configs will be used from LUCI builders. Old configs can be removed all
at once when chromium.webkit Buildbot master is brought down.

R=dpranke@chromium.org, jchinlee@chromium.org
BUG=864817,  818301 

Change-Id: Iddc309c9fcd7b5fdce168171524ed051bada9e0e
Reviewed-on: https://chromium-review.googlesource.com/c/1262974
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Cr-Commit-Position: refs/heads/master@{#596952}
[modify] https://crrev.com/9c8dd25f8884c59745524e5c105bfc2b2c76d254/testing/buildbot/chromium.mac.json
[modify] https://crrev.com/9c8dd25f8884c59745524e5c105bfc2b2c76d254/testing/buildbot/waterfalls.pyl
[modify] https://crrev.com/9c8dd25f8884c59745524e5c105bfc2b2c76d254/tools/mb/mb_config.pyl

Owner: jchin...@chromium.org
Assigning to jchinlee since she will be shutting down the master once the Linux builder issues are resolved. 
Cc: efoo@chromium.org
Project Member

Comment 30 by bugdroid1@chromium.org, Dec 19

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/e344b825ce176ac87ee11002379f235eae7f436e

commit e344b825ce176ac87ee11002379f235eae7f436e
Author: Jao-ke Chin-Lee <jchinlee@chromium.org>
Date: Wed Dec 19 20:28:24 2018

Remove WebKit builders and reorganize console view.

BUG= 818301 ,790309,790294,790286,907562

Change-Id: Ia6a8617575d650f81f5367725d05b97b95e2c3b4
Reviewed-on: https://chromium-review.googlesource.com/c/1381717
Commit-Queue: Jao-ke Chin-Lee <jchinlee@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Cr-Commit-Position: refs/heads/master@{#617930}
[modify] https://crrev.com/e344b825ce176ac87ee11002379f235eae7f436e/infra/config/global/luci-milo.cfg

Project Member

Comment 31 by bugdroid1@chromium.org, Dec 19

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/9daa3ffd692bd1e09f1adc094179988a888cd79d

commit 9daa3ffd692bd1e09f1adc094179988a888cd79d
Author: Jao-ke Chin-Lee <jchinlee@chromium.org>
Date: Wed Dec 19 20:31:28 2018

Revert "Remove WebKit builders and reorganize console view."

This reverts commit e344b825ce176ac87ee11002379f235eae7f436e.

Reason for revert: "error console not found"

Original change's description:
> Remove WebKit builders and reorganize console view.
> 
> BUG= 818301 ,790309,790294,790286,907562
> 
> Change-Id: Ia6a8617575d650f81f5367725d05b97b95e2c3b4
> Reviewed-on: https://chromium-review.googlesource.com/c/1381717
> Commit-Queue: Jao-ke Chin-Lee <jchinlee@chromium.org>
> Reviewed-by: John Budorick <jbudorick@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#617930}

TBR=jchinlee@chromium.org,jbudorick@chromium.org

Change-Id: Ia14ffb037ed036aea47b29017244bff45a2db4c4
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug:  818301 , 790309, 790294, 790286, 907562
Reviewed-on: https://chromium-review.googlesource.com/c/1385009
Reviewed-by: Jao-ke Chin-Lee <jchinlee@chromium.org>
Commit-Queue: Jao-ke Chin-Lee <jchinlee@chromium.org>
Cr-Commit-Position: refs/heads/master@{#617934}
[modify] https://crrev.com/9daa3ffd692bd1e09f1adc094179988a888cd79d/infra/config/global/luci-milo.cfg

Project Member

Comment 32 by bugdroid1@chromium.org, Dec 19

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c59af03d523766b9b34c29335b2df172de3cc14d

commit c59af03d523766b9b34c29335b2df172de3cc14d
Author: Jao-ke Chin-Lee <jchinlee@chromium.org>
Date: Wed Dec 19 21:17:59 2018

Reland "Remove WebKit builders and reorganize console view."

This reverts commit 9daa3ffd692bd1e09f1adc094179988a888cd79d.

Reason for revert: fix dangling reference

Original change's description:
> Revert "Remove WebKit builders and reorganize console view."
> 
> This reverts commit e344b825ce176ac87ee11002379f235eae7f436e.
> 
> Reason for revert: "error console not found"
> 
> Original change's description:
> > Remove WebKit builders and reorganize console view.
> > 
> > BUG= 818301 ,790309,790294,790286,907562
> > 
> > Change-Id: Ia6a8617575d650f81f5367725d05b97b95e2c3b4
> > Reviewed-on: https://chromium-review.googlesource.com/c/1381717
> > Commit-Queue: Jao-ke Chin-Lee <jchinlee@chromium.org>
> > Reviewed-by: John Budorick <jbudorick@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#617930}
> 
> TBR=jchinlee@chromium.org,jbudorick@chromium.org
> 
> Change-Id: Ia14ffb037ed036aea47b29017244bff45a2db4c4
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Bug:  818301 , 790309, 790294, 790286, 907562
> Reviewed-on: https://chromium-review.googlesource.com/c/1385009
> Reviewed-by: Jao-ke Chin-Lee <jchinlee@chromium.org>
> Commit-Queue: Jao-ke Chin-Lee <jchinlee@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#617934}

TBR=jchinlee@chromium.org,jbudorick@chromium.org

Change-Id: I2e4efb4fc923a4da5cdb87e24115d028084a31c8
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug:  818301 , 790309, 790294, 790286, 907562
Reviewed-on: https://chromium-review.googlesource.com/c/1384991
Reviewed-by: Jao-ke Chin-Lee <jchinlee@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Jao-ke Chin-Lee <jchinlee@chromium.org>
Cr-Commit-Position: refs/heads/master@{#617947}
[modify] https://crrev.com/c59af03d523766b9b34c29335b2df172de3cc14d/infra/config/global/cr-buildbucket.cfg
[modify] https://crrev.com/c59af03d523766b9b34c29335b2df172de3cc14d/infra/config/global/luci-milo.cfg

Project Member

Comment 33 by bugdroid1@chromium.org, Dec 20

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/master-manager/+/12978b7eea34ebc58c594f1e585a57e08ebd9d25

commit 12978b7eea34ebc58c594f1e585a57e08ebd9d25
Author: Jao-ke Chin-Lee <jchinlee@google.com>
Date: Thu Dec 20 01:34:15 2018

Project Member

Comment 34 by bugdroid1@chromium.org, Dec 20

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/a26a582c17d3352ce9eb3c7cc3dea9cdd055e6f3

commit a26a582c17d3352ce9eb3c7cc3dea9cdd055e6f3
Author: Jao-ke Chin-Lee <jchinlee@chromium.org>
Date: Thu Dec 20 02:01:57 2018

Remove chromium.webkit master.

Bug:  818301 ,790309,790294,790286,907562
Change-Id: I485021b3fcfd8c29e18fa4f360d9d48884631d88
Reviewed-on: https://chromium-review.googlesource.com/c/1381712
Reviewed-by: John Budorick <jbudorick@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Jao-ke Chin-Lee <jchinlee@chromium.org>

[delete] https://crrev.com/85ec97a85ef6ad642afb6481a495797eacc25a47/masters/master.chromium.webkit/templates/tree_status.html
[modify] https://crrev.com/a26a582c17d3352ce9eb3c7cc3dea9cdd055e6f3/tests/masters_test.py
[delete] https://crrev.com/85ec97a85ef6ad642afb6481a495797eacc25a47/masters/master.chromium.webkit/templates/change_macros.html
[delete] https://crrev.com/85ec97a85ef6ad642afb6481a495797eacc25a47/masters/master.chromium.webkit/slaves.cfg
[delete] https://crrev.com/85ec97a85ef6ad642afb6481a495797eacc25a47/masters/master.chromium.webkit/master_source_cfg.py
[delete] https://crrev.com/85ec97a85ef6ad642afb6481a495797eacc25a47/masters/master.chromium.webkit/Makefile
[delete] https://crrev.com/85ec97a85ef6ad642afb6481a495797eacc25a47/masters/master.chromium.webkit/master_site_config.py
[delete] https://crrev.com/85ec97a85ef6ad642afb6481a495797eacc25a47/masters/master.chromium.webkit/master_linux_webkit_latest_cfg.py
[delete] https://crrev.com/85ec97a85ef6ad642afb6481a495797eacc25a47/masters/master.chromium.webkit/master.cfg

Alright, looks like this is almost finished :-D

We've been wanting to remove chromium.webkit for a long time, and now after the WebKit->blink rename, this master is one of the few things left with the name "webkit". Thanks to Jao-ke, Vadim, Nodir, & Dirk for the above work :-)
Status: Fixed (was: Assigned)

Sign in to add a comment