New issue
Advanced search Search tips

Issue 660481 link

Starred by 2 users

Issue metadata

Status: Duplicate
Merged: issue 731558
Owner:
Closed: Jun 2017
Cc:
Components:
EstimatedDays: 1
NextAction: ----
OS: ----
Pri: 1
Type: Feature

Blocked on:
issue 691137
issue 694680
issue 694693
issue 694698
issue 694787
issue 696697
issue 700052
issue 700058
issue 706224
issue 727917
issue 728415
issue 729811



Sign in to add a comment

make linux_chromium_rel_ng WAI on LUCI

Project Member Reported by no...@chromium.org, Oct 28 2016

Issue description

Key result: we can schedule linux_chromium_rel_ng build on luci and it WAI within ~equivalent amount of time.

I've defined experimental builder in https://chromium.googlesource.com/infra/experimental/+/infra/config/cr-buildbucket-dev.cfg#174
and using [1] to schedule builds

[1]: https://apis-explorer.appspot.com/apis-explorer/?base=https://cr-buildbucket-dev.appspot.com/_ah/api#p/buildbucket/v1/buildbucket.put?_h=6&resource=%257B%250A++%2522bucket%2522%253A+%2522nodir%2522%252C%250A++%2522parameters_json%2522%253A+%2522%257B%255C%2522builder_name%255C%2522%253A+%255C%2522linux_chromium_rel_ng%255C%2522%252C+%255C%2522changes%255C%2522%253A+%255B%257B%255C%2522author%255C%2522%253A+%257B%255C%2522email%255C%2522%253A+%255C%2522brettw%2540chromium.org%255C%2522%257D%257D%255D%252C+%255C%2522properties%255C%2522%253A+%257B%255C%2522attempt_start_ts%255C%2522%253A+1477675516461730%252C+%255C%2522category%255C%2522%253A+%255C%2522cq%255C%2522%252C+%255C%2522issue%255C%2522%253A+%255C%25222460653003%255C%2522%252C+%255C%2522master%255C%2522%253A+%255C%2522master.tryserver.chromium.linux%255C%2522%252C+%255C%2522patch_project%255C%2522%253A+%255C%2522chromium%255C%2522%252C+%255C%2522patch_storage%255C%2522%253A+%255C%2522rietveld%255C%2522%252C+%255C%2522patchset%255C%2522%253A+%255C%25221%255C%2522%252C+%255C%2522reason%255C%2522%253A+%255C%2522CQ%255C%2522%252C+%255C%2522revision%255C%2522%253A+%255C%2522HEAD%255C%2522%252C+%255C%2522rietveld%255C%2522%253A+%255C%2522https%253A%252F%252Fcodereview.chromium.org%255C%2522%257D%257D%2522%250A%257D&

first version clones entire chromium repo just to run the recipe https://chromium-swarm-dev.appspot.com/task?id=32259ee9beed9d10&refresh=10&show_raw=1
 

Comment 1 by estaab@chromium.org, Oct 28 2016

Shouldn't this be using an ACL'ed pool so we can keep state? I'm in the process of setting that up still.

Comment 2 by no...@chromium.org, Oct 28 2016

yes, we should limit number of machines that this builder will run on. For this experimental builder I've limited it to swarm1-c4 using id dimension 
https://chromium.googlesource.com/infra/experimental/+/9c23951bf669d209f2296745f93f5578c4873e0f%5E%21/#F0

Comment 3 by estaab@chromium.org, Nov 21 2016

Owner: estaab@chromium.org
Status: Assigned (was: Started)

Comment 4 by estaab@chromium.org, Dec 29 2016

Issue 676440 has been merged into this issue.

Comment 5 by estaab@chromium.org, Dec 29 2016

Owner: d...@chromium.org
Status: Started (was: Assigned)

Comment 6 by estaab@chromium.org, Dec 29 2016

For future reference Nodir's experimental builder ran once here:
https://luci-milo.appspot.com/buildbucket/nodir/linux_chromium_rel_ng?server=cr-buildbucket-dev.appspot.com

I don't think it will work now since the recipe moved back to build but it should be easy to fix.
Project Member

Comment 7 by bugdroid1@chromium.org, Dec 30 2016

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config.git/+/746bbf32182c93a77deebe1084a86037fc871a4e

commit 746bbf32182c93a77deebe1084a86037fc871a4e
Author: Dan Jacques <dnj@chromium.org>
Date: Fri Dec 30 00:21:22 2016

Comment 8 by no...@chromium.org, Feb 6 2017

Cc: d...@chromium.org
Owner: no...@chromium.org
Project Member

Comment 9 by bugdroid1@chromium.org, Feb 7 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/205490f5a8f85cb70535d3b67b3cbc76206953e1

commit 205490f5a8f85cb70535d3b67b3cbc76206953e1
Author: nodir <nodir@chromium.org>
Date: Tue Feb 07 00:01:50 2017

Add experimental LUCI linux_chromium_rel_ng buidler

Run the builder on 1% of CLs

R=estaab@chromium.org, hinoka@chromium.org, dpranke@chromium.org
BUG= 660481 
NOTRY=true
NOPRESUBMIT=true

Review-Url: https://codereview.chromium.org/2679023002
Cr-Commit-Position: refs/heads/master@{#448451}

[modify] https://crrev.com/205490f5a8f85cb70535d3b67b3cbc76206953e1/infra/config/cq.cfg

Project Member

Comment 10 by bugdroid1@chromium.org, Feb 10 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/bc972209d2c28e9b85c2c926f3b9a4508026b218

commit bc972209d2c28e9b85c2c926f3b9a4508026b218
Author: nodir <nodir@chromium.org>
Date: Fri Feb 10 23:15:26 2017

cq.cfg: fix bucket name luci.chromium.try

cq.cfg specified luci.infra.try instead of luci.chromium.try

R=vadimsh@chromium.org
BUG= 660481 
NOTRY=true
NOPRESUBMIT=true

Review-Url: https://codereview.chromium.org/2693633002
Cr-Commit-Position: refs/heads/master@{#449779}

[modify] https://crrev.com/bc972209d2c28e9b85c2c926f3b9a4508026b218/infra/config/cq.cfg

Comment 11 by no...@chromium.org, Feb 10 2017

Blockedon: 691137

Comment 12 by no...@chromium.org, Feb 14 2017

Labels: Milestone-LinuxChromiumRelNg
Project Member

Comment 13 by bugdroid1@chromium.org, Feb 16 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/473b7382864a88e0850b2e2183dc09b39b3461fc

commit 473b7382864a88e0850b2e2183dc09b39b3461fc
Author: Nodir Turakulov <nodir@chromium.org>
Date: Thu Feb 16 23:50:55 2017

introduce bot_id and builder_id build properties

"slavename" property name is buildbot specific.
Introduce "bot_id" which has the same value.

"mastername" and "buildername" property names are buildbot-specific.
Introduce abstract "builder_id" name which is their combination.

Recipes will be updated to use bot_id and builder_id

R=iannucci@chromium.org, dnj@chromium.org
BUG= 660481 

Change-Id: I2fbad8b299eae79a6402609a8a825aa5e7b04904
Reviewed-on: https://chromium-review.googlesource.com/444207
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

[modify] https://crrev.com/473b7382864a88e0850b2e2183dc09b39b3461fc/scripts/slave/unittests/annotated_run_test.py
[modify] https://crrev.com/473b7382864a88e0850b2e2183dc09b39b3461fc/scripts/slave/annotated_run.py
[modify] https://crrev.com/473b7382864a88e0850b2e2183dc09b39b3461fc/scripts/slave/remote_run.py

We didn't talk about getting rid of master and builder concepts. Things like test-results need them as separate fields. Is combining them a blocker for getting off of buildbot?
Here are examples of where we would need to change the data model of another service if we drop master:
https://cs.chromium.org/chromium/infra/go/src/infra/appengine/test-results/model/test_file.go?l=74
https://chromium.googlesource.com/infra/infra/+/1ea5a72228409e908ce307e56a9e43fa5f1d6006/go/src/infra/monitoring/dispatcher/dispatcher.go#139

I'd prefer not to refactor these services as a prerequisite for the migration.

Comment 16 by no...@chromium.org, Feb 21 2017

yes, refactoring of these services must not be in this milestone

Comment 17 by no...@chromium.org, Feb 21 2017

Blockedon: 694680

Comment 18 by no...@chromium.org, Feb 21 2017

Blockedon: 694693

Comment 19 by no...@chromium.org, Feb 21 2017

Blockedon: 694698
Project Member

Comment 20 by bugdroid1@chromium.org, Feb 21 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/bae38ee5bba910f30733f2f82091fefa53acfac2

commit bae38ee5bba910f30733f2f82091fefa53acfac2
Author: Nodir Turakulov <nodir@chromium.org>
Date: Tue Feb 21 22:13:43 2017

kitchen: provide bot_id property

Read it from the SWARMING_BOT_ID environment variable.

R=dnj@chromium.org
BUG= 660481 

Change-Id: I5698f98c8493554572bd5f0b5d18231048ae48f8
Reviewed-on: https://chromium-review.googlesource.com/441946
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Daniel Jacques <dnj@chromium.org>

[modify] https://crrev.com/bae38ee5bba910f30733f2f82091fefa53acfac2/go/src/infra/tools/kitchen/cook.go
[modify] https://crrev.com/bae38ee5bba910f30733f2f82091fefa53acfac2/go/src/infra/tools/kitchen/cook_mode.go
[modify] https://crrev.com/bae38ee5bba910f30733f2f82091fefa53acfac2/go/src/infra/tools/kitchen/kitchen.go

Comment 21 by no...@chromium.org, Feb 21 2017

Blockedon: 694787

Comment 22 by no...@chromium.org, Feb 23 2017

new challenge is chromium_try's dependency on buildnumber property
What's the plan with buildernumber next?

Comment 24 by no...@chromium.org, Feb 24 2017

The plan is to introduce "swarming_run_id" property at the kitchen level and update recipes to use that if buildnumber is not available. Currently builds are failing because bulidnumber cannot be passed to scripts, such as "get compile targets".

Comment 25 by no...@chromium.org, Feb 27 2017

Blockedon: 696697
Project Member

Comment 26 by bugdroid1@chromium.org, Mar 2 2017

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config/+/ab4a315b52b606b76524e55d2ac471deac6a0431

commit ab4a315b52b606b76524e55d2ac471deac6a0431
Author: Nodir Turakulov <nodir@google.com>
Date: Thu Mar 02 17:57:53 2017

Project Member

Comment 27 by bugdroid1@chromium.org, Mar 6 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/698d4074084901a28bab0bfcbfb91de7ab882c05

commit 698d4074084901a28bab0bfcbfb91de7ab882c05
Author: Nodir Turakulov <nodir@chromium.org>
Date: Mon Mar 06 21:20:23 2017

kitchen: transform properties for migraiton

Simplify chromium's migration to LUCI by transforming property values at
the kitchen level. Clearly mark this code as migration code by putting
to a separate package "migration".

R=dnj@chromium.org
BUG= 660481 

Change-Id: I2bfc615645040f458ddb3a269ed6060b01bd5841
Reviewed-on: https://chromium-review.googlesource.com/450123
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Daniel Jacques <dnj@chromium.org>

[modify] https://crrev.com/698d4074084901a28bab0bfcbfb91de7ab882c05/go/src/infra/tools/kitchen/cook.go
[add] https://crrev.com/698d4074084901a28bab0bfcbfb91de7ab882c05/go/src/infra/tools/kitchen/migration/migration.go
[add] https://crrev.com/698d4074084901a28bab0bfcbfb91de7ab882c05/go/src/infra/tools/kitchen/migration/migration_test.go

Project Member

Comment 28 by bugdroid1@chromium.org, Mar 7 2017

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config/+/7077baff956df6ec29329809711b2d95fb8b43eb

commit 7077baff956df6ec29329809711b2d95fb8b43eb
Author: Nodir Turakulov <nodir@google.com>
Date: Tue Mar 07 18:20:39 2017

Project Member

Comment 29 by bugdroid1@chromium.org, Mar 7 2017

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config/+/7077baff956df6ec29329809711b2d95fb8b43eb

commit 7077baff956df6ec29329809711b2d95fb8b43eb
Author: Nodir Turakulov <nodir@google.com>
Date: Tue Mar 07 18:20:39 2017

Blockedon: 700052
Blockedon: 700058
Project Member

Comment 32 by bugdroid1@chromium.org, Mar 16 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/371c08762aeafbee0b0cc79c48f2f6c727ccecbe

commit 371c08762aeafbee0b0cc79c48f2f6c727ccecbe
Author: Nodir Turakulov <nodir@chromium.org>
Date: Thu Mar 16 20:38:20 2017

chromium_tests: remove luci.chromium.try

kitchen automatically converts mastername="luci.chromium.try"  and
buildername="LUCI linux_chromium_rel_ng" to
mastername="tryserver.chromium.linux" and
buildername="linux_chromium_rel_ng" respectively, to simpify the
transition. Remove luci.chromium.try entry from trybots.py

R=dnj@chromium.org, estaab@chromium.org
BUG= 660481 

Change-Id: Ia984f4b6ec78cb1352df10bb88b20e9d5bb0a3d3
Reviewed-on: https://chromium-review.googlesource.com/452698
Reviewed-by: Stephen Martinis <martiniss@chromium.org>
Reviewed-by: Erik Staab <estaab@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/371c08762aeafbee0b0cc79c48f2f6c727ccecbe/scripts/slave/recipe_modules/chromium_tests/trybots.py
[delete] https://crrev.com/c409d079b174aa0446ab6dd1d0247d03b586ae68/scripts/slave/recipes/chromium_trybot.expected/full_luci_chromium_try_LUCI_linux_chromium_rel_ng.json

Project Member

Comment 33 by bugdroid1@chromium.org, Mar 17 2017

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config/+/949d2ae26be650725a8c4e91f736ca96db5e5b43

commit 949d2ae26be650725a8c4e91f736ca96db5e5b43
Author: Nodir Turakulov <nodir@google.com>
Date: Fri Mar 17 16:43:13 2017

Project Member

Comment 34 by bugdroid1@chromium.org, Mar 17 2017

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config/+/e914320bed91cad7c7ec06db776dbea92bc1f689

commit e914320bed91cad7c7ec06db776dbea92bc1f689
Author: Nodir Turakulov <nodir@google.com>
Date: Fri Mar 17 18:40:32 2017

Project Member

Comment 35 by bugdroid1@chromium.org, Mar 17 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/fa444399668b881f3a2d813428bdc2c3026c5a16

commit fa444399668b881f3a2d813428bdc2c3026c5a16
Author: Nodir Turakulov <nodir@chromium.org>
Date: Fri Mar 17 19:20:46 2017

buildbucket: provide builder_hash template parameter

Will be used to in a name of cache for this builder. This is better
version of
https://chrome-internal-review.googlesource.com/c/338068/

R=estaab@chromium.org, vadimsh@chromium.org
BUG= 660481 

Change-Id: I0c771fc869bfa4b84627abd2b650f21d86dc11d0
Reviewed-on: https://chromium-review.googlesource.com/457116
Reviewed-by: Erik Staab <estaab@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/fa444399668b881f3a2d813428bdc2c3026c5a16/appengine/cr-buildbucket/swarming/swarming.py

Project Member

Comment 36 by bugdroid1@chromium.org, Mar 17 2017

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config/+/588746874466aa74cd6b1a9443e24af577922399

commit 588746874466aa74cd6b1a9443e24af577922399
Author: Nodir Turakulov <nodir@google.com>
Date: Fri Mar 17 19:23:37 2017

Project Member

Comment 37 by bugdroid1@chromium.org, Mar 17 2017

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config/+/588746874466aa74cd6b1a9443e24af577922399

commit 588746874466aa74cd6b1a9443e24af577922399
Author: Nodir Turakulov <nodir@google.com>
Date: Fri Mar 17 19:23:37 2017

Project Member

Comment 38 by bugdroid1@chromium.org, Mar 17 2017

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config/+/a4bff95fee8cf705d2bd465dc915199d3a147a52

commit a4bff95fee8cf705d2bd465dc915199d3a147a52
Author: Nodir Turakulov <nodir@google.com>
Date: Fri Mar 17 21:37:11 2017

Project Member

Comment 39 by bugdroid1@chromium.org, Mar 21 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/depot_tools/+/e718958ca03fc0766459fecf7820a300b206d7be

commit e718958ca03fc0766459fecf7820a300b206d7be
Author: Nodir Turakulov <nodir@chromium.org>
Date: Tue Mar 21 22:29:49 2017

define builder_cache base path

chromium_tests recipe module currently checks if builder_cache base path
is defined. If it is not, it let's the caller to decide where to put
the cache. All the callers choose start_dir.
https://cs.chromium.org/search/?q=get_checkout_dir&type=cs

In LUCI we want to use [CACHE]/builder as builder cache. To do that we
want to update chromium_tests recipe module to use [CACHE]/builder if
builder_cache is not defined.

Define builder_cache in buildbot to point to start dir, so that in LUCI
it is not defined and thus [CACHE]/builder is used.

R=iannucci@chromium.org
BUG= 660481 

Change-Id: I9018da5c43a5396add6a26ad2c12b1755fbe6307
Reviewed-on: https://chromium-review.googlesource.com/457439
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/e718958ca03fc0766459fecf7820a300b206d7be/recipe_modules/infra_paths/path_config.py

Project Member

Comment 40 by bugdroid1@chromium.org, Mar 22 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/depot_tools/+/14db1dfd88b987c9119f59f75f1f27b8e7c49602

commit 14db1dfd88b987c9119f59f75f1f27b8e7c49602
Author: Sergey Berezin <sergeyberezin@chromium.org>
Date: Wed Mar 22 01:05:24 2017

Revert "define builder_cache base path"

This reverts commit e718958ca03fc0766459fecf7820a300b206d7be.

Reason for revert: Suspected to have caused an outage http://o/370111
It seems to have shifted the current dir to a new place, created another checkout and some bots with small disk space ran out of disk.

Original change's description:
> define builder_cache base path
> 
> chromium_tests recipe module currently checks if builder_cache base path
> is defined. If it is not, it let's the caller to decide where to put
> the cache. All the callers choose start_dir.
> https://cs.chromium.org/search/?q=get_checkout_dir&type=cs
> 
> In LUCI we want to use [CACHE]/builder as builder cache. To do that we
> want to update chromium_tests recipe module to use [CACHE]/builder if
> builder_cache is not defined.
> 
> Define builder_cache in buildbot to point to start dir, so that in LUCI
> it is not defined and thus [CACHE]/builder is used.
> 
> R=​iannucci@chromium.org
> BUG= 660481 
> 
> Change-Id: I9018da5c43a5396add6a26ad2c12b1755fbe6307
> Reviewed-on: https://chromium-review.googlesource.com/457439
> Reviewed-by: Ryan Tseng <hinoka@chromium.org>
> Commit-Queue: Nodir Turakulov <nodir@chromium.org>
> 

TBR=iannucci@chromium.org,hinoka@chromium.org,estaab@chromium.org,nodir@chromium.org,chromium-reviews@chromium.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG= 660481 

Change-Id: Ia239ebe12a5bea1987c2ee23e543460efe753c81
Reviewed-on: https://chromium-review.googlesource.com/457778
Reviewed-by: Sergey Berezin <sergeyberezin@chromium.org>
Commit-Queue: Sergey Berezin <sergeyberezin@chromium.org>

[modify] https://crrev.com/14db1dfd88b987c9119f59f75f1f27b8e7c49602/recipe_modules/infra_paths/path_config.py

Project Member

Comment 41 by bugdroid1@chromium.org, Mar 27 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/depot_tools/+/8b5b594115480a6b4c4bbe384cc7afaecd814a4b

commit 8b5b594115480a6b4c4bbe384cc7afaecd814a4b
Author: Nodir Turakulov <nodir@chromium.org>
Date: Mon Mar 27 19:33:52 2017

add "generic" infra path config

It proved to be dangerous to introduce conditional logic into recipes
and recipe modules to alternate between buildbot and luci modes, namely
it touches buildbot code paths, which can be very dangerous.

Instead, introduce "generic" default path config that configures
existing base paths from the built-in ones.

Also remove swarmbucket path config because it is not used.

R=iannucci@chromium.org
BUG= 660481 

Change-Id: Ide0e1f64913fe35766b997ab5b55f01f4c63aa58
Reviewed-on: https://chromium-review.googlesource.com/459234
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

[add] https://crrev.com/8b5b594115480a6b4c4bbe384cc7afaecd814a4b/recipes/recipe_modules/infra_paths/example.expected/paths_generic_linux.json
[add] https://crrev.com/8b5b594115480a6b4c4bbe384cc7afaecd814a4b/recipes/recipe_modules/infra_paths/example.expected/paths_generic_win.json
[delete] https://crrev.com/d2db5a2240eaa5e661884314911689b0d022d5d9/recipes/recipe_modules/infra_paths/example.expected/paths_swarmbucket_linux.json
[modify] https://crrev.com/8b5b594115480a6b4c4bbe384cc7afaecd814a4b/recipes/recipe_modules/gclient/example.expected/basic.json
[add] https://crrev.com/8b5b594115480a6b4c4bbe384cc7afaecd814a4b/recipe_modules/infra_paths/example.expected/paths_generic_linux.json
[modify] https://crrev.com/8b5b594115480a6b4c4bbe384cc7afaecd814a4b/recipes/recipe_modules/infra_paths/path_config.py
[add] https://crrev.com/8b5b594115480a6b4c4bbe384cc7afaecd814a4b/recipe_modules/infra_paths/example.expected/paths_generic_mac.json
[modify] https://crrev.com/8b5b594115480a6b4c4bbe384cc7afaecd814a4b/recipes/recipe_modules/infra_paths/api.py
[delete] https://crrev.com/d2db5a2240eaa5e661884314911689b0d022d5d9/recipes/recipe_modules/infra_paths/example.expected/paths_swarmbucket_win.json
[modify] https://crrev.com/8b5b594115480a6b4c4bbe384cc7afaecd814a4b/recipes/recipe_modules/infra_paths/example.expected/paths_kitchen_win.json
[modify] https://crrev.com/8b5b594115480a6b4c4bbe384cc7afaecd814a4b/recipes/recipe_modules/infra_paths/example.expected/paths_buildbot_linux.json
[modify] https://crrev.com/8b5b594115480a6b4c4bbe384cc7afaecd814a4b/recipes/recipe_modules/gclient/example.expected/tryserver.json
[modify] https://crrev.com/8b5b594115480a6b4c4bbe384cc7afaecd814a4b/recipes/recipe_modules/infra_paths/example.expected/paths_kitchen_mac.json
[modify] https://crrev.com/8b5b594115480a6b4c4bbe384cc7afaecd814a4b/recipes/recipe_modules/infra_paths/example.expected/basic.json
[delete] https://crrev.com/d2db5a2240eaa5e661884314911689b0d022d5d9/recipes/recipe_modules/infra_paths/example.expected/paths_swarmbucket_mac.json
[modify] https://crrev.com/8b5b594115480a6b4c4bbe384cc7afaecd814a4b/recipes/recipe_modules/git/example.expected/git-cache-checkout.json
[add] https://crrev.com/8b5b594115480a6b4c4bbe384cc7afaecd814a4b/recipe_modules/infra_paths/example.expected/paths_generic_win.json
[modify] https://crrev.com/8b5b594115480a6b4c4bbe384cc7afaecd814a4b/recipes/recipe_modules/infra_paths/example.expected/paths_kitchen_linux.json
[add] https://crrev.com/8b5b594115480a6b4c4bbe384cc7afaecd814a4b/recipes/recipe_modules/infra_paths/example.expected/paths_generic_mac.json
[modify] https://crrev.com/8b5b594115480a6b4c4bbe384cc7afaecd814a4b/recipes/recipe_modules/infra_paths/example.expected/paths_buildbot_win.json
[modify] https://crrev.com/8b5b594115480a6b4c4bbe384cc7afaecd814a4b/recipes/recipe_modules/gclient/example.expected/revision.json
[modify] https://crrev.com/8b5b594115480a6b4c4bbe384cc7afaecd814a4b/recipes/recipe_modules/infra_paths/example.py
[modify] https://crrev.com/8b5b594115480a6b4c4bbe384cc7afaecd814a4b/recipes/recipe_modules/infra_paths/example.expected/paths_buildbot_mac.json

Project Member

Comment 42 by bugdroid1@chromium.org, Mar 27 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/depot_tools/+/284e34ae2878ddd2cda671554b444db8512fa2ab

commit 284e34ae2878ddd2cda671554b444db8512fa2ab
Author: Nodir Turakulov <nodir@chromium.org>
Date: Mon Mar 27 20:20:40 2017

Revert "add "generic" infra path config"

This reverts commit 8b5b594115480a6b4c4bbe384cc7afaecd814a4b.

Reason for revert: nothing wrong with this CL, but implementing a manual roll to build.git takes more than 20 min. I will reland this CL once my roll to build.git CL is ready

Original change's description:
> add "generic" infra path config
> 
> It proved to be dangerous to introduce conditional logic into recipes
> and recipe modules to alternate between buildbot and luci modes, namely
> it touches buildbot code paths, which can be very dangerous.
> 
> Instead, introduce "generic" default path config that configures
> existing base paths from the built-in ones.
> 
> Also remove swarmbucket path config because it is not used.
> 
> R=​iannucci@chromium.org
> BUG= 660481 
> 
> Change-Id: Ide0e1f64913fe35766b997ab5b55f01f4c63aa58
> Reviewed-on: https://chromium-review.googlesource.com/459234
> Commit-Queue: Nodir Turakulov <nodir@chromium.org>
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> 

TBR=iannucci@chromium.org,nodir@chromium.org,chromium-reviews@chromium.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG= 660481 

Change-Id: If27a537689dfe3669015bcbc7a5f07ca1f5c10af
Reviewed-on: https://chromium-review.googlesource.com/461223
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>

[delete] https://crrev.com/8b5b594115480a6b4c4bbe384cc7afaecd814a4b/recipes/recipe_modules/infra_paths/example.expected/paths_generic_linux.json
[delete] https://crrev.com/8b5b594115480a6b4c4bbe384cc7afaecd814a4b/recipes/recipe_modules/infra_paths/example.expected/paths_generic_win.json
[add] https://crrev.com/284e34ae2878ddd2cda671554b444db8512fa2ab/recipes/recipe_modules/infra_paths/example.expected/paths_swarmbucket_linux.json
[modify] https://crrev.com/284e34ae2878ddd2cda671554b444db8512fa2ab/recipes/recipe_modules/gclient/example.expected/basic.json
[delete] https://crrev.com/8b5b594115480a6b4c4bbe384cc7afaecd814a4b/recipe_modules/infra_paths/example.expected/paths_generic_linux.json
[modify] https://crrev.com/284e34ae2878ddd2cda671554b444db8512fa2ab/recipes/recipe_modules/infra_paths/path_config.py
[delete] https://crrev.com/8b5b594115480a6b4c4bbe384cc7afaecd814a4b/recipe_modules/infra_paths/example.expected/paths_generic_mac.json
[modify] https://crrev.com/284e34ae2878ddd2cda671554b444db8512fa2ab/recipes/recipe_modules/infra_paths/api.py
[add] https://crrev.com/284e34ae2878ddd2cda671554b444db8512fa2ab/recipes/recipe_modules/infra_paths/example.expected/paths_swarmbucket_win.json
[modify] https://crrev.com/284e34ae2878ddd2cda671554b444db8512fa2ab/recipes/recipe_modules/infra_paths/example.expected/paths_kitchen_win.json
[modify] https://crrev.com/284e34ae2878ddd2cda671554b444db8512fa2ab/recipes/recipe_modules/infra_paths/example.expected/paths_buildbot_linux.json
[modify] https://crrev.com/284e34ae2878ddd2cda671554b444db8512fa2ab/recipes/recipe_modules/gclient/example.expected/tryserver.json
[modify] https://crrev.com/284e34ae2878ddd2cda671554b444db8512fa2ab/recipes/recipe_modules/infra_paths/example.expected/paths_kitchen_mac.json
[modify] https://crrev.com/284e34ae2878ddd2cda671554b444db8512fa2ab/recipes/recipe_modules/infra_paths/example.expected/basic.json
[add] https://crrev.com/284e34ae2878ddd2cda671554b444db8512fa2ab/recipes/recipe_modules/infra_paths/example.expected/paths_swarmbucket_mac.json
[modify] https://crrev.com/284e34ae2878ddd2cda671554b444db8512fa2ab/recipes/recipe_modules/git/example.expected/git-cache-checkout.json
[delete] https://crrev.com/8b5b594115480a6b4c4bbe384cc7afaecd814a4b/recipe_modules/infra_paths/example.expected/paths_generic_win.json
[modify] https://crrev.com/284e34ae2878ddd2cda671554b444db8512fa2ab/recipes/recipe_modules/infra_paths/example.expected/paths_kitchen_linux.json
[delete] https://crrev.com/8b5b594115480a6b4c4bbe384cc7afaecd814a4b/recipes/recipe_modules/infra_paths/example.expected/paths_generic_mac.json
[modify] https://crrev.com/284e34ae2878ddd2cda671554b444db8512fa2ab/recipes/recipe_modules/infra_paths/example.expected/paths_buildbot_win.json
[modify] https://crrev.com/284e34ae2878ddd2cda671554b444db8512fa2ab/recipes/recipe_modules/gclient/example.expected/revision.json
[modify] https://crrev.com/284e34ae2878ddd2cda671554b444db8512fa2ab/recipes/recipe_modules/infra_paths/example.py
[modify] https://crrev.com/284e34ae2878ddd2cda671554b444db8512fa2ab/recipes/recipe_modules/infra_paths/example.expected/paths_buildbot_mac.json

Project Member

Comment 43 by bugdroid1@chromium.org, Mar 27 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/depot_tools/+/757f20796948ff4a9264dad5293bdff2f94d9c43

commit 757f20796948ff4a9264dad5293bdff2f94d9c43
Author: Nodir Turakulov <nodir@google.com>
Date: Mon Mar 27 23:42:20 2017

Reland: add "generic" infra path config

It proved to be dangerous to introduce conditional logic into recipes
and recipe modules to alternate between buildbot and luci modes, namely
it touches buildbot code paths, which can be very dangerous.

Instead, introduce "generic" path config that configures
existing base paths from the built-in ones.

Also remove swarmbucket path config because it is not used.

--

This CL is a simpler version of
https://chromium-review.googlesource.com/c/459234/
in that it does not make generic path config a default.
This avoids behavioral change in downstream recipe packages and
makes rolling into build.git trivial.

R=iannucci@chromium.org
BUG= 660481 

Change-Id: Ifc147e29e68a6e0c800da73a8e356238dfc2938a
Reviewed-on: https://chromium-review.googlesource.com/461359
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

[modify] https://crrev.com/757f20796948ff4a9264dad5293bdff2f94d9c43/recipes/recipe_modules/infra_paths/example.expected/basic.json
[modify] https://crrev.com/757f20796948ff4a9264dad5293bdff2f94d9c43/recipes/recipe_modules/infra_paths/example.expected/paths_kitchen_win.json
[delete] https://crrev.com/f7023e7cf0a6606080d9f53c5a9e6e8271443914/recipes/recipe_modules/infra_paths/example.expected/paths_swarmbucket_mac.json
[modify] https://crrev.com/757f20796948ff4a9264dad5293bdff2f94d9c43/recipes/recipe_modules/infra_paths/example.expected/paths_buildbot_linux.json
[add] https://crrev.com/757f20796948ff4a9264dad5293bdff2f94d9c43/recipes/recipe_modules/infra_paths/example.expected/paths_generic_win.json
[modify] https://crrev.com/757f20796948ff4a9264dad5293bdff2f94d9c43/recipes/recipe_modules/infra_paths/example.expected/paths_kitchen_linux.json
[delete] https://crrev.com/f7023e7cf0a6606080d9f53c5a9e6e8271443914/recipes/recipe_modules/infra_paths/example.expected/paths_swarmbucket_linux.json
[add] https://crrev.com/757f20796948ff4a9264dad5293bdff2f94d9c43/recipes/recipe_modules/infra_paths/example.expected/paths_generic_mac.json
[modify] https://crrev.com/757f20796948ff4a9264dad5293bdff2f94d9c43/recipes/recipe_modules/infra_paths/example.expected/paths_buildbot_win.json
[add] https://crrev.com/757f20796948ff4a9264dad5293bdff2f94d9c43/recipes/recipe_modules/infra_paths/example.expected/paths_generic_linux.json
[modify] https://crrev.com/757f20796948ff4a9264dad5293bdff2f94d9c43/recipes/recipe_modules/infra_paths/example.py
[modify] https://crrev.com/757f20796948ff4a9264dad5293bdff2f94d9c43/recipes/recipe_modules/infra_paths/example.expected/paths_kitchen_mac.json
[modify] https://crrev.com/757f20796948ff4a9264dad5293bdff2f94d9c43/recipes/recipe_modules/infra_paths/example.expected/paths_buildbot_mac.json
[delete] https://crrev.com/f7023e7cf0a6606080d9f53c5a9e6e8271443914/recipes/recipe_modules/infra_paths/example.expected/paths_swarmbucket_win.json
[modify] https://crrev.com/757f20796948ff4a9264dad5293bdff2f94d9c43/recipes/recipe_modules/infra_paths/path_config.py

Comment 44 by no...@chromium.org, Mar 29 2017

Blockedon: 706224
Project Member

Comment 45 by bugdroid1@chromium.org, Mar 29 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/e59143d175d0182ddb529cfb1638a5b5b133a8a8

commit e59143d175d0182ddb529cfb1638a5b5b133a8a8
Author: Nodir Turakulov <nodir@chromium.org>
Date: Wed Mar 29 22:36:55 2017

kitchen: use "generic" path config

See https://chromium-review.googlesource.com/c/461359/

R=iannucci@chromium.org
BUG= 660481 

Change-Id: I058f28e08788884f36784e1303ff9af438cbd0c5
Reviewed-on: https://chromium-review.googlesource.com/461300
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

[modify] https://crrev.com/e59143d175d0182ddb529cfb1638a5b5b133a8a8/go/src/infra/tools/kitchen/cook.go

Project Member

Comment 46 by bugdroid1@chromium.org, Mar 31 2017

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config/+/7e6eb6b85b62038484693fd468bfe531fe5b4a5e

commit 7e6eb6b85b62038484693fd468bfe531fe5b4a5e
Author: Nodir Turakulov <nodir@google.com>
Date: Fri Mar 31 18:02:23 2017

Comment 47 by no...@chromium.org, Mar 31 2017

The builder now should reuse build artifacts, but goma cache (issue 706224) still does not work
All but one LUCI build was correct since build numbers were implemented on April 4. One that was not correct failed with BOT_DIED swarming error code.

This builder is correct, but 2.5x slower because of issue 706224.

Comment 49 by no...@chromium.org, Apr 21 2017

Summary: make linux_chromium_rel_ng WAI on LUCI (was: experimental linux_chromium_rel_ng builder)

Comment 50 by efoo@chromium.org, Apr 27 2017

EstimatedDays: 5
Labels: LUCI-M1-Dev1 REQBY-LUCI-M2-Dev2

Comment 51 by efoo@chromium.org, May 4 2017

Labels: LUCI-M1-S1
Assigned to M1-S1

Comment 52 by efoo@chromium.org, May 4 2017

EstimatedDays: 1

Comment 53 by efoo@chromium.org, May 23 2017

Labels: LUCI-M1-S2
Project Member

Comment 54 by bugdroid1@chromium.org, May 31 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/dd8cdf65cb2104e8650cb0cee45a728e77d54e8f

commit dd8cdf65cb2104e8650cb0cee45a728e77d54e8f
Author: nodir <nodir@chromium.org>
Date: Wed May 31 00:40:12 2017

LUCI linux_chromium_rel_ng: increase to 10%

increase experiment percentage to 10% to get reliable correctness data sooner

Bug:  660481 
Review-Url: https://codereview.chromium.org/2913083003
Cr-Commit-Position: refs/heads/master@{#475708}

[modify] https://crrev.com/dd8cdf65cb2104e8650cb0cee45a728e77d54e8f/infra/config/cq.cfg

Comment 55 by no...@chromium.org, May 31 2017

Blockedon: 728415
Blockedon: 727917
Blockedon: 729811

Comment 58 by efoo@chromium.org, Jun 9 2017

Labels: LUCI-M1-S3
Mergedinto: 731558
Status: Duplicate (was: Started)

Comment 60 by efoo@chromium.org, Jan 31 2018

Labels: LUCI-Dev2

Comment 61 by efoo@chromium.org, Jan 31 2018

Labels: -luci -LUCI-M1-Dev1 LUCI-Dev1

Comment 62 by efoo@chromium.org, Jan 31 2018

Labels: -LUCI-Dev2

Comment 63 by efoo@chromium.org, Feb 13 2018

Labels: LUCI-Dev2

Comment 64 by efoo@chromium.org, Feb 13 2018

Labels: -REQBY-LUCI-M2-Dev2 REQBY-LUCI-Dev2

Sign in to add a comment