New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 775326 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Last visit > 30 days ago
Closed: Jun 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug

Blocking:
issue 774703



Sign in to add a comment

Get dimensions of waterfall builder programatically

Project Member Reported by robert...@chromium.org, Oct 17 2017

Issue description

In order for us to trigger tryjobs for analyzing failures in any new builder without the need for manual configuration, we need a way to dynamically get the swarming dimensions that would match any waterfall builder.

This issue is for tracking such work.

 
Project Member

Comment 1 by bugdroid1@chromium.org, Oct 18 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/663fd82b36d1b981c5719915654a03ad1d6830ee

commit 663fd82b36d1b981c5719915654a03ad1d6830ee
Author: Roberto Carrillo <robertocn@google.com>
Date: Wed Oct 18 18:53:43 2017

[Findit] Function to get a bot's dimensions via swarmbucket api.

Once the 'luci.chromium.ci' bucket is set-up, this new function will
allow us to get the dimensions of a waterfall master so that we can
schedule tryjobs on those dimensions without relying on a handcrafted
configuration.

R=chanli,lijeffrey,stgao,wylieb,nodir,tandrii
BUG= 775326 

Change-Id: If5f8c32cb09d2f3765e79000b4658e2a748b9908
Reviewed-on: https://chromium-review.googlesource.com/722341
Reviewed-by: Chan Li <chanli@chromium.org>
Reviewed-by: Jeffrey Li <lijeffrey@chromium.org>
Commit-Queue: Roberto Carrillo <robertocn@chromium.org>

[add] https://crrev.com/663fd82b36d1b981c5719915654a03ad1d6830ee/appengine/findit/common/swarmbucket/swarmbucket.py
[add] https://crrev.com/663fd82b36d1b981c5719915654a03ad1d6830ee/appengine/findit/common/swarmbucket/__init__.py
[add] https://crrev.com/663fd82b36d1b981c5719915654a03ad1d6830ee/appengine/findit/common/swarmbucket/test/__init__.py
[add] https://crrev.com/663fd82b36d1b981c5719915654a03ad1d6830ee/appengine/findit/common/swarmbucket/test/swarmbucket_test.py

Project Member

Comment 2 by bugdroid1@chromium.org, Nov 28 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/18f03641b841e47103cc03707586e740882872d0

commit 18f03641b841e47103cc03707586e740882872d0
Author: Roberto Carrillo <robertocn@google.com>
Date: Tue Nov 28 23:01:53 2017

[Findit] Support 'use_swarmbucket' config for defaults.

This new setting will let us simplify the extensive configuration for
builder-to-trybot mapping, by using default
luci.chromium.try/findit_variable when set.

R=chanli,stgao
BUG= 775326 

Change-Id: Ic0355b46d6dac546c6c8ce99dae869f68c4c1e2b
Reviewed-on: https://chromium-review.googlesource.com/794343
Reviewed-by: Chan Li <chanli@chromium.org>
Reviewed-by: Shuotao Gao <stgao@chromium.org>
Commit-Queue: Roberto Carrillo <robertocn@chromium.org>

[modify] https://crrev.com/18f03641b841e47103cc03707586e740882872d0/appengine/findit/handlers/test/config_test.py
[modify] https://crrev.com/18f03641b841e47103cc03707586e740882872d0/appengine/findit/model/wf_config.py
[modify] https://crrev.com/18f03641b841e47103cc03707586e740882872d0/appengine/findit/handlers/config.py

Project Member

Comment 3 by bugdroid1@chromium.org, Jun 7 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/eade713d74b47556aeb6d97c3457fc9b1b9c0b43

commit eade713d74b47556aeb6d97c3457fc9b1b9c0b43
Author: Roberto Carrillo <robertocn@google.com>
Date: Thu Jun 07 20:47:31 2018

[Findit] Capture luci/buildbucket info about build failures.

This will be needed to get the tryjob dimensions when the original
failure happens on a luci builder.

R=chanli,stgao

Bug:  775326 
Change-Id: I0a48b945c9889f09d38e2d7473868c2fab31a8dd
Reviewed-on: https://chromium-review.googlesource.com/1089916
Commit-Queue: Roberto Carrillo <robertocn@chromium.org>
Reviewed-by: Shuotao Gao <stgao@chromium.org>
Reviewed-by: Chan Li <chanli@chromium.org>

[modify] https://crrev.com/eade713d74b47556aeb6d97c3457fc9b1b9c0b43/appengine/findit/waterfall/build_info.py
[modify] https://crrev.com/eade713d74b47556aeb6d97c3457fc9b1b9c0b43/appengine/findit/waterfall/test/buildbot_test.py
[modify] https://crrev.com/eade713d74b47556aeb6d97c3457fc9b1b9c0b43/appengine/findit/waterfall/buildbot.py
[modify] https://crrev.com/eade713d74b47556aeb6d97c3457fc9b1b9c0b43/appengine/findit/waterfall/test/data/completed_build.json

Project Member

Comment 4 by bugdroid1@chromium.org, Jun 13 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/120e4bcf22f7f422fa77b87812dba7d17df49062

commit 120e4bcf22f7f422fa77b87812dba7d17df49062
Author: Roberto Carrillo <robertocn@google.com>
Date: Wed Jun 13 00:08:03 2018

[Findit] Get trybot dimensions from swarmbucket api.

This change gets the trybot dimensions dynamically from buildbucket via
the swarmbucket api when the original failure happened on a luci bot.

Note that this does not yet make the tryjob necessarily happen on a
swarming trybot as this is still determined by the static mapping when
triggering the tryjob, but only gets the dimensions (which may be
ignored if the static mapping points to a buildbot trybot)

A later change will make the tryjobs be triggered on swarming if the
matching dimensions are available on Findit's swarming pool and the
static mapping doesn't force buildbot trybot.

R=chanli,stgao,lijeffrey,liaoyuke

Bug:  775326 
Change-Id: I9c113b1ff8db28cad6528a887f3ba26bd34a47d7
Reviewed-on: https://chromium-review.googlesource.com/1093297
Commit-Queue: Roberto Carrillo <robertocn@chromium.org>
Reviewed-by: Chan Li <chanli@chromium.org>

[modify] https://crrev.com/120e4bcf22f7f422fa77b87812dba7d17df49062/appengine/findit/services/compile_failure/test/compile_try_job_test.py
[modify] https://crrev.com/120e4bcf22f7f422fa77b87812dba7d17df49062/appengine/findit/services/ci_failure.py
[modify] https://crrev.com/120e4bcf22f7f422fa77b87812dba7d17df49062/appengine/findit/services/test/try_job_test.py
[modify] https://crrev.com/120e4bcf22f7f422fa77b87812dba7d17df49062/appengine/findit/common/swarmbucket/swarmbucket.py
[modify] https://crrev.com/120e4bcf22f7f422fa77b87812dba7d17df49062/appengine/findit/services/parameters.py
[modify] https://crrev.com/120e4bcf22f7f422fa77b87812dba7d17df49062/appengine/findit/services/test/ci_failure_test.py
[modify] https://crrev.com/120e4bcf22f7f422fa77b87812dba7d17df49062/appengine/findit/services/compile_failure/compile_try_job.py
[modify] https://crrev.com/120e4bcf22f7f422fa77b87812dba7d17df49062/appengine/findit/services/test_failure/test_try_job.py
[modify] https://crrev.com/120e4bcf22f7f422fa77b87812dba7d17df49062/appengine/findit/services/try_job.py
[modify] https://crrev.com/120e4bcf22f7f422fa77b87812dba7d17df49062/appengine/findit/common/swarmbucket/test/swarmbucket_test.py

Status: Fixed (was: Started)

Sign in to add a comment