[WPT import] Checking for the wrong experimental tag in trybots? |
|||
Issue descriptionI was recently looking at why https://chromium-review.googlesource.com/c/chromium/src/+/953042 and https://ci.chromium.org/buildbot/chromium.infra.cron/wpt-importer/13446 failed, as all the trybots ultimately passed and the CL was ready to land. Apparently, it was the failure in the experimental win10_chromium_x64_rel_ng bot (which went purple) that caused wpt-importer to abort the job. I thought this was not supposed to happen, as since bug 792652 we ignore the results from experimental bots by looking at the "experimental" field in git-cl try-results' output. I was looking at the output for this CL, and noticed we have the following: { "result": "FAILURE", "status": "COMPLETED", "created_ts": "1520431535811420", "buildbucket_id": "8952705697864047024", "builder_name": "win10_chromium_x64_rel_ng", "tags": [ "bot_id:win1461-c4", "builder:win10_chromium_x64_rel_ng", "buildset:patch/gerrit/chromium-review.googlesource.com/953042/2", "cq_experimental:true", "master:master.tryserver.chromium.win", "user_agent:cq" ], "url": "http://build.chromium.org/p/tryserver.chromium.win/builders/win10_chromium_x64_rel_ng/builds/96268", "bucket": "master.tryserver.chromium.win", "experimental": false, "failure_reason": "INFRA_FAILURE" } That is, the "experimental" field is set to false, but there's a "cq_experimental:true" tag. I don't really know who sets those fields, so I'm not sure if it's the experimental field that's got a wrong value or if we should be checking the cq_experimental tag instead.
,
Mar 15 2018
OK, bug 792652 shows that the following CLs failed due to redness in the experimental trybots: * https://chromium-review.googlesource.com/c/chromium/src/+/953042 * https://chromium-review.googlesource.com/c/chromium/src/+/810928 * https://chromium-review.googlesource.com/c/chromium/src/+/811428 In all 3, the failing win7_chromium_rel_loc_exp bot has is_experimental set to false and cq_experimental:true, which can be seen in the build properties: * https://ci.chromium.org/buildbot/tryserver.chromium.win/win7_chromium_rel_loc_exp/4377 * https://ci.chromium.org/buildbot/tryserver.chromium.win/win7_chromium_rel_loc_exp/4426 * https://ci.chromium.org/buildbot/tryserver.chromium.win/win7_chromium_rel_loc_exp/4461 The Buildbucket Gerrit plugin also uses the cq_experimental tag to determine if a trybot is experimental or not: https://chromium.googlesource.com/infra/gerrit-plugins/buildbucket/+/c6d551e5a353161f64bf4ddc91f309600393a5df/src/main/resources/static/cr-buildbucket-view.js#151 Additionally, according to https://chromium.googlesource.com/infra/infra/+/master/appengine/cr-buildbucket/doc/swarming.md the "is_experimental" field that becomes the "experimental" field in the JSON returned by git-cl try-results seems to be LUCI/swarming-specific and not what we assumed it was. The cq_experimental tag also matches the description here https://cs.chromium.org/chromium/infra/go/src/infra/libs/bqschema/buildevent/completedbuild.go?type=cs&sq=package:chromium&l=31 What exactly sets those fields is still unclear to me, but I guess it's irrelevant at this point. I'll prepare a CL to change our checks.
,
Mar 15 2018
> Additionally, according to https://chromium.googlesource.com/infra/infra/+/master/appengine/cr-buildbucket/doc/swarming.md the "is_experimental" field that becomes the "experimental" field in the JSON returned by git-cl try-results seems to be LUCI/swarming-specific and not what we assumed it was. It's not LUCI/swarming-specific AFAICS, but it definitely does not mean what we need it to mean.
,
Mar 15 2018
Sounds like is_experimental is for LUCI migration https://chromium.googlesource.com/chromium/src/+/master/docs/luci_migration_faq.md (set when a bot is being migrated to LUCI and this particualr build runs on LUCI). Unless someone else more knowledgeable disagrees, I think your investigation & understanding are sound and correct. I'm gonna LGTM your CL. Thank you!
,
Mar 15 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/d0b9283b286a0ccf335643a399664ad5d276f680 commit d0b9283b286a0ccf335643a399664ad5d276f680 Author: Raphael Kubo da Costa <raphael.kubo.da.costa@intel.com> Date: Thu Mar 15 16:12:28 2018 WPT Import: Use the "cq_experimental" tag to check for experimental builds We were previously using the "experimental" field in the JSON returned by `git-cl try-results`, but that field actually corresponds to whether a recipe is running in experimental mode. See e.g. https://cs.chromium.org/chromium/infra/recipes-py/recipe_modules/runtime/api.py?q=is_experimental&sq=package:chromium&l=27&dr=CSs What we need is the "cq_experimental" tag that really indicates whether a trybot is scheduled in an experimental commit queue per https://cs.chromium.org/chromium/infra/go/src/infra/libs/bqschema/buildevent/completedbuild.go?type=cs&sq=package:chromium&l=31 This tag is also used by the Buildbucket Gerrit plugin to split bots into experimental and non-experimental ones. Bug: 792652 , 821445 Change-Id: I267bf033729b6a74382d018cb3da06c64dc7b207 Reviewed-on: https://chromium-review.googlesource.com/964121 Reviewed-by: Robert Ma <robertma@chromium.org> Reviewed-by: Quinten Yearsley <qyearsley@chromium.org> Commit-Queue: Raphael Kubo da Costa <raphael.kubo.da.costa@intel.com> Cr-Commit-Position: refs/heads/master@{#543385} [modify] https://crrev.com/d0b9283b286a0ccf335643a399664ad5d276f680/third_party/WebKit/Tools/Scripts/webkitpy/common/net/git_cl.py [modify] https://crrev.com/d0b9283b286a0ccf335643a399664ad5d276f680/third_party/WebKit/Tools/Scripts/webkitpy/common/net/git_cl_unittest.py
,
Mar 15 2018
|
|||
►
Sign in to add a comment |
|||
Comment 1 by foolip@chromium.org
, Mar 14 2018