chromite: config_dump.json doesn't match |
||||||
Issue descriptionhttps://uberchromegw.corp.google.com/i/chromeos/builders/caroline-release/builds/1118 chromite-0.0.2-r3428: FAIL: [chromite.cbuildbot.chromeos_config_unittest] ConfigDumpTest.testDump chromite-0.0.2-r3428: Make sure the json & config are kept in sync chromite-0.0.2-r3428: ---------------------------------------------------------------------- chromite-0.0.2-r3428: Traceback (most recent call last): chromite-0.0.2-r3428: File "/mnt/host/source/chromite/lib/timeout_util.py", line 191, in TimeoutWrapper chromite-0.0.2-r3428: return func(*args, **kwargs) chromite-0.0.2-r3428: File "/mnt/host/source/chromite/cbuildbot/chromeos_config_unittest.py", line 58, in testDump chromite-0.0.2-r3428: self.fail('config_dump.json does not match the ' chromite-0.0.2-r3428: AssertionError: config_dump.json does not match the defined configs. Run cbuildbot/chromeos_config_unittest --update It started to happen in 10026. The only change between 10025 and 10026 is crrev.com/9af4a9a23a7589ce4dec411604695c4572c75190 I'm not sure why so CC'ing ihf@ (the author).
,
Oct 12 2017
The chumped change must have done the wrong thing cherry-pick/merge wise. Strictly speaking, this is possible even when going through the CQ because there are multiple entities writing to the file.
,
Oct 12 2017
Oh... no, TOT is clean.
,
Oct 13 2017
I believe this is a really rare unittest flake. I'm not sure why it happened through.
,
Oct 13 2017
https://uberchromegw.corp.google.com/i/chromeos/builders/arkham-release/builds/1578 https://uberchromegw.corp.google.com/i/chromeos/builders/auron_paine-release/builds/1576 https://uberchromegw.corp.google.com/i/chromeos/builders/auron_yuna-release/builds/1575 https://uberchromegw.corp.google.com/i/chromeos/builders/banon-release/builds/1576 ... All release bots are now failing due to this error, though.
,
Oct 13 2017
I built caroline locally and did not hit this.
,
Oct 13 2017
chrome-bot updated the config dump at the same cycle as Ilja's CL https://chromium.googlesource.com/chromiumos/chromite/+/b740410d1b9ba8070f052699da5cac485d4d1185 and one more time in the next cycle https://chromium.googlesource.com/chromiumos/chromite/+/31ef8f074175b067e95f12a50403dd9374c0c284 Perhaps the second one for R63-10027.0.0 adjusted and fixed it for ToT?
,
Oct 13 2017
Good observation! This might be a race where the GE config change adding grunt (+VMTest) was overridden by my change to not run VMTest. Bernie, do GE config changes go through the CQ or are they chumped? If the latter, then I think this can be explained and the redness is for a single build. I tried to manually kick off a few more release builds.
,
Oct 13 2017
(Release bots that grabbed R63-10027.0.0 is getting greener. Lowering the priority.)
,
Oct 13 2017
+nxia The GE changes are 'chumped' much like how a PFQ run will 'chump' a Chrome uprev CL, they don't go through the formal CQ, but if the config updater builder lands something it should be consistent at the point in time the config updater synced, it is possible we have a race between the CQ and the config updater here. The config updater builder can be observed at https://uberchromegw.corp.google.com/i/chromeos.infra/builders/config-updater It is probably self correcting within an hour at least, I think the config updater will run again and correct any inconsistencies in the json.
,
Oct 13 2017
Ok, I figure this is a rare event. Closing. |
||||||
►
Sign in to add a comment |
||||||
Comment 1 by ihf@chromium.org
, Oct 12 2017Owner: ihf@chromium.org
Status: Started (was: Untriaged)