New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 711554 link

Starred by 1 user

Issue metadata

Status: Archived
Owner:
Last visit > 30 days ago
Closed: Apr 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug



Sign in to add a comment

CQ master failed at InsertBuildMessage

Project Member Reported by nxia@chromium.org, Apr 14 2017

Issue description

CQ master failed at InsertBuildMessage for ignored slaves. And the bug didn't get caught by CQ because of  crbug.com/711512 .

Going to revert the CL:471849.


https://uberchromegw.corp.google.com/i/chromeos/builders/master-paladin/builds/14288/steps/CommitQueueCompletion/logs/stdio


18:21:43: WARNING: No need to wait for the remaining running slaves given the results of relevant change triages.

@@@STEP_FAILURE@@@
18:21:43: ERROR: Traceback (most recent call last):
  File "/b/cbuild/repository/chromite/cbuildbot/stages/generic_stages.py", line 628, in Run
    self.PerformStage()
  File "/b/cbuild/repository/chromite/cbuildbot/stages/completion_stages.py", line 937, in PerformStage
    super(CommitQueueCompletionStage, self).PerformStage()
  File "/b/cbuild/repository/chromite/cbuildbot/stages/completion_stages.py", line 295, in PerformStage
    statuses = self._FetchSlaveStatuses()
  File "/b/cbuild/repository/chromite/cbuildbot/stages/completion_stages.py", line 213, in _FetchSlaveStatuses
    manager, build_id, db, builder_names, timeout))
  File "/b/cbuild/repository/chromite/cbuildbot/stages/completion_stages.py", line 933, in _GetSlaveBuildStatus
    timeout=timeout)
  File "/b/cbuild/repository/chromite/cbuildbot/manifest_version.py", line 614, in GetBuildersStatus
    side_effect_func=_PrintRemainingTime)
  File "/b/cbuild/repository/chromite/lib/timeout_util.py", line 297, in WaitForSuccess
    return retry()
  File "/b/cbuild/repository/chromite/lib/timeout_util.py", line 271, in retry
    if not retry_check(value):
  File "/b/cbuild/repository/chromite/cbuildbot/manifest_version.py", line 610, in <lambda>
    lambda x: slave_status.ShouldWait(),
  File "/b/cbuild/repository/chromite/cbuildbot/build_status.py", line 457, in ShouldWait
    message_value=self.all_cidb_status_dict[build].build_id)
  File "/b/cbuild/repository/chromite/lib/cidb.py", line 118, in wrapper
    return f(self, *args, **kwargs)
  File "/b/cbuild/repository/chromite/lib/cidb.py", line 811, in InsertBuildMessage
    message_value = message_value[:480]
TypeError: 'long' object has no attribute '__getitem__'
 

Comment 1 by nxia@chromium.org, Apr 14 2017

Cc: dgarr...@chromium.org jrbarnette@chromium.org

Comment 2 by ihf@chromium.org, Apr 14 2017

Cc: ihf@chromium.org

Comment 3 by nxia@chromium.org, Apr 14 2017

Labels: -Pri-1 Pri-2
Project Member

Comment 4 by bugdroid1@chromium.org, Apr 25 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/63de9457827b460d4565729e4143dcb788bbca0c

commit 63de9457827b460d4565729e4143dcb788bbca0c
Author: Ningning Xia <nxia@chromium.org>
Date: Tue Apr 25 23:32:52 2017

When CQ-master destructs itself, it needs to record ignored slaves.

When self-destruction is triggered, for each uncompleted slave builds,
the master build will insert a message with the slave build_id as the
value into the buildMessageTable. The slave builds may pass, fail or
abort after the master destructs itself.

BUG= chromium:707894 ;chromium:711554
TEST=unit_tests

Change-Id: I2083be86f8590a0ef9f4a5b4c4d29a472e09fbb3
Reviewed-on: https://chromium-review.googlesource.com/478414
Commit-Ready: Ningning Xia <nxia@chromium.org>
Tested-by: Ningning Xia <nxia@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>

[modify] https://crrev.com/63de9457827b460d4565729e4143dcb788bbca0c/cbuildbot/build_status_unittest.py
[modify] https://crrev.com/63de9457827b460d4565729e4143dcb788bbca0c/lib/constants.py
[modify] https://crrev.com/63de9457827b460d4565729e4143dcb788bbca0c/lib/fake_cidb.py
[modify] https://crrev.com/63de9457827b460d4565729e4143dcb788bbca0c/cbuildbot/build_status.py

Comment 5 by nxia@chromium.org, Apr 26 2017

Status: Fixed (was: Untriaged)

Comment 6 by dchan@google.com, May 30 2017

Labels: VerifyIn-60

Comment 7 by dchan@chromium.org, Aug 1 2017

Labels: VerifyIn-61

Comment 8 by dchan@chromium.org, Jan 22 2018

Status: Archived (was: Fixed)

Sign in to add a comment