New issue
Advanced search Search tips

Issue 909889 link

Starred by 1 user

Issue metadata

Status: Untriaged
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug



Sign in to add a comment

bvt-tast-chrome-pfq misreported as failing when bvt-inline fails

Project Member Reported by derat@chromium.org, Nov 28

Issue description

(I'm not sure if this falls under Test or CI.)

When I load http://cros-goldeneye/chromeos/legoland/builderHistory?buildConfig=eve-tot-chrome-pfq-informational&buildBranch=master right now, the row for the build at http://cros-goldeneye/chromeos/healthmonitoring/buildDetails?buildbucketId=8928622358409831392 indicates that both HWTest [bvt-inline] and  HWTest [bvt-tast-chrome-pfq] failed (see attached screenshot).

When I click through to the actual page for the build, though, it indicates that HWTest [bvt-inline] failed (due to platform_FilePerms), but HWTest [bvt-tast-chrome-pfq] appears as green, not red.

The stdout and suite links for bvt-tast-chrome-pfq also indicate that it (and all of the Tast tests that it runs) passed.

I'm guessing that the page at https://ci.chromium.org/p/chromeos/builders/luci.chromeos.general/Prod/b8928622358409831392, which also shows bvt-tast-chrome-pfq as red, is closer to the source of the problem. After indicating that the status.log file shows passing tests, it contains this:

----

 11-28-2018 [05:24:04] Output below this line is for buildbot consumption:
Will return from run_suite with status: OK

05:24:05: INFO: Running cidb query on pid 25297, repr(query) starts with <sqlalchemy.sql.expression.Insert object at 0x7fe0106c5690>
05:24:05: INFO: Running cidb query on pid 25297, repr(query) starts with <sqlalchemy.sql.expression.Update object at 0x7fe0106c5e90>
************************************************************
** Finished Stage HWTest [bvt-tast-chrome-pfq] - Wed, 28 Nov 2018 05:24:05 -0800 (PST)
************************************************************

06:41:50: ERROR: <class 'chromite.lib.failures_lib.TestFailure'>: ** HWTest failed (code 1) **
Traceback (most recent call last):
  File "/b/swarming/w/ir/cache/cbuild/repository/chromite/lib/parallel.py", line 441, in _Run
    self._task(*self._task_args, **self._task_kwargs)
  File "/b/swarming/w/ir/cache/cbuild/repository/chromite/cbuildbot/stages/generic_stages.py", line 704, in Run
    self.PerformStage()
  File "/b/swarming/w/ir/cache/cbuild/repository/chromite/cbuildbot/stages/test_stages.py", line 281, in PerformStage
    raise cmd_result.to_raise
TestFailure: ** HWTest failed (code 1) **

06:41:50: INFO: Translating result <class 'chromite.lib.failures_lib.TestFailure'>: ** HWTest failed (code 1) **
Traceback (most recent call last):
  File "/b/swarming/w/ir/cache/cbuild/repository/chromite/lib/parallel.py", line 441, in _Run
    self._task(*self._task_args, **self._task_kwargs)
  File "/b/swarming/w/ir/cache/cbuild/repository/chromite/cbuildbot/stages/generic_stages.py", line 704, in Run
    self.PerformStage()
  File "/b/swarming/w/ir/cache/cbuild/repository/chromite/cbuildbot/stages/test_stages.py", line 281, in PerformStage
    raise cmd_result.to_raise
TestFailure: ** HWTest failed (code 1) **
 to fail.

----

Don, any ideas if there's something about the way that we run these stages in parallel that's causing problems? The timeline at https://storage.cloud.google.com/chromeos-image-archive/eve-tot-chrome-pfq-informational/R72-11306.0.0-b3170604/timeline-stages.html indicates that most of these HWTest stages started at the same time, so I'm not sure why bvt-tast-chrome-pfq was reported as failing while bvt-cq and bvt-arc were reported as passing.
 
snap-133622.png
39.3 KB View Download

Sign in to add a comment