The builder uses this link for the bvt-cq test suite:

http://cautotest-prod.corp.google.com/afe/#tab_id=view_job&object_id=155266883

However, that link shows the suite is still in progress.


Ah.....


I found it.

The build configuration (at runtime), defines the test suite to be async.


https://logs.chromium.org/v/?s=chromeos%2Fbb%2Fchromiumos.tryserver%2Fchrome_pfq_informational%2F91%2F%2B%2Frecipes%2Fsteps%2FConfigDump%2F0%2Fstdout

Suite build config:

        "{\n    \"async\": true,\n    \"blocking\": false,\n    \"critical\": false,\n    \"file_bugs\": false,\n    \"max_retries\": 5,\n    \"minimum_duts\": 0,\n    \"num\": 1,\n    \"offload_failures_only\": false,\n    \"pool\": \"suites\",\n    \"priority\": \"PostBuild\",\n    \"retry\": true,\n    \"suite\": \"bvt-cq\",\n    \"suite_args\": null,\n    \"suite_min_duts\": 1,\n    \"timeout\": 15000,\n    \"warn_only\": false\n}",


This is because it's using the "override" test configurations, which is normal for tryjobs.

The fix is to remove "override" test configurations for all informational builders.

Are you using any non-informational build configs for bisect?

I've only ever used chrom[e|ium]-pfq-informational builders for bisects. I'm not even sure whether the primary PFQ slaves would function with 'cros tryjob'?

So far a board specific chrome regression not covered by pfq-informational has not happened on my watch.

P.S. We fixed the latest regression with a speculative revert, but I very much appreciate the quick response and fix!

I would recommend using "primary-pfq-tryjob" which is nearly the same config, but certain to not interfere with the real PFQ.

Good to know, I did not notice those before. Thanks!

It's only existed for a few days. ;>

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/7b6558e173ed4df2f76ac0ca1001f0a477f611ef

commit 7b6558e173ed4df2f76ac0ca1001f0a477f611ef
Author: Don Garrett <dgarrett@google.com>
Date: Sat Nov 11 02:45:57 2017

chromeos_config: Remove tryjob config overrides.

Both hw_test_overrides and vm_test_overrides are unhelpful for tryjob
safe build configurations. They will go away entirely, when we remove
OverrideConfigForTrybot, but in the mean time they are causing
confusion.

So... remove them in a quick/dirty and ham fisted way that is good
enough, for now.

BUG= chromium:783490 
TEST=cbuildbot/run_tests

Change-Id: Ia75d248b6e429340fd6a01310b03850b13bd9f8e
Reviewed-on: https://chromium-review.googlesource.com/762767
Commit-Ready: Don Garrett <dgarrett@chromium.org>
Tested-by: Don Garrett <dgarrett@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>

[modify] https://crrev.com/7b6558e173ed4df2f76ac0ca1001f0a477f611ef/cbuildbot/config_dump.json
[modify] https://crrev.com/7b6558e173ed4df2f76ac0ca1001f0a477f611ef/cbuildbot/chromeos_config_unittest.py
[modify] https://crrev.com/7b6558e173ed4df2f76ac0ca1001f0a477f611ef/cbuildbot/chromeos_config.py