What I think is happening: during the fix-it to migrate the old test disabling system to new test disabling test, we check a few CLs in with NOTRY=true, thinking that it wouldn't affect CQ (as benchmarks don't run on CQ).

However, *some* benchmarks are cherry-picked to run on CQ (https://cs.chromium.org/chromium/src/tools/perf/benchmarks/benchmark_smoke_unittest.py), and the way they get picked is due to python class discovery magic (one benchmark class per python file).

So what happening here is upon reenabling all benchmarks, we somehow change/ or add benchmarks that are smoke tested & one of them take very long to run, thus blocks CQ.

TO be safe, now I will disable benchmark_smoke_unittest.py completely, then try to reenable it in another CL by blacklisting tests (with CQ coverage).

Disabling telemetry benchmark smoke tests in https://chromium-review.googlesource.com/c/607170

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/e73f38e808b759b69d91ec3fab14eb40ac87d255

commit e73f38e808b759b69d91ec3fab14eb40ac87d255
Author: Ned Nguyen <nednguyen@google.com>
Date: Wed Aug 09 06:35:58 2017

Disable all benchmark smoke unittest

Reason: this suite contains some tests that are taking very long to run. Unclear which ones, so
disabling now to investigate later to unblock CQ.

TBR=kbr@chromium.org
NOTRY=true # almost all the tests in CQ passed, but CQ is stuck & this need to be landed ASAP

Bug:  753542 
Change-Id: I8f9a896b500e26531633d8ce837f06085f85e0b3
Reviewed-on: https://chromium-review.googlesource.com/607170
Commit-Queue: Ned Nguyen <nednguyen@google.com>
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Cr-Commit-Position: refs/heads/master@{#492859}
[modify] https://crrev.com/e73f38e808b759b69d91ec3fab14eb40ac87d255/tools/perf/benchmarks/benchmark_smoke_unittest.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/9e8d771f5e1b9a64d96542d0a12d9cd75d1e7ca9

commit 9e8d771f5e1b9a64d96542d0a12d9cd75d1e7ca9
Author: catapult-deps-roller@chromium.org <catapult-deps-roller@chromium.org>
Date: Wed Aug 09 09:37:12 2017

Roll src/third_party/catapult/ 4dd91f2ea..26fa809f1 (2 commits)

https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/4dd91f2eab39..26fa809f1faa

$ git log 4dd91f2ea..26fa809f1 --date=short --no-merges --format='%ad %ae %s'
2017-08-08 nednguyen Update ./telemetry/list_telemetry_unittests script to print details about which tests are skipped & unfinished
2017-08-08 xunjieli [wpr-go] Limit cert validity to below 39mths

Created with:
  roll-dep src/third_party/catapult
BUG= 753542 


Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, see:
http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls


CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel
TBR=sullivan@chromium.org

Change-Id: I0604c5ad6d0860363624b83ae8a5b545433e9f55
Reviewed-on: https://chromium-review.googlesource.com/607457
Reviewed-by: <catapult-deps-roller@chromium.org>
Commit-Queue: <catapult-deps-roller@chromium.org>
Cr-Commit-Position: refs/heads/master@{#492921}
[modify] https://crrev.com/9e8d771f5e1b9a64d96542d0a12d9cd75d1e7ca9/DEPS

The benchmark smoke test is disable, downgrade this to P1 to reenable benchmark smoke tests

We should keep this on the stack of things to look at/re-enable once the fallout of this re-enable fixit Randy is spear heading is well known.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/e9ed19547aa018949bfc5f72c17717c591d40abc

commit e9ed19547aa018949bfc5f72c17717c591d40abc
Author: nednguyen <nednguyen@google.com>
Date: Wed Aug 09 18:33:47 2017

Reenable benchmark smoke unittests

TBR=kbr@chromium.org

Bug:  753542 
Change-Id: If998b0ea0c0cd68e015271a38b6cfec705f43e68
Reviewed-on: https://chromium-review.googlesource.com/608587
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Ned Nguyen <nednguyen@google.com>
Cr-Commit-Position: refs/heads/master@{#493068}
[modify] https://crrev.com/e9ed19547aa018949bfc5f72c17717c591d40abc/tools/perf/benchmarks/benchmark_smoke_unittest.py