The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4d22812b67e2a514674e26b84c4c11e091e23170

commit 4d22812b67e2a514674e26b84c4c11e091e23170
Author: Ned Nguyen <nednguyen@google.com>
Date: Sat Aug 12 01:13:27 2017

Disable testRunTelemetryBenchmarkAsGoogletest on ChromeOS

TBR=ashleymarie@chromium.org, achuith@chromium.org, jdufault@chromium.org

Bug: 754913
Change-Id: Ideebb41c4894e5118de99addc8e6ede289201494
Reviewed-on: https://chromium-review.googlesource.com/612622
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Ned Nguyen <nednguyen@google.com>
Cr-Commit-Position: refs/heads/master@{#493923}
[modify] https://crrev.com/4d22812b67e2a514674e26b84c4c11e091e23170/tools/perf/scripts_smoke_unittest.py

I think this is the same error I was getting before updating the isolate dependencies so I'm guessing those need to be updated wherever this was failing as well

My best guess is that https://cs.chromium.org/chromium/src/chrome/test/BUILD.gn?l=1018 needs the following lines added:
    # For smoke testing run_telemetry_benchmark_as_googletest
    "//testing/scripts/run_telemetry_benchmark_as_googletest.py",

Any recommendations for a trybot I could trigger to verify my guess?

08/11 16:18:13.358 INFO |run_chromeos_tests:0052| [320/323] scripts_smoke_unittest.ScriptsSmokeTest.testRunTelemetryBenchmarkAsGoogletest queued
08/11 16:18:13.368 INFO |run_chromeos_tests:0052| [321/323] scripts_smoke_unittest.ScriptsSmokeTest.testRunTelemetryBenchmarkAsGoogletest failed unexpectedly 0.0529s:
08/11 16:18:13.369 INFO |run_chromeos_tests:0052|   Traceback (most recent call last):
08/11 16:18:13.370 INFO |run_chromeos_tests:0052|     File "/usr/local/telemetry/src/tools/perf/scripts_smoke_unittest.py", line 70, in testRunTelemetryBenchmarkAsGoogletest
08/11 16:18:13.371 INFO |run_chromeos_tests:0052|       self.assertEquals(return_code, 0, stdout)
08/11 16:18:13.372 INFO |run_chromeos_tests:0052|   AssertionError: /usr/bin/python: can't open file '../../testing/scripts/run_telemetry_benchmark_as_googletest.py': [Errno 2] No such file or directory

Ashley: I think this happen because how ChromeOS bots are setup is currently vastly different from usual CHromium bots. I reassign this to bccheng@ to triage/prioritize fixing the tests.

Ah that makes sense
Thanks for clarifying Ned!

ashleymarie@: If you have time, you can look at http://go/cros-vm to try this change out for yourself on ChromeOS. If you have any issues, please feel free to reach out.

To #6: the failure here is not the test failure but how the files are packaged on cros test environment:

08/11 16:18:13.372 INFO |run_chromeos_tests:0052|   AssertionError: /usr/bin/python: can't open file '../../testing/scripts/run_telemetry_benchmark_as_googletest.py': [Errno 2] No such file or directory

Somehow cros is not even using run_telemetry_benchmark_as_googletest test script.