There are two paths to this. We can make the bisect recipes use our test runner (which might require some other changes to the recipes and how they grab the results, but maybe not since our test runner allows you to specify a location for the results json) or we could move the cool down logic into telemetry itself.

Seems like having in telemetry would be nice, then future stuff (ie. pinpoint) will just have it available?

And as we move to swarming it would make sense since we will no longer be using the android test runner.

+eyaich

+benhenry this one should go on the speed/infra backlog

I have a CL out for letting a device cool in telemetry between pages here:
https://codereview.chromium.org/2541843007/

This is like the one we have in provision devices and the android test runner, only works even better because it will make sure between pages the phone is in a good temperature state instead of only between benchmarks. This will be helpful for long running benchmarks.

Apparently we already have code to wait for thermal throttling to be over.. So that may not be the issue.

After more digging this doesn't mean it isn't thermal throttling. It just waits for it to not be throttled and then continues. This means it could enter a throttled state mid test.

Simon: why do you think this is related to thermal? I think other possible causes are:
1) Telemetry leaks subprocesses. More processes -> cpu throttled.
2) Telemetry leaks files. There use to be a bug where we fill the browser directory with too many files & that slow down telemetry test significantly.

The best thing here is to enable tracing + cpu snapshotting so we can understand what causes the slow down, I think.

It was a suggestion based on the pattern, but those sound like good things to try too. I'll rename the bug as more general investigation.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/d461b1589b02a7e738a2157d4eda0f6b09ed0108

commit d461b1589b02a7e738a2157d4eda0f6b09ed0108
Author: catapult-deps-roller <catapult-deps-roller@chromium.org>
Date: Tue Dec 20 01:01:23 2016

Roll src/third_party/catapult/ d77eaf7f6..70f42a7c5 (1 commit).

https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/d77eaf7f69e8..70f42a7c55ca

$ git log d77eaf7f6..70f42a7c5 --date=short --no-merges --format='%ad %ae %s'
2016-12-19 rnephew [Telemetry][Android] Wait for device under test to cool between pages.

BUG=669923

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, see:
http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls

CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel
TBR=catapult-sheriff@chromium.org

Review-Url: https://codereview.chromium.org/2590683002
Cr-Commit-Position: refs/heads/master@{#439645}

[modify] https://crrev.com/d461b1589b02a7e738a2157d4eda0f6b09ed0108/DEPS

I landed cooling between pages. If that is what is causing the issue it should be fixed.

rnephew@, do you know how we could go about confirming this fix?

Sorry, this fell through the winter break black hole. Running a bisect that previously showed this behavior with a range before the change and one after should show if this is fixed.