Are the regressions in the same/similar range? Do we see similar regressions on system health?

Also reminder that on top_10_mobile stories are not independent, so --story-filter can affect reproducibility.

We should definitely try to kick off some bisects (particularly for the more clear regressions) without a --story-filter and see if that helps.

At this point we might also consider black-listing top_10_mobile so it never uses --story-filter for bisect. (And assume that system health already gives us enough coverage for regressions on small stories.)

That's a good question. I don't know, in general I've just been blocking issues to get a sense of how many bisects are seemingly affected by some (potential) root issue, helps me prioritize.

I didn't know they're not independent, so +1 to idea of blacklisting --story-filter. Do you know where that's done?

Yes, here: https://github.com/catapult-project/catapult/blob/master/dashboard/dashboard/start_try_job.py#L90

I can write a quick patch now.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/9caf2a3080089a596226aafec5de5c5518255777

commit 9caf2a3080089a596226aafec5de5c5518255777
Author: catapult-deps-roller <catapult-deps-roller@chromium.org>
Date: Mon Feb 06 15:09:00 2017

Roll src/third_party/catapult/ 3ea897790..1e9198268 (1 commit).

https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/3ea8977903c6..1e9198268428

$ git log 3ea897790..1e9198268 --date=short --no-merges --format='%ad %ae %s'
2017-02-06 perezju [dashboard] Disable --story-filter on memory.top_10_mobile

BUG=675639

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, see:
http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls

CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel
TBR=catapult-sheriff@chromium.org

Review-Url: https://codereview.chromium.org/2672343002
Cr-Commit-Position: refs/heads/master@{#448251}

[modify] https://crrev.com/9caf2a3080089a596226aafec5de5c5518255777/DEPS