Some example command lines:

https://build.chromium.org/p/tryserver.chromium.perf/builders/winx64ati_perf_bisect/builds/1638
C:\b\c\b\winx64ati_perf_bisect\src\tools\perf\run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter='http...msn\.co\.il.' page_cycler_v2.intl_ar_fa_he --output-dir=c:\users\chrome~1\appdata\local\temp\perf-test-outputskrp0u

https://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus6_perf_bisect/builds/2725
/b/c/b/android_nexus6_perf_bisect/src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter='http...ynet\.co\.il.' page_cycler_v2.intl_ar_fa_he --device ZX1G229B6Q --output-dir=/tmp/perf-test-outputfv7HcL

https://build.chromium.org/p/tryserver.chromium.perf/builders/linux_perf_bisect/builds/6818
/usr/bin/python /b/c/b/linux_perf_bisect/src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter='http...bookish\.com' page_cycler_v2.tough_layout_cases --output-dir=/tmp/perf-test-outputFjgEHu

https://uberchromegw.corp.google.com/i/internal.tryserver.clankium/builders/clankium-tablet-perf-bisect/builds/9
/usr/bin/python /b/build/slave/clankium-tablet-perf-bisect/build/src/tools/perf/run_benchmark -v --browser=android-chrome --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter='http...kapook\.com' start_with_url.warm.startup_pages --device 09e26f10 --output-dir=/tmp/perf-test-outputJuWG3l

https://build.chromium.org/p/tryserver.chromium.perf/builders/winx64intel_perf_bisect/builds/1223
C:\b\depot_tools\python276_bin\python.exe C:\b\c\b\winx64intel_perf_bisect\src\tools\perf\run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter='http...us\.24h\.com\.vn.' page_cycler_v2.intl_ko_th_vi --output-dir=c:\users\chrome~1\appdata\local\temp\perf-test-outputgdppkd

https://build.chromium.org/p/tryserver.chromium.perf/builders/winx64ati_perf_bisect/builds/1639
C:\b\depot_tools\python276_bin\python.exe C:\b\rr\tmpheiz9e\rw\checkout\scripts\slave\runisolatedtest.py --test_name "" --builder_name winx64ati_perf_bisect --checkout_dir C:\b\c\b\winx64ati_perf_bisect C:\b\c\b\winx64ati_perf_bisect\src\tools\perf\run_benchmark -- C:\b\depot_tools\python276_bin\python.exe C:\b\c\b\winx64ati_perf_bisect\src\tools\perf\run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter='http...ru\.wikipedia\.org.' page_cycler_v2.intl_hi_ru --output-dir=c:\users\chrome~1\appdata\local\temp\perf-test-outputuclv5_

Looks like there is something funny with the way the command line is interepreted in this one:
https://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus9_perf_bisect/builds/2234/steps/Gathering%20reference%20values.Performance%20Test%201%20of%205/logs/Failure%20Output

/usr/bin/python /b/c/b/android_nexus9_perf_bisect/src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter='Wikipedia\ \(1\ tab\)' smoothness.sync_scroll.key_mobile_sites_smooth --device HT4CTJT03478 --output-dir=/tmp/perf-test-outputAapHgb

No benchmark named "\(1\".

That's interesting. I'll take this to investigate further what is going on.

Thanks, Juan!

Catapult side bug: https://github.com/catapult-project/catapult/issues/2967

A short term fix is in place: https://codereview.chromium.org/2481773002/

I've re-started some bisects on all of the blocked bugs.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/cfa43000d1547324bc97b8c1bf6a1b9303e6e8fb

commit cfa43000d1547324bc97b8c1bf6a1b9303e6e8fb
Author: catapult-deps-roller <catapult-deps-roller@chromium.org>
Date: Tue Nov 08 00:10:04 2016

Roll src/third_party/catapult/ 8ac279fd8..d18e525a0 (6 commits).

https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/8ac279fd8db3..d18e525a0681

$ git log 8ac279fd8..d18e525a0 --date=short --no-merges --format='%ad %ae %s'
2016-11-07 aiolos Revert of [Telemetry] Remove the logic of killing stray adb processes (patchset #1 id:1 of https://codereview.chromium.org/2424843002/ )
2016-11-07 aiolos Add missing import in model track.
2016-11-07 jbudorick [devil + systrace] Set up logging in profile_chrome.
2016-11-07 jbudorick [devil] Add explicit None checking to device_temp_file.
2016-11-07 perezju [Dashboard] Fix regex for complex story names
2016-11-07 nednguyen [Telemetry] Remove the logic of killing stray adb processes

BUG=662301, 662472 , 656224 

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, see:
http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls

CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel
TBR=catapult-sheriff@chromium.org

Review-Url: https://codereview.chromium.org/2477333004
Cr-Commit-Position: refs/heads/master@{#430433}

[modify] https://crrev.com/cfa43000d1547324bc97b8c1bf6a1b9303e6e8fb/DEPS

Lowering pri- here as the workaround is now in place. Also have a CL ready for one of the places where the command line was being interpreted incorrectly.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build.git/+/870e91d32948d0c30e4bb5833e6088a3c2a50a23

commit 870e91d32948d0c30e4bb5833e6088a3c2a50a23
Author: Juan A. Navarro Perez <perezju@chromium.org>
Date: Wed Nov 09 10:36:58 2016

[Bisect Script] Correctly interpret quoted arguments in test command

Allow test commands to use shell-like quoting for passing arguments with
special characters (e.g. spaces, etc.)

BUG= 662472 

Change-Id: Ib2794bcce1dba8f5dd195f9defddd3dc3a16a2fc
Reviewed-on: https://chromium-review.googlesource.com/408139
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>

[modify] https://crrev.com/870e91d32948d0c30e4bb5833e6088a3c2a50a23/scripts/slave/recipe_modules/bisect_tester/perf_test.py
[modify] https://crrev.com/870e91d32948d0c30e4bb5833e6088a3c2a50a23/scripts/slave/recipe_modules/bisect_tester/perf_test_unittest.py

Okay, so the priority has been lowered, but what are the next steps here? If there's a better long-term fix, maybe that should be filed as a fresh bug and this one should be closed?

Yeah, I agree with closing this for now, the current fixes in place have served us well.

The longer term fix would fit within the realm of the data pipe from benchmarks to dashboard.