Looks like this is related to the OBBS migration.

The builders infer what targets to build based on what tests are run. Because all Android and Windows bots were migrated at once, the builders built only telemetry_perf_tests previously, and now only build performance_test_suite. Therefore, Pinpoint jobs before the migration must be run with telemetry_perf_tests, and jobs after the migration must be run with performance_test_suite. (That's r572268 for Android and r571917 for Windows.)

We didn't notice this during the Linux migration, because the builder rename and sharding issues masked this problem.

We didn't see the same behavior on Mac, because they weren't migrated all at once, so there was a time period when the builders built both targets.

We don't need to be building telemetry_perf_tests anymore do we?  We only need it built up until the revision we are still using it, but maybe I am missing something.  

Can  we do a check in pinpoint to use the different built isolate depending on what revision range it falls in?

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/eedd3cd97815988eda12865d375d29ab6a2e6284

commit eedd3cd97815988eda12865d375d29ab6a2e6284
Author: Dave Tu <dtu@chromium.org>
Date: Thu Jul 12 21:13:54 2018

[pinpoint] Use old isolate target for jobs before OBBS migration.

Just a hack to make bisect jobs in the migration range work. Bisect
jobs are generally run with commit position, so this should work for
most of them.

Bug:  chromium:862654 
Change-Id: Ib6e8e592ce5cad3dbae11d2b06b312947c958751
Reviewed-on: https://chromium-review.googlesource.com/1134524
Commit-Queue: David Tu <dtu@chromium.org>
Reviewed-by: Simon Hatch <simonhatch@chromium.org>
Reviewed-by: Juan Antonio Navarro Pérez <perezju@chromium.org>

[modify] https://crrev.com/eedd3cd97815988eda12865d375d29ab6a2e6284/dashboard/dashboard/pinpoint_request.py
[modify] https://crrev.com/eedd3cd97815988eda12865d375d29ab6a2e6284/dashboard/dashboard/pinpoint_request_test.py

Yes, I've added a hack in dashboard to use the different isolate based on revision range. I'll also work on coming up with more robust handling of infrastructure changes and processes to handle these breakages before they occur.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/75331317f1a04f44d09d69fe56bb1588629c8f3b

commit 75331317f1a04f44d09d69fe56bb1588629c8f3b
Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Fri Jul 13 01:08:19 2018

Roll src/third_party/catapult 66447ba64fcc..8495de1eb482 (3 commits)

https://chromium.googlesource.com/catapult.git/+log/66447ba64fcc..8495de1eb482


git log 66447ba64fcc..8495de1eb482 --date=short --no-merges --format='%ad %ae %s'
2018-07-12 dtu@chromium.org [pinpoint] Fix: Use new results path for performance_webview_test_suite.
2018-07-12 dtu@chromium.org [pinpoint] Use old isolate target for jobs before OBBS migration.
2018-07-12 benjhayden@chromium.org Add /api/timeseries2


Created with:
  gclient setdep -r src/third_party/catapult@8495de1eb482

The AutoRoll server is located here: https://catapult-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.

CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel

BUG= chromium:862654 
TBR=sullivan@chromium.org

Change-Id: Ifb6b94cad8056d5d2b668fe3acde70ea8d00fbeb
Reviewed-on: https://chromium-review.googlesource.com/1135218
Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#574796}
[modify] https://crrev.com/75331317f1a04f44d09d69fe56bb1588629c8f3b/DEPS

I've deployed a new dashboard with the hack in.
I tested it by re-kicking the jobs in  issue 862379 . Looks good!