I see this in the test output:

media.mobile is disabled on the selected browser
This platform is not supported for this benchmark. If this is in error please add it to the benchmark's supported platforms.


I have a bug open to give a more useful error message in this scenario.
https://github.com/catapult-project/catapult/issues/4527

+nednugyen, +eyaich

I dug a little bit deeper. This benchmark is disabled on Android WebView. But, the perf dashboard clearly shows that it found an alert on this benchmark on Android Nexus6 WebView Perf.
https://chromium.googlesource.com/chromium/src/+/HEAD/tools/perf/benchmarks/media.py#84
https://chromeperf.appspot.com/group_report?bug_id=865160

I looked at a recent run. It's passing `--browser=android-chromium` instead of `--browser=android-webview`. Could this be related to the OBBS migration?
Build: https://ci.chromium.org/buildbot/chromium.perf/Android%20Nexus6%20WebView%20Perf/2064
Swarming task: https://chrome-swarming.appspot.com/task?id=3eaa8da38a4d3310
Benchmark output: https://chrome-isolated.appspot.com/browse?digest=51f7f39d6873a19d107fefd1bec052fc8c5605f7&as=benchmark_log.txt

Looks like this is a mistake with the OBBS migration. 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/66ee3b7d073873beb5c86ec2c6574dca4dbaf049

commit 66ee3b7d073873beb5c86ec2c6574dca4dbaf049
Author: Ned Nguyen <nednguyen@google.com>
Date: Thu Jul 19 12:05:05 2018

Fix json config of webview builder to use webview browser type

This CL also add extra validation to perf_json_config_validator script (run on PRESUBMIT) to prevent this bug from happening again

Bug:865181
Cq-Include-Trybots: master.tryserver.chromium.perf:obbs_fyi
Change-Id: Ie68f426d3e5f073c5239ef05e14b67ac89b6911b

NOTRY=true  # webkit_layout_tests flaky

Change-Id: Ie68f426d3e5f073c5239ef05e14b67ac89b6911b
Reviewed-on: https://chromium-review.googlesource.com/1142950
Commit-Queue: Ned Nguyen <nednguyen@google.com>
Reviewed-by: Juan Antonio Navarro Pérez <perezju@chromium.org>
Cr-Commit-Position: refs/heads/master@{#576451}
[modify] https://crrev.com/66ee3b7d073873beb5c86ec2c6574dca4dbaf049/testing/buildbot/chromium.perf.json
[modify] https://crrev.com/66ee3b7d073873beb5c86ec2c6574dca4dbaf049/tools/perf/core/perf_data_generator.py
[modify] https://crrev.com/66ee3b7d073873beb5c86ec2c6574dca4dbaf049/tools/perf/core/perf_json_config_validator.py

I'm perf sheriffing and I noticed some of the webview builds affected have ref builds, which were unaffected by the change. Example:

https://chromeperf.appspot.com/report?sid=fd816ff6a54eb7a5fa1670537b192c99ef3a109215649e092537cea6dc5ea9fa&start_rev=572456&end_rev=577119

Does anyone understand how the ref builds work for these bots? Should we reopen to investigate?

Hmm does seem a bit strange that the refs didn't move at all. They're specified by setting --browser=reference right? Maybe Ned can chime in on how telemetry knows exactly what to use at that point, could be that they were working fine it was only the non-reference that was wonky.

I might be wrong, but I think webview reference is not a browser option that Telemetry currently supports.

#12: you're right

I filed  bug 866899  about the erroneous upload of webview ref builds.