I don't think you are at this point, but just an FYI.  Stephen pointed me to this manageable file (https://cs.chromium.org/chromium/src/tools/perf/chromium.perf.fyi.extras.json?q=fyi.extras&sq=package:chromium&l=1) to add the src side json to for bots on the fyi waterfall.  Might be a place you could stick a test to trigger on that waterfall.

Disregard last comment, looks like martiniss@ is one step ahead of me as per usual.  

https://chromium-review.googlesource.com/c/chromium/src/+/634264

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/6b54b248a2a21e6eeb30580cd163fdafd5f65001

commit 6b54b248a2a21e6eeb30580cd163fdafd5f65001
Author: Simon <simonhatch@chromium.org>
Date: Tue Aug 29 21:24:39 2017

Schedule histogram benchmark

Bug: 758702
Change-Id: I96b6206a25c0601a1fe83526d176d273afc1854d
Reviewed-on: https://chromium-review.googlesource.com/641397
Reviewed-by: Emily Hanley <eyaich@chromium.org>
Reviewed-by: Ethan Kuefner <eakuefner@chromium.org>
Commit-Queue: Simon Hatch <simonhatch@chromium.org>
Cr-Commit-Position: refs/heads/master@{#498233}
[modify] https://crrev.com/6b54b248a2a21e6eeb30580cd163fdafd5f65001/testing/buildbot/chromium.perf.fyi.json
[modify] https://crrev.com/6b54b248a2a21e6eeb30580cd163fdafd5f65001/tools/perf/chromium.perf.fyi.extras.json

Hey I'm still getting this error from presubmit when I try to submit the new (disabled) benchmark: https://chromium-review.googlesource.com/c/chromium/src/+/634216



Traceback (most recent call last):
  <module> at /usr/local/google/home/simonhatch/chromium/src/tools/perf/generate_perf_data:12
    sys.exit(perf_data_generator.main(sys.argv[1:]))
  main at /usr/local/google/home/simonhatch/chromium/src/tools/perf/core/perf_data_generator.py:1130
    if tests_are_up_to_date([fyi_waterfall, waterfall]):
  tests_are_up_to_date at /usr/local/google/home/simonhatch/chromium/src/tools/perf/core/perf_data_generator.py:930
    tests = generate_all_tests(w)
  generate_all_tests at /usr/local/google/home/simonhatch/chromium/src/tools/perf/core/perf_data_generator.py:870
    BENCHMARK_REF_BUILD_BLACKLIST)
  generate_telemetry_tests at /usr/local/google/home/simonhatch/chromium/src/tools/perf/core/perf_data_generator.py:775
    benchmark.Name()))
ValueError: No sharding map for benchmark 'dummy_benchmark.histogram_benchmark_1' found. Please disable the benchmark with @Disabled('all'), and file a bug with Speed>Benchmarks>Waterfall component and cc martiniss@ and nednguyen@ to execute the benchmark on the waterfall.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/a3265d0ca04d6bb7415687c3d7cde3f158da0504

commit a3265d0ca04d6bb7415687c3d7cde3f158da0504
Author: Simon <simonhatch@chromium.org>
Date: Fri Sep 15 20:50:01 2017

Add --xvfb to histogram benchmark bot.

Looking at logs for recent runs of this benchmark, see failures about not having a display environment set.

Bug: 758702
Change-Id: Ica3c76d80c456b3ec43f0a58ce0bb865b7a764d7
Reviewed-on: https://chromium-review.googlesource.com/669145
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Simon Hatch <simonhatch@chromium.org>
Cr-Commit-Position: refs/heads/master@{#502360}
[modify] https://crrev.com/a3265d0ca04d6bb7415687c3d7cde3f158da0504/testing/buildbot/chromium.perf.fyi.json
[modify] https://crrev.com/a3265d0ca04d6bb7415687c3d7cde3f158da0504/tools/perf/chromium.perf.fyi.extras.json