The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infra/puppet/+/1c172e46dfb3fa7b9d6285c27c397d5c2273e4d7

commit 1c172e46dfb3fa7b9d6285c27c397d5c2273e4d7
Author: Ethan Kuefner <eakuefner@chromium.org>
Date: Wed Jul 19 21:19:22 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/9d00951723b85c2085613c6655023f5a094812d5

commit 9d00951723b85c2085613c6655023f5a094812d5
Author: Ethan Kuefner <eakuefner@chromium.org>
Date: Mon Aug 14 20:56:42 2017

Add --isolated-script-test-perf-output flag to test harnesses

In preparation for renaming the isolated-script-test-chartjson-output flag
that Swarming passes to test harnesses, we add this argument to all harnesses
in src/. This is a two-sided patch, so the next piece will rename the arg on
the swarming side, and then we can remove all references on the src side to
the old name.

Doing the rename on the swarming side may be blocked on reconciling a few
cases that this CL leaves (like chartjson being required in cases where it
should not be, or harnesses needing to temporarily check for the presence of
at most one arg), but in the interest of simplicity, this CL only does the
add.

Bug:  744736 
Change-Id: Ic37a53a8f82e33ab1c04f291f42408ab048dbb4e
Reviewed-on: https://chromium-review.googlesource.com/587291
Commit-Queue: Ethan Kuefner <eakuefner@chromium.org>
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Reviewed-by: Emily Hanley <eyaich@chromium.org>
Reviewed-by: Christopher Grant <cjgrant@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Cr-Commit-Position: refs/heads/master@{#494183}
[modify] https://crrev.com/9d00951723b85c2085613c6655023f5a094812d5/chrome/test/vr/perf/vr_test_arg_parser.py
[modify] https://crrev.com/9d00951723b85c2085613c6655023f5a094812d5/testing/scripts/content_shell_crash_test.py
[modify] https://crrev.com/9d00951723b85c2085613c6655023f5a094812d5/testing/scripts/run_devtools_check.py
[modify] https://crrev.com/9d00951723b85c2085613c6655023f5a094812d5/testing/scripts/run_gpu_integration_test_as_googletest.py
[modify] https://crrev.com/9d00951723b85c2085613c6655023f5a094812d5/testing/scripts/run_gtest_perf_test.py
[modify] https://crrev.com/9d00951723b85c2085613c6655023f5a094812d5/testing/scripts/run_isolated_script_test.py
[modify] https://crrev.com/9d00951723b85c2085613c6655023f5a094812d5/testing/scripts/run_telemetry_as_googletest.py
[modify] https://crrev.com/9d00951723b85c2085613c6655023f5a094812d5/testing/scripts/run_telemetry_benchmark_as_googletest.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/ac7cf203c7e48a49bb324d25fae0f3759ddf296f

commit ac7cf203c7e48a49bb324d25fae0f3759ddf296f
Author: Ethan Kuefner <eakuefner@chromium.org>
Date: Tue Aug 22 19:08:37 2017

[Upload] Add histogram support to results_dashboard.py

This CL adds support for uploading histograms to results_dashboard.py.
Followup CLs will:

  1. Plumb this up to upload_perf_dashboard_results.py, and
  2. Make use of this functionality when calling
     upload_perf_dashboard_results.py from the recipe.

Bug:744736
Change-Id: Iba8a3b30904a364e86410405ca7818aefdb6d3ee
Reviewed-on: https://chromium-review.googlesource.com/611795
Commit-Queue: Ethan Kuefner <eakuefner@chromium.org>
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Stephen Martinis <martiniss@chromium.org>

[modify] https://crrev.com/ac7cf203c7e48a49bb324d25fae0f3759ddf296f/scripts/slave/results_dashboard.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1571ba8f8e1329923d40766b84ebf3671c7cce5a

commit 1571ba8f8e1329923d40766b84ebf3671c7cce5a
Author: Ethan Kuefner <eakuefner@chromium.org>
Date: Tue Aug 29 23:39:25 2017

Make chartjson flag optional in test harnesses

One of the test harnesses marked chart JSON required even though it is
unused, and the other does use chart JSON, but will need to switch to the
other flag once https://chromium-review.googlesource.com/c/617607 lands.

This CL prepares for the aforementioned CL to land by marking the flag
optional in both harnesses, and updating the gtest perf harness to use the
old arg as a fallback if the new one is not present.

Bug:  744736 
Change-Id: I6a10d5befc0b42cdf50f4ec80ef63fc8c72205af
Reviewed-on: https://chromium-review.googlesource.com/617986
Commit-Queue: Ethan Kuefner <eakuefner@chromium.org>
Reviewed-by: Ned Nguyen (ooo til 08/30) <nednguyen@google.com>
Reviewed-by: Christopher Grant <cjgrant@chromium.org>
Reviewed-by: Emily Hanley <eyaich@chromium.org>
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Cr-Commit-Position: refs/heads/master@{#498288}
[modify] https://crrev.com/1571ba8f8e1329923d40766b84ebf3671c7cce5a/chrome/test/vr/perf/latency/webvr_latency_test.py
[modify] https://crrev.com/1571ba8f8e1329923d40766b84ebf3671c7cce5a/chrome/test/vr/perf/vrcore_fps/vrcore_fps_test.py
[modify] https://crrev.com/1571ba8f8e1329923d40766b84ebf3671c7cce5a/testing/scripts/content_shell_crash_test.py
[modify] https://crrev.com/1571ba8f8e1329923d40766b84ebf3671c7cce5a/testing/scripts/run_gtest_perf_test.py
[modify] https://crrev.com/1571ba8f8e1329923d40766b84ebf3671c7cce5a/testing/scripts/run_telemetry_benchmark_as_googletest.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c20a4959d178171ec50e4a98d22fc9da4f98ab4b

commit c20a4959d178171ec50e4a98d22fc9da4f98ab4b
Author: Ethan Kuefner <eakuefner@chromium.org>
Date: Thu Aug 31 03:06:52 2017

Add histogram output whitelist to perf_data_generator.py

As we ramp up to a soft launch of the histogram pipeline, we want to
gradually enable histogram uploads for specific benchmarks. This change
enables us to do that manually to start.

Bug:  744736 
Change-Id: Iaee447e8fa0f352c034aebed5383844158800564
Reviewed-on: https://chromium-review.googlesource.com/578474
Commit-Queue: Ethan Kuefner <eakuefner@chromium.org>
Reviewed-by: Emily Hanley <eyaich@chromium.org>
Cr-Commit-Position: refs/heads/master@{#498741}
[modify] https://crrev.com/c20a4959d178171ec50e4a98d22fc9da4f98ab4b/testing/buildbot/chromium.perf.fyi.json
[modify] https://crrev.com/c20a4959d178171ec50e4a98d22fc9da4f98ab4b/testing/buildbot/chromium.perf.json
[modify] https://crrev.com/c20a4959d178171ec50e4a98d22fc9da4f98ab4b/tools/perf/core/perf_data_generator.py
[modify] https://crrev.com/c20a4959d178171ec50e4a98d22fc9da4f98ab4b/tools/perf/core/perf_data_generator_unittest.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/3e6978a8c02dabd44a76e239554131a1014065a7

commit 3e6978a8c02dabd44a76e239554131a1014065a7
Author: Ethan Kuefner <eakuefner@chromium.org>
Date: Wed Sep 06 20:18:37 2017

Rename chartjson flag in swarming

This CL renames --isolated-script-test-chartjson-output and the
associated filename to --isolated-script-test-perf-output and
perftest-output.json respectively, in preparation for adding
histogram support to swarming/api.py.

This depends on
https://chromium-review.googlesource.com/c/chromium/src/+/617986
which is waiting on review.

Bug:744736
Change-Id: I7557fd851b57b90fbd664fb82d10bbcc7e3a05bb
Reviewed-on: https://chromium-review.googlesource.com/617607
Commit-Queue: Ethan Kuefner <eakuefner@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>

[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/tryserver_webrtc_win_x64_win10.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipe_modules/test_utils/test_api.py
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/client_webrtc_mac64_debug.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/tryserver_webrtc_win_experimental.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/tryserver_webrtc_linux_tsan2.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/client_webrtc_win32_debug.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium_trybot.expected/add_swarming_layout_tests_via_manual_diff_inspection_win.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/client_webrtc_linux_ubsan_vptr.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium.expected/dynamic_swarmed_sharded_passed_isolated_script_perf_test_disabled.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/tryserver_webrtc_linux32_rel.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/tryserver_webrtc_linux_asan.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium_trybot.expected/dynamic_swarmed_isolated_script_test_failure_no_result_json.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium_trybot.expected/swarmed_webkit_tests_interrupted.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium.expected/dynamic_swarmed_sharded_passed_isolated_script_perf_test.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipe_modules/swarming/examples/full.expected/isolated_script_expired_new.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium.expected/dynamic_swarmed_sharded_passed_isolated_script_perf_test_empty.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/client_webrtc_win64_debug.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/tryserver_webrtc_linux_ubsan_vptr.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipe_modules/chromium_tests/tests/steps/swarming_isolated_script_test.expected/chartjson_max_failures.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/tryserver_webrtc_win_rel.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium.expected/dynamic_swarmed_sharded_isolated_chartjson_test_missing_all_shards.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium.expected/dynamic_swarmed_isolated_script_perf_test_ignore_task_failure.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipe_modules/chromium_tests/tests/steps/swarming_isolated_script_test.expected/basic.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium_trybot.expected/swarmed_webkit_tests_minimal_pass_continues.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/client_webrtc_linux_memcheck.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/tryserver_webrtc_mac_rel.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipe_modules/chromium_tests/tests/steps/swarming_isolated_script_test.expected/dimensions_windows.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/tryserver_webrtc_win_asan.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/tryserver_webrtc_win_x64_dbg.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium.expected/dynamic_swarmed_failed_isolated_script_test.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/client_webrtc_fyi_win64_debug__win10_.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/tryserver_webrtc_linux_dbg_gerrit.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipe_modules/swarming/examples/full.expected/isolated_script_with_custom_merge.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/client_webrtc_win32_asan.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium_trybot.expected/add_swarming_layout_tests_via_manual_diff_inspection_linux.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/client_webrtc_linux_tsan_v2.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium.expected/dynamic_swarmed_sharded_isolated_script_test_harness_failure.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipe_modules/chromium_tests/tests/steps/generate_isolated_script.expected/swarming.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/tryserver_webrtc_linux_dbg.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium_trybot.expected/dynamic_isolated_script_test_with_args_on_trybot.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipe_modules/chromium_tests/tests/steps/swarming_isolated_script_test.expected/dimensions_mac.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/tryserver_webrtc_win_x64_rel.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/client_webrtc_linux64_debug_forced.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipe_modules/swarming/examples/full.expected/isolated_script_expired_old.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/client_webrtc_mac_asan.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipe_modules/swarming/examples/full.expected/isolated_script_with_custom_trigger_script.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/client_webrtc_linux_msan.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/client_webrtc_win64_release.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/client_webrtc_linux_asan.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/client_webrtc_win32_release.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium_trybot.expected/dynamic_isolated_script_test_on_trybot_passing.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium.expected/dynamic_swarmed_sharded_isolated_chartjson_test_missing_shard.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium.expected/dynamic_swarmed_sharded_corrupt_json_isolated_script_test.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipe_modules/chromium_tests/tests/steps/swarming_isolated_script_test.expected/override_compile_targets.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium.expected/dynamic_swarmed_isolated_script_test_mac_gpu.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/client_webrtc_linux_ubsan.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipe_modules/swarming/examples/full.expected/isolated_script_timeout_old.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/tryserver_webrtc_mac_asan.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/tryserver_webrtc_linux_experimental.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipe_modules/chromium_tests/tests/steps/generate_isolated_script.expected/swarming_dimension_sets.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium_trybot.expected/dynamic_isolated_script_test_on_trybot_failing.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/tryserver_webrtc_linux_ubsan.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/tryserver_webrtc_linux_msan.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipe_modules/swarming/api.py
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium_trybot.expected/swarmed_webkit_tests_compile_without_patch_fails.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/tryserver_webrtc_win_dbg.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium.expected/dynamic_swarmed_isolated_script_test_linux_gpu.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium_trybot.expected/swarmed_layout_tests_too_many_failures_for_retcode.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipe_modules/chromium_tests/tests/steps/swarming_isolated_script_test.expected/chartjson_no_results_failure.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium.expected/dynamic_swarmed_sharded_failed_isolated_script_test.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipe_modules/chromium_tests/tests/steps/swarming_isolated_script_test.expected/dimensions_mac_hidpi.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/client_webrtc_fyi_webrtc_perf_tests__linux_trusty_swarming_.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipe_modules/chromium_tests/tests/steps/generate_isolated_script.expected/swarming_trigger_script.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/tryserver_webrtc_linux_rel.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/client_webrtc_linux64_release.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/tryserver_webrtc_linux32_dbg.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium_trybot.expected/add_swarming_layout_tests_via_manual_diff_inspection_that_fails.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipe_modules/chromium_tests/tests/steps/swarming_isolated_script_test.expected/chartjson_ignore_task_failure.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/client_webrtc_linux64_debug_failing_test.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipe_modules/chromium_tests/tests/steps/swarming_isolated_script_test.expected/chartjson_invalid.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipe_modules/chromium_tests/tests/steps/swarming_isolated_script_test.expected/chartjson_no_results.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipe_modules/chromium_tests/tests/steps/swarming_isolated_script_test.expected/chartjson_disabled.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium.expected/dynamic_swarmed_passed_isolated_script_test.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipe_modules/swarming/examples/full.expected/isolated_large_outdir.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/client_webrtc_mac64_release.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipe_modules/chromium_tests/tests/steps/generate_isolated_script.expected/fake_results_handler.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium.expected/dynamic_swarmed_passed_with_bad_retcode_isolated_script_test.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium.expected/dynamic_swarmed_passed_isolated_script_test_with_swarming_failure.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipe_modules/swarming/examples/full.expected/isolated_script_timeout_new.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium.expected/dynamic_swarmed_isolated_script_test_win_non_gpu.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium.expected/dynamic_swarmed_isolated_script_test_failure_no_result_json.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/tryserver_webrtc_win_x64_win8.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/tryserver_webrtc_linux_memcheck.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium_trybot.expected/swarmed_layout_tests_with_and_without_patch_fail.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium.expected/dynamic_swarmed_sharded_isolated_chartjson_test_disabled.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium.expected/dynamic_swarmed_sharded_isolated_chartjson_test_harness_failure.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium.expected/dynamic_swarmed_sharded_invalid_json_isolated_script_test.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium_trybot.expected/swarmed_webkit_tests_unexpected_error.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/tryserver_webrtc_mac_dbg.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipe_modules/chromium_tests/tests/steps/swarming_isolated_script_test.expected/chartjson_not_uploading.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium.expected/dynamic_swarmed_sharded_passed_isolated_script_perf_test_no_chartjson.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/client_webrtc_linux64_debug.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipe_modules/chromium_tests/tests/steps/swarming_isolated_script_test.expected/chartjson.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium.expected/dynamic_swarmed_sharded_isolated_script_test_missing_shard.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/client_webrtc_fyi_win64_debug__win8_.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium.expected/dynamic_swarmed_isolated_script_test_win_gpu.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/webrtc/standalone.expected/tryserver_webrtc_mac_experimental.json
[modify] https://crrev.com/3e6978a8c02dabd44a76e239554131a1014065a7/scripts/slave/recipes/chromium.expected/dynamic_swarmed_sharded_passed_isolated_script_test.json

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/60ab2bba2366227984e6b434eaf95e6987a2bd7c

commit 60ab2bba2366227984e6b434eaf95e6987a2bd7c
Author: Ethan Kuefner <eakuefner@chromium.org>
Date: Fri Sep 08 02:36:53 2017

Add histogram support to swarming/api.py

This CL adds support for histogram output to swarming/api.py. It does
this by generalizing the concept of chart JSON output already present
for isolated script tests to "perftest output", which can either be
chart JSON or HistogramSet. We distinguish between what type of test
results are being dealt with by using a type heuristic (dict for chart
JSON, list for HistogramSet) and merge appropriately.

Bug:744736
Change-Id: Ifbaa4c04593a9d2485a175d28fae278f080647fd
Reviewed-on: https://chromium-review.googlesource.com/625683
Commit-Queue: Simon Hatch <simonhatch@chromium.org>
Reviewed-by: Stephen Martinis <martiniss@chromium.org>

[add] https://crrev.com/60ab2bba2366227984e6b434eaf95e6987a2bd7c/scripts/slave/recipes/chromium.expected/dynamic_swarmed_sharded_passed_isolated_script_perf_test_histograms.json
[modify] https://crrev.com/60ab2bba2366227984e6b434eaf95e6987a2bd7c/scripts/slave/recipes/chromium.expected/dynamic_swarmed_sharded_isolated_chartjson_test_harness_failure.json
[modify] https://crrev.com/60ab2bba2366227984e6b434eaf95e6987a2bd7c/scripts/slave/recipe_modules/test_utils/test_api.py
[modify] https://crrev.com/60ab2bba2366227984e6b434eaf95e6987a2bd7c/scripts/slave/README.recipes.md
[modify] https://crrev.com/60ab2bba2366227984e6b434eaf95e6987a2bd7c/scripts/slave/recipes/chromium.py
[modify] https://crrev.com/60ab2bba2366227984e6b434eaf95e6987a2bd7c/scripts/slave/recipe_modules/swarming/api.py
[modify] https://crrev.com/60ab2bba2366227984e6b434eaf95e6987a2bd7c/scripts/slave/recipe_modules/chromium_tests/steps.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/ae2292d50507da44c5f971de25619a5acf63ccdd

commit ae2292d50507da44c5f971de25619a5acf63ccdd
Author: Ethan Kuefner <eakuefner@chromium.org>
Date: Mon Sep 11 18:17:55 2017

Add histogram support to upload_perf_dashboard_results

This CL adds two new flags to upload_perf_dashboard_results,
--send-as-histograms, and --oauth-token-file. These two arguments are
accepted directly by results_dashboard.SendResults, which
upload_perf_dashboard_results calls.

We use a file to obfuscate the actual oauth token, so that it is not printed
into logs and cannot be used for spoofing.

These new flags will be used by a CL that adds support for the histogram
uploading flow to chromium_tests/steps.py.

Bug:744736
Change-Id: Ie89ab021730ff0d56e68b654e665ddb8c3f00e4a
Reviewed-on: https://chromium-review.googlesource.com/653202
Commit-Queue: Ethan Kuefner <eakuefner@chromium.org>
Reviewed-by: Stephen Martinis <martiniss@chromium.org>

[modify] https://crrev.com/ae2292d50507da44c5f971de25619a5acf63ccdd/scripts/slave/upload_perf_dashboard_results.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/cb81fabf71b17e70f4ac10ad2d995034019c5625

commit cb81fabf71b17e70f4ac10ad2d995034019c5625
Author: Ethan Kuefner <eakuefner@chromium.org>
Date: Wed Sep 13 18:17:37 2017

Add json3 to histogram benchmark bot

The missing --output-format=json-test-results is causing the FYI bot to be
red.

Bug:  744736 
Change-Id: Icd6e93d86346c48ff9bc99d5b61500c39dc3d5dc
Reviewed-on: https://chromium-review.googlesource.com/663423
Reviewed-by: Simon Hatch <simonhatch@chromium.org>
Reviewed-by: Stephen Martinis <martiniss@chromium.org>
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Ethan Kuefner <eakuefner@chromium.org>
Cr-Commit-Position: refs/heads/master@{#501694}
[modify] https://crrev.com/cb81fabf71b17e70f4ac10ad2d995034019c5625/testing/buildbot/chromium.perf.fyi.json
[modify] https://crrev.com/cb81fabf71b17e70f4ac10ad2d995034019c5625/tools/perf/chromium.perf.fyi.extras.json

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/91d53fd3718d59bbbfbfe668dfa47c7a17d591e8

commit 91d53fd3718d59bbbfbfe668dfa47c7a17d591e8
Author: Ethan Kuefner <eakuefner@chromium.org>
Date: Wed Sep 20 05:41:12 2017

Wire up histogram pipeline in chromium_tests

This CL modifies the uploading portion of chromium_tests/steps.py to support
the histogram pipeline. In particular, it:

  * Generates an oauth token to be passed to upload_perf_dashboard_results
  * Calls the Catapult-side add_reserved_diagnostics script to add revision
    and builder annotations
  * Passes the is_histogramset flag properly.

Bug:744736
Change-Id: I3c868029ca55305bac9d474cb574c54c4a45fdbb
Reviewed-on: https://chromium-review.googlesource.com/653206
Commit-Queue: Ethan Kuefner <eakuefner@chromium.org>
Reviewed-by: Stephen Martinis <martiniss@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>

[add] https://crrev.com/91d53fd3718d59bbbfbfe668dfa47c7a17d591e8/scripts/slave/recipe_modules/chromium_tests/tests/steps/swarming_isolated_script_test.expected/histograms.json
[modify] https://crrev.com/91d53fd3718d59bbbfbfe668dfa47c7a17d591e8/scripts/slave/recipes/chromium.expected/dynamic_swarmed_isolated_script_perf_test_ignore_task_failure.json
[modify] https://crrev.com/91d53fd3718d59bbbfbfe668dfa47c7a17d591e8/scripts/slave/recipe_modules/chromium_tests/tests/steps/swarming_isolated_script_test.expected/chartjson_ignore_task_failure.json
[modify] https://crrev.com/91d53fd3718d59bbbfbfe668dfa47c7a17d591e8/scripts/slave/recipe_modules/chromium_tests/tests/steps/swarming_isolated_script_test.expected/basic.json
[modify] https://crrev.com/91d53fd3718d59bbbfbfe668dfa47c7a17d591e8/scripts/slave/recipe_modules/chromium_tests/tests/steps/swarming_isolated_script_test.expected/chartjson_no_results.json
[modify] https://crrev.com/91d53fd3718d59bbbfbfe668dfa47c7a17d591e8/scripts/slave/recipe_modules/chromium_tests/tests/steps/swarming_isolated_script_test.expected/chartjson_simplified_ignore_task_failure.json
[modify] https://crrev.com/91d53fd3718d59bbbfbfe668dfa47c7a17d591e8/scripts/slave/recipe_modules/chromium_tests/tests/steps/swarming_isolated_script_test.py
[modify] https://crrev.com/91d53fd3718d59bbbfbfe668dfa47c7a17d591e8/scripts/slave/recipe_modules/chromium_tests/tests/steps/swarming_isolated_script_test.expected/override_compile_targets.json
[modify] https://crrev.com/91d53fd3718d59bbbfbfe668dfa47c7a17d591e8/scripts/slave/recipe_modules/chromium_tests/tests/steps/swarming_isolated_script_test.expected/dimensions_windows.json
[modify] https://crrev.com/91d53fd3718d59bbbfbfe668dfa47c7a17d591e8/scripts/slave/recipe_modules/chromium_tests/tests/steps/swarming_isolated_script_test.expected/chartjson_disabled.json
[modify] https://crrev.com/91d53fd3718d59bbbfbfe668dfa47c7a17d591e8/scripts/slave/recipe_modules/chromium_tests/tests/steps/swarming_isolated_script_test.expected/chartjson_simplified_disabled.json
[modify] https://crrev.com/91d53fd3718d59bbbfbfe668dfa47c7a17d591e8/scripts/slave/recipe_modules/chromium_tests/__init__.py
[modify] https://crrev.com/91d53fd3718d59bbbfbfe668dfa47c7a17d591e8/scripts/slave/recipes/chromium.expected/dynamic_swarmed_sharded_passed_isolated_script_perf_test_histograms.json
[modify] https://crrev.com/91d53fd3718d59bbbfbfe668dfa47c7a17d591e8/scripts/slave/recipe_modules/chromium_tests/tests/steps/swarming_isolated_script_test.expected/chartjson_invalid.json
[modify] https://crrev.com/91d53fd3718d59bbbfbfe668dfa47c7a17d591e8/scripts/slave/recipe_modules/chromium_tests/steps.py
[modify] https://crrev.com/91d53fd3718d59bbbfbfe668dfa47c7a17d591e8/scripts/slave/recipes/chromium.expected/dynamic_swarmed_sharded_passed_isolated_script_perf_test.json
[modify] https://crrev.com/91d53fd3718d59bbbfbfe668dfa47c7a17d591e8/scripts/slave/recipe_modules/chromium_tests/tests/steps/swarming_isolated_script_test.expected/chartjson_not_uploading.json
[modify] https://crrev.com/91d53fd3718d59bbbfbfe668dfa47c7a17d591e8/scripts/slave/recipe_modules/chromium_tests/tests/steps/swarming_isolated_script_test.expected/dimensions_mac.json
[modify] https://crrev.com/91d53fd3718d59bbbfbfe668dfa47c7a17d591e8/scripts/slave/recipe_modules/chromium_tests/tests/steps/swarming_isolated_script_test.expected/chartjson_no_results_failure.json
[modify] https://crrev.com/91d53fd3718d59bbbfbfe668dfa47c7a17d591e8/scripts/slave/README.recipes.md
[modify] https://crrev.com/91d53fd3718d59bbbfbfe668dfa47c7a17d591e8/scripts/slave/recipe_modules/chromium_tests/tests/steps/swarming_isolated_script_test.expected/chartjson.json
[modify] https://crrev.com/91d53fd3718d59bbbfbfe668dfa47c7a17d591e8/scripts/slave/recipe_modules/chromium_tests/tests/steps/swarming_isolated_script_test.expected/dimensions_mac_hidpi.json
[modify] https://crrev.com/91d53fd3718d59bbbfbfe668dfa47c7a17d591e8/scripts/slave/recipe_modules/chromium_tests/tests/steps/swarming_isolated_script_test.expected/chartjson_max_failures.json

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/42161623ec7697ced713dfe309d8db1c68be1b27

commit 42161623ec7697ced713dfe309d8db1c68be1b27
Author: Ethan Kuefner <eakuefner@chromium.org>
Date: Thu Sep 21 03:08:43 2017

Pass --also-run-disabled-tests for histogram bot

The histogram dummy benchmark is disabled by default, so we need to pass
--also-run-disabled-tests so that it can be run on FYI.

Bug:  744736 
Change-Id: If60106bc68a034c75a516eae9b15f69aec7a727d
TBR: sullivan,martiniss,simonhatch
Reviewed-on: https://chromium-review.googlesource.com/676534
Reviewed-by: Ethan Kuefner <eakuefner@chromium.org>
Commit-Queue: Ethan Kuefner <eakuefner@chromium.org>
Cr-Commit-Position: refs/heads/master@{#503343}
[modify] https://crrev.com/42161623ec7697ced713dfe309d8db1c68be1b27/testing/buildbot/chromium.perf.fyi.json
[modify] https://crrev.com/42161623ec7697ced713dfe309d8db1c68be1b27/tools/perf/chromium.perf.fyi.extras.json

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/eaf808252d7fa54f19edb09946a2071eff2dabc3

commit eaf808252d7fa54f19edb09946a2071eff2dabc3
Author: Ethan Kuefner <eakuefner@chromium.org>
Date: Tue Sep 26 21:13:20 2017

Pick up histogram output file if we're outputting histograms

HistogramSet has a different default output filename when using Telemetry; we
need to account for this in run_telemetry_benchmark_as_googletest so that that
script can find the histogram output if present.

Bug:  744736 
Change-Id: I1fec4a2d86e7e0a9700bc0b44160b78f0484d866
Reviewed-on: https://chromium-review.googlesource.com/683780
Reviewed-by: Stephen Martinis <martiniss@chromium.org>
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Ned Nguyen <nednguyen@google.com>
Cr-Commit-Position: refs/heads/master@{#504486}
[modify] https://crrev.com/eaf808252d7fa54f19edb09946a2071eff2dabc3/testing/scripts/run_telemetry_benchmark_as_googletest.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/bd3be85eab146c8e8e71b448784cfabe3ff89ce9

commit bd3be85eab146c8e8e71b448784cfabe3ff89ce9
Author: Ethan Kuefner <eakuefner@chromium.org>
Date: Wed Nov 15 23:10:28 2017

[Perf] Switch loading.* to histogram pipeline

This CL enables the histogram pipeline for loading.desktop and loading.mobile.

Bug:  744736 
Change-Id: Id12cf136d78e4cae6b552e5713d6aba9c27b6d2a
Reviewed-on: https://chromium-review.googlesource.com/769329
Commit-Queue: Ethan Kuefner <eakuefner@chromium.org>
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Cr-Commit-Position: refs/heads/master@{#516878}
[modify] https://crrev.com/bd3be85eab146c8e8e71b448784cfabe3ff89ce9/testing/buildbot/chromium.perf.fyi.json
[modify] https://crrev.com/bd3be85eab146c8e8e71b448784cfabe3ff89ce9/testing/buildbot/chromium.perf.json
[modify] https://crrev.com/bd3be85eab146c8e8e71b448784cfabe3ff89ce9/tools/perf/core/perf_data_generator.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/569f942d19bd6b5b730d7292d201a3bc2d0593f4

commit 569f942d19bd6b5b730d7292d201a3bc2d0593f4
Author: Ethan Kuefner <eakuefner@chromium.org>
Date: Thu Nov 16 13:24:05 2017

Revert "[Perf] Switch loading.* to histogram pipeline"

This reverts commit bd3be85eab146c8e8e71b448784cfabe3ff89ce9.

Reason for revert: <INSERT REASONING HERE>

Original change's description:
> [Perf] Switch loading.* to histogram pipeline
> 
> This CL enables the histogram pipeline for loading.desktop and loading.mobile.
> 
> Bug:  744736 
> Change-Id: Id12cf136d78e4cae6b552e5713d6aba9c27b6d2a
> Reviewed-on: https://chromium-review.googlesource.com/769329
> Commit-Queue: Ethan Kuefner <eakuefner@chromium.org>
> Reviewed-by: Ned Nguyen <nednguyen@google.com>
> Cr-Commit-Position: refs/heads/master@{#516878}

TBR=eakuefner@chromium.org,nednguyen@google.com,eyaich@chromium.org,ashleymarie@chromium.org

Change-Id: Iba3674f272545c1a6771feeb423fb097d5bbcd9d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug:  744736 
Reviewed-on: https://chromium-review.googlesource.com/774698
Reviewed-by: Ethan Kuefner <eakuefner@chromium.org>
Commit-Queue: Ethan Kuefner <eakuefner@chromium.org>
Cr-Commit-Position: refs/heads/master@{#517069}
[modify] https://crrev.com/569f942d19bd6b5b730d7292d201a3bc2d0593f4/testing/buildbot/chromium.perf.fyi.json
[modify] https://crrev.com/569f942d19bd6b5b730d7292d201a3bc2d0593f4/testing/buildbot/chromium.perf.json
[modify] https://crrev.com/569f942d19bd6b5b730d7292d201a3bc2d0593f4/tools/perf/core/perf_data_generator.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/70decc915c6864b4e563655155fd2b67a72c20c9

commit 70decc915c6864b4e563655155fd2b67a72c20c9
Author: Ethan Kuefner <eakuefner@chromium.org>
Date: Tue Nov 21 23:58:55 2017

Enable loading.desktop on histogram FYI bot

Bug:  744736 
Change-Id: Ibd0bd94b8e6a481892fcd968839da667c5746ca2
Reviewed-on: https://chromium-review.googlesource.com/782360
Reviewed-by: Simon Hatch <simonhatch@chromium.org>
Reviewed-by: Emily Hanley <eyaich@chromium.org>
Commit-Queue: Simon Hatch <simonhatch@chromium.org>
Cr-Commit-Position: refs/heads/master@{#518456}
[modify] https://crrev.com/70decc915c6864b4e563655155fd2b67a72c20c9/testing/buildbot/chromium.perf.fyi.json
[modify] https://crrev.com/70decc915c6864b4e563655155fd2b67a72c20c9/tools/perf/chromium.perf.fyi.extras.json

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/5b797fb49c51ca6f148e8c2f34bfbb84c55ba916

commit 5b797fb49c51ca6f148e8c2f34bfbb84c55ba916
Author: Ethan Kuefner <eakuefner@chromium.org>
Date: Thu Nov 30 15:11:35 2017

[Perf] Enable speedometer2 and smoothness.key_silk_cases on histogram FYI

This CL enables speedometer2, which is a LegacyPageTest-based benchmark,
and smoothness.key_silk_cases, which uses the TIR label feature, to give
us more coverage on the histogram FYI bot.

Bug:  744736 
Change-Id: I4343273cbf33eab10a3c803f2b4bbff71dccf962
Reviewed-on: https://chromium-review.googlesource.com/798124
Reviewed-by: Emily Hanley <eyaich@chromium.org>
Commit-Queue: Simon Hatch <simonhatch@chromium.org>
Cr-Commit-Position: refs/heads/master@{#520545}
[modify] https://crrev.com/5b797fb49c51ca6f148e8c2f34bfbb84c55ba916/testing/buildbot/chromium.perf.fyi.json
[modify] https://crrev.com/5b797fb49c51ca6f148e8c2f34bfbb84c55ba916/tools/perf/chromium.perf.fyi.extras.json

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c3db22be0cdd161ec137b081f34775bd8fd84c3e

commit c3db22be0cdd161ec137b081f34775bd8fd84c3e
Author: Ethan Kuefner <eakuefner@chromium.org>
Date: Wed Dec 06 16:57:31 2017

[Perf] Enable system_health.common_desktop and memory_desktop on histogram FYI

This CL enables the aforementioned two benchmarks on our histogram FYI testing
bot.

We also remove smoothness.key_silk_cases since that's a mobile-only benchmark.

Bug:  744736 
Change-Id: I65c21d8769625a34b73d038c093c1b472eacd3ee
Reviewed-on: https://chromium-review.googlesource.com/808667
Reviewed-by: Emily Hanley <eyaich@chromium.org>
Commit-Queue: Simon Hatch <simonhatch@chromium.org>
Cr-Commit-Position: refs/heads/master@{#522109}
[modify] https://crrev.com/c3db22be0cdd161ec137b081f34775bd8fd84c3e/testing/buildbot/chromium.perf.fyi.json
[modify] https://crrev.com/c3db22be0cdd161ec137b081f34775bd8fd84c3e/tools/perf/chromium.perf.fyi.extras.json

Simon has a spreadsheet for tracking the chromium.perf rollout here https://docs.google.com/spreadsheets/d/1slGyK5VA8-K2W07CXglO1k_0SOQfhjVsw-SFQkpDu-8/edit#gid=2027334032

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/98391d32c58ec6944fd489875efa2dd76348ee53

commit 98391d32c58ec6944fd489875efa2dd76348ee53
Author: Ethan Kuefner <eakuefner@chromium.org>
Date: Wed Apr 11 14:58:06 2018

[Perf] Enable histogram uploads for blink_perf.*

We're ready to try again with enabling some chromium.perf benchmarks to upload
histograms to the perf dashboard, so let's try with blink_perf.

Bug:  744736 
Change-Id: I859b63a27ad0b3aaa825a520a20d00bd7afe1ab4
Reviewed-on: https://chromium-review.googlesource.com/1005674
Reviewed-by: Emily Hanley <eyaich@chromium.org>
Commit-Queue: Ethan Kuefner <eakuefner@chromium.org>
Cr-Commit-Position: refs/heads/master@{#549882}
[modify] https://crrev.com/98391d32c58ec6944fd489875efa2dd76348ee53/testing/buildbot/chromium.perf.json
[modify] https://crrev.com/98391d32c58ec6944fd489875efa2dd76348ee53/testing/scripts/run_performance_tests.py
[modify] https://crrev.com/98391d32c58ec6944fd489875efa2dd76348ee53/tools/perf/core/perf_data_generator.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/767eac43b12f56b088eed35e5f1ba8d3a3b3e30c

commit 767eac43b12f56b088eed35e5f1ba8d3a3b3e30c
Author: Ethan Kuefner <eakuefner@chromium.org>
Date: Thu Apr 12 16:20:50 2018

[Perf] Enable histogram uploads for memory.top_10_mobile

Bug:  744736 
Change-Id: Ia1b895010fbb174476a29799b2151c4bc1e0265a
Reviewed-on: https://chromium-review.googlesource.com/1010284
Reviewed-by: Emily Hanley <eyaich@chromium.org>
Commit-Queue: Ethan Kuefner <eakuefner@chromium.org>
Cr-Commit-Position: refs/heads/master@{#550229}
[modify] https://crrev.com/767eac43b12f56b088eed35e5f1ba8d3a3b3e30c/testing/buildbot/chromium.perf.json
[modify] https://crrev.com/767eac43b12f56b088eed35e5f1ba8d3a3b3e30c/testing/scripts/run_performance_tests.py
[modify] https://crrev.com/767eac43b12f56b088eed35e5f1ba8d3a3b3e30c/tools/perf/core/perf_data_generator.py

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/f718fb12b8cfd16b07509674747abf56cf330ac8

commit f718fb12b8cfd16b07509674747abf56cf330ac8
Author: Emily Hanley <eyaich@google.com>
Date: Fri Apr 13 15:06:36 2018

Adding valid output for disabled benchmarks that output hisograms

Bug= chromium:744736 
Change-Id: Iec36a0406bcfe30ffe5ed791daa1e0ecb7b35696
Reviewed-on: https://chromium-review.googlesource.com/1012124
Reviewed-by: Ethan Kuefner <eakuefner@chromium.org>
Commit-Queue: Emily Hanley <eyaich@chromium.org>

[modify] https://crrev.com/f718fb12b8cfd16b07509674747abf56cf330ac8/telemetry/telemetry/internal/results/histogram_set_json_output_formatter.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/767eac43b12f56b088eed35e5f1ba8d3a3b3e30c

commit 767eac43b12f56b088eed35e5f1ba8d3a3b3e30c
Author: Ethan Kuefner <eakuefner@chromium.org>
Date: Thu Apr 12 16:20:50 2018

[Perf] Enable histogram uploads for memory.top_10_mobile

Bug:  744736 
Change-Id: Ia1b895010fbb174476a29799b2151c4bc1e0265a
Reviewed-on: https://chromium-review.googlesource.com/1010284
Reviewed-by: Emily Hanley <eyaich@chromium.org>
Commit-Queue: Ethan Kuefner <eakuefner@chromium.org>
Cr-Commit-Position: refs/heads/master@{#550229}
[modify] https://crrev.com/767eac43b12f56b088eed35e5f1ba8d3a3b3e30c/testing/buildbot/chromium.perf.json
[modify] https://crrev.com/767eac43b12f56b088eed35e5f1ba8d3a3b3e30c/testing/scripts/run_performance_tests.py
[modify] https://crrev.com/767eac43b12f56b088eed35e5f1ba8d3a3b3e30c/tools/perf/core/perf_data_generator.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f3f894934642a8189d08b532c18d49645c3ad6b3

commit f3f894934642a8189d08b532c18d49645c3ad6b3
Author: Ethan Kuefner <eakuefner@chromium.org>
Date: Tue May 08 17:10:51 2018

[Perf] Enable system_health.* for histogram uploads

This CL switches all system_health benchmarks to the histogram pipeline.

Bug:  744736 
Change-Id: Icb97397ec417d27220e489651cc51591757d8c9e
Reviewed-on: https://chromium-review.googlesource.com/1048626
Reviewed-by: Emily Hanley <eyaich@chromium.org>
Commit-Queue: Ethan Kuefner <eakuefner@chromium.org>
Cr-Commit-Position: refs/heads/master@{#556842}
[modify] https://crrev.com/f3f894934642a8189d08b532c18d49645c3ad6b3/testing/buildbot/chromium.perf.json
[modify] https://crrev.com/f3f894934642a8189d08b532c18d49645c3ad6b3/testing/scripts/run_performance_tests.py
[modify] https://crrev.com/f3f894934642a8189d08b532c18d49645c3ad6b3/tools/perf/core/perf_data_generator.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/983e4035400618b2fc9335160b82a71d29bce2b2

commit 983e4035400618b2fc9335160b82a71d29bce2b2
Author: Ethan Kuefner <eakuefner@chromium.org>
Date: Wed May 09 16:16:37 2018

Revert "[Perf] Enable system_health.* for histogram uploads"

This reverts commit f3f894934642a8189d08b532c18d49645c3ad6b3.

Reason for revert: Causing failures

Original change's description:
> [Perf] Enable system_health.* for histogram uploads
> 
> This CL switches all system_health benchmarks to the histogram pipeline.
> 
> Bug:  744736 
> Change-Id: Icb97397ec417d27220e489651cc51591757d8c9e
> Reviewed-on: https://chromium-review.googlesource.com/1048626
> Reviewed-by: Emily Hanley <eyaich@chromium.org>
> Commit-Queue: Ethan Kuefner <eakuefner@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#556842}

TBR=eakuefner@chromium.org,eyaich@chromium.org

Change-Id: I53f1788cc1f488b207a0718e8fbe14e2899af5b7
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug:  744736 
Reviewed-on: https://chromium-review.googlesource.com/1052407
Reviewed-by: Ethan Kuefner <eakuefner@chromium.org>
Commit-Queue: Ethan Kuefner <eakuefner@chromium.org>
Cr-Commit-Position: refs/heads/master@{#557211}
[modify] https://crrev.com/983e4035400618b2fc9335160b82a71d29bce2b2/testing/buildbot/chromium.perf.json
[modify] https://crrev.com/983e4035400618b2fc9335160b82a71d29bce2b2/testing/scripts/run_performance_tests.py
[modify] https://crrev.com/983e4035400618b2fc9335160b82a71d29bce2b2/tools/perf/core/perf_data_generator.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/84af3a5b198eba903a67a4d751ff69008d852da6

commit 84af3a5b198eba903a67a4d751ff69008d852da6
Author: Simon <simonhatch@chromium.org>
Date: Tue May 15 16:32:32 2018

Reland "[Perf] Enable system_health.* for histogram uploads"

This reverts commit 983e4035400618b2fc9335160b82a71d29bce2b2.

Reason for revert: Trying again

Original change's description:
> Revert "[Perf] Enable system_health.* for histogram uploads"
> 
> This reverts commit f3f894934642a8189d08b532c18d49645c3ad6b3.
> 
> Reason for revert: Causing failures
> 
> Original change's description:
> > [Perf] Enable system_health.* for histogram uploads
> > 
> > This CL switches all system_health benchmarks to the histogram pipeline.
> > 
> > Bug:  744736 
> > Change-Id: Icb97397ec417d27220e489651cc51591757d8c9e
> > Reviewed-on: https://chromium-review.googlesource.com/1048626
> > Reviewed-by: Emily Hanley <eyaich@chromium.org>
> > Commit-Queue: Ethan Kuefner <eakuefner@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#556842}
> 
> TBR=eakuefner@chromium.org,eyaich@chromium.org
> 
> Change-Id: I53f1788cc1f488b207a0718e8fbe14e2899af5b7
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Bug:  744736 
> Reviewed-on: https://chromium-review.googlesource.com/1052407
> Reviewed-by: Ethan Kuefner <eakuefner@chromium.org>
> Commit-Queue: Ethan Kuefner <eakuefner@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#557211}

TBR=eakuefner@chromium.org,eyaich@chromium.org

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug:  744736 
Change-Id: Ie8e2f4a2e1c2d4b3e84a9edce1ca88e4d7005f39
Reviewed-on: https://chromium-review.googlesource.com/1059733
Commit-Queue: Simon Hatch <simonhatch@chromium.org>
Reviewed-by: Simon Hatch <simonhatch@chromium.org>
Reviewed-by: Emily Hanley <eyaich@chromium.org>
Cr-Commit-Position: refs/heads/master@{#558738}
[modify] https://crrev.com/84af3a5b198eba903a67a4d751ff69008d852da6/testing/buildbot/chromium.perf.json
[modify] https://crrev.com/84af3a5b198eba903a67a4d751ff69008d852da6/testing/scripts/run_performance_tests.py
[modify] https://crrev.com/84af3a5b198eba903a67a4d751ff69008d852da6/tools/perf/core/perf_data_generator.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c8f39ea4ccdbff0d445da0a3a5c897b5a456ad80

commit c8f39ea4ccdbff0d445da0a3a5c897b5a456ad80
Author: Simon <simonhatch@chromium.org>
Date: Wed May 16 03:54:25 2018

[Perf] Enable smoothness.* benchmarks for histogram uploads

Landing this for eakuefner@ since he's OOO this week.

This CL switches smoothness.* benchmarks to the histogram pipeline.

Bug:  744736 
Change-Id: Ic9a52a0785deb7df3b3b2e065c4db7ba3b6a2fcf
Reviewed-on: https://chromium-review.googlesource.com/1059933
Commit-Queue: Simon Hatch <simonhatch@chromium.org>
Reviewed-by: Emily Hanley <eyaich@chromium.org>
Cr-Commit-Position: refs/heads/master@{#558965}
[modify] https://crrev.com/c8f39ea4ccdbff0d445da0a3a5c897b5a456ad80/testing/buildbot/chromium.perf.json
[modify] https://crrev.com/c8f39ea4ccdbff0d445da0a3a5c897b5a456ad80/testing/scripts/run_performance_tests.py
[modify] https://crrev.com/c8f39ea4ccdbff0d445da0a3a5c897b5a456ad80/tools/perf/core/perf_data_generator.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/5cb833b8f3f1967c8bf5317139e9caf968da8d72

commit 5cb833b8f3f1967c8bf5317139e9caf968da8d72
Author: Simon <simonhatch@chromium.org>
Date: Wed May 16 12:51:31 2018

[Perf] Enable various small benchmarks for histogram uploads

Landing this for eakuefner@ since he's OOO this week.

This CL switches various small (mostly V8 "press") benchmarks to the
histogram pipeline.

Bug:  744736 
Change-Id: I2391ed93f53bd5471498513c9dc6effa8c25dfba
Reviewed-on: https://chromium-review.googlesource.com/1059586
Commit-Queue: Simon Hatch <simonhatch@chromium.org>
Reviewed-by: Emily Hanley <eyaich@chromium.org>
Cr-Commit-Position: refs/heads/master@{#559059}
[modify] https://crrev.com/5cb833b8f3f1967c8bf5317139e9caf968da8d72/testing/buildbot/chromium.perf.json
[modify] https://crrev.com/5cb833b8f3f1967c8bf5317139e9caf968da8d72/testing/scripts/run_performance_tests.py
[modify] https://crrev.com/5cb833b8f3f1967c8bf5317139e9caf968da8d72/tools/perf/core/perf_data_generator.py
[modify] https://crrev.com/5cb833b8f3f1967c8bf5317139e9caf968da8d72/tools/perf/core/perf_data_generator_unittest.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/e320062a8adeabedc679bf73f652fa316467021f

commit e320062a8adeabedc679bf73f652fa316467021f
Author: Simon <simonhatch@chromium.org>
Date: Wed May 16 18:26:14 2018

[Perf] Enable some smaller benchmarks for histogram uploads

Landing this for eakuefner@ since he's OOO this week.

This CL switches some smaller benchmarks to the histogram pipeline.
Ie. thread_times, battor, tracing_with_*, etc.

Bug:  744736 
Change-Id: I5bafea53e9d8b82bb3d38559192ab54565b63f44
Reviewed-on: https://chromium-review.googlesource.com/1059937
Reviewed-by: Emily Hanley <eyaich@chromium.org>
Commit-Queue: Simon Hatch <simonhatch@chromium.org>
Cr-Commit-Position: refs/heads/master@{#559192}
[modify] https://crrev.com/e320062a8adeabedc679bf73f652fa316467021f/testing/buildbot/chromium.perf.json
[modify] https://crrev.com/e320062a8adeabedc679bf73f652fa316467021f/testing/scripts/run_performance_tests.py
[modify] https://crrev.com/e320062a8adeabedc679bf73f652fa316467021f/tools/perf/core/perf_data_generator.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ff06bc7fb63a25d10da687d857ef9806df4f49f2

commit ff06bc7fb63a25d10da687d857ef9806df4f49f2
Author: Ethan Kuefner <eakuefner@chromium.org>
Date: Thu May 24 16:43:53 2018

[Perf] Enable v8.browsing_* for histogram uploads

This CL switches the v8.browsing_* benchmarks to the histogram pipeline.

NOTRY=true

Bug:  744736 
Change-Id: Ied3d2bc472251e875e45b2d932d2951ff1cba06b
Reviewed-on: https://chromium-review.googlesource.com/1048849
Commit-Queue: Ethan Kuefner <eakuefner@chromium.org>
Reviewed-by: Emily Hanley <eyaich@chromium.org>
Cr-Commit-Position: refs/heads/master@{#561524}
[modify] https://crrev.com/ff06bc7fb63a25d10da687d857ef9806df4f49f2/testing/buildbot/chromium.perf.json
[modify] https://crrev.com/ff06bc7fb63a25d10da687d857ef9806df4f49f2/testing/scripts/run_performance_tests.py
[modify] https://crrev.com/ff06bc7fb63a25d10da687d857ef9806df4f49f2/tools/perf/core/perf_data_generator.py

Dave, is there anything blocking us from ensuring that Pinpoint always uses histograms for chromium.perf and chromium.perf.fyi?

https://chromium-review.googlesource.com/c/chromium/src/+/1070219 flips the flag for the waterfalls, and Emily mentioned that we need to keep the logic that maybe appends --output-format=histograms until Pinpoint doesn't need to make a decision as to whether to use histograms or not.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4b0d7decd3fdf8bd44ff79af4b6f168a69296cb1

commit 4b0d7decd3fdf8bd44ff79af4b6f168a69296cb1
Author: Ethan Kuefner <eakuefner@chromium.org>
Date: Tue May 29 21:53:50 2018

[Perf] Flip flag for histogram pipeline

We have already whitelisted enough benchmarks that we want to flip the flag
for all Telemetry benchmarks. system_health was the largest benchmark so we
don't expect to encounter request deadline issues on the perf dashboard
side.

Bug:  744736 
Change-Id: I62ce9511a32a56b3eb33bc35bdd6e487f135083f
Reviewed-on: https://chromium-review.googlesource.com/1070219
Commit-Queue: Ethan Kuefner <eakuefner@chromium.org>
Reviewed-by: Emily Hanley <eyaich@chromium.org>
Cr-Commit-Position: refs/heads/master@{#562610}
[modify] https://crrev.com/4b0d7decd3fdf8bd44ff79af4b6f168a69296cb1/testing/buildbot/chromium.perf.json
[modify] https://crrev.com/4b0d7decd3fdf8bd44ff79af4b6f168a69296cb1/testing/scripts/run_performance_tests.py
[modify] https://crrev.com/4b0d7decd3fdf8bd44ff79af4b6f168a69296cb1/tools/perf/core/perf_data_generator.py
[modify] https://crrev.com/4b0d7decd3fdf8bd44ff79af4b6f168a69296cb1/tools/perf/core/perf_data_generator_unittest.py

Pinpoint has been using histograms exclusively for a few months already!
Pinpoint doesn't use that appending logic, it creates the test commands from scratch here:
https://chromium.googlesource.com/catapult.git/+/HEAD/dashboard/dashboard/pinpoint/models/quest/run_telemetry_test.py#13


Is `--output-format=histograms` becoming the default? We'll want to retain the flag in Pinpoint for a few months since we may bisect on old versions of Chromium/Catapult that may require the flag.

Thanks Dave. It's not becoming the default in the sense that Telemetry intends to output HistogramSet JSON if --output-format is not passed; HTML will remain the default.

That said, run_performance_tests now (as of the above CL) always appends --output-format=histograms as long as no other --output-format flag has been passed (e.g. by Pinpoint). It sounds like this is the sense in which you mean it, and if so, yes, I think it would be good to clean up the flag on the Pinpoint side in a few months and simplify the logic on the src side.

Filed https://bugs.chromium.org/p/chromium/issues/detail?id=848077 about cleaning this up.