The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/1d2e7600b0b457ab18c533122efc97a1d35c858c

commit 1d2e7600b0b457ab18c533122efc97a1d35c858c
Author: Emily Hanley <eyaich@google.com>
Date: Fri May 11 17:30:15 2018

Adding --smoke_test_mode to telemetry to not repeat stories

Bug:  chromium:840427 
Change-Id: I8d88507f36f7becf4f3503037788ee12f81fb371
Reviewed-on: https://chromium-review.googlesource.com/1047793
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Emily Hanley <eyaich@chromium.org>

[modify] https://crrev.com/1d2e7600b0b457ab18c533122efc97a1d35c858c/telemetry/telemetry/internal/story_runner.py
[modify] https://crrev.com/1d2e7600b0b457ab18c533122efc97a1d35c858c/telemetry/telemetry/internal/story_runner_unittest.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/403ed1551d4d6e1c90ebf7dced75e2dabf7cbb86

commit 403ed1551d4d6e1c90ebf7dced75e2dabf7cbb86
Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Fri May 11 19:58:36 2018

Roll src/third_party/catapult/ b6e96e16f..1d2e7600b (1 commit)

https://chromium.googlesource.com/catapult.git/+log/b6e96e16f364..1d2e7600b0b4

$ git log b6e96e16f..1d2e7600b --date=short --no-merges --format='%ad %ae %s'
2018-05-11 eyaich Adding --smoke_test_mode to telemetry to not repeat stories

Created with:
  roll-dep src/third_party/catapult
BUG= chromium:840427 


The AutoRoll server is located here: https://catapult-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.


TBR=sullivan@chromium.org

Change-Id: Iee4ae15aea0c7dc2e76c18f19f21cdcff03624c8
Reviewed-on: https://chromium-review.googlesource.com/1055800
Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#557988}
[modify] https://crrev.com/403ed1551d4d6e1c90ebf7dced75e2dabf7cbb86/DEPS

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/18ae7760e18c810aaa8b49aac5bd450fb589f433

commit 18ae7760e18c810aaa8b49aac5bd450fb589f433
Author: Emily Hanley <eyaich@google.com>
Date: Mon May 14 17:07:58 2018

Creating a flag for perf jobs to run on the reference build.

Bug:  840427 
Change-Id: Icd9263f3f03ef9f38a4bc2fbc52ea5da28e520a2
Reviewed-on: https://chromium-review.googlesource.com/1057435
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Emily Hanley <eyaich@chromium.org>
Cr-Commit-Position: refs/heads/master@{#558345}
[modify] https://crrev.com/18ae7760e18c810aaa8b49aac5bd450fb589f433/testing/buildbot/chromium.perf.fyi.json
[modify] https://crrev.com/18ae7760e18c810aaa8b49aac5bd450fb589f433/testing/buildbot/chromium.perf.json
[modify] https://crrev.com/18ae7760e18c810aaa8b49aac5bd450fb589f433/testing/scripts/run_performance_tests.py
[modify] https://crrev.com/18ae7760e18c810aaa8b49aac5bd450fb589f433/tools/perf/core/perf_data_generator.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/2c8ffb55a9d5a8b7ce7eaac3a45e0789f428ab1a

commit 2c8ffb55a9d5a8b7ce7eaac3a45e0789f428ab1a
Author: Emily Hanley <eyaich@google.com>
Date: Mon May 14 17:19:44 2018

Update benchmark_smoke_test to set the enable_smoke_test flag

Bug:  840427 
Change-Id: I80c13fefb4e5594ad29a6511845415584d7ea11a
Reviewed-on: https://chromium-review.googlesource.com/1047854
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Emily Hanley <eyaich@chromium.org>
Cr-Commit-Position: refs/heads/master@{#558352}
[modify] https://crrev.com/2c8ffb55a9d5a8b7ce7eaac3a45e0789f428ab1a/tools/perf/benchmarks/benchmark_smoke_unittest.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/6156ddee9c553723c0bdbfb2bae14865cecca53f

commit 6156ddee9c553723c0bdbfb2bae14865cecca53f
Author: Emily Hanley <eyaich@google.com>
Date: Wed May 16 00:35:24 2018

Implement smoke test mode for performance_test_suite

1) Updates the script that executes the tests on the swarming bot by
accepting a flag for the benchmark to shard map.  We now have at least
three use cases for this: main perf waterfall android and desktop,
smoke testing, testing on our trybot and potentially android go.

2) Updates the custom merge script to not upload perf results when
we are in smoke test mode.

Bug:840427
Change-Id: Idec8f45cc2d1b5a3cd336ef3d1698d3d86161ba3

NOTRY=true  # win_7_chromium_rel_ng flaky

Change-Id: Idec8f45cc2d1b5a3cd336ef3d1698d3d86161ba3
Reviewed-on: https://chromium-review.googlesource.com/1055614
Commit-Queue: Ned Nguyen <nednguyen@google.com>
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Cr-Commit-Position: refs/heads/master@{#558900}
[modify] https://crrev.com/6156ddee9c553723c0bdbfb2bae14865cecca53f/testing/buildbot/chromium.perf.fyi.json
[modify] https://crrev.com/6156ddee9c553723c0bdbfb2bae14865cecca53f/testing/buildbot/chromium.perf.json
[modify] https://crrev.com/6156ddee9c553723c0bdbfb2bae14865cecca53f/testing/scripts/run_performance_tests.py
[modify] https://crrev.com/6156ddee9c553723c0bdbfb2bae14865cecca53f/tools/perf/core/perf_data_generator.py
[modify] https://crrev.com/6156ddee9c553723c0bdbfb2bae14865cecca53f/tools/perf/process_perf_results.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1f4fc1f7d335fa403e6192c759d50e9aa755c24b

commit 1f4fc1f7d335fa403e6192c759d50e9aa755c24b
Author: Emily Hanley <eyaich@google.com>
Date: Wed May 16 13:56:05 2018

Add dummy benchmarks to chromium waterfalls

Bug:  840427 
Change-Id: I44364a9c3954b14118c77c51cfd454be3919c1f2
Reviewed-on: https://chromium-review.googlesource.com/1055776
Reviewed-by: John Budorick <jbudorick@chromium.org>
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Emily Hanley <eyaich@chromium.org>
Cr-Commit-Position: refs/heads/master@{#559076}
[modify] https://crrev.com/1f4fc1f7d335fa403e6192c759d50e9aa755c24b/testing/buildbot/chromium.mac.json
[modify] https://crrev.com/1f4fc1f7d335fa403e6192c759d50e9aa755c24b/testing/buildbot/chromium.win.json
[modify] https://crrev.com/1f4fc1f7d335fa403e6192c759d50e9aa755c24b/testing/buildbot/test_suites.pyl
[add] https://crrev.com/1f4fc1f7d335fa403e6192c759d50e9aa755c24b/tools/perf/core/smoke_test_benchmark_shard_map.json

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/c7cb085ec9f89d6ed9996f035a39b53cc30fae0e

commit c7cb085ec9f89d6ed9996f035a39b53cc30fae0e
Author: Emily Hanley <eyaich@google.com>
Date: Wed May 16 18:57:22 2018

Updating telemetry smoke-test-mode flag to not take a value

Bug:  chromium:840427 
Change-Id: I61333cf8e573cedde9d15cca87850b914bf66b3a
Reviewed-on: https://chromium-review.googlesource.com/1062215
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Emily Hanley <eyaich@chromium.org>

[modify] https://crrev.com/c7cb085ec9f89d6ed9996f035a39b53cc30fae0e/telemetry/telemetry/internal/story_runner.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/2028c1da1380e7f4f508675ca8d98083952c69b3

commit 2028c1da1380e7f4f508675ca8d98083952c69b3
Author: Emily Hanley <eyaich@google.com>
Date: Wed May 16 20:43:22 2018

Updating the smoke test flag to not require a value

Bug:  840427 
Change-Id: I1d658ed8c005ab29ec4754b1586257aec8b2c106
Reviewed-on: https://chromium-review.googlesource.com/1062286
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Emily Hanley <eyaich@chromium.org>
Cr-Commit-Position: refs/heads/master@{#559275}
[modify] https://crrev.com/2028c1da1380e7f4f508675ca8d98083952c69b3/tools/perf/process_perf_results.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/9c7817080b55cfcaea3531ed9a0d1a9f0d64a2e9

commit 9c7817080b55cfcaea3531ed9a0d1a9f0d64a2e9
Author: Lukasz Anforowicz <lukasza@chromium.org>
Date: Wed May 16 23:20:20 2018

Revert "Add dummy benchmarks to chromium waterfalls"

This reverts commit 1f4fc1f7d335fa403e6192c759d50e9aa755c24b.

Reason for revert: Speculating that this is the root cause of  https://crbug.com/843725 

Original change's description:
> Add dummy benchmarks to chromium waterfalls
>
> Bug:  840427 
> Change-Id: I44364a9c3954b14118c77c51cfd454be3919c1f2
> Reviewed-on: https://chromium-review.googlesource.com/1055776
> Reviewed-by: John Budorick <jbudorick@chromium.org>
> Reviewed-by: Ned Nguyen <nednguyen@google.com>
> Commit-Queue: Emily Hanley <eyaich@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#559076}

TBR=nednguyen@google.com,eyaich@chromium.org,jbudorick@chromium.org

Bug:  843725 
Change-Id: I10fa2c1c631e3d43b31f610f4af40756840e9ba2
Reviewed-on: https://chromium-review.googlesource.com/1062711
Reviewed-by: Łukasz Anforowicz <lukasza@chromium.org>
Reviewed-by: Emily Hanley <eyaich@chromium.org>
Commit-Queue: Emily Hanley <eyaich@chromium.org>
Cr-Commit-Position: refs/heads/master@{#559327}
[modify] https://crrev.com/9c7817080b55cfcaea3531ed9a0d1a9f0d64a2e9/testing/buildbot/chromium.mac.json
[modify] https://crrev.com/9c7817080b55cfcaea3531ed9a0d1a9f0d64a2e9/testing/buildbot/chromium.win.json
[modify] https://crrev.com/9c7817080b55cfcaea3531ed9a0d1a9f0d64a2e9/testing/buildbot/test_suites.pyl
[delete] https://crrev.com/393cfcaff4c98af880deff6fbd69cd78b203f5dd/tools/perf/core/smoke_test_benchmark_shard_map.json

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/63c450575899b85d59064c0c83c86fb0833c28b8

commit 63c450575899b85d59064c0c83c86fb0833c28b8
Author: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Wed May 16 23:24:39 2018

Roll src/third_party/catapult/ d95db22a6..a7e134aa0 (8 commits)

https://chromium.googlesource.com/catapult.git/+log/d95db22a613c..a7e134aa0dbf

$ git log d95db22a6..a7e134aa0 --date=short --no-merges --format='%ad %ae %s'
2018-05-16 szager Refactor browser_interval_profiling_controller
2018-05-16 benjhayden Add google-signin to third_party/polymer2.
2018-05-16 nednguyen Add logging of the apk name picked by android backend settings
2018-05-16 benjhayden Add paper-checkbox to third_party/polymer2.
2018-05-16 benjhayden Add polymer-redux to third_party.
2018-05-16 eyaich Updating telemetry smoke-test-mode flag to not take a value
2018-05-16 benjhayden Add paper-input to third_party/polymer2.
2018-05-16 nednguyen Make sure Telemetry use <benchmark>/<story_name> consistenly when printing to stdout

Created with:
  roll-dep src/third_party/catapult
BUG=chromium:839645,chromium:843516, chromium:840427 


The AutoRoll server is located here: https://catapult-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.


TBR=sullivan@chromium.org

Change-Id: I25330250acecec6a49829102e25e07ff8b35db93
Reviewed-on: https://chromium-review.googlesource.com/1062933
Commit-Queue: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Reviewed-by: catapult-chromium-autoroll <catapult-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#559328}
[modify] https://crrev.com/63c450575899b85d59064c0c83c86fb0833c28b8/DEPS

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/81715b6193a1841807e99bc416712b62a3df0c58

commit 81715b6193a1841807e99bc416712b62a3df0c58
Author: Emily Hanley <eyaich@google.com>
Date: Tue May 22 14:36:48 2018

Re-adding dummy benchamrks to chromium debug waterfalls.

Previously
reviewed here: https://chromium-review.googlesource.com/c/chromium/src/+/1055776
and then reverted.

We have to set the browser for each configuration that we run on.
It will be one of debug, release or android-chromium.  We can do this one
of two ways:

1) multiple entries in test_suites.pyl
2) add each entry in test_suite_exceptions.pyl

Although #1 is less verbose I think #2 is how these files are intended
to be used.

Bug:  840427 
Change-Id: Idff4c36a248215dbfdf0faed45b483800046bf26
Reviewed-on: https://chromium-review.googlesource.com/1066088
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Emily Hanley <eyaich@chromium.org>
Cr-Commit-Position: refs/heads/master@{#560571}
[modify] https://crrev.com/81715b6193a1841807e99bc416712b62a3df0c58/testing/buildbot/chromium.mac.json
[modify] https://crrev.com/81715b6193a1841807e99bc416712b62a3df0c58/testing/buildbot/chromium.win.json
[modify] https://crrev.com/81715b6193a1841807e99bc416712b62a3df0c58/testing/buildbot/test_suite_exceptions.pyl
[modify] https://crrev.com/81715b6193a1841807e99bc416712b62a3df0c58/testing/buildbot/test_suites.pyl
[add] https://crrev.com/81715b6193a1841807e99bc416712b62a3df0c58/tools/perf/core/smoke_test_benchmark_shard_map.json

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4e653b28d15b1f801007c670bada8e64149211bc

commit 4e653b28d15b1f801007c670bada8e64149211bc
Author: Jeremy Roman <jbroman@chromium.org>
Date: Tue May 22 20:03:03 2018

Revert "Re-adding dummy benchamrks to chromium debug waterfalls."

This reverts commit 81715b6193a1841807e99bc416712b62a3df0c58.

Reason for revert: Breaks on Win10 Tests x64 (dbg).

"""
BrowserFinderException: Cannot find browser of type debug. 

Available browsers:
debug_x64
reference
"""

and

"""
Traceback (most recent call last):
  File "..\..\testing\scripts\run_performance_tests.py", line 312, in <module>
    sys.exit(main())
  File "..\..\testing\scripts\run_performance_tests.py", line 284, in main
    sharding = sharding_map[shard_index]['benchmarks']
KeyError: '7'
"""

depending on shard number

Original change's description:
> Re-adding dummy benchamrks to chromium debug waterfalls.
> 
> Previously
> reviewed here: https://chromium-review.googlesource.com/c/chromium/src/+/1055776
> and then reverted.
> 
> We have to set the browser for each configuration that we run on.
> It will be one of debug, release or android-chromium.  We can do this one
> of two ways:
> 
> 1) multiple entries in test_suites.pyl
> 2) add each entry in test_suite_exceptions.pyl
> 
> Although #1 is less verbose I think #2 is how these files are intended
> to be used.
> 
> Bug:  840427 
> Change-Id: Idff4c36a248215dbfdf0faed45b483800046bf26
> Reviewed-on: https://chromium-review.googlesource.com/1066088
> Reviewed-by: John Budorick <jbudorick@chromium.org>
> Commit-Queue: Emily Hanley <eyaich@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#560571}

TBR=nednguyen@google.com,eyaich@chromium.org,jbudorick@chromium.org

Change-Id: I89ca43c0eb6292ee252c459e1db94749bfb603b0
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug:  840427 , 845543 
Reviewed-on: https://chromium-review.googlesource.com/1069420
Reviewed-by: Jeremy Roman <jbroman@chromium.org>
Commit-Queue: Jeremy Roman <jbroman@chromium.org>
Cr-Commit-Position: refs/heads/master@{#560744}
[modify] https://crrev.com/4e653b28d15b1f801007c670bada8e64149211bc/testing/buildbot/chromium.mac.json
[modify] https://crrev.com/4e653b28d15b1f801007c670bada8e64149211bc/testing/buildbot/chromium.win.json
[modify] https://crrev.com/4e653b28d15b1f801007c670bada8e64149211bc/testing/buildbot/test_suite_exceptions.pyl
[modify] https://crrev.com/4e653b28d15b1f801007c670bada8e64149211bc/testing/buildbot/test_suites.pyl
[delete] https://crrev.com/ebc94b58f609b461e0799e10797080d4c8add639/tools/perf/core/smoke_test_benchmark_shard_map.json

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/b64ac7c8bee2f677d0a2dffa83c9740094a0ed7f

commit b64ac7c8bee2f677d0a2dffa83c9740094a0ed7f
Author: Emily Hanley <eyaich@google.com>
Date: Fri May 25 13:36:55 2018

Re-landing dummy benchmarks for the third time.

NOTRY=true

Bug:  840427 
Change-Id: I3c1336cccc6e0af88aea9b7ad5e7d10a8cdd5b6f
Reviewed-on: https://chromium-review.googlesource.com/1069659
Commit-Queue: Emily Hanley <eyaich@chromium.org>
Reviewed-by: Stephen Martinis <martiniss@chromium.org>
Cr-Commit-Position: refs/heads/master@{#561849}
[modify] https://crrev.com/b64ac7c8bee2f677d0a2dffa83c9740094a0ed7f/testing/buildbot/chromium.mac.json
[modify] https://crrev.com/b64ac7c8bee2f677d0a2dffa83c9740094a0ed7f/testing/buildbot/chromium.win.json
[modify] https://crrev.com/b64ac7c8bee2f677d0a2dffa83c9740094a0ed7f/testing/buildbot/test_suite_exceptions.pyl
[modify] https://crrev.com/b64ac7c8bee2f677d0a2dffa83c9740094a0ed7f/testing/buildbot/test_suites.pyl
[add] https://crrev.com/b64ac7c8bee2f677d0a2dffa83c9740094a0ed7f/tools/perf/core/smoke_test_benchmark_shard_map.json

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/992302415beb6de6ea6a3090e16dbadc78ed513c

commit 992302415beb6de6ea6a3090e16dbadc78ed513c
Author: Emily Hanley <eyaich@google.com>
Date: Wed Jun 27 15:46:23 2018

Adding ability to generate sharding maps for a subset of benchmarks

Bug:  840427 
Change-Id: I1f11ed443c31afe50607cde766df13e610a07762
Reviewed-on: https://chromium-review.googlesource.com/1104793
Commit-Queue: Emily Hanley <eyaich@chromium.org>
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Cr-Commit-Position: refs/heads/master@{#570787}
[modify] https://crrev.com/992302415beb6de6ea6a3090e16dbadc78ed513c/tools/perf/core/sharding_map_generator.py
[modify] https://crrev.com/992302415beb6de6ea6a3090e16dbadc78ed513c/tools/perf/generate_perf_sharding

What is the difference between this bug and issue 739863? Seem like the same thing. Setting this to be a dupe speculatively. If I'm wrong please undupe and explain why.