thanks for CC. On android we use only start_with_url.*, startup on desktop is more relevant to +gab@. I think the --pageset-repeat is incorrect as you describe for all of telemetry tests, including TBM. So +nednguyen@ as potential owner.

This is only applied for the "startup warm" benchmarks because of https://cs.chromium.org/chromium/src/tools/perf/benchmarks/startup.py?rcl=0&l=32

With the new architecture of Telemetry, I think the startup benchmarks should be redesigned to get cold & warm work better, s.t like a single startup story will contain both cold & warm start-up of the browser & output both cold & warm metrics. Right now we have to separate cold & warm at the benchmark level, which is awkward.

re #2: ah, you are right, having two benchmarks and ignoring parts of results is weird.

A solution would be to have one benchmark for "startup" and one for "start_with_url", each one would report both cold and warm times.  On the other hand, we need to be able to bisect cold starts, in which case we do not want to run the warm ones at all. Is there an elegant way to do this from the perf dashboard in the current telemetry architecture?

Pasko, if we can get a story report both cold & warm times, it would also be better to combine start_with_url & startup. To me, they are just start_with_about_blank and start_with_some_url.

For ignoring the warm ones when bisecting, I think if the total overhead for a startup run is not bad, there is no need to optimize for this. Our page_cycler_v2 benchmarks also have the cold vs warm situation & we don't do anything special there when only bisecting cold.

I don't know how startup.* works, there are a few confusions around it that make justifying work for it harder:  Issue 571613 .

> if the total overhead for a startup run is not bad, there is no need to optimize for this

The overhead is basically 2x. So if we decide to bisect cold startup we would get results 2 times faster (1 hour instead of 2 hours?) compared to the bisect that does both cold and warm. I think it's okay for now, and we can live with it, just wanted to know if you have more ideas how to make us more productive using this particular aspect of bisects.

.. and please call me pasko@ or Egor, not Pasko, thanks :)

I don't know much about telemetry. I think erikchen@ is the only one to know the intersection between telemetry and desktop startup well.

@erikchen to triage or address.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/eedd7c15d7970a5b81a2e6060c8f18d5242885a4

commit eedd7c15d7970a5b81a2e6060c8f18d5242885a4
Author: catapult-deps-roller <catapult-deps-roller@chromium.org>
Date: Fri Jan 27 04:03:47 2017

Roll src/third_party/catapult/ 683b84f3b..fbdf6f597 (2 commits).

https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/683b84f3b681..fbdf6f597f89

$ git log 683b84f3b..fbdf6f597 --date=short --no-merges --format='%ad %ae %s'
2017-01-26 charliea Record the time required to import the trace in the model stats.
2017-01-26 simonhatch Dashboard - Workaround startup.warm.* producing no values with --pageset-repeat=1.

BUG= 665550 ,677843

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, see:
http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls

CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel
TBR=catapult-sheriff@chromium.org

Review-Url: https://codereview.chromium.org/2655853008
Cr-Commit-Position: refs/heads/master@{#446585}

[modify] https://crrev.com/eedd7c15d7970a5b81a2e6060c8f18d5242885a4/DEPS

 Issue 665023  has been merged into this issue.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/8736cb0f30d613eda3ad4fc61387364931f4d1dc

commit 8736cb0f30d613eda3ad4fc61387364931f4d1dc
Author: catapult-deps-roller@chromium.org <catapult-deps-roller@chromium.org>
Date: Thu Jun 29 03:43:18 2017

Roll src/third_party/catapult/ 3b76c885e..a70ee6f1c (1 commit)

https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/3b76c885e5d4..a70ee6f1c1fb

$ git log 3b76c885e..a70ee6f1c --date=short --no-merges --format='%ad %ae %s'
2017-06-28 simonhatch Dashboard - Extend startup pageset-repeat hack.

Created with:
  roll-dep src/third_party/catapult
BUG=677843


Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, see:
http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls


CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel
TBR=sullivan@chromium.org

Change-Id: I3cc7177be59e0f20688db50f354b4958d8fe0a50
Reviewed-on: https://chromium-review.googlesource.com/554259
Reviewed-by: <catapult-deps-roller@chromium.org>
Commit-Queue: <catapult-deps-roller@chromium.org>
Cr-Commit-Position: refs/heads/master@{#483284}
[modify] https://crrev.com/8736cb0f30d613eda3ad4fc61387364931f4d1dc/DEPS