Currently the ChromeOS and the Windows Official builds of Chrome are optimized with PGO (or equivalent), and they rely on Telemetry for the profiling step.
The list of benchmarks used to train the Windows builds is here: https://cs.chromium.org/chromium/src/build/win/run_pgo_profiling_benchmarks.py , note that we don't really care if these benchmarks are 'important' for the performance measurements, they've been hand-picked because using them during the profiling step led to some nice performance gains.
This creates an implicit dependency between the official build and Telemetry (removing one of these benchmarks from Chrome might have a significant impact on performance), so it might be worthwhile to create a new benchmark (or benchmark suite) dedicated to PGO (or AFDO). This benchmark will probably be a superset of some of the existing ones (e.g. it'll visit the top25 sites, do some scrolling, run a few V8 benchmarks etc).
An important feature for this benchmark is that it should be fast, currently the Windows PGO build spend ~3h in the benchmarking step and it impact our ability to use this config on our perf bots etc, which means that we're not testing the performance of the 'real' Chrome. It look like the existing Telemetry benchmarks sometime wait for a few seconds on each page (probably to mimic what a real user do), and this slow them down a lot, I don't think that we need this kind of feature during the profiling step.
Comment 1 by nduca@chromium.org
, Dec 12 2016