Note that there are some existing limitation about how long a single benchmark can take. Can you make a simple design doc so that folks can chime in different concerns?

I've put together a rough doc: https://docs.google.com/document/d/1QKlZIoURAxZk-brrXsKYZl9O8ieqXht3ogeF9yLNFCI/edit?usp=sharing
Please feel free to share / invite people.

Here's URL list from Kenji's spreadsheet and {easy,tough} cases of {TTFMP,TTI}:
https://docs.google.com/spreadsheets/d/1wU_eYc_kBxvR7I0f93UID23eC8dVTnFsXbuHWiexvTs/edit?usp=sharing

I'll create initial story set (loading only, no interactions) with these URLs.

Awesome, but we first need to add tag attribute to telemetry story & "--tag-filter" to telemetry?

Can we use the label attribute of Story and --story-label-filter flag?
I think it would suffice our needs.

https://cs.chromium.org/chromium/src/third_party/catapult/telemetry/telemetry/story/story_filter.py?l=51

Or, I totally forgot that we had story 'labels'. Yes, current implementation of labels & --story-label-filter seems to behave as how we described tagging to me.

Maybe just update your doc so we refer to "labels" instead of "tags".

WIP CL is here: https://codereview.chromium.org/2363343003/

I'd like to ask for advice about record_wpr.

- Flipkart (https://www.flipkart.com/) page doesn't load when recording the page by record_wpr. It looks like the server (proxy?) was not responding. It loads if I visit the page without record_wpr. I'm not sure how to debug this.

- Facebook (timeline) and Inbox require signed-in access. How hard it is to record a page as logged-in user?

For the fb case, we should be able to use facebook_login.LoginWithMobileSite. There is an equivalent one for Google logins.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/31d0a108f1783a04f2b981d668883a43a30ddc32

commit 31d0a108f1783a04f2b981d668883a43a30ddc32
Author: ksakamoto <ksakamoto@chromium.org>
Date: Fri Oct 07 22:50:10 2016

Add loading mobile story set

This adds a story set and corresponding benchmark for mobile loading
performance loading team is interested in.

Design doc: https://docs.google.com/document/d/1QKlZIoURAxZk-brrXsKYZl9O8ieqXht3ogeF9yLNFCI/edit

BUG=643524
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.perf:linux_perf_cq;master.tryserver.chromium.perf:mac_retina_perf_cq;master.tryserver.chromium.perf:winx64_10_perf_cq

Review-Url: https://codereview.chromium.org/2363343003
Cr-Commit-Position: refs/heads/master@{#423993}

[modify] https://crrev.com/31d0a108f1783a04f2b981d668883a43a30ddc32/tools/perf/OWNERS
[add] https://crrev.com/31d0a108f1783a04f2b981d668883a43a30ddc32/tools/perf/benchmarks/loading.py
[modify] https://crrev.com/31d0a108f1783a04f2b981d668883a43a30ddc32/tools/perf/benchmarks/page_cycler_v2.py
[add] https://crrev.com/31d0a108f1783a04f2b981d668883a43a30ddc32/tools/perf/page_sets/data/loading_mobile.json
[add] https://crrev.com/31d0a108f1783a04f2b981d668883a43a30ddc32/tools/perf/page_sets/data/loading_mobile_000.wpr.sha1
[add] https://crrev.com/31d0a108f1783a04f2b981d668883a43a30ddc32/tools/perf/page_sets/data/loading_mobile_001.wpr.sha1
[add] https://crrev.com/31d0a108f1783a04f2b981d668883a43a30ddc32/tools/perf/page_sets/loading_mobile.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/b524efc65e507803ec42e4d73509c420cb715081

commit b524efc65e507803ec42e4d73509c420cb715081
Author: zhenw <zhenw@chromium.org>
Date: Sat Oct 08 17:33:08 2016

Revert of Add loading mobile story set (patchset #6 id:140001 of https://codereview.chromium.org/2363343003/ )

Reason for revert:
Failure on Android Perf bot

BUG= chromium:654215 

Original issue's description:
> Add loading mobile story set
>
> This adds a story set and corresponding benchmark for mobile loading
> performance loading team is interested in.
>
> Design doc: https://docs.google.com/document/d/1QKlZIoURAxZk-brrXsKYZl9O8ieqXht3ogeF9yLNFCI/edit
>
> BUG=643524
> CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.perf:linux_perf_cq;master.tryserver.chromium.perf:mac_retina_perf_cq;master.tryserver.chromium.perf:winx64_10_perf_cq
>
> Committed: https://crrev.com/31d0a108f1783a04f2b981d668883a43a30ddc32
> Cr-Commit-Position: refs/heads/master@{#423993}

TBR=kouhei@chromium.org,nednguyen@google.com,ksakamoto@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=643524

Review-Url: https://codereview.chromium.org/2409433002
Cr-Commit-Position: refs/heads/master@{#424083}

[modify] https://crrev.com/b524efc65e507803ec42e4d73509c420cb715081/tools/perf/OWNERS
[delete] https://crrev.com/ccd977fa5650bf395c59bbbb0afc81f5dbd3531c/tools/perf/benchmarks/loading.py
[modify] https://crrev.com/b524efc65e507803ec42e4d73509c420cb715081/tools/perf/benchmarks/page_cycler_v2.py
[delete] https://crrev.com/ccd977fa5650bf395c59bbbb0afc81f5dbd3531c/tools/perf/page_sets/data/loading_mobile.json
[delete] https://crrev.com/ccd977fa5650bf395c59bbbb0afc81f5dbd3531c/tools/perf/page_sets/data/loading_mobile_000.wpr.sha1
[delete] https://crrev.com/ccd977fa5650bf395c59bbbb0afc81f5dbd3531c/tools/perf/page_sets/data/loading_mobile_001.wpr.sha1
[delete] https://crrev.com/ccd977fa5650bf395c59bbbb0afc81f5dbd3531c/tools/perf/page_sets/loading_mobile.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/513268e432c40c7e2f0e5ab40ca8aad499479f41

commit 513268e432c40c7e2f0e5ab40ca8aad499479f41
Author: nednguyen <nednguyen@google.com>
Date: Tue Oct 11 13:43:39 2016

Add loading mobile benchmark (but disable it)

This is a reland of ksakamoto@'s work in https://codereview.chromium.org/2363343003/

THe loading benchmark however is disabled due to  crbug.com/654215 

------

This adds a story set and corresponding benchmark for mobile loading
performance loading team is interested in.

Design doc: https://docs.google.com/document/d/1QKlZIoURAxZk-brrXsKYZl9O8ieqXht3ogeF9yLNFCI/edit

BUG=643524
TBR=kouhei@chromium.org, ksakamoto@chromium.org

Review-Url: https://codereview.chromium.org/2409073003
Cr-Commit-Position: refs/heads/master@{#424415}

[modify] https://crrev.com/513268e432c40c7e2f0e5ab40ca8aad499479f41/tools/perf/OWNERS
[add] https://crrev.com/513268e432c40c7e2f0e5ab40ca8aad499479f41/tools/perf/benchmarks/loading.py
[modify] https://crrev.com/513268e432c40c7e2f0e5ab40ca8aad499479f41/tools/perf/benchmarks/page_cycler_v2.py
[add] https://crrev.com/513268e432c40c7e2f0e5ab40ca8aad499479f41/tools/perf/page_sets/data/loading_mobile.json
[add] https://crrev.com/513268e432c40c7e2f0e5ab40ca8aad499479f41/tools/perf/page_sets/data/loading_mobile_000.wpr.sha1
[add] https://crrev.com/513268e432c40c7e2f0e5ab40ca8aad499479f41/tools/perf/page_sets/data/loading_mobile_001.wpr.sha1
[add] https://crrev.com/513268e432c40c7e2f0e5ab40ca8aad499479f41/tools/perf/page_sets/loading_mobile.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/eff0660d223203c7e081c9fbdb7ad0e4cf425d15

commit eff0660d223203c7e081c9fbdb7ad0e4cf425d15
Author: ksakamoto <ksakamoto@chromium.org>
Date: Thu Oct 13 03:47:34 2016

Enable loading.mobile benchmark on Android except reference and webview

Note to sheriffs: If there are failures/crashes on waterfall bots,
please comment out individual pages in tools/perf/page_sets/loading_mobile.py
instead of disabling the whole benchmark.

BUG=643524, 654215 
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.perf:linux_perf_cq;master.tryserver.chromium.perf:mac_retina_perf_cq;master.tryserver.chromium.perf:winx64_10_perf_cq

Review-Url: https://codereview.chromium.org/2409103002
Cr-Commit-Position: refs/heads/master@{#424949}

[modify] https://crrev.com/eff0660d223203c7e081c9fbdb7ad0e4cf425d15/tools/perf/benchmarks/loading.py