Should we wait for deleting the old benchmarks before doing this? I worry that trying to update these pages now will make it's harder for you to do the data comparision work for removing legacy benchmarks

By data comparison, do you mean checking to make sure that the renames don't affect the perf data? If so, I could wait to submit the CLs for the actual updating of recordings for now, and only merge in small fixes (like the LinkedIn login) that won't affect the benchmark results. 

Hi Amy, by data comparison, I mean comparing the metrics data of the pages between the old benchmark & new benchmark. The basic blocker that prevent us from removing all the old benchmarks right now is making sure that the the new ones are as good as the old ones in catching regressions.

Okay, I see now. Since Simon has completed the renames, I can start comparing the data from old benchmarks vs. new, and work on refreshing the page sets after that's done. I'll file another bug to keep track of the data comparison. 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/7ef822fe35a91be5ef9116a84cf93fb830a1eddf

commit 7ef822fe35a91be5ef9116a84cf93fb830a1eddf
Author: Amy Qiu <amyqiu@google.com>
Date: Tue Jul 03 19:37:41 2018

Create linkedin login helper

As part of refreshing page sets, the new linkedin page requires logging
in before viewing profiles. Create a linkedin login helper and update
the credentials.json file.

Bug:  852530 
Change-Id: Ibce40dc6cf0c3d09f4fed5cd8f34f0d7cf14d8e4
Reviewed-on: https://chromium-review.googlesource.com/1099728
Commit-Queue: Amy Qiu <amyqiu@google.com>
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Cr-Commit-Position: refs/heads/master@{#572322}
[modify] https://crrev.com/7ef822fe35a91be5ef9116a84cf93fb830a1eddf/tools/perf/page_sets/data/credentials.json.sha1
[add] https://crrev.com/7ef822fe35a91be5ef9116a84cf93fb830a1eddf/tools/perf/page_sets/login_helpers/linkedin_login.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/0131dcca85a720678ded88d843088de9e0e263c8

commit 0131dcca85a720678ded88d843088de9e0e263c8
Author: Amy Qiu <amyqiu@google.com>
Date: Mon Jul 16 21:29:53 2018

Refresh top real world desktop pages

Update all top real world desktop pages to current websites. In some cases,
this required modifying html selectors or removing unnecessary logins. Certain
websites were replaced due to no longer existing (yahoo games with twitch.tv)
or unusable results (weather.com with accuweather). Expectations related to
fixed recordings were deleted, since the tests appear now to be passing.

Fixed expectations are tracked here:
https://docs.google.com/spreadsheets/d/1GBXuDNh1tZmZWcPZFLtm5UB8MyVPavfdVmi712XP6og/edit?usp=sharing
The associate bugs can be resolved if the pages appear to pass consistently later.

Bug:  852530 
Change-Id: I760f0933d24cf067a4d8148f44c4995d32e1e768
Reviewed-on: https://chromium-review.googlesource.com/1125211
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Amy Qiu <amyqiu@google.com>
Cr-Commit-Position: refs/heads/master@{#575428}
[modify] https://crrev.com/0131dcca85a720678ded88d843088de9e0e263c8/tools/perf/page_sets/data/rendering_desktop.json
[modify] https://crrev.com/0131dcca85a720678ded88d843088de9e0e263c8/tools/perf/page_sets/data/rendering_mobile.json
[modify] https://crrev.com/0131dcca85a720678ded88d843088de9e0e263c8/tools/perf/page_sets/data/top_25_012.wprgo.sha1
[modify] https://crrev.com/0131dcca85a720678ded88d843088de9e0e263c8/tools/perf/page_sets/login_helpers/google_login.py
[modify] https://crrev.com/0131dcca85a720678ded88d843088de9e0e263c8/tools/perf/page_sets/rendering/top_real_world_desktop.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/529ef69330f6b45bb897fb5f9648cadcd04047d2

commit 529ef69330f6b45bb897fb5f9648cadcd04047d2
Author: Amy Qiu <amyqiu@google.com>
Date: Thu Jul 19 15:44:06 2018

Refresh top real world mobile pages

Update all top real world mobile pages to current websites by modifying
links or html selectors. The following pages were replaced:

cuteoverload_mobile: with google images

Fixed expectations tracked here:
https://docs.google.com/spreadsheets/d/1qx76DCRf-j4xNVfHOPII02FQnQWWXMi-FpcAtn4boAk/edit?usp=sharing

Bug:  852530 
Change-Id: Ib34ba6c72f0b52fb01452581d4d8e8c2c96c0242
Reviewed-on: https://chromium-review.googlesource.com/1129459
Reviewed-by: Sadrul Chowdhury <sadrul@chromium.org>
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Amy Qiu <amyqiu@google.com>
Cr-Commit-Position: refs/heads/master@{#576507}
[add] https://crrev.com/529ef69330f6b45bb897fb5f9648cadcd04047d2/tools/perf/page_sets/data/key_mobile_sites_016.wprgo.sha1
[add] https://crrev.com/529ef69330f6b45bb897fb5f9648cadcd04047d2/tools/perf/page_sets/data/key_mobile_sites_017.wprgo.sha1
[add] https://crrev.com/529ef69330f6b45bb897fb5f9648cadcd04047d2/tools/perf/page_sets/data/key_mobile_sites_018.wprgo.sha1
[add] https://crrev.com/529ef69330f6b45bb897fb5f9648cadcd04047d2/tools/perf/page_sets/data/key_mobile_sites_019.wprgo.sha1
[add] https://crrev.com/529ef69330f6b45bb897fb5f9648cadcd04047d2/tools/perf/page_sets/data/key_mobile_sites_020.wprgo.sha1
[modify] https://crrev.com/529ef69330f6b45bb897fb5f9648cadcd04047d2/tools/perf/page_sets/data/rendering_mobile.json
[modify] https://crrev.com/529ef69330f6b45bb897fb5f9648cadcd04047d2/tools/perf/page_sets/login_helpers/linkedin_login.py
[modify] https://crrev.com/529ef69330f6b45bb897fb5f9648cadcd04047d2/tools/perf/page_sets/rendering/rendering_story.py
[modify] https://crrev.com/529ef69330f6b45bb897fb5f9648cadcd04047d2/tools/perf/page_sets/rendering/top_real_world_mobile.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/6ede15e170dba7d7305d4d6103b7ce35a39594f5

commit 6ede15e170dba7d7305d4d6103b7ce35a39594f5
Author: Amy Qiu <amyqiu@google.com>
Date: Tue Jul 24 01:39:53 2018

Refresh simple mobile sites recordings

Update recordings for pages in simple_mobile_sites for rendering.mobile.
Change the flickr url to an album to bypass the initial login screen
(doesn't allow scrolling).

Bug:  852530 
Change-Id: I2bd71f512492cb306c47fea0f9bd857fae953338
Reviewed-on: https://chromium-review.googlesource.com/1146499
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Amy Qiu <amyqiu@google.com>
Cr-Commit-Position: refs/heads/master@{#577398}
[modify] https://crrev.com/6ede15e170dba7d7305d4d6103b7ce35a39594f5/tools/perf/page_sets/data/rendering_mobile.json
[add] https://crrev.com/6ede15e170dba7d7305d4d6103b7ce35a39594f5/tools/perf/page_sets/data/simple_mobile_sites_003.wprgo.sha1
[modify] https://crrev.com/6ede15e170dba7d7305d4d6103b7ce35a39594f5/tools/perf/page_sets/rendering/simple_mobile_sites.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/b1a67cfabfc5e894795bf611486e8aa1e0d3b56b

commit b1a67cfabfc5e894795bf611486e8aa1e0d3b56b
Author: Amy Qiu <amyqiu@google.com>
Date: Thu Jul 26 15:57:26 2018

Delete replaced top_real_world_desktop pages

Now that it's been a week since pages in top_real_world_desktop were
refreshed, remove the old pages. Also delete expectations for replaced
pages.

Bug:  852530 
Change-Id: I3a702e877a56fe94a47add8b2254774d9c93fa28
Reviewed-on: https://chromium-review.googlesource.com/1150680
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Amy Qiu <amyqiu@google.com>
Cr-Commit-Position: refs/heads/master@{#578314}
[modify] https://crrev.com/b1a67cfabfc5e894795bf611486e8aa1e0d3b56b/tools/perf/expectations.config
[modify] https://crrev.com/b1a67cfabfc5e894795bf611486e8aa1e0d3b56b/tools/perf/page_sets/data/rendering_desktop.json
[modify] https://crrev.com/b1a67cfabfc5e894795bf611486e8aa1e0d3b56b/tools/perf/page_sets/data/rendering_mobile.json
[modify] https://crrev.com/b1a67cfabfc5e894795bf611486e8aa1e0d3b56b/tools/perf/page_sets/rendering/top_real_world_desktop.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/61a2eaa3b73387606f4978dc0b463c4d7d494051

commit 61a2eaa3b73387606f4978dc0b463c4d7d494051
Author: Amy Qiu <amyqiu@google.com>
Date: Thu Jul 26 15:58:31 2018

Refresh pages for pathological mobile sites

Create new recordings for pages in pathological mobile sites to ensure
current sites are used for the rendering benchmark. Update URL for espn
and add login for linkedin.

Bug:  852530 
Change-Id: I3f30d1b4daf7c4a850f8f759129f27be01a1751a
Reviewed-on: https://chromium-review.googlesource.com/1148949
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Amy Qiu <amyqiu@google.com>
Cr-Commit-Position: refs/heads/master@{#578315}
[add] https://crrev.com/61a2eaa3b73387606f4978dc0b463c4d7d494051/tools/perf/page_sets/data/pathological_mobile_sites_012.wprgo.sha1
[modify] https://crrev.com/61a2eaa3b73387606f4978dc0b463c4d7d494051/tools/perf/page_sets/data/rendering_mobile.json
[modify] https://crrev.com/61a2eaa3b73387606f4978dc0b463c4d7d494051/tools/perf/page_sets/rendering/pathological_mobile_sites.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/7af2637c10be7648e2730ea2fa0ce4c92d3f55d1

commit 7af2637c10be7648e2730ea2fa0ce4c92d3f55d1
Author: Amy Qiu <amyqiu@google.com>
Date: Fri Jul 27 00:45:16 2018

Refresh pages for key_desktop_move_cases

Create new recordings for pages in key_desktop_move_cases to ensure
current websites are used.

Bug:  852530 
Change-Id: Ia216e388edf269a0462f014f90bbe22be0a68916
Reviewed-on: https://chromium-review.googlesource.com/1150691
Reviewed-by: Sadrul Chowdhury <sadrul@chromium.org>
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Ned Nguyen <nednguyen@google.com>
Cr-Commit-Position: refs/heads/master@{#578497}
[add] https://crrev.com/7af2637c10be7648e2730ea2fa0ce4c92d3f55d1/tools/perf/page_sets/data/key_desktop_move_cases_003.wprgo.sha1
[modify] https://crrev.com/7af2637c10be7648e2730ea2fa0ce4c92d3f55d1/tools/perf/page_sets/data/rendering_desktop.json
[modify] https://crrev.com/7af2637c10be7648e2730ea2fa0ce4c92d3f55d1/tools/perf/page_sets/rendering/key_desktop_move_cases.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ba64fdd84ffcaf92713f060d6c21baf4a4d0d315

commit ba64fdd84ffcaf92713f060d6c21baf4a4d0d315
Author: Amy Qiu <amyqiu@google.com>
Date: Fri Jul 27 14:23:27 2018

Delete replaced top_real_world_mobile pages

Since it's been a week after pages in top_real_world_mobile were
refreshed, delete the old pages and their corresponding expectations.

Bug:  852530 
Change-Id: Ic4d9cb9503fa22dd2b73735404a60c69e2a8a2ba
Reviewed-on: https://chromium-review.googlesource.com/1151104
Commit-Queue: Amy Qiu <amyqiu@google.com>
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Cr-Commit-Position: refs/heads/master@{#578630}
[modify] https://crrev.com/ba64fdd84ffcaf92713f060d6c21baf4a4d0d315/tools/perf/expectations.config
[modify] https://crrev.com/ba64fdd84ffcaf92713f060d6c21baf4a4d0d315/tools/perf/page_sets/data/rendering_mobile.json
[modify] https://crrev.com/ba64fdd84ffcaf92713f060d6c21baf4a4d0d315/tools/perf/page_sets/rendering/top_real_world_mobile.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/e30141a348d52fefbce0fbd6df951245b61abf25

commit e30141a348d52fefbce0fbd6df951245b61abf25
Author: Amy Qiu <amyqiu@google.com>
Date: Mon Jul 30 22:13:47 2018

Remove updated simple mobile sites

It's been a week since simple_mobile_sites was refreshed, remove the old
pages and corresponding expectation

Bug:  852530 
Change-Id: I8f18f80794ae711f36215ee50b50d8bb40e7dbf9
Reviewed-on: https://chromium-review.googlesource.com/1154997
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Amy Qiu <amyqiu@google.com>
Cr-Commit-Position: refs/heads/master@{#579193}
[modify] https://crrev.com/e30141a348d52fefbce0fbd6df951245b61abf25/tools/perf/expectations.config
[modify] https://crrev.com/e30141a348d52fefbce0fbd6df951245b61abf25/tools/perf/page_sets/data/rendering_mobile.json
[modify] https://crrev.com/e30141a348d52fefbce0fbd6df951245b61abf25/tools/perf/page_sets/rendering/simple_mobile_sites.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3ee1c24c7867a9ea70d8999079d9bc0adc70efc8

commit 3ee1c24c7867a9ea70d8999079d9bc0adc70efc8
Author: Amy Qiu <amyqiu@google.com>
Date: Mon Jul 30 22:40:34 2018

Refresh tough pinch zoom pages

Update recordings in tough_pinch_zoom_cases to current websites. Add
login for sites that need it and replace non-existant websites with
comparable ones.

Bug:  852530 
Change-Id: I2e02c1c0374ae8d9ba8f8cd32095172c827eeb7f
Reviewed-on: https://chromium-review.googlesource.com/1153027
Reviewed-by: Sadrul Chowdhury <sadrul@chromium.org>
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Amy Qiu <amyqiu@google.com>
Cr-Commit-Position: refs/heads/master@{#579207}
[modify] https://crrev.com/3ee1c24c7867a9ea70d8999079d9bc0adc70efc8/tools/perf/page_sets/data/rendering_desktop.json
[modify] https://crrev.com/3ee1c24c7867a9ea70d8999079d9bc0adc70efc8/tools/perf/page_sets/data/tough_pinch_zoom_cases.json
[add] https://crrev.com/3ee1c24c7867a9ea70d8999079d9bc0adc70efc8/tools/perf/page_sets/data/tough_pinch_zoom_cases_001.wprgo.sha1
[modify] https://crrev.com/3ee1c24c7867a9ea70d8999079d9bc0adc70efc8/tools/perf/page_sets/rendering/tough_pinch_zoom_cases.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/7fb3c39aee4fcd1066e44bdb6a9fc7f5baa793b2

commit 7fb3c39aee4fcd1066e44bdb6a9fc7f5baa793b2
Author: Amy Qiu <amyqiu@google.com>
Date: Fri Aug 03 18:08:04 2018

Remove old key desktop move cases pages

Now that it's been a week since key_desktop_move_cases was refreshed,
remove the old pages.

Bug:  852530 
Change-Id: Ic439646390d3acafc9d59f26a089e7ca94fc2466
Reviewed-on: https://chromium-review.googlesource.com/1162341
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Amy Qiu <amyqiu@google.com>
Cr-Commit-Position: refs/heads/master@{#580596}
[modify] https://crrev.com/7fb3c39aee4fcd1066e44bdb6a9fc7f5baa793b2/tools/perf/expectations.config
[modify] https://crrev.com/7fb3c39aee4fcd1066e44bdb6a9fc7f5baa793b2/tools/perf/page_sets/data/rendering_desktop.json
[modify] https://crrev.com/7fb3c39aee4fcd1066e44bdb6a9fc7f5baa793b2/tools/perf/page_sets/rendering/key_desktop_move_cases.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/040ad24ceb2ff9094f413ceb07fc74723d3896da

commit 040ad24ceb2ff9094f413ceb07fc74723d3896da
Author: Amy Qiu <amyqiu@google.com>
Date: Fri Aug 03 19:18:56 2018

Remove old pathological mobile sites pages

Since it's been a week after refreshing pathological_mobile_sites,
remove the old pages.

Bug:  852530 
Change-Id: I75096a10510f8d9bec5838deeb6dcd00e0c55fc0
Reviewed-on: https://chromium-review.googlesource.com/1162344
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Amy Qiu <amyqiu@google.com>
Cr-Commit-Position: refs/heads/master@{#580621}
[modify] https://crrev.com/040ad24ceb2ff9094f413ceb07fc74723d3896da/tools/perf/expectations.config
[modify] https://crrev.com/040ad24ceb2ff9094f413ceb07fc74723d3896da/tools/perf/page_sets/data/rendering_mobile.json
[modify] https://crrev.com/040ad24ceb2ff9094f413ceb07fc74723d3896da/tools/perf/page_sets/rendering/pathological_mobile_sites.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1aceb44e1f71e7f44581a9744cbf94bdb49ab793

commit 1aceb44e1f71e7f44581a9744cbf94bdb49ab793
Author: Amy Qiu <amyqiu@google.com>
Date: Tue Aug 07 16:12:42 2018

Remove old tough pinch zoom pages

Now that it's been a week since tough_pinch_zoom_cases was refreshed,
delete the old pages.

Bug:  852530 
Change-Id: If22c834a48eb6db0847f34ec9b76005e9f96bd58
Reviewed-on: https://chromium-review.googlesource.com/1165266
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Amy Qiu <amyqiu@google.com>
Cr-Commit-Position: refs/heads/master@{#581243}
[modify] https://crrev.com/1aceb44e1f71e7f44581a9744cbf94bdb49ab793/tools/perf/expectations.config
[modify] https://crrev.com/1aceb44e1f71e7f44581a9744cbf94bdb49ab793/tools/perf/page_sets/data/rendering_desktop.json
[modify] https://crrev.com/1aceb44e1f71e7f44581a9744cbf94bdb49ab793/tools/perf/page_sets/data/tough_pinch_zoom_cases.json
[modify] https://crrev.com/1aceb44e1f71e7f44581a9744cbf94bdb49ab793/tools/perf/page_sets/rendering/tough_pinch_zoom_cases.py

This is finished other than crbug.com/865602, which is for renaming a few pages on the perf dashboard.