It looks like, under the hood, this benchmark uses a select few stories from the top_25 page set, which at least gmail was updated in the CL. However, it looks like the Docs page wasn't updated. I'm going to try and reproduce this locally: I expect that we need to do something like what we did for gmail (https://cs.chromium.org/chromium/src/tools/perf/page_sets/top_pages.py?type=cs&q=gmail+top_pages.py&sq=package:chromium&l=68) for Docs.

Urgh, looks like it only fails on Windows, which makes this a pain to reproduce (I don't have my local Windows machine up and running yet.)

Also, it looks like since that CL was submitted, a follow-up one was that also migrated Docs to the new login method, so I'm not quite sure that's our answer.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/82102d061fcc284e520803bad8670ed5ba939990

commit 82102d061fcc284e520803bad8670ed5ba939990
Author: Charlie Andrews <charliea@chromium.org>
Date: Thu Sep 14 23:19:27 2017

Update top_25 WPR recordings

I'm fairly sure that they should have been updated in
http://crrev.com/c/648252, but it was overlooked.

Bug:  756117 ,  747021 , 765383
Change-Id: I4ebb7a5cbaf9fa52c99ba9f05d990ca639f3d3e7
Reviewed-on: https://chromium-review.googlesource.com/667250
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Charlie Andrews <charliea@chromium.org>
Cr-Commit-Position: refs/heads/master@{#502092}
[modify] https://crrev.com/82102d061fcc284e520803bad8670ed5ba939990/tools/perf/page_sets/data/top_25.json
[add] https://crrev.com/82102d061fcc284e520803bad8670ed5ba939990/tools/perf/page_sets/data/top_25_011.wprgo.sha1

Doh. It looks like my rerecording caused the Google barackobama story to start failing. CL to revert that to its prior state is now in the CQ: https://chromium-review.googlesource.com/c/chromium/src/+/668685

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/d15c79924126a8ed94ac652fcd4c760347d142d6

commit d15c79924126a8ed94ac652fcd4c760347d142d6
Author: catapult-deps-roller@chromium.org <catapult-deps-roller@chromium.org>
Date: Sat Sep 16 01:32:18 2017

Roll src/third_party/catapult/ 5c13bd84f..c49949828 (3 commits)

https://chromium.googlesource.com/external/github.com/catapult-project/catapult.git/+log/5c13bd84fb39..c49949828b72

$ git log 5c13bd84f..c49949828 --date=short --no-merges --format='%ad %ae %s'
2017-09-15 nednguyen [common] Update discover.py to use importlib for importing module instead of builtin __import__
2017-09-15 charliea Disable power monitoring on all desktop platforms
2017-09-15 bpastene devil: Fix args help text of video_recorder.py

Created with:
  roll-dep src/third_party/catapult
BUG= 765783 ,765383


Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, see:
http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls


CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel
TBR=sullivan@chromium.org

Change-Id: I948184a41bc8572be7266d2c176c03ef49562381
Reviewed-on: https://chromium-review.googlesource.com/669684
Reviewed-by: <catapult-deps-roller@chromium.org>
Commit-Queue: <catapult-deps-roller@chromium.org>
Cr-Commit-Position: refs/heads/master@{#502465}
[modify] https://crrev.com/d15c79924126a8ed94ac652fcd4c760347d142d6/DEPS

Doh. That CL was unrelated, and intended for https://bugs.chromium.org/p/chromium/issues/detail?id=763263.

I'm going to assign this to flackr@ now after the disables that I did