All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=756217

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=bf4c190831918ce070a195f8329c1444d47edb9e113755104a0af9935908c48a


Bot(s) for this bug's original alert(s):

chromium-rel-win7-dual

Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8971065822756860944

=== Auto-CCing suspected CL author thakis@chromium.org ===

Hi thakis@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Nico Weber
  Commit : 2af1673dd06e01a9adc15d34776f1100ca4e7f5c
  Date   : Sat Aug 12 23:23:01 2017
  Subject: Roll clang 309984:310694.

Bisect Details
  Configuration: win_perf_bisect
  Benchmark    : rasterize_and_record_micro.top_25
  Metric       : rasterize_time/Weather.com
  Change       : 17.40% | 3.27933333333 -> 3.85

Revision             Result                    N
chromium@493982      3.27933 +- 0.055112       6      good
chromium@493984      3.27883 +- 0.0224239      6      good
chromium@493985      3.85 +- 0.0578446         6      bad       <--

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=Weather.com rasterize_and_record_micro.top_25

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8971065822756860944


For feedback, file a bug with component Speed>Bisection

thakis: any chance to look into this one? cc-ing rasterize_and_record owners wkorman and vmpstr to help understand priority.

Note that we switched to static pages for this benchmark on Sep 7:

https://chromium.googlesource.com/chromium/src/+/2a87e954ef8849a96627048487928b381d4b37ed

Would be interesting to see whether we can still repro perf regression with the clang roll but with the new page set. Above command line should still work but --story-filter is probably now 'file://static_top_25/weather.html'.

+simonhatch is there any incantation possible to do a perf tryjob as in #5?

To clarify, you want to test the clang roll in #c3 but with the newer pageset?

Yes, on Windows in particular, with and without  https://chromium.googlesource.com/chromium/src/+/2af1673dd06e01a9adc15d34776f1100ca4e7f5c applied

Sorry, forgot about this. I made an attempt earlier this week. Basically just tried to do a perf try job at the revision that had the new page sets, and reverted the clang roll.

https://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/7773

Failed during initial setup:

"Updating Clang to 309984-2...
Downloading prebuilt clang
Downloading https://commondatastorage.googleapis.com/chromium-browser-clang/Win/clang-309984-2.tgz 
HTTP Error 404: Not Found
Failed to download prebuilt clang clang-309984-2.tgz
Use --force-local-build if you want to build locally.
Exiting.
Error: Command 'C:\\b\\depot_tools\\win_tools-2_7_6_bin\\python\\bin\\python.exe src/tools/clang/scripts/update.py --if-needed' returned non-zero exit status 1 in C:\b\c\b\win_perf_bisect
step returned non-zero exit code: 2"

Also tried just reverting from ToT but that wasn't clean, and I don't really know what I'm doing so I left that alone.

I have a Win7 machine at work. I can take a look at this on Monday. Picking up from thakis@ so I don't forget, or if he's able to get to it sooner feel free to take it back.

15 / 25 of the static pages are failing at ToT on Win7 for me for some reason, though the perf waterfall looks green. I need to look more closely but may not get to it again until next week. If that's not soon enough we should ask thakis@ or someone else with a Windows machine to investigate.

I was also looking at  issue 757342  thinking it could relate, but from perf graphs it appears the regression tracked by this bug is present regardless of that other clang-by-default reverted-or-not flip.

thakis@ how can I view the changelog for clang 309984:310694?

I usually run something like

svn log -r309984:310694  https://llvm.org/svn/llvm-project

The regression appeared to only be on weather.com. There is a new static snapshot of weather.com in CQ as:

https://chromium-review.googlesource.com/c/chromium/src/+/755541

I'll try this again following that landing.

Transferring these to vmpstr@ as owner of the involved benchmark.

Note that the r_and_r benchmark is in consideration to be replaced (for perf regression monitoring purposes at least) by an updated repaint benchmark. More on that at issue 770800.

vmpstr: perf sheriff ping--any update on this bug?

wkorman@, do you know what the next steps are here?

If we are going ahead with replacing r_and_r with repaint, I would consider this a duplicate of that work as we'll end up with a new benchmark/baselines in any case.

I think we've switched to Clang for Windows entirely at this point and maybe too long ago to easily switch back, to compare/contrast, so in any case investigating this further may be more time than it's worth.

The issue was specific to weather.com so we could look at the snapshot we had at the time of the regression and see if there is anything apparent in its content.