New issue
Advanced search Search tips

Issue 771307 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Closed: Oct 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression

Blocking:
issue 683729



Sign in to add a comment

12.7%-27.8% regression in blink_perf.dom at 503847:503935

Project Member Reported by majidvp@chromium.org, Oct 3 2017

Issue description

See the link to graphs below.
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=771307

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=42a4ce346a8ffbd62a30d0be1a2f609a3259b55cc5e50ee84b1a7e8738db0c79


Bot(s) for this bug's original alert(s):

chromium-rel-win8-dual
win-high-dpi
Cc: brucedaw...@chromium.org
Owner: brucedaw...@chromium.org
Status: Assigned (was: Untriaged)

=== Auto-CCing suspected CL author brucedawson@chromium.org ===

Hi brucedawson@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Bruce Dawson
  Commit : d7af1e3aba05d9839ecffa52585b6e4808212142
  Date   : Sat Sep 23 00:35:11 2017
  Subject: Make VS 2017 the default compiler on Windows

Bisect Details
  Configuration: winx64_high_dpi_perf_bisect
  Benchmark    : blink_perf.dom
  Metric       : move-down-with-hidden-elements/move-down-with-hidden-elements

Revision             Result                     N
chromium@503871      0.413 +- 0.0677052         6       good
chromium@503903      0.432833 +- 0.0945983      6       good
chromium@503911      0.424 +- 0.0881703         9       good
chromium@503913      0.443071 +- 0.227158       14      good
chromium@503914      0.432167 +- 0.0631414      6       good
chromium@503915      0.548222 +- 0.192872       9       bad       <--
chromium@503919      0.513556 +- 0.0462193      9       bad
chromium@503935      0.529667 +- 0.0314219      6       bad

Please refer to the following doc on diagnosing blink_perf regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/speed/benchmark_harnesses/blink_perf.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.dom

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8966726225403201456


For feedback, file a bug with component Speed>Bisection
Given the nature of my change (altering the default compiler for all Windows builds) it is inevitable that there will be some winners and some losers. Given the benefits of the change (conformance, mostly) we would probably be okay with a slight net loss in performance, across all our benchmarks.

When I look at all of the results for this change:

https://chromeperf.appspot.com/group_report?rev=503915

I see eighteen test results for various flavors of Windows. Four are regressions and fourteen are improvements. One of the regressions (sinc_resampler_convolve/unoptimized_aligned,  crbug.com/768394 ) was analyzed and found to be unimportant, so that leaves three regressions.

Well, it actually leaves four because this one doesn't show up on that list for some reason.

But, regardless, my analysis suggests that this is a reasonable consequence of the compiler switch and we should close it as WontFix. If I'm missing anything then please let me know - I'm not the most expert at navigating our perf results.

Blocking: 683729
Status: WontFix (was: Assigned)
See comment #4 for details, but basically the switch to VS 2017 randomized the performance results. This test was a loser, but the overall trend was positive. WontFix.

Sign in to add a comment