New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 780187 link

Starred by 1 user

Issue metadata

Status: Duplicate
Merged: issue 778135
Owner:
Closed: Nov 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

15.7% regression in blink_perf.layout at 510887:510999

Project Member Reported by briander...@chromium.org, Oct 31 2017

Issue description

See the link to graphs below.
 
Project Member

Comment 1 by 42576172...@developer.gserviceaccount.com, Oct 31 2017

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=780187

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=af7e73118821847fe0eff3e1836708d49f40559354a1d4335dc3830c980f097f


Bot(s) for this bug's original alert(s):

chromium-rel-win7-gpu-nvidia
Cc: b...@chromium.org
Owner: b...@chromium.org
Status: Assigned (was: Untriaged)

=== Auto-CCing suspected CL author bnc@chromium.org ===

Hi bnc@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Bence Béky
  Commit : b22c2f1ddd9301b8ad2886d057fa45b574c35e31
  Date   : Mon Oct 23 21:16:38 2017
  Subject: Deprecate ERR_INVALID_SPDY_STREAM net error code.

Bisect Details
  Configuration: winx64nvidia_perf_bisect
  Benchmark    : blink_perf.layout
  Metric       : SimpleTextPathLineLayout/SimpleTextPathLineLayout
  Change       : 3.22% | 172160.756406 -> 166610.695522

Revision             Result                 N
chromium@510886      172161 +- 2416.99      6       good
chromium@510915      168989 +- 10226.6      14      good
chromium@510916      164290 +- 8211.13      6       good
chromium@510917      157213 +- 5632.96      6       bad       <--
chromium@510919      159639 +- 11343.1      6       bad
chromium@510922      164049 +- 13073.9      14      bad
chromium@510929      163874 +- 2006.61      9       bad
chromium@510943      156749 +- 12844.2      6       bad
chromium@510999      166611 +- 2857.44      6       bad

Please refer to the following doc on diagnosing blink_perf regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/speed/benchmark_harnesses/blink_perf.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.layout

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8964186510196860256


For feedback, file a bug with component Speed>Bisection

Comment 4 by b...@chromium.org, Nov 2 2017

Cc: -b...@chromium.org benjhayden@chromium.org
Status: Started (was: Assigned)
cc'ing benjhayden@ who helped me so much with  issue 762300 .

I ran the benchmark locally:
tools/perf/run_benchmark run -v --browser=exact --story-filter=SimpleTextPathLineLayout --pageset-repeat=21 --also-run-disabled-tests blink_perf.layout --browser-executable=out/gn/chrome

Results for revert of suspected culprit:
avg	2,763.592
count	105
max	3,716.705
min	2,067.594
std	322.618
sum	290,177.182

Results for tip of tree:
Δavg	-2.644
%Δavg	-0.096%
Δstd	+28.481
%Δstd	+8.828%
z-score	-0.0 σ
p-value	0.674
U	5,327.000
avg	2,760.948
count	105
max	3,757.412
min	2,083.510
std	351.098
sum	289,899.586

Emoticon has flat mouth, "insignificant regression".  I know, it's a different architecture.

Now running the following command:
tools/perf/run_benchmark try winx66nvidia --story-filter=SimpleTextPathLineLayout --pageset-repeat=51 --also-run-disabled-tests blink_perf.layout
on https://crrev.com/c/751301.




Comment 5 by b...@chromium.org, Nov 3 2017

Result of benchmark on https://crrev.com/c/751301 is "significant".
Sorry, what's the question?
The HTML Results for winx64nvidia_perf_bisect for Patchset 2 looks to me like 751301 significantly improves SimpleTextPathLineLayout.
https://console.developers.google.com/m/cloudstorage/b/chromium-telemetry/o/html-results/results-2017-11-02_09-57-06#r=TOT&s=%25Δavg
https://chromium-review.googlesource.com/c/chromium/src/+/751301
It looks like blink_perf.layout is owned by eae, if you have questions about blink_perf.layout or SimpleTextPathLineLayout.

Comment 7 by b...@chromium.org, Nov 6 2017

Cc: e...@chromium.org
eae@: Please help me understand how https://crrev.com/c/751301 could possibly have an effect on SimpleTextPathLineLayout/blink_perf.layout.  Thanks.

Comment 8 by e...@chromium.org, Nov 6 2017

I can't see how it possibly could.
The graph is pretty noisy, and goes down over several revisions. Trying a wider bisect.
Mergedinto: 778135
Status: Duplicate (was: Started)

=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Hans Wennborg
  Commit : 22525134c75ca2a005997a717f6887a23af86210
  Date   : Mon Oct 23 17:10:25 2017
  Subject: win: Re-enable Clang by default

Bisect Details
  Configuration: winx64nvidia_perf_bisect
  Benchmark    : blink_perf.layout
  Metric       : SimpleTextPathLineLayout/SimpleTextPathLineLayout
  Change       : 14.63% | 185527.631758 -> 158393.889082

Revision             Result                 N
chromium@510721      185528 +- 2853.81      6      good
chromium@510810      180691 +- 6319.48      6      good
chromium@510821      184330 +- 3269.91      6      good
chromium@510823      180622 +- 5463.26      6      good
chromium@510824      161300 +- 6153.52      6      bad       <--
chromium@510827      165905 +- 4466.94      6      bad
chromium@510832      158200 +- 20344.9      6      bad
chromium@510854      160461 +- 33170.1      6      bad
chromium@510898      166078 +- 2000.66      6      bad
chromium@511074      157413 +- 10816.2      6      bad
chromium@511426      158394 +- 8584.61      6      bad

Please refer to the following doc on diagnosing blink_perf regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/speed/benchmark_harnesses/blink_perf.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.layout

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8963645346259012032


For feedback, file a bug with component Speed>Bisection

Sign in to add a comment