New issue
Advanced search Search tips

Issue 710203 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner: ----
Closed: Jun 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression

Blocked on:
issue 711039



Sign in to add a comment

70.6% regression in performance_browser_tests at 462561:462739

Project Member Reported by m...@chromium.org, Apr 10 2017

Issue description

See the link to graphs below.
 

Comment 1 by m...@chromium.org, Apr 10 2017

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=710203

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgvPSx_AkM


Bot(s) for this bug's original alert(s):

chromium-rel-win7-dual
Project Member

Comment 5 by 42576172...@developer.gserviceaccount.com, Apr 13 2017


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: win_perf_bisect
  Benchmark    : performance_browser_tests
  Metric       : TabCapturePerformance_comp_gpu_novsync_webrtc/Capture

Revision             Result                  N
chromium@462560      52.7114 +- 3.14636      21      good
chromium@462739      52.4825 +- 1.95393      21      bad

To Run This Test
  .\src\out\Release\performance_browser_tests.exe --test-launcher-print-test-stdio=always --enable-gpu

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8982483424393696416

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=4982480689430528


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Blockedon: 711039
Project Member

Comment 8 by 42576172...@developer.gserviceaccount.com, Apr 13 2017

Cc: xiy...@chromium.org
Owner: xiy...@chromium.org

=== Auto-CCing suspected CL author xiyuan@chromium.org ===

Hi xiyuan@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : xiyuan
  Commit : bb330d1b8f803ccb942c7ca2e7e2510c612c1828
  Date   : Thu Apr 06 18:49:14 2017
  Subject: cros: Call GetPrivateSlotForChromeOSUser properly

Bisect Details
  Configuration: win_perf_bisect
  Benchmark    : performance_browser_tests
  Metric       : TabCapturePerformance_comp_gpu/CaptureSucceeded
  Change       : 8.91% | 45.5211281667 -> 49.5768733333

Revision             Result                   N
chromium@462560      45.5211 +- 2.74253       6      good
chromium@462561      49.4993 +- 0.999657      6      bad       <--
chromium@462562      49.5915 +- 0.700449      6      bad
chromium@462563      49.3079 +- 0.513956      6      bad
chromium@462566      49.2024 +- 0.543092      6      bad
chromium@462572      49.5171 +- 0.935002      6      bad
chromium@462583      49.2889 +- 0.914639      6      bad
chromium@462605      49.3482 +- 0.59582       6      bad
chromium@462650      49.3659 +- 0.601749      6      bad
chromium@462739      49.5769 +- 0.517818      6      bad

To Run This Test
  .\src\out\Release\performance_browser_tests.exe --test-launcher-print-test-stdio=always --enable-gpu

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8982418267435551968

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5826301383933952


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!

Comment 10 by m...@chromium.org, Apr 13 2017

Owner: m...@chromium.org
Hmm...This doesn't make sense, given a CrOS-only change regressed on the Windows platform.

I've kicked-off a new bisect job with a wider range, to see if something earlier is causing the problem (or perhaps there is a flaky performance regression?).

Comment 11 by m...@chromium.org, Apr 13 2017

Status: Assigned (was: Untriaged)
Project Member

Comment 12 by 42576172...@developer.gserviceaccount.com, Apr 14 2017


=== BISECT JOB RESULTS ===
Bisect was unable to run to completion

Error: INFRA_FAILURE

The bisect was able to narrow the range, you can try running with:
  good_revision: 0cfc2fe30fabcd67f5b8b5fe158393ca194d8b45
  bad_revision : 5a988db10ba9ed054abe390821248a795f470713

If failures persist contact the team (see below) and report the error.


Bisect Details
  Configuration: win_perf_bisect
  Benchmark    : performance_browser_tests
  Metric       : TabCapturePerformance_comp_gpu_novsync_webrtc/Capture

Revision             Result                   N
chromium@462430      35.9 +- 2.0955           14      good
chromium@462585      35.5395 +- 1.0537        14      bad
chromium@462739      35.3673 +- 0.549258      6       bad

To Run This Test
  .\src\out\Release\performance_browser_tests.exe --test-launcher-print-test-stdio=always --enable-gpu

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8982392489242891008

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5006540571934720


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!

Comment 13 by m...@chromium.org, Apr 14 2017

Le sigh....

Comment 15 by m...@chromium.org, Apr 14 2017

Kicked it off AGAIN. >:-/
Project Member

Comment 17 by 42576172...@developer.gserviceaccount.com, Apr 15 2017

Cc: brat...@opera.com
Owner: brat...@opera.com

=== Auto-CCing suspected CL author bratell@opera.com ===

Hi bratell@opera.com, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : bratell
  Commit : 2bd991090f462a64efe0f6b26bf0ad35e19386b0
  Date   : Thu Apr 06 11:46:41 2017
  Subject: Avoid duplicate functions/code in core/editing: MatchResultICU

Bisect Details
  Configuration: win_perf_bisect
  Benchmark    : performance_browser_tests
  Metric       : TabCapturePerformance_comp_gpu_novsync_webrtc/Capture
  Change       : 4.68% | 36.5245418889 -> 35.181281

Revision             Result                   N
chromium@462430      36.5245 +- 1.89357       9       good
chromium@462431      35.7425 +- 3.05746       14      bad       <--
chromium@462432      35.7539 +- 0.459339      9       bad
chromium@462433      35.6703 +- 0.719396      6       bad
chromium@462435      35.6848 +- 0.40932       6       bad
chromium@462440      35.6184 +- 0.220225      6       bad
chromium@462450      35.7855 +- 0.372834      6       bad
chromium@462469      35.7756 +- 0.670598      6       bad
chromium@462508      35.7235 +- 0.604564      6       bad
chromium@462585      35.8534 +- 0.524546      6       bad
chromium@462739      35.1813 +- 0.491885      6       bad

To Run This Test
  .\src\out\Release\performance_browser_tests.exe --test-launcher-print-test-stdio=always --enable-gpu

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8982299669675671040

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5006540571934720


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 18 by 42576172...@developer.gserviceaccount.com, Apr 15 2017


=== BISECT JOB RESULTS ===
Perf regression found but unable to narrow commit range

Build failures prevented the bisect from narrowing the range further.


Bisect Details
  Configuration: win_perf_bisect
  Benchmark    : performance_browser_tests
  Metric       : TabCapturePerformance_comp_gpu_novsync_webrtc/Capture
  Change       : 2.54% | 53.2274891429 -> 52.4015985

Suspected Commit Range
  2 commits in range
  https://chromium.googlesource.com/chromium/src/+log/d4e0737157002b83b6293fe352062c18a24fa926..8d547e161dbe03d5887207c72c3bc4af33fbfea0


Revision             Result                   N
chromium@462500      53.2275 +- 2.03708       14       good
chromium@462501      ---                      ---      build failure
chromium@462502      52.8836 +- 0.944331      14       bad
chromium@462503      52.8817 +- 0.749228      6        bad
chromium@462505      52.7517 +- 0.532535      6        bad
chromium@462510      52.9592 +- 1.00437       9        bad
chromium@462519      52.7994 +- 0.724405      6        bad
chromium@462538      52.8271 +- 0.827345      6        bad
chromium@462575      53.0903 +- 0.860095      9        bad
chromium@462650      52.5051 +- 0.745249      6        bad
chromium@462800      52.4016 +- 0.619534      6        bad

To Run This Test
  .\src\out\Release\performance_browser_tests.exe --test-launcher-print-test-stdio=always --enable-gpu

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8982300394275345728

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5834938294730752


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!

Comment 19 by brat...@opera.com, Apr 15 2017

Owner: ----
Status: Available (was: Assigned)
I'm sorry, this looks like a very elusive regression. The commit it found 2 hours ago is a renamed variable in documentedit. Extremely unlikely to have caused a webrtc performance regression. The commits it found 30 minutes ago also seem very unlikely (removed a build dependency and an header include).

Seems that in 4-5 bisects it has found different commits or failed so is the testcase now so random that the bisect script can't see whether it's before or after the problems started?

Comment 20 by m...@chromium.org, Jun 5 2017

Status: WontFix (was: Available)
Metrics recovered around r463850.

Sign in to add a comment