New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 675671 link

Starred by 1 user
Status: WontFix
Owner:
jam@chromium.org
Closed: Sep 2017
Cc:
jam@chromium.org
█ jasontiller@chromium.org
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

4.9% regression in thread_times.key_mobile_sites_smooth at 438264:438299

Project Member Reported by jasontiller@chromium.org, Dec 19 2016 
Graph is a bit noisy, but this appears to be real.

Comment 1 by jasontiller@chromium.org, Dec 19 2016 

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=675671

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgj9-EpgkM


Bot(s) for this bug's original alert(s):

android-nexus5X
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Dec 19 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev   N   Good?
chromium@438263  2.98751  0.837517  25  good
chromium@438299  3.05249  0.994593  30  bad

Bisect job ran on: android_nexus5X_perf_bisect
Bug ID: 675671

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests thread_times.key_mobile_sites_smooth
Test Metric: thread_browser_cpu_time_per_frame/thread_browser_cpu_time_per_frame
Relative Change: 2.17%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5X_perf_bisect/builds/1006
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8992821398191931008


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5345272387862528

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 5 by 42576172...@developer.gserviceaccount.com, Dec 19 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@438263  2.95751  1.25719  30  good
chromium@438299  3.07882  1.20748  30  bad

Bisect job ran on: android_nexus5X_perf_bisect
Bug ID: 675671

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests thread_times.key_mobile_sites_smooth
Test Metric: thread_browser_cpu_time_per_frame/thread_browser_cpu_time_per_frame
Relative Change: 4.10%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5X_perf_bisect/builds/1010
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8992816439365151024


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5237126185091072

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 7 by 42576172...@developer.gserviceaccount.com, Dec 19 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev   N   Good?
chromium@438263  2.97201  0.916673  30  good
chromium@438299  3.08288  1.0445    30  bad

Bisect job ran on: android_nexus5X_perf_bisect
Bug ID: 675671

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests thread_times.key_mobile_sites_smooth
Test Metric: thread_browser_cpu_time_per_frame/thread_browser_cpu_time_per_frame
Relative Change: 3.73%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5X_perf_bisect/builds/1011
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8992811919478428912


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5791794099388416

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 9 by 42576172...@developer.gserviceaccount.com, Dec 20 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev   N   Good?
chromium@438263  2.98668  0.976791  30  good
chromium@438299  3.09826  1.13675   30  bad

Bisect job ran on: android_nexus5X_perf_bisect
Bug ID: 675671

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests thread_times.key_mobile_sites_smooth
Test Metric: thread_browser_cpu_time_per_frame/thread_browser_cpu_time_per_frame
Relative Change: 3.74%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5X_perf_bisect/builds/1012
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8992798747802718816


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=4940183420534784

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 11 by 42576172...@developer.gserviceaccount.com, Mar 24 2017 


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: android_nexus5X_perf_bisect
  Benchmark    : thread_times.key_mobile_sites_smooth
  Metric       : thread_browser_cpu_time_per_frame/thread_browser_cpu_time_per_frame

Revision             Result                   N
chromium@438263      2.98668 +- 0.976791      30      good
chromium@438299      3.09826 +- 1.13675       30      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests thread_times.key_mobile_sites_smooth

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8984378555899710976

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=4940183420534784


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 13 by 42576172...@developer.gserviceaccount.com, Mar 30 2017

Cc: jam@chromium.org
Owner: jam@chromium.org

=== Auto-CCing suspected CL author jam@chromium.org ===

Hi jam@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : jam
  Commit : 09f47a9a5ed4541b4c9cb48602bbff79fc9fb087
  Date   : Tue Dec 13 21:55:06 2016
  Subject: Run PDFExtensionTest.PdfAccessibilityInOOPIF with PlzNavigate.

Bisect Details
  Configuration: android_nexus5X_perf_bisect
  Benchmark    : thread_times.key_mobile_sites_smooth
  Metric       : thread_browser_cpu_time_per_frame/thread_browser_cpu_time_per_frame
  Change       : 3.33% | 2.37543004132 -> 2.45452154399

Revision             Result                   N
chromium@438263      2.37543 +- 0.854048      14      good
chromium@438281      2.32778 +- 0.572359      9       good
chromium@438290      2.3599 +- 0.606117       9       good
chromium@438295      2.39864 +- 0.706629      14      good
chromium@438296      2.8357 +- 0.561317       6       bad       <--
chromium@438297      2.63188 +- 1.14446       14      bad
chromium@438299      2.45452 +- 0.894194      14      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests thread_times.key_mobile_sites_smooth

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8983687233023668288

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=4940183420534784


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!

Comment 17 by benhenry@chromium.org, Jul 27 2017

Status: Assigned (was: Untriaged)
Explictly assigning. A CL you landed tripped one of the speed metrics we measure in the lab. If this is the first time this has happened to one of your CLs, or if it's been a while, please read: https://chromium.googlesource.com/chromium/src/+/master/docs/speed/addressing_performance_regressions.md

We're looking for one of the following:
1. Justification via explanation
2. Plan to revert or fix
3. Angry rage throwing of equipment at my head

Just be aware that I'm trained in trumpet playing and First Aid and am not afraid to use it.

Note: This was a bulk edit message and not very personal.

Comment 18 by sullivan@chromium.org, Sep 14 2017

Status: WontFix (was: Assigned)
Only happened on one page/device, and bisect cannot reproduce (the CL from #13 changes an unrelated test).
►

Sign in to add a comment