New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 671177 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Closed: Dec 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression

Blocked on:
issue 671481



Sign in to add a comment

28.9%-36.1% regression in speedometer at 435864:435964

Project Member Reported by hablich@chromium.org, Dec 5 2016

Issue description

See the link to graphs below.
 

===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@435863  884.242  453.433  50  good
chromium@435964  860.235  168.33   50  bad

Bisect job ran on: winx64_zen_perf_bisect
Bug ID: 671177

Test Command: src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests speedometer
Test Metric: AngularJS-TodoMVC/AngularJS-TodoMVC
Relative Change: 2.72%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64_zen_perf_bisect/builds/623
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8994110572339879632


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5003433642295296

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: 
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@435863  383.767  334.178  50  good
chromium@435876  403.145  166.258  50  good
chromium@435883  416.123  319.67   50  unknown
chromium@435889  427.089  376.686  50  bad
chromium@435914  404.212  115.343  50  bad
chromium@435964  404.383  180.371  50  bad

Bisect job ran on: winx64_zen_perf_bisect
Bug ID: 671177

Test Command: src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests speedometer
Test Metric: FlightJS-TodoMVC/FlightJS-TodoMVC
Relative Change: 5.37%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64_zen_perf_bisect/builds/624
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8994108616826603920


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5818593470578688

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Blockedon: 671481
Cc: pkalinnikov@chromium.org
Owner: pkalinnikov@chromium.org

=== PERF REGRESSION ===


=== Auto-CCing suspected CL author pkalinnikov@chromium.org ===

Hi pkalinnikov@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : Add high-precision timing histograms.
Author  : pkalinnikov
Commit description:
  
BUG=609747

Review-Url: https://codereview.chromium.org/2503283003
Cr-Commit-Position: refs/heads/master@{#435890}
Commit  : 7c84dd6c2273cea47031237046ec001ac7fb75ea
Date    : Fri Dec 02 09:07:18 2016


===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@435863  503.72   205.978  50  good
chromium@435889  531.628  486.721  50  good
chromium@435890  491.257  70.7267  50  bad    <--
chromium@435891  488.275  73.9187  50  bad
chromium@435893  493.689  193.847  50  bad
chromium@435896  489.856  138.197  50  bad
chromium@435902  494.269  125.575  50  bad
chromium@435914  491.643  137.152  50  bad
chromium@435964  492.882  124.03   50  bad

Bisect job ran on: winx64_zen_perf_bisect
Bug ID: 671177

Test Command: src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests speedometer
Test Metric: BackboneJS-TodoMVC/BackboneJS-TodoMVC
Relative Change: 2.15%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64_zen_perf_bisect/builds/628
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8993843734441711088


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5260521358491648

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Cc: engedy@chromium.org
I think the CL 2503283003 is not relevant to this performance regression for 2 reasons:

1. The component that is modified in the CL is not active, when the aforementioned benchmark runs, therefore it can't influence its results. To prove it, I added CHECK(false) to DocumentSubresourceFilter constructor, and run the benchmark, and it worked successfully without failures.

2. The performance graph (https://chromeperf.appspot.com/group_report?bug_id=671177) reached its normal level after a while, so that spike was probably flaky and unrelated to my CL.

Michael, what do you think? Can we mark this as resolved?
Owner: hablich@chromium.org
Status: WontFix (was: Untriaged)
Yeah, probably zenbooks missbehaving.
Project Member

Comment 13 by 42576172...@developer.gserviceaccount.com, Apr 10 2017


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: winx64_zen_perf_bisect
  Benchmark    : speedometer
  Metric       : AngularJS-TodoMVC/AngularJS-TodoMVC

Revision             Result                  N
chromium@435863      897.899 +- 190.669      21      good
chromium@435964      975.763 +- 745.913      21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests speedometer

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8982667341478765424

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5003433642295296


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!

Sign in to add a comment