New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 671027 link

Starred by 0 users
Status: WontFix
Owner: ----
Closed: Jul 2017
Cc:
█ svaldez@chromium.org
qyears...@chromium.org
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression

Blocked on:
issue 674689


Sign in to add a comment

10.1%-17.5% regression in blink_perf.parser at 434856:435481

Project Member Reported by qyears...@chromium.org, Dec 4 2016 
See the link to graphs below.
Project Member

Comment 4 by 42576172...@developer.gserviceaccount.com, Dec 5 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@434855  6251.37  283.038  25  good
chromium@435481  6215.64  357.767  25  bad

Bisect job ran on: android_nexus5X_perf_bisect
Bug ID: 671027

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.parser
Test Metric: html-parser/html-parser
Relative Change: 0.57%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5X_perf_bisect/builds/954
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8994155688303361088


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=6395736076320768

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 5 by 42576172...@developer.gserviceaccount.com, Dec 5 2016

Cc: svaldez@chromium.org
Owner: svaldez@chromium.org

=== PERF REGRESSION ===


=== Auto-CCing suspected CL author svaldez@chromium.org ===

Hi svaldez@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : Roll src/third_party/boringssl/src e1cc35e58..f086df9f5
Author  : svaldez
Commit description:
  
https://boringssl.googlesource.com/boringssl/+log/e1cc35e581a6d42f618d8c783f36faebc6023bb7..f086df9f5f28cf9bbfc37055f1cc2f02a4f9e259

Review-Url: https://codereview.chromium.org/2539933002
Cr-Commit-Position: refs/heads/master@{#435330}
Commit  : 9a751f7d6489ae992391da8b72161860d0d98da9
Date    : Wed Nov 30 18:07:46 2016


===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@434855  31676.6  1043.35  25  good
chromium@435168  31719.6  1071.76  25  good
chromium@435325  31506.6  941.009  25  good
chromium@435328  39606.4  48025.2  25  good
chromium@435329  45317.9  46298.9  25  good
chromium@435330  31241.1  852.713  25  bad    <--
chromium@435335  31152.2  1124.76  25  bad
chromium@435345  31303.2  927.064  20  bad
chromium@435364  31298.5  935.415  25  bad
chromium@435403  31308.6  1949.54  25  bad
chromium@435481  31186.9  948.013  25  bad

Bisect job ran on: android_nexus5X_perf_bisect
Bug ID: 671027

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.parser
Test Metric: html-parser-threaded/html-parser-threaded
Relative Change: 1.55%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5X_perf_bisect/builds/952
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8994183209744382608


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5828044948766720

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 8 by qyears...@chromium.org, Dec 6 2016

Owner: ----
Rolling an SSL library doesn't sound likely to affect the blink_perf.parser benchmark. Retrying bisect.
Project Member

Comment 9 by 42576172...@developer.gserviceaccount.com, Dec 7 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@434855  44.2942  7.19179  25  good
chromium@435481  44.4549  5.84302  25  bad

Bisect job ran on: android_nexus5X_perf_bisect
Bug ID: 671027

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.parser
Test Metric: query-selector-all-attribute-complex/query-selector-all-attribute-complex
Relative Change: 0.36%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5X_perf_bisect/builds/965
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8993985757535777200


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=4968327250706432

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 10 by 42576172...@developer.gserviceaccount.com, Dec 7 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: 
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@434855  31728.5  734.284  25  good
chromium@435168  31794.3  705.794  25  good
chromium@435325  31475.8  951.803  25  good
chromium@435403  31399.8  1045.17  25  good
chromium@435423  31272.0  728.472  20  good
chromium@435428  31167.2  1073.22  25  unknown
chromium@435433  31101.4  967.065  25  bad
chromium@435442  30993.9  902.732  25  bad
chromium@435481  31132.1  1118.86  25  bad

Bisect job ran on: android_nexus5X_perf_bisect
Bug ID: 671027

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.parser
Test Metric: html-parser-threaded/html-parser-threaded
Relative Change: 1.88%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5X_perf_bisect/builds/964
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8993985773125564032


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5907991704895488

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 13 by pmeenan@chromium.org, Dec 13 2016 

Kicked a bisect on the html-parser-threaded test: https://chromeperf.appspot.com/buildbucket_job_status/8993379455392389456

and html-parser test: https://chromeperf.appspot.com/buildbucket_job_status/8993379399751844720

Both of which had very clean, large, step-function regressions and should bisect easily.
Project Member

Comment 14 by 42576172...@developer.gserviceaccount.com, Dec 13 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@434855  6118.41  325.117  30  good
chromium@435481  6145.4   411.649  30  bad

Bisect job ran on: android_nexus5X_perf_bisect
Bug ID: 671027

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.parser
Test Metric: html-parser/html-parser
Relative Change: 0.44%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5X_perf_bisect/builds/986
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8993379399751844720


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5777362220744704

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 15 by 42576172...@developer.gserviceaccount.com, Dec 14 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: 
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@434855  32463.7  1357.51  30  good
chromium@435168  32552.7  1095.69  30  good
chromium@435325  32187.3  1147.89  30  good
chromium@435403  32125.9  1164.05  30  good
chromium@435423  31925.8  1205.99  30  good
chromium@435433  32318.1  8107.48  30  good
chromium@435435  31697.5  1201.9   30  unknown
chromium@435436  31582.7  1288.97  30  bad
chromium@435438  32732.0  21318.6  30  bad
chromium@435442  31576.7  956.616  25  bad
chromium@435481  31684.2  924.708  30  bad

Bisect job ran on: android_nexus5X_perf_bisect
Bug ID: 671027

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.parser
Test Metric: html-parser-threaded/html-parser-threaded
Relative Change: 2.40%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5X_perf_bisect/builds/985
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8993379455392389456


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5905931932532736

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 16 by simonhatch@chromium.org, Dec 15 2016

Blockedon: 674689
Project Member

Comment 19 by 42576172...@developer.gserviceaccount.com, Apr 11 2017 


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: android_nexus5X_perf_bisect
  Benchmark    : blink_perf.parser
  Metric       : query-selector-all-attribute-complex/query-selector-all-attribute-complex

Revision             Result                  N
chromium@434855      44.2942 +- 7.19179      25      good
chromium@435481      44.4549 +- 5.84302      25      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.parser

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8982667480808434080

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=4968327250706432


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!

Comment 20 by benhenry@chromium.org, Jul 27 2017

Status: WontFix (was: Untriaged)
This alert was found before M-60 branched. Closing as WontFix as this is believed to either be invalid or non-reproducible.
►

Sign in to add a comment