Issue metadata
Sign in to add a comment
|
25.1%-25.3% regression in blink_perf.parser at 430486:430557 |
||||||||||||||||||||
Issue description(Also testing new bisect recipe)
,
Nov 10 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8996382686693440816
,
Nov 11 2016
===== BISECT JOB RESULTS ===== Status: failed ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@430485 16.2181 9.03602 25 good chromium@430521 14.978 7.42545 25 good chromium@430539 14.9303 7.44399 25 good chromium@430548 15.9192 7.97257 25 good chromium@430553 12.1459 1.24524 25 bad chromium@430557 10.7326 0.901072 25 bad Bisect job ran on: android_nexus5_perf_bisect Bug ID: 664102 Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.parser Test Metric: query-selector-all-attribute/query-selector-all-attribute Relative Change: 33.82% Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/4331 Job details: https://chromeperf.appspot.com/buildbucket_job_status/8996382686693440816 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5903368231321600 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Nov 11 2016
Bisect failed.
,
Nov 11 2016
,
Nov 12 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8996215233985005552
,
Nov 12 2016
===== BISECT JOB RESULTS ===== Status: failed ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@430485 15.67 9.89456 25 good chromium@430521 14.9908 7.65282 25 good chromium@430539 15.219 6.60582 25 good chromium@430548 15.4198 7.63327 25 good chromium@430553 12.2283 1.39446 25 bad chromium@430557 10.7464 0.78597 25 bad Bisect job ran on: android_nexus5_perf_bisect Bug ID: 664102 Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.parser Test Metric: query-selector-all-attribute/query-selector-all-attribute Relative Change: 31.42% Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/4334 Job details: https://chromeperf.appspot.com/buildbucket_job_status/8996215233985005552 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5229426390859776 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Nov 14 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8996046410849454448
,
Nov 14 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8996013812381474944
,
Nov 14 2016
===== BISECT JOB RESULTS ===== Status: failed ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@430485 16.1704 8.41239 25 good chromium@430521 14.7542 7.57002 25 good chromium@430539 15.4631 7.13546 25 good chromium@430548 15.2793 7.53196 25 good chromium@430553 12.1027 1.09366 25 bad chromium@430557 10.7133 1.02192 25 bad Bisect job ran on: android_nexus5_perf_bisect Bug ID: 664102 Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.parser Test Metric: query-selector-all-attribute-complex/query-selector-all-attribute-complex Relative Change: 33.75% Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/4335 Job details: https://chromeperf.appspot.com/buildbucket_job_status/8996046410849454448 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5841171333513216 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Nov 14 2016
,
Nov 14 2016
=== Auto-CCing suspected CL author vogelheim@chromium.org === Hi vogelheim@chromium.org, the bisect results pointed to your CL below as possibly causing a regression. Please have a look at this info and see whether your CL be related. ===== BISECT JOB RESULTS ===== Status: completed ===== SUSPECTED CL(s) ===== Subject : [CachedAccessor] for window.document. Author : vogelheim Commit description: - Implemend [CachedAccessor] attribute to enable cached accessors. - Implement a cached accessor for window.document. This is based on crrev.com/2335203006 by peterssen@google.com, with only a minor fix added. Intent to implement & ship: https://groups.google.com/a/chromium.org/forum/#!topic/blink-dev/OhIFnre7ytQ Previous (reviewed) CL: https://codereview.chromium.org/2335203006/ Depends on V8 CL: https://codereview.chromium.org/2405213002/ BUG=chromium:634276 Review-Url: https://codereview.chromium.org/2429343004 Cr-Commit-Position: refs/heads/master@{#430551} Commit : 460ec9e9068eb8bf57ce3fae68bb277ca4106f9a Date : Tue Nov 08 09:18:55 2016 ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@430548 16.1762 7.77477 25 good chromium@430550 16.9649 9.16329 25 good chromium@430551 12.1172 1.04847 25 bad <-- chromium@430553 12.1317 1.0854 25 bad Bisect job ran on: android_nexus5_perf_bisect Bug ID: 664102 Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.parser Test Metric: query-selector-all-attribute/query-selector-all-attribute Relative Change: 25.00% Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/4336 Job details: https://chromeperf.appspot.com/buildbucket_job_status/8996013812381474944 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5839146575200256 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Nov 22 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8995299122867384592
,
Nov 22 2016
I've look at this, but am very unsure what to make it of it.
- The graphs are rather convincing, so there's something up with my CL.
- The CL itself is generally fine, so wouldn't want to revert it as-is.
- The observed performance drop puzzles me:
- It's only on Nexus 5. It's *not* on Nexus 10 or other Androids.
- The affected tests indeed use the document property, but only peripherally.
- There are a number of other tests is the query-selector-*/* range that use
the document property in exactly the same way, but aren't affected, even
on Nexus 5.
I'm at a loss what would be specific to this setup.
I'll look into this a bit more, but am tempted to 'WontFix'. Either way, I think priority 0/Emergency is quite exaggerated.
@perezju/sullivan: Please let me know if you disagree on priority.
,
Nov 22 2016
I'm so sorry about this, Daniel! The P0 was for the bisect failures, not for the actual performance test regression. I think we can actually mark "Fixed" because the bisect works correctly now. I agree the actual regression is minor.
,
Nov 22 2016
===== BISECT JOB RESULTS ===== Status: failed ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@430502 76.8899 5.24566 25 good chromium@430530 73.432 3.34807 25 good chromium@430544 77.6142 3.81502 25 good chromium@430548 77.277 8.53103 25 good chromium@430551 66.2582 3.36291 25 bad chromium@430558 65.9379 4.85307 25 bad Bisect job ran on: android_nexus7_perf_bisect Bug ID: 664102 Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.parser Test Metric: query-selector-last/query-selector-last Relative Change: 14.24% Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus7_perf_bisect/builds/3509 Job details: https://chromeperf.appspot.com/buildbucket_job_status/8995299122867384592 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5320735902400512 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Apr 11 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8982651612202801680
,
Apr 11 2017
=== BISECT JOB RESULTS === Perf regression found but unable to narrow commit range Build failures prevented the bisect from narrowing the range further. Bisect Details Configuration: android_nexus5_perf_bisect Benchmark : blink_perf.parser Metric : query-selector-all-attribute/query-selector-all-attribute Suspected Commit Range 1 commits in range https://chromium.googlesource.com/chromium/src/+log/8a65ad68ecff0e06b76c412c8ad539053b21dc81..e62f1a5d8886fc1086a180f4da94e0a3d1ca03dc Revision Result N chromium@430485 15.67 +- 9.89456 25 good chromium@430521 14.9908 +- 7.65282 25 good chromium@430539 15.219 +- 6.60582 25 good chromium@430548 15.4198 +- 7.63327 25 good chromium@430553 12.2283 +- 1.39446 25 bad chromium@430557 10.7464 +- 0.78597 25 bad To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.parser Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8982651612202801680 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=5229426390859776 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Apr 11 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8982620147188801008
,
Apr 11 2017
=== BISECT JOB RESULTS === Perf regression found but unable to narrow commit range Build failures prevented the bisect from narrowing the range further. Bisect Details Configuration: android_nexus7_perf_bisect Benchmark : blink_perf.parser Metric : query-selector-last/query-selector-last Suspected Commit Range 1 commits in range https://chromium.googlesource.com/chromium/src/+log/8a65ad68ecff0e06b76c412c8ad539053b21dc81..460ec9e9068eb8bf57ce3fae68bb277ca4106f9a Revision Result N chromium@430502 76.8899 +- 5.24566 25 good chromium@430530 73.432 +- 3.34807 25 good chromium@430544 77.6142 +- 3.81502 25 good chromium@430548 77.277 +- 8.53103 25 good chromium@430551 66.2582 +- 3.36291 25 bad chromium@430558 65.9379 +- 4.85307 25 bad To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.parser Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8982620147188801008 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=5320735902400512 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you! |
|||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||
Comment 1 by perezju@chromium.org
, Nov 10 2016