Issue metadata
Sign in to add a comment
|
28.6%-58.1% regression in v8.browsing_mobile_ignition at 429415:431763 |
||||||||||||||||||||||
Issue descriptionSee the link to graphs below.
,
Nov 21 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8995371065437911456
,
Nov 22 2016
===== BISECT JOB RESULTS ===== Status: failed === Bisection aborted === The bisect was aborted because Bisect cannot identify a culprit: No values were found while testing the reference range. Please contact the the team (see below) if you believe this is in error. ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@429450 N/A N/A 0 good chromium@431695 N/A N/A 0 bad Bisect job ran on: android_nexus5X_perf_bisect Bug ID: 667313 Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests v8.browsing_mobile_ignition Test Metric: memory:chrome:renderer_processes:reported_by_chrome:v8:effective_size_avg/browse_social Relative Change: None Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5X_perf_bisect/builds/897 Job details: https://chromeperf.appspot.com/buildbucket_job_status/8995371065437911456 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=6688241946722304 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Nov 22 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8995307956168741072
,
Nov 22 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8995307799745202496
,
Nov 22 2016
Some of the benchmark recovered but some didn't. I restarted two bisections with fresh (non-archive) builds. Let's see.
,
Nov 22 2016
===== BISECT JOB RESULTS ===== Status: failed === Bisection aborted === The bisect was aborted because Bisect cannot identify a culprit: No values were found while testing the reference range. Please contact the the team (see below) if you believe this is in error. ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@429448 N/A N/A 0 good chromium@431695 N/A N/A 0 bad Bisect job ran on: android_nexus5_perf_bisect Bug ID: 667313 Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests v8.browsing_mobile_ignition Test Metric: memory:chrome:renderer_processes:reported_by_chrome:v8:code_and_metadata_size_avg/browse_social Relative Change: None Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/4366 Job details: https://chromeperf.appspot.com/buildbucket_job_status/8995307956168741072 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5809151630901248 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Nov 23 2016
===== BISECT JOB RESULTS ===== Status: failed === Bisection aborted === The bisect was aborted because Bisect cannot identify a culprit: No values were found while testing the reference range. Please contact the the team (see below) if you believe this is in error. ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@429448 N/A N/A 0 good chromium@431695 N/A N/A 0 bad Bisect job ran on: android_nexus5_perf_bisect Bug ID: 667313 Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests v8.browsing_mobile_ignition Test Metric: memory:chrome:renderer_processes:reported_by_chrome:code_and_metadata_size_avg/browse_social Relative Change: None Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/4367 Job details: https://chromeperf.appspot.com/buildbucket_job_status/8995307799745202496 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5255615708921856 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Nov 23 2016
Annie, seems like this is another missbehaving bisect.
,
Nov 23 2016
Simon, can you take a look at this one?
,
Nov 24 2016
The error says no values were found while testing the reference range, but looking at the individual json outputs I can see that the metric is being generated:
"browse_social@@memory:chrome:renderer_processes:reported_by_chrome:code_and_metadata_size_avg": {
"browse:social:facebook": {
"description": "total size of code and metadata reported by Chrome for renderer processes in Chrome",
"grouping_keys": {
"case": "browse",
"group": "social"
},
"important": false,
"improvement_direction": "down",
"name": "memory:chrome:renderer_processes:reported_by_chrome:code_and_metadata_size_avg",
"page_id": 1,
"std": 0.0,
"tir_label": "browse_social",
"type": "list_of_scalar_values",
"units": "sizeInBytes",
"values": [
1885922.7848101268
]
},
"summary": {
"description": "total size of code and metadata reported by Chrome for renderer processes in Chrome",
"grouping_keys": {
"case": "browse",
"group": "social"
},
"important": false,
"improvement_direction": "down",
"name": "memory:chrome:renderer_processes:reported_by_chrome:code_and_metadata_size_avg",
"std": 0.0,
"tir_label": "browse_social",
"type": "list_of_scalar_values",
"units": "sizeInBytes",
"values": [
1885922.7848101268
]
}
},
Opened a bug (668536) to investigate why compare_samples.py isn't finding the metric in the json.
,
Nov 25 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8994987379075017568
,
Nov 26 2016
===== BISECT JOB RESULTS ===== Status: failed === Bisection aborted === The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence. Please contact the the team (see below) if you believe this is in error. ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@429448 17073987 3773636 27 good chromium@431695 16967552 2682998 27 bad Bisect job ran on: android_nexus5_perf_bisect Bug ID: 667313 Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests v8.browsing_mobile_ignition Test Metric: memory:chrome:renderer_processes:reported_by_chrome:v8:effective_size_avg/browse_social/browse_social_facebook Relative Change: None Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/4381 Job details: https://chromeperf.appspot.com/buildbucket_job_status/8994987379075017568 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5795733951741952 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Nov 28 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8994746720076852400
,
Nov 29 2016
===== BISECT JOB RESULTS ===== Status: failed === Bisection aborted === The bisect was aborted because Bisect cannot identify a culprit: No values were found while testing the reference range. Please contact the the team (see below) if you believe this is in error. ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@429414 N/A N/A 0 good chromium@431763 N/A N/A 0 bad Bisect job ran on: android_nexus9_perf_bisect Bug ID: 667313 Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests v8.browsing_mobile_ignition Test Metric: memory:chrome:renderer_processes:reported_by_chrome:v8:code_and_metadata_size_avg/browse_social Relative Change: None Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus9_perf_bisect/builds/2284 Job details: https://chromeperf.appspot.com/buildbucket_job_status/8994746720076852400 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5847758487945216 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Nov 30 2016
Graph recovered although I am still not sure what the culprit was. Closing this for now ...
,
Nov 30 2016
Unfortunately still waiting for last fix to land for compare_samples, I'll rekick this bisect after just to find out what caused it.
,
Nov 30 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8994540037015222704
,
Dec 1 2016
===== BISECT JOB RESULTS ===== Status: failed === Bisection aborted === The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence. Please contact the the team (see below) if you believe this is in error. ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@429414 1748836 3924163 54 good chromium@431763 2260165 171469 27 bad Bisect job ran on: android_nexus9_perf_bisect Bug ID: 667313 Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests v8.browsing_mobile_ignition Test Metric: memory:chrome:renderer_processes:reported_by_chrome:v8:code_and_metadata_size_avg/browse_social Relative Change: None Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus9_perf_bisect/builds/2295 Job details: https://chromeperf.appspot.com/buildbucket_job_status/8994540037015222704 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5899311274000384 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Dec 1 2016
Well at least it can pull the values out, but looks like the problem is now crbug.com/669608
,
Dec 6 2016
,
Jan 12 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8990641051735829792
,
Jan 14 2017
=== BISECT JOB RESULTS === NO Perf regression found Bisect Details Configuration: android_nexus9_perf_bisect Benchmark : v8.browsing_mobile_ignition Metric : memory:chrome:renderer_processes:reported_by_chrome:v8:effective_size_avg/browse_social Revision Result N chromium@429414 23090676 +- 2439942 14 good chromium@431763 34840908 +- 4301614 14 bad To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests v8.browsing_mobile_ignition Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8990641051735829792 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=4606161703141376 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Mar 22 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8984378726922718768
,
Mar 22 2017
=== BISECT JOB RESULTS === NO Perf regression found Bisect Details Configuration: android_nexus9_perf_bisect Benchmark : v8.browsing_mobile_ignition Metric : memory:chrome:renderer_processes:reported_by_chrome:v8:effective_size_avg/browse_social Revision Result N chromium@429414 23090676 +- 2439942 14 good chromium@431763 34840908 +- 4301614 14 bad To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests v8.browsing_mobile_ignition Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8984378726922718768 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=4606161703141376 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Mar 23 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8984287872905774736
,
Mar 24 2017
=== BISECT JOB RESULTS === NO Perf regression found Bisect Details Configuration: android_nexus5X_perf_bisect Benchmark : v8.browsing_mobile_ignition Metric : memory:chrome:renderer_processes:reported_by_chrome:v8:effective_size_max/browse_social Revision Result N chromium@429414 23090676 +- 2439942 14 good chromium@431763 34840908 +- 4301614 14 bad To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests v8.browsing_mobile_ignition Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8984287872905774736 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=4606161703141376 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Mar 31 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8983626849584949184
,
Apr 1 2017
=== BISECT JOB RESULTS === Bisect was unable to run to completion Please try rerunning the bisect. If failures persist contact the team (see below) and report the error. Bisect Details Configuration: android_nexus7_perf_bisect Benchmark : v8.browsing_mobile_ignition Metric : memory:chrome:renderer_processes:reported_by_chrome:v8:effective_size_avg/browse_social Revision Result N chromium@429414 11557013 +- 942308 13 good chromium@431763 16662549 +- 1648375 12 bad To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests v8.browsing_mobile_ignition Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8983626849584949184 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=4606161703141376 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Apr 11 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8982632427790556960
,
Apr 11 2017
=== BISECT JOB RESULTS === NO Perf regression found, tests failed to produce values Bisect Details Configuration: android_nexus5_perf_bisect Benchmark : v8.browsing_mobile_ignition Metric : memory:chrome:renderer_processes:reported_by_chrome:code_and_metadata_size_avg/browse_social To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests v8.browsing_mobile_ignition Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8982632427790556960 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=5255615708921856 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Apr 11 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8982580669025069392
,
Apr 11 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8982576192381733184
,
Apr 11 2017
=== BISECT JOB RESULTS === NO Perf regression found Bisect Details Configuration: android_nexus5_perf_bisect Benchmark : v8.browsing_mobile_ignition Metric : memory:chrome:renderer_processes:reported_by_chrome:v8:effective_size_avg/browse_social/browse_social_facebook Revision Result N chromium@429448 17073987 +- 3773636 27 good chromium@431695 16967552 +- 2682998 27 bad To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests v8.browsing_mobile_ignition Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8982580669025069392 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=5795733951741952 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Apr 12 2017
=== BISECT JOB RESULTS === Bisect was unable to run to completion Please try rerunning the bisect. If failures persist contact the team (see below) and report the error. Bisect Details Configuration: android_nexus5_perf_bisect Benchmark : v8.browsing_mobile_ignition Metric : memory:chrome:renderer_processes:reported_by_chrome:v8:code_and_metadata_size_avg/browse_social Revision Result N chromium@429448 1413202 +- 83005.3 14 good chromium@431695 1840037 +- 94359.0 12 bad To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests v8.browsing_mobile_ignition Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8982576192381733184 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=5809151630901248 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you! |
|||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||
Comment 1 by rmcilroy@chromium.org
, Nov 21 2016