Issue metadata
Sign in to add a comment
|
4.1% regression in startup.warm.blank_page at 397045:397076 |
||||||||||||||||||||
Issue descriptionSee the link to graphs below.
,
Jul 7 2016
Moving this nonessential bug to the next milestone. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Jul 19 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9006697708837878336
,
Jul 19 2016
Last bisect has narrowed down the range to 397026:397035 but mothing in that range jumps out to me! Kicking another bisect with that range. http://test-results.appspot.com/revision_range?start=397026&end=397035 2016-07-03 18:08:34 ===== BISECT JOB RESULTS ===== Status: started ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@397026 552.6 6.10737 5 good chromium@397035 808.4 223.742 5 bad chromium@397043 643.6 4.219 5 bad chromium@397064 718.8 18.6869 5 bad Bisect job ran on: winx64_zen_perf_bisect Bug ID: 616936 Test Command: src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --also-run-disabled-tests startup.warm.blank_page Test Metric: foreground_tab_request_start/foreground_tab_request_start Relative Change: 30.08% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64_zen_perf_bisect/builds/241 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9008138049139558800
,
Aug 5 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9005127659133418576
,
Aug 5 2016
That bisect doesn't look super clean. Either way, trying another.
,
Aug 6 2016
=== Auto-CCing suspected CL author hiroshige@chromium.org === Hi hiroshige@chromium.org, the bisect results pointed to your CL below as possibly causing a regression. Please have a look at this info and see whether your CL be related. ===== BISECT JOB RESULTS ===== Status: completed ===== SUSPECTED CL(s) ===== Subject : Do not initiate revalidation of Resource with redirects from MemoryCache Author : hiroshige Commit description: BUG= 613971 Review-Url: https://codereview.chromium.org/2011283002 Cr-Commit-Position: refs/heads/master@{#397064} Commit : 64ad58e1d8451857f6a763651727c9d39375ea3b Date : Wed Jun 01 05:16:57 2016 ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@397026 659.583 60.8612 12 good chromium@397043 666.583 42.9703 12 good chromium@397054 506.2 5.54076 5 good chromium@397059 508.8 7.04982 5 good chromium@397062 494.2 7.39594 5 good chromium@397063 546.8 12.7945 5 good chromium@397064 731.75 35.3018 8 bad <-- Bisect job ran on: winx64_zen_perf_bisect Bug ID: 616936 Test Command: src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --also-run-disabled-tests startup.warm.blank_page Test Metric: foreground_tab_load_complete/foreground_tab_load_complete Relative Change: 2.46% Score: 99.9 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64_zen_perf_bisect/builds/362 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9005127659133418576 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5900869707497472 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Aug 18 2016
Perf sheriff ping: reminder to follow up on possible performance issues
,
Sep 9 2016
Friendly sheriff ping: Have you had a chance to look into this regression? Thanks!
,
Sep 10 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9001943254950245936
,
Sep 10 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9001943244892946416
,
Sep 10 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9001942002198859504
,
Sep 10 2016
Hmm. My change only affects revalidation of resources that contains redirects, and thus is unlikely to affect the test. When tested on Linux, the code I changed doesn't seem to be executed (inserted CHECK(0) there and wasn't executed). Started bisects again...
,
Sep 10 2016
===== BISECT JOB RESULTS ===== Status: completed ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@397049 136.884 0.826668 5 good chromium@397062 137.674 0.564901 5 good chromium@397069 137.8 0.385686 5 good chromium@397071 138.642 0.478629 5 bad chromium@397074 139.547 0.87089 5 bad Bisect job ran on: winx64nvidia_perf_bisect Bug ID: 616936 Test Command: src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --also-run-disabled-tests startup.warm.blank_page Test Metric: first_main_frame_load_time/first_main_frame_load_time Relative Change: 1.95% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64nvidia_perf_bisect/builds/1863 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9001942002198859504 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=6138928912924672 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Sep 10 2016
===== BISECT JOB RESULTS ===== Status: completed ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@397026 537.6 21.3612 5 good chromium@397027 536.4 6.22896 5 good chromium@397035 641.2 27.335 5 bad chromium@397043 655.0 8.30662 5 bad chromium@397064 759.0 31.9687 5 bad Bisect job ran on: winx64_zen_perf_bisect Bug ID: 616936 Test Command: src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --also-run-disabled-tests startup.warm.blank_page Test Metric: foreground_tab_load_complete/foreground_tab_load_complete Relative Change: 41.18% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64_zen_perf_bisect/builds/473 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9001943254950245936 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=4574400840466432 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Sep 10 2016
===== BISECT JOB RESULTS ===== Status: completed ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@397026 503.407 21.5411 27 good chromium@397027 498.583 15.9343 12 good chromium@397035 520.37 24.5422 27 bad chromium@397043 512.0 28.0651 18 bad chromium@397064 526.75 10.3475 8 bad Bisect job ran on: winx64_zen_perf_bisect Bug ID: 616936 Test Command: src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --also-run-disabled-tests startup.warm.blank_page Test Metric: foreground_tab_request_start/foreground_tab_request_start Relative Change: 3.03% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64_zen_perf_bisect/builds/474 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9001943244892946416 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5860786293964800 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Oct 11 2016
Perf sheriff ping
,
Oct 17 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8998517862180072672
,
Oct 17 2016
Only one bot, but there is a ref build result and it appears that the ref build performance didn't regress; started another bisect job but if we can't reproduce it with bisect bots then I'm not sure how we'll find the cause.
,
Oct 17 2016
Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64_zen_perf_bisect/builds/553 Failure reason: the build has failed. Additional errors: The metric was not found in the test output. Either of the initial "good" or "bad" revisions failed to be tested or built.
,
Nov 17 2016
Unassign myself as my change is not executed in the tests (Comment #13).
,
May 18 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8979227173964723680
,
May 18 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8979227152916851216
,
May 18 2017
=== Auto-CCing suspected CL author hiroshige@chromium.org === Hi hiroshige@chromium.org, the bisect results pointed to your CL, please take a look at the results. === BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : hiroshige Commit : 64ad58e1d8451857f6a763651727c9d39375ea3b Date : Wed Jun 01 05:16:57 2016 Subject: Do not initiate revalidation of Resource with redirects from MemoryCache Bisect Details Configuration: winx64_zen_perf_bisect Benchmark : startup.warm.blank_page Metric : foreground_tab_request_start/foreground_tab_request_start Change : 13.88% | 513.833333333 -> 585.166666667 Revision Result N chromium@397026 513.833 +- 38.7406 6 good chromium@397043 512.333 +- 43.8102 6 good chromium@397054 451.833 +- 77.1416 6 good chromium@397059 484.0 +- 59.716 6 good chromium@397062 451.5 +- 41.7073 6 good chromium@397063 505.222 +- 76.7565 9 good chromium@397064 585.167 +- 71.7136 6 bad <-- To Run This Test src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=2 --also-run-disabled-tests startup.warm.blank_page Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8979227173964723680 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=5810328853544960 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
May 18 2017
Unassign myself again (According to my Comment #13 my changed code path is not executed in the tests).
,
May 19 2017
=== BISECT JOB RESULTS === Perf regression found but unable to narrow commit range Build failures prevented the bisect from narrowing the range further. Bisect Details Configuration: win_perf_bisect Benchmark : startup.warm.blank_page Metric : first_main_frame_load_time/first_main_frame_load_time Change : 1.43% | 132.5 -> 133.571428571 Suspected Commit Range 2 commits in range https://chromium.googlesource.com/chromium/src/+log/d64a9dbc5611dcab7a850f8bbadfcc12d6e7b465..04b26cac6e20af2aff50b8a064691c605d79ae13 Revision Result N chromium@397042 132.5 +- 4.41588 14 good chromium@397043 --- --- build failure chromium@397044 135.0 +- 13.2665 21 bad chromium@397050 134.071 +- 4.11443 14 bad chromium@397062 133.762 +- 7.3355 21 bad chromium@397071 133.571 +- 3.6645 14 bad To Run This Test src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=2 --also-run-disabled-tests startup.warm.blank_page Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8979227152916851216 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=5869298486083584 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Aug 16 2017
Doesn't look like we can find the cause of this. Metric has gone back down. |
|||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||
Comment 1 by briander...@chromium.org
, Jun 2 2016