Issue metadata
Sign in to add a comment
|
2.6%-44.9% regression in system_health.common_desktop at 463520:463939 |
||||||||||||||||||||
Issue descriptionSee the link to graphs below.
,
Apr 19 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8981888053706168768
,
Apr 19 2017
=== BISECT JOB RESULTS === Bisect failed for unknown reasons Please contact the team (see below) and report the error. Bisect Details Configuration: mac_pro_perf_bisect Benchmark : system_health.common_desktop Metric : timeToFirstMeaningfulPaint_avg/load_search/load_search_amazon To Run This Test src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.search.amazon system_health.common_desktop Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8981888053706168768 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=5798384163618816 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Apr 19 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8981884365727554400
,
Apr 19 2017
,
Apr 19 2017
=== BISECT JOB RESULTS === Bisect failed for unknown reasons Please contact the team (see below) and report the error. Bisect Details Configuration: mac_pro_perf_bisect Benchmark : system_health.common_desktop Metric : timeToFirstMeaningfulPaint_avg/load_search/load_search_amazon To Run This Test src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.search.amazon system_health.common_desktop Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8981884365727554400 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=5798384163618816 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Apr 19 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8981843579626372752
,
Apr 20 2017
=== Auto-CCing suspected CL author samans@chromium.org === Hi samans@chromium.org, the bisect results pointed to your CL, please take a look at the results. === BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : samans Commit : 2040988b1345e6ce738bd03ab7ccd5f4b73a956b Date : Tue Apr 11 23:58:35 2017 Subject: Use MojoCompositorFrameSink in RendererCompositorFrameSink Bisect Details Configuration: mac_10_11_perf_bisect Benchmark : system_health.common_desktop Metric : timeToFirstMeaningfulPaint_avg/load_search/load_search_amazon Change : 29.79% | 284.604857144 -> 369.381777777 Revision Result N chromium@463726 284.605 +- 204.853 14 good chromium@463833 287.265 +- 215.799 14 good chromium@463840 288.258 +- 287.867 21 good chromium@463844 268.282 +- 185.723 14 good chromium@463845 362.558 +- 49.8569 6 bad <-- chromium@463846 357.905 +- 72.2337 14 bad chromium@463847 370.153 +- 114.55 14 bad chromium@463860 364.71 +- 53.7142 6 bad chromium@463886 364.041 +- 67.4524 9 bad chromium@463939 369.382 +- 70.8859 9 bad To Run This Test src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.search.amazon system_health.common_desktop Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8981843579626372752 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=6164589449838592 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Apr 20 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8981785652682379504
,
Apr 20 2017
=== BISECT JOB RESULTS === Bisect failed for unknown reasons Please contact the team (see below) and report the error. Bisect Details Configuration: mac_retina_perf_bisect Benchmark : system_health.common_desktop Metric : after_load:power_avg/load_social/load_social_vk To Run This Test src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.social.vk system_health.common_desktop Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8981785652682379504 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=5798384163618816 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Apr 24 2017
samans - please either justify, fix your regression or revert your CL.
,
Apr 25 2017
I'm trying to understand what's going on. Not sure if reverting is a good option. There are metrics that improved as a result of my CL. For example, on the same bot (chromium-rel-mac11) and same test (system_health.common_desktop) but a different subtest (load_media vs load_social), the same metric (timeToFirstMeaningfulPaint_avg) improved 57% (See the link below). So basically the revert can cause regression. Not sure if the metric is broken or things legitimately improved/regressed. I'll keep you posted. https://bugs.chromium.org/p/chromium/issues/detail?id=713716
,
Apr 26 2017
We should not trade big regressions in some cases for big speedups in others, at least not without a good reason.
,
Apr 27 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8981142590678512320
,
Apr 27 2017
=== BISECT JOB RESULTS === NO Perf regression found Bisect Details Configuration: mac_retina_perf_bisect Benchmark : system_health.common_desktop Metric : after_load:power_avg/load_social/load_social_vk Revision Result N chromium@463519 14.3637 +- 3.57519 21 good chromium@463909 14.4562 +- 5.65395 21 bad To Run This Test src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.social.vk system_health.common_desktop Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8981142590678512320 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=5798384163618816 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Apr 28 2017
We had some discussion on another bug that might also apply here. We suspect that some small variation in delivery of BeginFrames can sometimes cause big changes in scheduling tasks in the renderer and some metrics can randomly get better or worse. Unless things get worse on average, I don't believe we should be worried. load_search_amazon is worse, but some other metrics are doing better. 10.10% improvement in load_social_twitter: http://crbug.com/715767 52.3% improvement in startup.warm.blank_page: http://crbug.com/715831 38.60% improvement in load_media_dailymotion: http://crbug.com/713716 29.71% improvement in load_tools_stackoverflow: http://crbug.com/711856 68.1% improvement in startup.cold.blank_page: http://crbug.com/715766
,
Aug 10 2017
As I explained before this is just noise so marking as WontFix. |
|||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||
Comment 1 by alexclarke@chromium.org
, Apr 19 2017