Issue metadata
Sign in to add a comment
|
2.2%-117.1% regression in performance_browser_tests at 385569:385654 |
||||||||||||||||||||
Issue descriptionSee the link to graphs below.
,
Apr 8 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Value Std. Dev. Num Values Good? chromium@385616 19.844749 0.44046 18 good chromium@385631 19.700073 0.317633 18 bad Bisect job ran on: win_perf_bisect Bug ID: 601792 Test Command: .\src\out\Release\performance_browser_tests.exe --test-launcher-print-test-stdio=always --enable-gpu Test Metric: CastV2Performance_gpu_30fps_fast/capture_duration Relative Change: 2.36% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/6451 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9015937875199100672 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=601792 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Apr 15 2016
Some of these regressions recovered, some didn't. We're likely dealing with multiple regressions. Submitted more bisects.
,
Apr 15 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Value Std. Dev. Num Values Good? chromium@385397 105.013827 5.216789 18 good chromium@385402 104.005262 3.358495 18 bad Bisect job ran on: winx64ati_perf_bisect Bug ID: 601792 Test Command: python src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --also-run-disabled-tests page_cycler.morejs Test Metric: warm_times/page_load_time Relative Change: 0.75% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64ati_perf_bisect/builds/1323 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9015282726717894096 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=601792 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Apr 16 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Value Std. Dev. Num Values Good? chromium@385616 18.28441 0.129076 18 good chromium@385653 18.282943 0.150417 18 bad Bisect job ran on: winx64ati_perf_bisect Bug ID: 601792 Test Command: .\src\out\Release_x64\performance_browser_tests.exe --test-launcher-print-test-stdio=always --enable-gpu Test Metric: TabCapturePerformance_comp_gpu_webrtc/CaptureSucceeded Relative Change: 0.10% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64ati_perf_bisect/builds/1322 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9015282729569522160 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=601792 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Apr 22 2016
Hrmph. More of these recovering, still trying more bisects.
,
May 6 2016
Kick off another bisect in https://chromeperf.appspot.com/buildbucket_job_status/9013373310007736640
,
May 13 2016
Previous bisect jobs failed due to error in bisect recipe: crbug.com/606281 Re-launched another bisect:https://chromeperf.appspot.com/buildbucket_job_status/9012758313767151632
,
Jun 3 2016
prasadv@: The bisect in #8 seemed to succeed but didn't post anything on the bug, and I can't find the results in the buildbot link. Do you know where the results are?
,
Jun 24 2016
Ping Prasa: can you answer rmcilroy's question in #9?
,
Jul 8 2016
Still not recovered. Kicked off another bisect.
,
Jul 15 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9007037833921935888
,
Jul 16 2016
=== Auto-CCing suspected CL author wez@chromium.org === Hi wez@chromium.org, the bisect results pointed to your CL below as possibly causing a regression. Please have a look at this info and see whether your CL be related. ===== BISECT JOB RESULTS ===== Status: completed ===== SUSPECTED CL(s) ===== Subject : Patch to try dump-on-DCHECK. Author : wez Commit description: This patch does two things: 1. Adds a flag to switch DCHECK from logging, dumping, and then crashing the process, to only uploading a crash dump, and only on the first failed DCHECK in each process. 2. Forces that flag, and DCHECK_ALWAYS_ON, on in Windows official builds. All non-debug e.g. CHECK behaviours remain unchanged; the intended effect is for DCHECKs to switch from no-ops to uploading dumps without crashing, in Windows official builds. Note that this CL is intended to be landed, a branch cut to release from, and then immediately reverted; it is not intended to be landed in Chromium for any longer period. BUG=596231 Committed: https://crrev.com/6436ac7ddec4b2b3aba4ee38aabe7dffe238a077 Cr-Commit-Position: refs/heads/master@{#383894} Committed: https://crrev.com/bac26c8a840909a679a5a74557fa6f4f60ae9e07 Cr-Commit-Position: refs/heads/master@{#384011} Committed: https://crrev.com/16502cb143f737bafad5d035b8ed6d76aabce288 Cr-Commit-Position: refs/heads/master@{#384675} Review URL: https://codereview.chromium.org/1814423002 Cr-Commit-Position: refs/heads/master@{#385413} Commit : b80aa8f5c269c9ee4f3a00b03840ea3df68f77d1 Date : Wed Apr 06 09:13:42 2016 ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@385390 120.168 1.21779 5 good chromium@385404 121.732 2.21756 5 good chromium@385409 121.356 1.43411 5 good chromium@385411 121.009 2.52168 12 good chromium@385412 123.688 6.91187 12 good chromium@385413 129.745 1.72946 8 bad <-- chromium@385420 130.828 3.15989 5 bad Bisect job ran on: win_8_perf_bisect Bug ID: 601792 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests page_cycler.morejs Test Metric: warm_times/page_load_time Relative Change: 8.87% Score: 99.9 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_8_perf_bisect/builds/2041 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9007037833921935888 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5824636831399936 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Jul 18 2016
crrev.com/1814423002 was a one-canary experiment, expected to impact performance adversely. It was reverted on Apr 7th by crrev.com/1870633003. Did performance not recover after the revert?
,
Jul 19 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9006701361830643760
,
Jul 19 2016
On this particular test it looks like it came down on April 22. So I'm not sure what happened there - but we're good on that test. Many of the performance_browser_tests haven't recovered or haven't recovered fully. I'll try bisecting on those.
,
Jul 19 2016
,
Jul 19 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9006701315147526192
,
Jul 19 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@385568 19.7775 0.297232 18 good chromium@385631 19.8096 0.253373 17 bad Bisect job ran on: win_perf_bisect Bug ID: 601792 Test Command: .\src\out\Release\performance_browser_tests.exe --test-launcher-print-test-stdio=always --enable-gpu Test Metric: CastV2Performance_gpu_30fps_fast/capture_duration Relative Change: 0.93% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/6713 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9006701361830643760 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5863113363030016 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Jul 19 2016
=== Auto-CCing suspected CL author xjz@chromium.org === Hi xjz@chromium.org, the bisect results pointed to your CL below as possibly causing a regression. Please have a look at this info and see whether your CL be related. ===== BISECT JOB RESULTS ===== Status: completed ===== SUSPECTED CL(s) ===== Subject : Remove --tab-capture-upscale/downscale-quality. Author : xjz Commit description: Historically three scalers were designed for tab-capture upscaling and downscaling: fast, good, best. Experiments show that "good" or "best" down-scalers may cause downstream encoders to perform much worse, as the detail they maintain adds excessive entropy to the images. This change removes the command-line switches and always uses "fast" for video down-scaling and "best" for upscaling. BUG= 350135 Review URL: https://codereview.chromium.org/1690013002 Cr-Commit-Position: refs/heads/master@{#385626} Commit : 9000b88d353b4929a21b4cc5fcb2b87abadd37a1 Date : Thu Apr 07 02:28:51 2016 ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@385616 7.4951 0.152772 5 good chromium@385625 7.47794 0.0761176 5 good chromium@385626 10.5866 0.210105 5 bad <-- chromium@385627 10.5622 0.148259 5 bad chromium@385628 10.522 0.174821 5 bad chromium@385630 10.5646 0.225541 5 bad chromium@385634 10.4706 0.0761088 5 bad chromium@385653 10.6022 0.14883 5 bad Bisect job ran on: winx64ati_perf_bisect Bug ID: 601792 Test Command: .\src\out\Release_x64\performance_browser_tests.exe --test-launcher-print-test-stdio=always --enable-gpu Test Metric: CastV2Performance_gpu_24fps/capture_duration Relative Change: 41.45% Score: 99.9 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64ati_perf_bisect/builds/1438 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9006701315147526192 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5789075643039744 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Jul 20 2016
The CL removes the command-line switches for tab capture scalers and sets default up/down scalers. Now the down-scaling uses the same scaler as before, but the up-scaling changes to use the "best" scaler from the "fast" one. So in up-scaling cases, we sacrifice the performance for higher quality. |
|||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||
Comment 1 by rsch...@chromium.org
, Apr 8 2016