Issue metadata
Sign in to add a comment
|
64.4% regression in smoothness.tough_filters_cases at 401756:401809 |
||||||||||||||||||||
Issue descriptionSee the link to graphs below.
,
Jun 24 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@401755 56.7094 3.08298 18 good chromium@401809 56.3306 1.5653 18 bad Bisect job ran on: win_8_perf_bisect Bug ID: 623052 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.tough_filters_cases Test Metric: frame_times/http___rawgit.com_WebKit_webkit_master_PerformanceTests_Animometer_developer.html?test-interval_20_display_minimal_controller_fixed_frame-rate_50_kalman-process-error_1_kalman-measurement-error_4_time-measurement_performance_suite-name_Animometer_test-name_Focus_complexity_100 Relative Change: 0.29% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_8_perf_bisect/builds/1977 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9008967636131924784 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5876971326668800 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Jul 2 2016
Moving this nonessential bug to the next milestone. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Jul 11 2016
Mustaq, can you follow up on this? From the instructions (https://chromium.googlesource.com/chromium/src/+/master/tools/perf/docs/perf_regression_sheriffing.md): "After your shift, please try to follow up on the bugs you filed weekly. Kick off new bisects if the previous ones failed, and if the bisect picks a likely culprit follow up to ensure the CL author addresses the problem. If you are certain that a specific CL caused a performance regression, and the author does not have an immediate plan to address the problem, please revert the CL."
,
Jul 11 2016
Removed recovered plots, kicked off another bisect.
,
Jul 12 2016
I notice that the regression range (https://chromium.googlesource.com/chromium/src/+log/df8088d6e360427e2520b7cc8b9e2452d03ec17b%5E..5d8c668952c6972eb1f833fddccef7d2fd66048b?pretty=fuller) includes https://chromium.googlesource.com/chromium/src/+/f2d7f5e1891703ec4384ededd80f896816921204. I also note that it only seems to have regressed on desktop, and not on Android. It also *may* only be in the software compositor, since the affected Win bots don't have GPUs. enne@, is it possible that begin frame scheduling could have caused this regression in Animometer "Focus"?
,
Jul 12 2016
Strangely, it seems like my Mac Canary doesn't have --enable-begin-frame-scheduling turned on. Which makes this actually easier to test. Running Animometer "Focus" with fixed-complexity 100, --disable-gpu gives ~37FPS in the DevTools FPS meter. Running Animometer "Focus" with fixed-complexity 100, --disable-gpu --enable-begin-frame-scheduling gives ~24-30FPS in the DevTools FPS meter. So there's definitely a regression there. Is that expected? Is it possible that the software compositor is now clamping to vsync where it wasn't before, or something like that? Note that --enable-begin-frame-scheduling doesn't seem to affect the performance of the GPU path (~30FPS).
,
Jul 12 2016
--enable-begin-frame-scheduling was enabled in r401796 and reverted in r402294. This graph did not change, so maybe something else is going on. On Linux this recovered: https://chromeperf.appspot.com/report?sid=77794a9aedebacc2cbac7bf78b84da0682604ba245d8581efdec3825f6edb4bd&rev=401809
,
Jul 13 2016
Good point. It looks like it actually did regress and recover on some other, real GPU bots too (chromium-win7-gpu-intel, chromium-rel-mac-retina): https://chromeperf.appspot.com/report?sid=f66e1935f1103b2ff88a18816e3e63d41117917ee999c22203a9cf77493b49e1&start_rev=399396&end_rev=403557 I'm a little stumped. The non-GPU bots regressed at the same time, but didn't recover. Weird.
,
Jul 18 2016
This seems to have recovered at http://test-results.appspot.com/revision_range?start=405152&end=405220. I'm still no nearer to understanding what the cause was, but I think we can close this.
,
Jul 18 2016
For posterity, graphs showing regression & recovery: https://chromeperf.appspot.com/report?sid=f8990f595beb5894059da2d94b0231823c6c441d67e5e19f0afbdaf4e29857a0&start_rev=401080&end_rev=405932 Bots: chromium-rel-win10, chromium-rel-win8-dual, chromium-rel-win7-x64-dual. Note that the single-core bots were unaffected, and during the regression, dual-core bots dropped to performance of the single-core bot. So perhaps there was a change to renderer threading (guess). |
|||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||
Comment 1 by mustaq@chromium.org
, Jun 24 2016