Issue metadata
Sign in to add a comment
|
6.7%-18.3% regression in media.tough_video_cases_tbmv2 at 507199:507282 |
||||||||||||||||||||
Issue descriptionSee the link to graphs below.
,
Oct 9 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8966187959242384016
,
Oct 9 2017
=== Auto-CCing suspected CL author miu@chromium.org === Hi miu@chromium.org, the bisect results pointed to your CL, please take a look at the results. === BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : Yuri Wiitala Commit : 081f9dcc223a535202a52c351392514ad5635650 Date : Fri Oct 06 23:50:16 2017 Subject: Add scaling capability to CopyOutputRequests. (Part 2 of 2) Bisect Details Configuration: winx64_high_dpi_perf_bisect Benchmark : media.tough_video_cases_tbmv2 Metric : memory:chrome:all_processes:reported_by_os:system_memory:proportional_resident_size_avg/video.html?src_tulip2.ogg_type_audio Change : 18.29% | 187707297.5 -> 222034335.167 Revision Result N chromium@507198 187707298 +- 1622057 6 good chromium@507219 187229869 +- 4232813 6 good chromium@507230 187687969 +- 1702872 6 good chromium@507231 187960821 +- 2078815 6 good chromium@507232 220428621 +- 3686156 6 bad <-- chromium@507233 221162699 +- 3827999 6 bad chromium@507235 222334997 +- 2263746 6 bad chromium@507240 220979525 +- 2954700 6 bad chromium@507282 222034335 +- 3335706 6 bad To Run This Test src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=video.html.src.tulip2.ogg.type.audio media.tough_video_cases_tbmv2 More information on addressing performance regressions: http://g.co/ChromePerformanceRegressions Debug information about this bisect: https://chromeperf.appspot.com/buildbucket_job_status/8966187959242384016 For feedback, file a bug with component Speed>Bisection
,
Oct 10 2017
crouleau: Looks like my change regressed the memory usage of your perf test. The thing is, it should not be possible unless your perf test is doing screen video capture (i.e., making CopyOutputRequests with the "source" property set). Can you shed some light on how that might happen? If your perf test does, in fact, intend to capture the screen; then please resolve this as WAI. The memory increase is attributed to the extra caching of textures introduced by my change (for performance reasons).
,
Oct 10 2017
Yuri, John filed this bug since he was the sheriff during this time. As part of his sheriff responsibilities, he will help you look into it and answer questions. There are instructions above on how to run the test and the code for it is here https://cs.chromium.org/chromium/src/tools/perf/page_sets/tough_video_cases/video.html and here https://cs.chromium.org/chromium/src/tools/perf/page_sets/tough_video_cases.py If there are questions that John cannot answer, then I would be happy to help. I don't think the tests should be doing any video capture, but it's possible that something in Telemetry that I don't know about is doing it. This particular test intends to simply play an audio file.
,
Oct 11 2017
,
Oct 12 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8965962803868298512
,
Oct 12 2017
Hmm...An unrelated Android test too. Weird. I wonder if this has to do with tab snapshots of backgrounded tabs...
,
Oct 12 2017
=== BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : Yuri Wiitala Commit : 081f9dcc223a535202a52c351392514ad5635650 Date : Fri Oct 06 23:50:16 2017 Subject: Add scaling capability to CopyOutputRequests. (Part 2 of 2) Bisect Details Configuration: android_nexus5_perf_bisect Benchmark : system_health.memory_mobile Metric : memory:chrome:all_processes:reported_by_os:gpu_memory:proportional_resident_size_avg/long_running_tools/long_running_tools_gmail-background Change : 11.99% | 60965278.9542 -> 68272442.5621 Revision Result N chromium@507175 60965279 +- 178605 6 good chromium@507208 60945093 +- 177492 6 good chromium@507225 60890473 +- 59804.0 6 good chromium@507229 60901336 +- 139.107 6 good chromium@507231 61076145 +- 598305 6 good chromium@507232 68359770 +- 529662 6 bad <-- chromium@507233 68316060 +- 358911 6 bad chromium@507241 68272443 +- 75581.8 6 bad Please refer to the following doc on diagnosing memory regressions: https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=long.running.tools.gmail.background system_health.memory_mobile More information on addressing performance regressions: http://g.co/ChromePerformanceRegressions Debug information about this bisect: https://chromeperf.appspot.com/buildbucket_job_status/8965962803868298512 For feedback, file a bug with component Speed>Bisection
,
Oct 17 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8965465797213315024
,
Oct 17 2017
=== BISECT JOB RESULTS === NO Perf regression found, tests failed to produce values Bisect Details Configuration: winx64_high_dpi_perf_bisect Benchmark : media.desktop Metric : memory:chrome:all_processes:reported_by_os:system_memory:proportional_resident_size_avg/video.html?src_crowd.ogg_type_audio To Run This Test src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=video.html.src.crowd.ogg.type.audio media.desktop More information on addressing performance regressions: http://g.co/ChromePerformanceRegressions Debug information about this bisect: https://chromeperf.appspot.com/buildbucket_job_status/8965465797213315024 For feedback, file a bug with component Speed>Bisection
,
Oct 24 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8964816103938388528
,
Oct 24 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8964816085618881584
,
Oct 24 2017
=== BISECT JOB RESULTS === NO Perf regression found, tests failed to produce values Bisect Details Configuration: winx64_high_dpi_perf_bisect Benchmark : media.desktop Metric : memory:chrome:all_processes:reported_by_os:system_memory:native_heap:proportional_resident_size_avg/video.html?src_crowd1080_vp9.webm To Run This Test src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=video.html.src.crowd1080.vp9.webm media.desktop More information on addressing performance regressions: http://g.co/ChromePerformanceRegressions Debug information about this bisect: https://chromeperf.appspot.com/buildbucket_job_status/8964816103938388528 For feedback, file a bug with component Speed>Bisection
,
Oct 24 2017
Bisects aren't going to work anymore for this because we changed the benchmark's name :( Sorry. Bisect is very confident that your CL was the culprit. You can rerun the benchmark manually. It is now called media.desktop instead of media.tough_video_cases.
,
Oct 24 2017
=== BISECT JOB RESULTS === NO Perf regression found, tests failed to produce values Bisect Details Configuration: winx64_high_dpi_perf_bisect Benchmark : media.desktop Metric : memory:chrome:all_processes:reported_by_os:system_memory:native_heap:proportional_resident_size_avg/video.html?src_tulip2.ogg_type_audio To Run This Test src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=video.html.src.tulip2.ogg.type.audio media.desktop More information on addressing performance regressions: http://g.co/ChromePerformanceRegressions Debug information about this bisect: https://chromeperf.appspot.com/buildbucket_job_status/8964816085618881584 For feedback, file a bug with component Speed>Bisection
,
Nov 3 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8963909174533902448
,
Nov 3 2017
=== BISECT JOB RESULTS === NO Perf regression found, tests failed to produce values Bisect Details Configuration: winx64_high_dpi_perf_bisect Benchmark : media.desktop Metric : memory:chrome:all_processes:reported_by_os:system_memory:proportional_resident_size_avg/video.html?src_tulip2.vp9.webm To Run This Test src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=video.html.src.tulip2.vp9.webm media.desktop More information on addressing performance regressions: http://g.co/ChromePerformanceRegressions Debug information about this bisect: https://chromeperf.appspot.com/buildbucket_job_status/8963909174533902448 For feedback, file a bug with component Speed>Bisection
,
Jan 25 2018
Closing, as metrics are much better nowadays than pre-regression. I'm pretty sure the media metrics regressed for non-media reasons. |
|||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||
Comment 1 by 42576172...@developer.gserviceaccount.com
, Oct 9 2017