Issue metadata
Sign in to add a comment
|
1294% regression in time-to-play media.tough_video_cases around 456705:456986 |
||||||||||||||||||||||
Issue descriptionSpike in time to play for two tests on win8. Similar spike in Issue 702332 (also time-to-play on win), but the ranges don't overlap (though they are close).
,
Mar 16 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8984924113811559904
,
Mar 17 2017
=== Auto-CCing suspected CL author sugoi@chromium.org === Hi sugoi@chromium.org, the bisect results pointed to your CL, please take a look at the results. === BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : sugoi Commit : cf232fd17d6a4137620296afebef13eac0d39f18 Date : Tue Mar 14 21:24:41 2017 Subject: SwiftShader DEPS update Bisect Details Configuration: win_8_perf_bisect Benchmark : media.tough_video_cases Metric : time_to_play/tulip2.m4a Change : 1.89% | 94.9889285714 -> 86.4941666667 Revision Result N chromium@456834 94.9889 +- 94.8262 14 good chromium@456837 87.4379 +- 1.83898 14 good chromium@456838 86.6208 +- 0.633223 6 bad <-- chromium@456839 86.77 +- 3.2565 14 bad chromium@456844 86.5467 +- 1.16204 6 bad chromium@456853 86.7133 +- 1.23298 6 bad chromium@456872 87.6279 +- 9.37582 14 bad chromium@456910 86.455 +- 1.5516 9 bad chromium@456986 86.4942 +- 2.25769 6 bad To Run This Test src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests media.tough_video_cases Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8984924113811559904 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=6442930077696000 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Mar 20 2017
,
Mar 20 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8984559461009706992
,
Mar 20 2017
This bisect looks like it found the wrong culprit. I started another bisect from 456834 to 456837.
,
Mar 21 2017
Assigning this to chcunningham@ to find the cause of the regression and assign appropriately.
,
Mar 21 2017
Why is the initial bisect result suspected wrong?
,
Mar 21 2017
The original change is from ~86 ms to ~95 ms, but The bisect identified a difference from ~86 ms to ~87 ms as the culprit. Where are the other 7 ms?
,
Mar 21 2017
We just need to wait and see what the bisect comes back with.
,
Mar 21 2017
=== BISECT JOB RESULTS === NO Perf regression found Bisect Details Configuration: mac_10_12_perf_bisect Benchmark : media.tough_video_cases Metric : vm_resident_set_size_delta_size/crowd.wav_total Revision Result N chromium@456834 7122.86 +- 5317.83 21 good chromium@456837 6782.1 +- 5326.01 21 bad To Run This Test src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests media.tough_video_cases Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8984559461009706992 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=5552382349410304 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Mar 21 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8984494230468833968
,
Mar 21 2017
Whoops, I bisected the wrong thing. I'm trying it again.
,
Mar 21 2017
=== BISECT JOB RESULTS === NO Perf regression found Bisect Details Configuration: win_8_perf_bisect Benchmark : media.tough_video_cases Metric : time_to_play/video.html?src_tulip2.m4a_type_audio Revision Result N chromium@456834 404.45 +- 2290.69 21 good chromium@456837 475.418 +- 2815.14 21 bad To Run This Test src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=video.html.src.tulip2.m4a.type.audio media.tough_video_cases Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8984494230468833968 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=5244048660496384 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Mar 21 2017
A few observations 1. The mac alerts are all now ignored (matching ref graph spike). The windows alerts remain. 2. The "all graphs for this alert" link is busted here (or I'm missing something) - currently shows no alerts. So to see the windows alerts, use this link: https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg9ITc_AgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg9Je0rAkM 3. The range in the title is misleading (I'm changing it now). The windows graphs both show the alert at rev 456986. The last good value on these graphs comes from rev 456705. 4. Given the above, I think this is a duplicate of another time-to-play windows regression in this range: Issue 702332 . The bisects in that issue found the following CL to be at fault: Author : sugoi Moving SwiftShader from component to bundled library Review-Url: https://codereview.chromium.org/2715563002 Cr-Commit-Position: refs/heads/master@{#456764} Will attempt another bisect with this range in mind, but I think we have a good case for resolving as duplicate either way. Caleb, yell if you disagree.
,
Mar 21 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8984486515702711312
,
Mar 21 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8984486497372455376
,
Mar 21 2017
> The "all graphs for this alert" link is busted Now fixed. Turns out I had ignored the alerts by accident
,
Mar 21 2017
Issue 702416 has been merged into this issue.
,
Mar 21 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8984483875981838624
,
Mar 21 2017
=== BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : sugoi Commit : 842efc6922b0cd66febd6f36f04318ebf396c184 Date : Tue Mar 14 18:10:39 2017 Subject: Moving SwiftShader from component to bundled library Bisect Details Configuration: win_8_perf_bisect Benchmark : media.tough_video_cases Metric : time_to_play/video.html?src_tulip2.m4a_type_audio Change : 220.69% | 26.9433333333 -> 86.4033333333 Revision Result N chromium@456705 26.9433 +- 53.838 6 good chromium@456741 20.1328 +- 65.922 9 good chromium@456759 71.0656 +- 509.95 9 good chromium@456762 14.4733 +- 20.6233 6 good chromium@456763 14.0408 +- 16.2005 6 good chromium@456764 87.2483 +- 7.10498 9 bad <-- chromium@456768 88.1006 +- 13.9772 9 bad chromium@456776 86.84 +- 2.72597 9 bad chromium@456846 85.5433 +- 2.04148 6 bad chromium@456986 86.4033 +- 8.43244 6 bad To Run This Test src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=video.html.src.tulip2.m4a.type.audio media.tough_video_cases Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8984486515702711312 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=5823099909439488 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Mar 22 2017
=== BISECT JOB RESULTS === NO Perf regression found Bisect Details Configuration: win_8_perf_bisect Benchmark : media.tough_video_cases Metric : time_to_play/tulip2.m4a Revision Result N chromium@456834 87.5381 +- 3.2447 21 good chromium@456837 88.9705 +- 21.7783 21 bad To Run This Test src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests media.tough_video_cases Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8984483875981838624 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=5864198115950592 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Mar 22 2017
=== BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : sugoi Commit : 842efc6922b0cd66febd6f36f04318ebf396c184 Date : Tue Mar 14 18:10:39 2017 Subject: Moving SwiftShader from component to bundled library Bisect Details Configuration: win_8_perf_bisect Benchmark : media.tough_video_cases Metric : time_to_play/tulip2.m4a Change : 716.18% | 10.6033333333 -> 86.5425 Revision Result N chromium@456705 10.6033 +- 1.13317 6 good chromium@456741 10.4025 +- 0.448874 6 good chromium@456759 11.5325 +- 2.0725 6 good chromium@456762 11.0883 +- 1.51759 6 good chromium@456763 10.8492 +- 0.975664 6 good chromium@456764 87.6842 +- 1.29733 6 bad <-- chromium@456768 87.9933 +- 1.5058 6 bad chromium@456776 87.8367 +- 1.17547 6 bad chromium@456846 86.1633 +- 1.67282 6 bad chromium@456986 86.5425 +- 1.73066 6 bad To Run This Test src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests media.tough_video_cases Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8984486497372455376 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=5856786948554752 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Mar 22 2017
I was wrong. sugoi@'s commit seems to be the culprit.
,
Mar 22 2017
turned out to be a different sugoi@ commit, so you were *technically* correct ;)
,
Mar 22 2017
Still looking into it. It looks like the GPU process is getting created for no reason in these tests. I need to prevent that from happening without breaking anything, which is the tricky part :)
,
Mar 25 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/7ae2ba1892a8b4dee8d7c231ac0e0fbe66588912 commit 7ae2ba1892a8b4dee8d7c231ac0e0fbe66588912 Author: sugoi <sugoi@chromium.org> Date: Sat Mar 25 13:59:47 2017 Solving some telemetry tests' slowness After investigation, it turns out that most telemetry test perfomance regressions were linked to an observer indefinitely waiting for gpu info. This comes from SystemInfoHandler::GetInfo(), where, when SwiftShader was being used, GpuDataManager::IsEssentialGpuInfoAvailable() would always return false, so the gpu info collection would think it needs to set a GpuObserver in order to wait for the info to be loaded. By having GpuDataManager::IsEssentialGpuInfoAvailable() return true when SwiftShader is used, the gpu info is returned immediately and this solves the hang. BUG= 702417 Review-Url: https://codereview.chromium.org/2770933008 Cr-Commit-Position: refs/heads/master@{#459651} [modify] https://crrev.com/7ae2ba1892a8b4dee8d7c231ac0e0fbe66588912/content/browser/gpu/gpu_data_manager_impl_private.cc
,
Apr 11 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8982640125828895344
,
Apr 11 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8982611970075228416
,
Apr 11 2017
=== Auto-CCing suspected CL author jbudorick@chromium.org === Hi jbudorick@chromium.org, the bisect results pointed to your CL, please take a look at the results. === BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : jbudorick Commit : 761f60864c37e7af6fe9dba49256a8b014860049 Date : Tue Mar 14 21:20:28 2017 Subject: [android] Clean up test_runner.py arguments. Bisect Details Configuration: win_8_perf_bisect Benchmark : media.tough_video_cases Metric : time_to_play/video.html?src_tulip2.m4a_type_audio Change : 65.12% | 1410.28277778 -> 491.912222222 Revision Result N chromium@456834 1410.28 +- 1868.72 9 good chromium@456835 88.46 +- 1.00591 6 bad <-- chromium@456836 89.85 +- 5.24778 6 bad chromium@456837 491.912 +- 1219.0 9 bad To Run This Test src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=video.html.src.tulip2.m4a.type.audio media.tough_video_cases Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8982640125828895344 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=5244048660496384 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you! |
|||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||
Comment 1 by chcunningham@google.com
, Mar 16 2017