Issue metadata
Sign in to add a comment
|
50.5% regression in media.android.tough_video_cases_tbmv2 at 464134:464233 |
||||||||||||||||||||
Issue descriptionSee the link to graphs below.
,
Apr 19 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8981888069724745408
,
Apr 19 2017
=== BISECT JOB RESULTS === Bisect failed for unknown reasons Please contact the team (see below) and report the error. Bisect Details Configuration: android_nexus5X_perf_bisect Benchmark : media.android.tough_video_cases_tbmv2 Metric : cpu_time_percentage_avg/video.html?src_tulip2.ogv To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=video.html.src.tulip2.ogv media.android.tough_video_cases_tbmv2 Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8981888069724745408 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=5812046991458304 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Apr 19 2017
,
Apr 19 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8981876813986874016
,
Apr 19 2017
=== Auto-CCing suspected CL author johnchen@chromium.org === Hi johnchen@chromium.org, the bisect results pointed to your CL, please take a look at the results. === BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : johnchen Commit : 97a6fea550e9d2fcd81649cffbb0de8b4750af9a Date : Wed Apr 12 23:49:54 2017 Subject: Disable atrace in TBMv2 media benchmarks Bisect Details Configuration: android_nexus5X_perf_bisect Benchmark : media.android.tough_video_cases_tbmv2 Metric : cpu_time_percentage_avg/video.html?src_tulip2.ogv Change : 38.67% | 0.809635391184 -> 1.12268116024 Revision Result N chromium@464133 0.809635 +- 0.0395952 6 good chromium@464183 0.816349 +- 0.036941 6 good chromium@464208 0.818206 +- 0.0568559 6 good chromium@464209 0.812239 +- 0.0603066 6 good chromium@464210 1.14372 +- 0.0793586 6 bad <-- chromium@464212 1.12888 +- 0.0901339 6 bad chromium@464215 1.12617 +- 0.0623402 6 bad chromium@464221 1.1257 +- 0.0687016 6 bad chromium@464233 1.12268 +- 0.0731864 6 bad To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=video.html.src.tulip2.ogv media.android.tough_video_cases_tbmv2 Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8981876813986874016 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=5812046991458304 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Apr 19 2017
The affected media, tulip2.ogv, isn't supported on Android, and the results reported for this media isn't actually meaningful. This isn't a true perf regression. I will stop testing this media on Android.
,
Apr 20 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/5b42152cb32481bca8a330b023a1f2c57ad38cd6 commit 5b42152cb32481bca8a330b023a1f2c57ad38cd6 Author: johnchen <johnchen@chromium.org> Date: Thu Apr 20 05:00:06 2017 Remove theora from Android media benchmarks Some media benchmark pages use theora format, which isn't supported on Android. One of the media, tulip2.ogv was measured on Android and consumed lab resources while reporting meaningless data. This CL modifies media benchmarks to stop playing theora format media on Android. BUG= 713113 Review-Url: https://codereview.chromium.org/2826293003 Cr-Commit-Position: refs/heads/master@{#465901} [modify] https://crrev.com/5b42152cb32481bca8a330b023a1f2c57ad38cd6/tools/perf/benchmarks/media.py
,
Apr 20 2017
|
|||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||
Comment 1 by alexclarke@chromium.org
, Apr 19 2017